Cracow Grid Workshop, November 5-6, 2001 Concepts for implementing adaptive finite element codes for grid computing Krzysztof Banaś, Joanna Płażek Cracow University of Technology
Cracow Grid Workshop, November 5-6, 2001 Outline Motivation FEM code as grid application Current project Future plans
Cracow Grid Workshop, November 5-6, 2001 Grid environment from application developer perspective Open environment (coupling with external modules, standardization of interfaces) High performance communication libraries Simpler code development Cooperation with other groups Broad user base
Cracow Grid Workshop, November 5-6, 2001 State of the art for the FEM Complex problems (nonlinear, coupled, multi- scale, multi-physics) Complex codes (adaptive meshes, multi-level solvers, parallel and distributed execution) Steep learning curves for users Unmanageable software for developers
Cracow Grid Workshop, November 5-6, 2001 Preprocessing geometrical modeling mesh generation Postprocessing GUI interactive visualization, collaboration and steering ODE integration field discretization adaptive mesh services linear solver Computational Module Example finite element PSE
Cracow Grid Workshop, November 5-6, 2001 Current project - existing software Adaptive FE codes in Fortran 77, Fortran 90, C Areas of application: acoustics, elasticity, compressible fluid flow, multi-phase flow, electro- magnetism hp-adaptivity Multi-level solvers Parallel versions for PVM, DSM and MPI
Cracow Grid Workshop, November 5-6, 2001 Current project - application areas Scattering of acoustic and electro-magnetic waves
Cracow Grid Workshop, November 5-6, 2001 Current project - application areas Subsurface flow simulation (discontinuous approximation, multi-level solvers) Solver/preconditioner Number of mesh levels 2345 GMRES/single-level block ILU GMRES/multi-level block ILU7899 Table1. Number of linear solver iterations to reach convergence
Cracow Grid Workshop, November 5-6, 2001 Current project - application areas Compressible fluid flow
Cracow Grid Workshop, November 5-6, 2001 Current project Modularization Standardization of interfaces Testing communication mechanisms Coupling with visualization software
Cracow Grid Workshop, November 5-6, 2001 FE code modules Assumptions: –code is built from small specialized modules –modules form separate libraries –modules have their own data structure –interfaces are designed for F90, C and C++ –modules may define their own data distribution –modules are glued together by problem dependent code
Cracow Grid Workshop, November 5-6, 2001 FE code modules Adaptive mesh procedures: –Read_mesh_data( mesh_id, read_type) –Create_node(mesh_id, parameters, &node_id) –Divide_element(mesh_id, element_id, element_sons_ids) –Get_element_father(mesh_id, element_id, &father_id) –Get_face_edges(mesh_id, face_id, face_edges_ids) –Get_element_neighbors(mesh_id, element_id, neighbors_ids) –...
Cracow Grid Workshop, November 5-6, 2001 FE code modules Field discretization procedures: –Get_face_dofs( field_id, face_id, dofs) –L2_project_dofs_from_element_to_element( field_id, element_from_id, element_to_id) –Get_solution_at_point( field_id, point_coordinates, solution) –Set_uniform_degree_p( field_id) –Write_element_solution( field_id, element_id, solution_dofs) –...
Cracow Grid Workshop, November 5-6, 2001 FE code modules Linear equations solver procedures: –Set_preconditioner_options(solver_id, preconditioner_type, preconditioner_parameters) –Assemble_element_stiffness_matrix( solver_id, assembly_type, block_sizes, block_positions, stiffness matrices, load_vectors) –Solve_system(solver_id) –Store_face_solution( solver_id, face_id, dofs) –Free_solver_memory(solver_id) –...
Cracow Grid Workshop, November 5-6, 2001 Why adaptive FE code can be attractive as grid application? Powerful tool for scientific and engineering analysis Complex, multi-component code Large amounts of data produced and exchanged among modules Adaptive meshes require adaptive resources
Cracow Grid Workshop, November 5-6, 2001 Future plans Sky is the limit... Open, flexible, extensible FE software Large-scale distributed computing Integrated problem solving environment