ITAPS

Mesh Adapt Service

The Mesh Adapt service creates unstructured anisotropic meshes using local mesh modification that satisfy a prescribed anisotropic size field. Parallel mesh adapt works with large scale problems and focuses on carefully selected local mesh operators, such as refinement, coarsening, swapping and node repositioning, to increase the quality of the mesh while satisfying the desired size field. Parallel adaptive methods dramatically reduce the number of degrees of freedom needed to obtain a given level of accuracy. Performing parallel mesh adaptation requires local mesh migration algorithm to move mesh entities on or near by the common boundary between several partitions [1, 2].

The Mesh Adapt service is the intelligent control of mesh modification procedures which are comprised of four related high level components: mesh refinement, mesh coarsening, projecting boundary vertices onto curved geometry and element shape correction. The mesh is efficiently aligned to the mesh metric field by incremental refinement and coarsening based on edge length analysis with respect to the mesh metric field. The Mesh Adapt service supports anisotropic mesh adaptation accounting for mixed element types and boundary layer meshes. It can adapt meshes on domains of interest to accurately and efficiently compute key quantities, especially near wall quantities like wall shear stress [3].

Given a geometric domain (which can be a mesh model or a CAD model), a current mesh and a desired mesh metric field defined over that mesh, a series of controlled mesh modification steps are applied to obtain a new mesh that satisfies the given mesh metric field. The mesh modification algorithm is composed of three stages:

  • mesh coarsening to eliminate the initial short edges;
  • mesh refinement that includes operations to ensure proper geometric approximation of the mesh to the geometric domain and proper shape to match the metric field;
  • mesh optimization to improve the quality of the final mesh

In parallel, the refinement stage requires a single communication step. Collapse, swap and node repositioning can require mesh migration [4]. Mesh migration [5] and involves the exchange of irregularly structured messages. IPComMan [6] is used to boost the communication performance and hide communication delays.

Mesh Adapt supports FMDB and partially iMesh and iMeshP ITAPS interface implementation with fill iMesh/iMeshP implementation underway. The Mesh Adapt Service has been tested in serial and parallel in the variety of applications. In the "whole body" simulation, the solution based, anisotropic, boundary layer mesh adaptation was used to improve the mesh quality. After four sets of flow computation and mesh modification, an adapted mesh with 9.0 million nodes and 42.8 million elements was obtained. Figure 1 depicts the "whole" body model with the labels of sections and the boundary conditions. Figure 2 shows the magnified views of selected arteries in comparison with the initial mesh [7]. The bigger parallel mesh adaptation run is the AAA test case with creation and moving of five air bubbles, which is shown in Figure 3. To construct air bubbles, one step of uniform refinement, predictive load balancing [8] and analytical size field were used. The run is dominated by refinement with the limited number of coarsening and swapping operations and mesh migration to support them. In Table 1, the strong scaling results for construction of air bubbles on Ranger are presented, with the initial uniform mesh of 4.3 million elements and the final mesh of 730 million elements. The biggest test in terms of processors for moving already created air bubbles was run on 32k cores on the IBM Blue Gene/P computer, with the initial mesh of 165 million tetrahedra, and 188 million tetrahedra in the final mesh. It involves the movement of air bubbles by a distance of 1/5th of their radius. Other examples of meshes as large as 8 billion elements have been run.

whole body model

Figure 1. 4 The "whole" body model with the labels of sections (left) and the boundary conditions (right).

comparison meshes

Figure 2. The comparison of initial and adapted meshes.

parellel mesh

Figure 3. Parallel mesh adaptation for moving air bubbles.

# of Procs

Time (sec)

Scalability

2048

110.649

1

4096

57.421

0.96

8192

35.408

0.79

16384

20.645

0.67

Table 1. Strong scaling studies for the air bubble construction test on Range

References

[1] F. Alauzet, X. Li, S. Seol, and M.S. Shephard, Parallel anisotropic 3D mesh adaptation by mesh modification, Engng. Comput., 21(3):247-258, 2006.

[2] M.S. Shephard, K.E. Jansen, O. Sahni and L.A. Diachin, Parallel Adaptive Simulations on Unstructured Meshes, Journal of Physics: Conference Series, 78-012053, 10 pages, 2007.

[3] O. Sahni, K.E. Jansen, M.S. Shephard, C.A. Taylor M.W. and Beal, Adaptive boundary layer meshing for viscous flow simulations, Engng. Comput., 24 (3): 267-285, 2008.

[4] H. L. de Cougny and Mark S. Shephard, Parallel refinement and coarsening of tetrahedral meshes, Int. J. Numer. Meth. Eng., 46: 1101-1125, 1999.

[5] A. Ovcharenko, O. Sahni, K.E. Jansen, C.D. Carothers and M.S. Shephard, Neighborhood communication paradigm to increase scalability in large-scale dynamic scientific applications. Parallel Comput., under review, 2010.

[6] E.S. Seol, M.S. Shephard, Efficient distributed mesh data structure for parallel automated adaptive analysis, Eng. Comput., 22: 197-213, 2006.

[7] M. Zhou, O. Sahni, H. J. Kim, C. A. Figueroa, C. A. Taylor, M. S. Shephard and K. E. Jansen, Cardiovascular flow simulation at extreme scale, Comput. Mech., 46 (1): 71-82, 2009. [8] M. Zhou, T. Xie, S. Seol, M.S. Shephard, O. Sahni, K.E. Jansen, Tools to Support Mesh Adaptation on Massively Parallel Computers, Engng. Comput., 2010, under review.

 

Contact points

Aleksandr Ovcharenko, Mark S. Shephard
Email: {shurik@scorec.rpi.edu, shephard@scorec.rpi.edu}

Funding sources

NSF Grant No. 0749152, DOE Grant No. DE-FC02-06ER25769, NASA SBIR, KAPL, SLAC

Download

The repository download is accessible at the following URL:
http://redmine.scorec.rpi.edu/anonsvn/meshadapt/trunk/