Implement general 2D forest meshes

[lroberts36]

PR Summary

This PR implements the minimum necessary technology for defining meshes based on generally connected forests of octrees. I have left details of coordinates to a future PR, so for now each tree must cover a hyper-rectangular domain defined by $\vec{x}_\text{min}$ and $\Delta \vec{x}$. Some rough notes about my implementation are in ForestOfOctreesNotes.pdf

The four major things this PR accomplishes:

• Add forest::Node, forest::Edge, and forest::Face for defining the forest mesh topology. forest::Face contains routines for finding neighbors and then calculating the correct logical coordinate transformations to those neighbors. For now, I have stuck to defining 2D forests but the mechanisms used here should generalize to 3D relatively easily in a future PR.
• Add forest::Forest::Make2D that builds a Forest from a forest::ForestDefinition. ForestDefinition contains a set of Faces, boundary conditions defined on forest::Edges, RegionSizes associated with each block, and a set of initial LogicalLocations to refine.
• Add Mesh constructor as well as ParthenonManager::ParthenonInitPackagesAndMesh that takes a forest::ForestDefinition and constructs a mesh based on a general 2D forest.
• Thread logical coordinate transformations through boundary communication routines so that neighbor blocks that are non-simply oriented can communicate boundaries.

Smaller things also included in the PR:

• Rename RelativeOrientation -> LogicalLocationCoordinateTransformation for clarity.
• Move MeshBndryFnctn to Mesh::forest::trees instead of Mesh and add appropriate registration routines.
• Adds an example and associated regression test that communicates gids and other information through boundaries to ensure that neighbor communication is occurring correctly. The gold file has just been verified by hand.
• Adds a unit test that performs refinement on non-simply connected forests.

Things this PR does not do (but maybe a future PR should):

• Deal with physical coordinates on trees beyond the simplest choice.
• Implement periodic boundary conditions for general forests.
• Allow for 3D meshes.
• Allow for 6+ valent points in the macro-mesh.
• Deal with corner communication for 5-valent corners or generally store information about valence of corners and edges in MeshBlock.
• Make a better interface for users to define general forests.

Some examples of properly nested refinement near singular mesh points (this mesh configuration is tested in the newly added unit test):

Boundaries filled with neighbor gids in a 2x2 forest where the lower left tree coordinates are rotated by 90 degrees relative to the other trees (results of the boundary communication regression test):

PR Checklist

• Refactor Mesh Constructors #1055 is merged
• Particle BCs without relocatable device code #1026 is merged
• Code passes cpplint
• New features are documented.
• Adds a test for any bugs fixed. Adds tests for new features.
• Code is formatted
• Changes are summarized in CHANGELOG.md
• CI has been triggered on Darwin for performance regression tests.
• Docs build
• (@lanl.gov employees) Update copyright on changed files

[lroberts36]

Passes riot and parthenon-mhd tests.

[lroberts36]

Agrees with old athenapk blast wave results to 1.e-13.

[Yurlungur]

This is really cool. And I think I understood most of what's going on here. But this is pretty complicated and introduces a lot of new concepts to parthenon that are hard to visualize and reason about. I feel pretty strongly that this MR needs documentation associated with it.

[Yurlungur]

how should I interpret this initialization? These are coordinates, but of what?

[Yurlungur]

It would probably be nice to have some automated machinery for common useful topologies to save users from having to manually define mesh structures.

[lroberts36]

Right now, nodes store a position in the x-y plane. This was convenient for making plots like the refinement plot shown above but the positions are unused otherwise. Probably in the future we don't actually want to associate nodes with positions since they are just there for specifying the topology of the mesh.

[Yurlungur]

In a forest setting, how are nx1, nx2, nx3 interpreted?

[lroberts36]

They are ignored. These should probably be removed from this input deck.

[Yurlungur]

same Q as above

[lroberts36]

Static refinement of a region is not currently supported for general forests, so this will be ignored. Currently, to statically refine a general forest you just have to initially tell it the the LogicalLocations that you want to be refined.

[Yurlungur]

this (and above) would probably be good to explain. And the extraneous lines removed from the input deck.

[lroberts36]

This is really cool. And I think I understood most of what's going on here. But this is pretty complicated and introduces a lot of new concepts to parthenon that are hard to visualize and reason about. I feel pretty strongly that this MR needs documentation associated with it.

Are you satisfied with the latex document that is linked in the PR description or do you think it needs more detail? I don't really want to write this up in Sphinx, since I don't know how to deal with tikz and equations there. Do we have a way of including latex documentation in the repo?

[Yurlungur]

Are you satisfied with the latex document that is linked in the PR description or do you think it needs more detail?

I think it works well as a description of your design and implementation. I don't think it suffices as a user-level explanation of how to actually build a forest or use it. Ideally what I would like to see is a link to your latex writeup in the docs folder---and maybe the tex source under version control in the repo---as well as some user-focused references for how to actually build a tree.

I don't really want to write this up in Sphinx, since I don't know how to deal with tikz and equations there. Do we have a way of including latex documentation in the repo?

There seems to be some way to do this... but a cursory skim doesn't provide a clean answer. It does support latex syntax for equations, but your doc has quite a bit of inline math.

[lroberts36]

I think it works well as a description of your design and implementation. I don't think it suffices as a user-level explanation of how to actually build a forest or use it. Ideally what I would like to see is a link to your latex writeup in the docs folder---and maybe the tex source under version control in the repo---as well as some user-focused references for how to actually build a tree.

I think that a lot of the practical interface with forests is going to change in near future PRs. For instance, the simplest way to specify forests would be to have blocks for each face in the parameter input that contain a list of the nodes defining the face and some information about the coordinates of the face (although this would become unwieldy for forests with lots of trees). I also haven't completely figured out how to deal with singular valence points, more general coordinates, and visualization. As a result, I don't think it makes sense to write up detailed documentation of how to use forests in this PR, since it is really focused on under the hood changes (plus the minimum necessary interface to write some tests).

There seems to be some way to do this... but a cursory skim doesn't provide a clean answer. It does support latex syntax for equations, but your doc has quite a bit of inline math.

Also tikz needs to work for the diagrams (which I think are key to understanding what is going on).

[Yurlungur]

I think that a lot of the practical interface with forests is going to change in near future PRs. For instance, the simplest way to specify forests would be to have blocks for each face in the parameter input that contain a list of the nodes defining the face and some information about the coordinates of the face (although this would become unwieldy for forests with lots of trees). I also haven't completely figured out how to deal with singular valence points, more general coordinates, and visualization. As a result, I don't think it makes sense to write up detailed documentation of how to use forests in this PR, since it is really focused on under the hood changes (plus the minimum necessary interface to write some tests).

Fair enough. Can we at least stick the tex source file in the repo under version control and link to the pdf in the docs?

Also tikz needs to work for the diagrams (which I think are key to understanding what is going on).

I think there's some way to just use a tex file but I couldn't figure it out from the docs... And yes tikz is life.

[lroberts36]

I think there's some way to just use a tex file but I couldn't figure it out from the docs... And yes tikz is life.

Yeah, Sphinx is extremely frustrating. Stack Overflow suggests you can't include a latex file directly, but the comment is pretty old.

[Yurlungur]

ah that's frustrating