WIP: Store pmb->gid in VariableState

[pdmullen]

PR Summary

In addition to sparse_id, it would be also nice to be able to access the current block gid in a SparsePack via

pack(b, var()).gid

This utility is added in this MR, by adding gid to VariableState.

PR Checklist

• Code passes cpplint
• New features are documented.
• Adds a test for any bugs fixed. Adds tests for new features.
• Code is formatted
• Changes are summarized in CHANGELOG.md
• Change is breaking (API, behavior, ...)
  • Change is additionally added to CHANGELOG.md in the breaking section
  • PR is marked as breaking
  • Short summary API changes at the top of the PR (plus optionally with an automated update/fix script)
• CI has been triggered on Darwin for performance regression tests.
• Docs build
• (@lanl.gov employees) Update copyright on changed files

[Yurlungur]

This is a nice solution... Better than putting it directly into the pack. 👍

[pdmullen]

This is a nice solution... Better than putting it directly into the pack. 👍

Credit to @lroberts36

[lroberts36]

See my comment below.

[lroberts36]

It is possible that after a re-mesh when amr is turned on a block can have its gid changed. I don't think there is any mechanism to propagate this through to the variables, so I think that the VariableState could end up with a stale gid in this case.

The two possible solutions I think of. First, maybe adding a method to MeshBlock called UpdateGid(int gid) which updates the block gid and the VariableState of all variables (but I guess this depends on iterating through all of the containers). This method would need to be called in amr_loadbalance.cpp instead of just setting the block gids by hand. The other possibility is to just make sure that the values are updated during sparse packing, which is a little less clean I think but probably easier to implement.

[pgrete]

Just double checking: are our gids always guaranteed to be contiguous (starting from 0)?
We have some code paths that implicitly assume that there's always a block with gid 0.

[lroberts36]

Yes, gids are defined to always start at zero and be contiguous (i.e. they are just the indices of all the blocks in a std::vector that has been sorted by the Morton number of the blocks' logical locations). The issue is that after remeshing, the Morton number of a block that hasn't been refined or derefined doesn't change but it's gid can change since blocks with lower Morton numbers may have been added/removed from the grid.