How to handle multiple domains per MPI process

[benjaminjeliot]

Hi everyone,

I was hoping someone could help me to understand how to handle multiple domains per process in Ascent.

I am currently trying to use Ascent in an app (very similar to the CloverLeaf 3D proxy) with 3D rectilinear mesh, multiple MPI processes and multiple domains per MPI process. Everything works fine with one domain per MPI process but fails when I try to use multiple domains per process.

• What do I need to change to handle multiple domains?
• How should I define the domain ID?
• How do I set multiple mesh domain data from a single process?
• How do I publish/execute multiple domains?

Many thanks,
Jim

[cyrush]

Hi Jim,

There is still one publish per MPI task, each MPI Task presents a conduit tree with children that represent the domains.

Here is one way to create such a tree:

Node n;
Node & domain_0 = n.append();
// setup domain_0
Node & domain_1 = n.append();
// setup domain_1

For each domain, you can provide the domain id under state/domain_id:

domain_0["state/domain_id"] = 0;

Here are some test snippets that demo MPI use:

ascent/src/tests/ascent/t_ascent_mpi_render_3d.cpp

Line 62 in 60dcf7d

create_3d_example_dataset(data,32,par_rank,par_size);

ascent/src/tests/t_utils.hpp

Line 336 in 60dcf7d

create_3d_example_dataset(Node &data,

Hope this helps, and I will also admit we need better docs and examples that show the MPI case.

-Cyrus

[benjaminjeliot]

Hi Cyrus,

Thanks for your reply - it made sense when you explained it.

It only took a few changes and now my code is working great. :-)

Kind regards,
Jim

[cyrush]

Glad that helped!