ExportDB ignores array variables

[JustinPrivitera]

As far as I can tell, array mesh variables get ignored when using our exporter. I know Blueprint can support data organized like this. Not sure about Silo. It would be nice if VisIt

1. made a good faith effort to export array variables
2. put up some kind of error message to explain why it didn't export the variables you asked for

[markcmiller86]

FYI...Silo does support "array" variables. But, when the HDF5 driver was implemented (it came after the PDB driver), it placed a compile-time upper limit on the number of component variables which defaults to 16. Thats sad 😢. So, the way this would be handled in Silo is with multiple DBPutXxxvar() calls and then a DBPutDefvars() call that created an array exprssion for the array variable.

[JustinPrivitera]

Does that mean if I have array variables in VisIt with more than 16 components, VisIt will silently fail to write them? Or do you think VisIt silently fails to write all array variables to Silo?

[markcmiller86]

Does that mean if I have array variables in VisIt with more than 16 components, VisIt will silently fail to write them? Or do you think VisIt silently fails to write all array variables to Silo?

Didn't you say above that array variables are not exported to any writers to begin with?

[JustinPrivitera]

Didn't you say above that array variables are not exported to any writers to begin with?

I do not know. What I know is that I had multiple 85-component array variables that I asked VisIt to export and they were skipped by Blueprint and by Silo. Examining the underlying infrastructure, my opinion at first glance is that the array variable names are getting scrubbed long before the Blueprint reader ever is asked to export specific variables. There's more digging to be done here.

[markcmiller86]

I haven't looked at the code but because "array" varaiables are really somewhat unique to VisIt itself (they don't really appear in many of the third-party library file formats where as "scalar", "vector" and "tensor" variables do and they do not have some equivalent algebraic type interpretation), I have the suspicion that we've never really added logic to properly export array variables to any writer.

When we do, we have questions to answer such as how do we do this with VTK? How do we do this with netCDF? Etc., In Silo, given the issues I mention above, I think the right answer is to iterate over the array components calling DBPutXxxvar1() methods and then finally an associated DBPutDefvar() call that forms the expression for the array variable.

[JustinPrivitera]

I would at least prefer that VisIt says something when it skips exporting array variables. Perhaps the approach could be that for most writers, it complains that it doesn't know what to do, and then for certain writers where we know what we want to do (like Blueprint, for example), it performs as desired?

[markcmiller86]

In Silo, given the issues I mention above, I think the right answer is to iterate over the array components

I think this is probably the right answer for most writers. In fact, its probably worth adding some general export option (e.g. applies to any writer)...

• write array variables component-by-component
• write an expression to reconstruct the array variable

Which should probably default to checked and are only unchecked for the rare cases (e.g. Blueprint) where array variables are indeed handled natively by the data model.