Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically cache noneq params (Evib, Erot, etc.) to line database cache file #176

Open
2 tasks
erwanp opened this issue Jan 5, 2021 · 2 comments · May be fixed by #511
Open
2 tasks

Automatically cache noneq params (Evib, Erot, etc.) to line database cache file #176

erwanp opened this issue Jan 5, 2021 · 2 comments · May be fixed by #511
Assignees
Labels
databases related to line databases enhancement New feature or request performance refactor requires changes in code architecture ⭐⭐⭐_ High Priority
Milestone

Comments

@erwanp
Copy link
Member

erwanp commented Jan 5, 2021

🎯 Describe what you'd like

  • Automatically add Evib and Erot to the .h5 line cache.

Good issue to understand how RADIS work, because it touches most files of the Line-by-line module !

This and #45 will make noneq calculations practically as fast as equilibrium, and ~10-100 times faster than now !

💡 Possible implementations

Modifications could be done to SpectrumFactory.non_eq_spectrum, in particular here check_noneq_parameters which does the job of fetching the molecule Evib/Erot, and assign to each line of the line database the corresponding Evib, Erot.

Attention points :

  • Make sure the spectroscopic constants are stored in metadata : Evib/Erot will change if user change the set of spectroscopic constants used : cache files should be regenerated automatically. See check_cache_file and how metadata is used. Add a test on that.

  • When these parameters are computed some lines may already have been discarded from the database. For instance, isotope selection is done during the loading of each database file. If modifications are done at calculation time as suggested above, the cache file would not contain all the lines of the database. Two options : also store the isotopes in metadata. Same goes for the waverange computed (again : store it in the metadata, and recompute if needed?). 2nd option : cache everything initally, at loading, for all files, so whenever a database is first initialized. Problem : people running equilibrium computations only (most users!) would end up caching all their databases. Or : have a "nonequilibrium=True/False" boolean for each database ? (could be infered from the presence of "levelsfmt" already today.

💡 a very limited H5 file for each calculation could totally replace the current mechanism where the SpectrumFactory.df0 Pandas dataframe containing all lines loaded, is copied at calculation time to SpectrumFactory.df1 (where populations will be computed based on temperature, etc.). Using a .h5 file may induce a (limited ?) performance overhead, but we could also divide the memory usage by up to 2 (see #118 ). Major refactor, though !

  • Same for the "nonequiliibrum model" used for 2T approximation in polyatomic molecules (ex CO2 : a single Tvib can be defined as T1=T2=T3 or T3 only). See group_energy_modes in partfunc.py

Also :


Tip : you can vote for New Features on https://feathub.com/radis/radis

@erwanp erwanp added enhancement New feature or request performance databases related to line databases labels Jan 5, 2021
@erwanp erwanp added this to the 1.0 milestone Jan 5, 2021
@erwanp erwanp changed the title Automatically cache nonequilibrium parameters to line cache Automatically cache noneq params (Evib, Erot, etc.) to line database cache file Jan 5, 2021
@erwanp erwanp added the refactor requires changes in code architecture label Jan 5, 2021
This was referenced Jan 5, 2021
@erwanp erwanp added this to ⭐⭐⭐ High Priority in Roadmap 2021 Jan 11, 2021
@erwanp erwanp added the ⭐⭐⭐_ High Priority label Jan 11, 2021
@dcmvdbekerom
Copy link
Member

Just so you know, I'm looking into this but I'm in the process of figuring all the code out...

@erwanp
Copy link
Member Author

erwanp commented Jan 18, 2021

Amazing. I think the starting point is the RovibParFuncCalculator class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
databases related to line databases enhancement New feature or request performance refactor requires changes in code architecture ⭐⭐⭐_ High Priority
Projects
Roadmap 2021
⭐⭐⭐ High Priority
Development

Successfully merging a pull request may close this issue.

3 participants