Modules missing when installing with conda

[jkosinski]

Hi,

When I install with conda create -n my_env -c conda-forge cctbx-base, modules probe, reduce, and chem_data are not being installed. Is there a way of installing them in the environment?

Best,
Jan

[bkpoon]

For probe and reduce, we are migrating to the probe and reduce code inside mmtbx. These are the mmtbx.probe2 and mmtbx.reduce2 commands. The work and testing is being done by the MolProbity folks at Duke (@chrisjwilliams).

The chem_data repository will not be included in the conda package because it is about 2 GB. We are in the planning stages of cleaning up and hosting chem_data in a public repository and then you could use an environment variable to point to that location or make a symbolic link to the directory inside your conda environment. We'll probably add a command to simplify this process.

If you have a Phenix installation, you can make a symbolic link to the path to chem_data in ${CONDA_PREFIX}/lib/python3.10/site-packages. That is for Python 3.10. Replace the python3.10 part with the version of Python in your active conda environment.

ln -s <Phenix installation>/modules/chem_data ${CONDA_PREFIX}/lib/python3.10/site-packages/

And then you can verify that there is a chem_data directory in ${CONDA_PREFIX}/lib/python3.10/site-packages/. After that, you can use the new mmtbx.probe2 and smtbx.reduce2 commands. But keep in mind that they are still being tested.

[bkpoon]

One additional note about chem_data, there are databases stored as pickle files that should be regenerated for Python 3. So if you are using the Phenix copy of chem_data, you should copy that directory somewhere first and then link the copied version to the conda environment.

Inside the chem_data directory, there are directories named cablam_data and rotarama_data. The files ending in .pickle and .dlite should be deleted. Then run the mmtbx.rebuild_cablam_cache and mmtbx.rebuild_rotarama_data commands in your environment to rebuild the databases.

[wenchangzhou-qtx]

Hey @bkpoon,

Thanks for having such a wonderful repo! I actually have a similiar issue with my installation. After I install with conda, I cannot see mmtxb.refinement package in it, am I missing something? Thanks

[bkpoon]

Can you describe which specific command or file is missing? I can do this with a newly created conda environment with cctbx-base. Are you having problems importing mmtbx.refinement? The chem_data directory still needs to come from Phenix right now.

[wenchangzhou-qtx]

Thanks @bkpoon I can run mmtbx.reduce but not mmtbx.refinement.

[bkpoon]

The refinement programs are in Phenix. We have phenix.refine for diffraction data and phenix.real_space_refine for fitting into maps. The underlying algorithms are in the mmtbx.refinement module (and other parts of cctbx), but the tools are part of Phenix.

[wenchangzhou-qtx]

Got it. I'm trying to use mmtbx.reifnement module to accomplish the same task as phenix.refine, is this possible within mmtbx?

[bkpoon]

Yes, it is possible, but it can get complicated if you are trying to reproduce all the functionality in phenix.refine. For example, phenix.refine refines specific things inside each macro cycle, like the coordinates or B-factors. The individual parts are in cctbx-base, but you would have to combine all the parts together. I would suggest that you get access to Phenix if you are trying to do something for production use.