Our site is trying to deal with Anaconda’s ongoing changes to its terms of service, which might make it prohibitively expensive to provide Anaconda as a software module on our clusters. Have any of your sites dealt with this?
Here at Georgia Tech, Anaconda first contacted to our IT governance staff in Sept 2023. At one point, they gave our campus a 7-figure quote for a site license, which is still up in the air. In the meantime, at our HPC facility, we’re continuing to offer Anaconda as a software module. However, we’d like to be prepared for what might be coming, and we’d appreciate any info from sites in a similar situation!
Many thanks,
Ron
Ron Rahaman
Research Scientist II, Research Software Engineer
Partnership for an Advanced Computing Environment (PACE)
Open Source Programming Office (OSPO)
Georgia Institute of Technology
Our users have a choice to use either regular Python modules (and set their environment using virtualenv) or use miniconda modules and use mamba and/or pip for the package installation.
The user home directories on our cluster are relatively small – 10 GB, which is not enough to store the Python packages, so when a new user loads a miniconda module for the first time, we prompt them to select a directory in their project space.
We also provide separate modules for the ML academic classes. Many of them use the same set of packages (tensorflow, pytorch and similar) and to avoid 500+ students installing them in their space, we install them before each semester and provide them with directions on how they can add their own packages and use them for their class.
At UAB, we’re making a transition to Miniforge and are planning to document the how and why of the transition. We use Lmod, which helps us out a bit, because at some point we’ll set up our Anaconda module(s) to simply emit a warning and load Miniforge instead.
Obviously, we can’t stop researchers from using the anaconda or r channels, but we can educate them about the how and why for the transition, and what they’ll need to do to avoid TOS violations.
As far as “forcing” users to avoid the restricted channels, that is a leadership decision and way outside our hands. But if legal action causes us to move in that direction, we’ll be ready with an alternative and training already in place.
Thanks! I got similar advice from the ACCESS Campus Champions mailing list, too. I think it might be reasonable for us to:
Discontinue our current anaconda module, which provides the conda program and the Python packages that are downloaded from the anaconda channel
Introduce a module for mamba or miniforge, which will provide the installer and an equivalent set of Python packages downloaded from conda-forge
As I understand it from several helpful folks and an Anaconda article I found: The license covers the usage of the “anaconda” channel, and conda defaults to the “anaconda” channel. Other installers like mamba and miniforge default to the “conda-forge” channel, so they would expose us to much less risk.
Users do appreciate having the set of packages from Anaconda available via a module without having to install the packages themselves. But I’m sure they don’t care if the packages come from “anaconda” or “conda-forge”, and I suspect most of the important ones are mirrored on “conda-forge” already.
For a while now, we’ve been gently encouraging people to use venv and pip more. We’ll keep doing that and maybe start being a bit less gentle
Update from Boston University: we’ve installed a new miniconda module (which provides conda and mamba) using miniforge (GitHub - conda-forge/miniforge: A conda-forge distribution.). We’re still calling the module “miniconda” for now. We have a setup script for .condarc files that users are prompted to run when the first time they load the module, and that is configured to set the channels to conda-forge and nodefaults.
We have for a long time provided a python3 module that includes a large number of Python libraries which is roughly equivalent to the Anaconda “base” environment. During the setup of those modules all Python libraries are installed from Pypi using pip, and we encourage users to use the python3 modules unless they need conda environments.
