We have recently got the new HPC cluster. I’m exploring to standarize the application stack for end-users using Lmod, Spack, EasyBuild etc., Can you suggest best pratices and what are the recommended hierarchy of tools/compilers/libraries/runtimes to provide in user environments.
Thanks.
One place to start is to look at the Linux Foundation OpenHPC project - OpenHPC maintains and community-supported repository of common HPC cluster software and tools. See https://openhpc.community/ - OpenHPC recently released version 2.0 of their distribution, which supports CentOS 8.2 or OpenSUSE Leap 15 as the base OS on x86_64 or aarch64 hardware. Take a look at the installation recipe at https://openhpc.community/downloads/
We have started down the path of using Spack for most of our packages. For some of our binary (usually commercial tools) we are still using Modules, but I am pretty convinced that Spack is the way to go long term.
This question seems to show up regularly, including from me… I can’t honestly find anyone who can articulate why someone might want to explore Spack vs. EasyBuild. I guess it could ultimately be that either would work for most use cases, but adherents do seem passionate about whichever they use. We also use modulefiles for most packages at the moment, but I can see how either Spack or EasyBuild will have to become an option in order to facilitate the ability for users to craft a specific environment without worrying about the overall system. My major problem with either option is that the documentation - I’m sorry to say - is really poor. I’m certain people will disagree, however, but it jumps rapidly from basic installation instructions to incredibly complex, nuanced use cases. Can we please just have a clear, simple example of how to create some environments? Rant over… 