HomeHighlightsTowards reproducible software environments in HPC with Guix

Towards reproducible software environments in HPC with Guix

In an effort to make our research and the software used within it more reproducible Utrecht Bioinformatics Center is collaborating on Guix-HPC, an open source software project, with the Inria Bordeaux – Sud-Ouest Research Centre and Max Delbrück Center for Molecular Medicine in Berlin. In this joint venture all partners use HPC software, and the ability to reproduce experiments is a stringent necessity.

The project is scheduled to last two years. By that time, the project’s initiators hope to have met the software reproducibility needs of their institutions. The wider objective is to convince other HPC decision makers of the advance that this approach represents.

Necessity of reproducibility

Reproducibility is an important topic of scientific discussion. In computer science, more and more experiments rely on sets of complex software. The ability to reproduce an experiment thus depends on the ability to reproduce the software environment. In this context, the National Science Foundation (NSF) in the United States now encourages experiments carried out by the HPC community to be reproduced, and journals such as Nature are also insisting on the importance of sharing source codes and supporting reproducibility. Some HPC conferences such as SuperComputing have directives on the subject. Finally, Inria’s latest strategic plan devotes an entire chapter to the topic.

Why Guix?

Guix appears to be one of the solutions, is free software, developed under the auspices of the GNU Project by a growing community of enthusiasts and organizations: currently between 40 and 50 people contribute each month. It is used to reproduce software environments.
Guix offers an alternative solution to traditional package managers and containers. It can be used to reproduce a software environment without the need for the system’s package manager. The three partners in this new cooperative project seek to optimise this software for HPC. This is done by adding packages for HPC software that were developed and used at each of the institutions, but also and above all by adding functionalities that facilitate its use on a computing cluster and implementing reproducible workflows.

Experienced Guix user

Ricardo Wurmus, system administrator of the Scientific Bioinformatics Platform at the Max Delbrück Center for Molecular Medicine, uses Guix: “Before Guix the installation of scientific software was necessarily ad-hoc. Groups would build their own software, statically link it,and hope that it will never have to change (because management of software environments was virtually impossible).  Not only can we now manage a single environment per group in a reliable fashion; we use Guix for environments at all levels: group, project, user,  workflow… etc. ”

At the Inria Bordeaux – Sud-Ouest research centre, Ludovic Courtès, an engineer in the Experimentation and development Section, is responsible for optimising the software for HPC. With support provided by a technology development initiative from Inria, its long-term objectives are that the software meet the HPC requirements of the Centre’s research teams and that it is compatible with computer clusters like the one hosted and used at the Bordeaux centre: PlaFRIM.

See the Guix-HPC Web site for more information.

Read the extended news item on the Inria website.