Using Python on Red Hat 8

What’s Changing?

Not all versions of Python currently installed on ARC clusters will be available after the update from CentOS 7 to Red Hat 8 in August.

Python 2 will no longer be provided and is no longer supported on ARC resources. Code written for Python 2 is most often not directly compatible with Python 3. Changes to code written for Python 2 will likely be required.

Why that matters

Any scripts that load a version-specific Python module will likely have to be updated to specify a version that is available on the updated cluster.

Python packages are installed for the specific version of Python that is in use during installation. If you switch from using a module for one version of Python to a different one with where either the major or minor version changes (for python3.7, the major version is 3 and the minor version is 7), then you will have to re-install any packages/libraries in order to make them available in the library of the new version of Python.

You only need to install Python packages once for each cluster on which you wish to use the library and, separately, for each version of Python that you use.

Please note, Python packages should be installed using the command line from a login node, not from within Jupyter Notebook or the JupyterLab app.

What should I do?

Check available versions

Log onto the Preview cluster prior to maintenance and use the following command (the $ is the prompt; do not type it):

$ module available python

The output from that command will show available options (versions subject to change) as follows:

python/3.9.12 python/3.10.4 (D) python3.9-anaconda/2021.11

Note that the lowest Python version is 3.9, so if libraries were previously installed with a Python of a lower version, you will need to reinstall those libraries. The (D) next to version 3.10.4 indicates that it is the default Python version. If you load the Python module without specifying a version number, version 3.10.4 will load by default.

Update version of Python module being loaded

You should modify version numbers in any of your scripts that load Python modules, as appropriate. ARC recommends that you take advantage of the Preview cluster during the pre-maintenance period to review changes in available module versions and to run some test jobs with your revised scripts.

Re-install user-installed libraries/packages

If you have previously been using any version of Python that is 3.8 or earlier, you will need to upgrade Python libraries after the update. To install a Python package into your personal library using pip, enter the following command, replacing <package_name> with the actual package name:

$ pip install --user <package_name>

The --user tag will, by default, place packages in
$HOME/.local/lib/python?.?/site-packages

where ?.? indicates the versioning of the Python release. The library will then be available to you for this and future sessions.

Port Python 2 code to Python 3

If you have code that was written for Python 2, you will need to update it so that it is compatible with Python 3.

More information and suggested tools

Please see the following links for further information and some suggested tools that you may find helpful:

Migrating from Python 2 to 3: Automated Tools and Strategies
How To Migrate Python 2 Applications To Python 3
2to3 – Automated Python 2 to 3 code translation