Jupyter#

Jupyter Lab (and notebook) are now enabled in Liger. Check out the all jupyter containers in the liger-ai-tools registry (container description on the main README).

Running Jupyter#

Refer to the liger-ai-tools repository for below-mentioned scripts and Jupyter container build files.

Via jupyter.run#

The easiest way to run Jupyter with TensorFlow 2.4 enabled is via the jupyter.run script. This file runs all the underlying scheduling and container commands letting you start Jupyter with one command.

Login to Liger:

ssh <login>@liger.ec-nantes.fr

Get the script by cloning the repository:

git clone https://gitlab.in2p3.fr/ecn-collaborations/liger-ai-tools.git
cd liger-ai-tools

Run Jupyter Lab (default):

./jupyter.run       # defaults to jupyter Lab

To run jupyter notebook instead:

./jupyter.run notebook       # defaults to jupyter Lab

Click on the link.

The job is submitted via the underlying batch scheduler (slurm) to a GPU node and it will run for the "DURATION" option time specified in the script. Default is 5h.

To keep Jupyter running for long jobs, you can use the & option to run it in the background. This will prevent Jupyter to be killed if you log out of Liger.

./jupyter.run &

Make sure to cancel the scheduled job if you finish before the time limit by running

Mysqueue # get the job id
scancel <jobID>

Failing to do so could result in keeping unused resources busy preventing other users from using GPU resources.

The jupyter.run script has the following default scheduling settings:

NODE=turing02       # GPU server hosting the job. For more info on available servers check Liger docs 
GPUS=1              # number of GPUs. Max depending on the server ^
DURATION=05:00:00   # Format: DD-HH:MM:SS. Max value = 4 days. 

You can change these settings with your chosen ones by editing jupyter.run and running the script again. This can be particularly useful when the chosen node is already busy.

Manually#

It is possible to run Jupyter manually if you want to have further custom settings (such as QOS, secondary accounts, container isolation or other specific singularity or Slurm options).

The jupyter.run automatically reserves the space and runs the Jupyter tesnorFlow the respective container present in the container registry. Either edit the jupyter.run script further or reserve the Slurm resources interactively and run the Jupyter containers through singularity as a standard AI job, explained several times in this documentation. Note that the Jupyter containers are meant to be run interactively, batch submission will fail.

This option will also allow you to build containers on top of the ones provided ans run your custom ones instead.