Refer to the liger-ai-tools repository for below-mentioned scripts and Jupyter container build files.
The easiest way to run Jupyter with TensorFlow 2.4 enabled is via the
jupyter.run script. This file runs all the underlying scheduling and container commands letting you start Jupyter with one command.
Login to Liger:
Get the script by cloning the repository:
git clone https://gitlab.in2p3.fr/ecn-collaborations/liger-ai-tools.git cd liger-ai-tools
Run Jupyter Lab (default):
./jupyter.run # defaults to jupyter Lab
To run jupyter notebook instead:
./jupyter.run notebook # defaults to jupyter Lab
Click on the link.
The job is submitted via the underlying batch scheduler (slurm) to a GPU node and it will run for the "DURATION" option time specified in the script. Default is 5h.
To keep Jupyter running for long jobs, you can use the
& option to run it in the background. This will prevent Jupyter to be killed if you
log out of Liger.
Make sure to cancel the scheduled job if you finish before the time limit by running
Mysqueue # get the job id scancel <jobID>
Failing to do so could result in keeping unused resources busy preventing other users from using GPU resources.
jupyter.run script has the following default scheduling settings:
NODE=turing02 # GPU server hosting the job. For more info on available servers check Liger docs GPUS=1 # number of GPUs. Max depending on the server ^ DURATION=05:00:00 # Format: DD-HH:MM:SS. Max value = 4 days.
You can change these settings with your chosen ones by editing
jupyter.run and running the script again. This can be particularly useful when the chosen node is already busy.
It is possible to run Jupyter manually if you want to have further custom settings (such as QOS, secondary accounts, container isolation or other specific singularity or Slurm options).
jupyter.run automatically reserves the space and runs the Jupyter tesnorFlow the respective container present in the container registry. Either edit the
jupyter.run script further or reserve the Slurm resources interactively and run the Jupyter containers through singularity as a standard AI job, explained several times in this documentation. Note that the Jupyter containers are meant to be run interactively, batch submission will fail.
This option will also allow you to build containers on top of the ones provided ans run your custom ones instead.