Using Jupyter Notebook on Matilda

Overview

This tutorial assumes you have some basic knowledge of conda/miniconda and can setup a basic environment. On Matilda, we have installed miniconda3 as a module to help you setup your conda environments. To use:

module load miniconda3

Once you have initialized conda and created a basic environment (in this tutorial called "test-env") you may install the jupyter package and it's dependencies into the environment and you are ready to run Jupyter Notebook on Matilda. For example (using our test-env environment, with Python v. 3.9):

conda create -n test-env python==3.9
conda activate test-env
conda install notebook

This tutorial covers 2 basic methods for running Jupyter Notebook on the Matilda HPC compute nodes.

Interactive Scheduled Job

srun

The first method involves invoking a schedule interactive job to begin the process. For example:

srun -N 1 -c 1 -t 5:00:00 --pty /bin/bash --login

(For more information on the specifics of interactive runs, please refer to Job Script Examples.)

Once your job starts and you are logged into the compute node use the following to initiate your notebook session:

module load miniconda3
source ~/.bashrc
conda activate test-env
jupyter-notebook --no-browser --port=8889 --ip=0.0.0.0

Substitute the actual name of your environment for "test-env" and choose a port (here we used 8889, default is 8888 if none specified). You should see something like the following:

[I 13:49:58.728 NotebookApp] Serving notebooks from local directory: /home/u/username
[I 13:49:58.728 NotebookApp] Jupyter Notebook 6.3.0 is running at:
[I 13:49:58.728 NotebookApp] http://hpc-throughput-p01:8889/?token=47ef2216d4ce8e14f30967def52d6e8dd6a0db0514692b00
[I 13:49:58.728 NotebookApp]  or http://127.0.0.1:8889/?token=47ef2216d4ce8e14f30967def52d6e8dd6a0db0514692b00
[I 13:49:58.728 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 13:49:58.741 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///home/u/username/.local/share/jupyter/runtime/nbserver-1239796-open.html
    Or copy and paste one of these URLs:
        http://hpc-throughput-p01:8889/?token=47ef2216d4ce8e14f30967def52d6e8dd6a0db0514692b00
     or http://127.0.0.1:8889/?token=47ef2216d4ce8e14f30967def52d6e8dd6a0db0514692b00

From another shell on your workstation, initiate a port forwarding session to the compute node as shown below:

ssh -N -L 8889:hpc-throughput-p01:8889 username@hpc-login.oakland.edu

Make sure to substitute in the actual port used as well as the actual node you are logged into (in this example we are running on hpc-throughput-p01).

You will be asked for your password. Enter it, and please note, your session will appear to "hang" - this is normal, so go to the next step (see below for example):

Password Screen

DO NOT CLOSE THE FORWARDED TERMINAL WINDOW UNTIL YOU FINISH USING NOTEBOOK.

Now open a browser and enter:

http://127.0.0.1:8889/?token=47ef2216d4ce8e14f30967def52d6e8dd6a0db0514692b00

You should now be connected to the Jupyter notebook. Please again note to substitute the actual port used for "8889" above (also your token will be different than the one shown above). Just copy and paste the appropriate link.

salloc

If you want to attach and detach from your notebook without killing the interactive job session, you can use the salloc approach to allocate resources, and then connect to the scheduled resources using srun. For example:

salloc -N 1 -c 1 -t 30:00
srun --jobid=<#####> --pty /bin/bash

Once attached to the allocated resource session, simply follow the instructions for instantiating your Jupyter Notebook as described above (under srun). The salloc/srun interactive job technique is described in more detail in the Example Job Scripts wiki page.

** Please note that if you exit the interactive session it will kill the notebook but your node reservation will remain. When reconnecting with srun you will have to restart the notebook. **

SBATCH Job

You can schedule a Jupyter Notebook run using a conventional "sbatch" job script as shown in the example below:

Example Job Script

{{{#!/bin/bash # ====Sample Job Script==== #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --cpus-per-task=1 #SBATCH --mem=4G #SBATCH --time=01:00:00 #SBATCH --job-name=jupyter-notebook

# get tunneling info XDG_RUNTIME_DIR="" node=$(hostname -s) user=$(whoami) cluster="hpc-login" port=8889

# print tunneling instructions jupyter-log echo -e " Command to create ssh tunnel: ssh -N -L ${port}:${node}:${port} ${user}@${cluster}.oakland.edu

Use a Browser on your local machine to go to: localhost:${port} (prefix w/ https:// if using password) " > jupyter-log

# load modules or conda environments here module load miniconda3

# activate conda environment cd source ~/.bashrc conda activate test-env

# Run Jupyter jupyter-notebook --no-browser --port=${port} --ip=${node} }}} By viewing the accompanying jupyter-log file generated by this script you can display instructions for connecting to the batch job. To summarize these - once the job is running open a separate terminal and ssh tunnel to the node where the job is running. For example:

ssh -N -L 8889:hpc-throughput-p01:8889 username@hpc-login.oakland.edu

You will be asked for your password. Enter it, and please note, your session will appear to "hang" - this is normal, so go to the next step (see below for example):

Password Screen

DO NOT CLOSE THE FORWARDED TERMINAL WINDOW UNTIL YOU FINISH USING NOTEBOOK.

Once the tunnel is established, open a local browser session and enter:

http://localhost:8889

Substitute the actual port used for "8889" above. Note this can be adjusted in the example job script.

Overall, this is very similar to the procedure used for the interactive run except the notebook session is established via the job script instead of interactively.

Precautions and Notes

There are several issues to keep in mind when running Jupyter Notebooks on Matilda. These are highlighted below.

Ports

The default port for Jupyter notebooks is "8888". Since others may be running a notebook on the node on which your job is running, it is possible that port will be in-use and an error will occur. If this happens, increment the port number until you find one that is not being used. This can be controlled either in the interactive job session, or by making a change to the job script for port.

Login Node

Please exercise care if running Jupyter Notebooks on the login node. Login is not meant for intensive jobs and notebooks consuming inordinate resources may be killed without warning.

When ever practical, you should be running your notebook on a compute node as a schedule job (interactive or batch).

Compute Node

Installing pakages on the Compute node with "pip --install" need additional commands.

pip install "PACKAGENAME" --trusted-host pypi.org --trusted-host files.pythonhosted.org

Security

Tokens

When running a Jupyter Notebook it is possible for someone else to attach to your session. If you have not setup a password for your notebook, running the "jupyter-notebook" command will generate an authentication token which you can use to access the session. You will see something like the following:

    To access the notebook, open this file in a browser:
        file:///home/u/username/.local/share/jupyter/runtime/nbserver-1237569-open.html
    Or copy and paste one of these URLs:
        http://hpc-throughput-p01:8889/?token=e9624d77e74d00265f88abdd0691510311b94e4355a77a04
     or http://127.0.0.1:8889/?token=e9624d77e74d00265f88abdd0691510311b94e4355a77a04

In the case above, use one of the following in your browser:

http://127.0.0.1:8889/?token=e9624d77e74d00265f88abdd0691510311b94e4355a77a04
OR
http://localhost:8889/?token=e9624d77e74d00265f88abdd0691510311b94e4355a77a04

DO NOT USE the compute node name in your browser as this will result in a connection failure.

Passwords

Alternately, you may setup a password to use with your Jupyter Notebook sessions. To create the necessary configuration file enter the following on your Matilda account:

module load miniconda3
conda activate test-env
jupyter notebook --generate-config

(Substitute the actual name of your conda environment for "test-env" where Jupyter packages are installed.) This will create a file named:

~/.jupyter/jupyter_notebook_config.py

Now to generate a password for your notebooks:

jupyter notebook password

You will be prompted to enter a password and again, for confirmation. Use this password to login to your notebook sessions.


CategoryHPC