Jupyter Notebook Configuration and Usage on RUPC

Configuration

Introduction

The simplest traditional way to start the Jupyter Notebook on a cluster is to make ssh -Y connection to the cluster and simply type 
jupyter-notebook
command in terminal. Next one has to wait for a minute or two before the browser window appears and one can start to work. The browser which was launched on the cluster will be very slow and hardly responsive, depending on the network speed and the load. The approach described below is different. It is based on usage of Virtual Private Network (VPN) to connect the cluster and your computer using private network which provides secure connection to the cluster. Moreover in this approach one can use local browser to access the Jupyter Notebook launched on the cluster. This way one transfers only a small amount of information between the cluster and local computer. It also doesn't require to configure the firewall as it uses private i.e. not public network for the cluster access.

Prerequisites

To use Jupyter Notebook on the cluster one has to install openVPN on your computer. To install openVPN please do the following steps . OpenVPN allows secure private connection to the cluster from anywhere.

One Time Configuration on RUPC

To configure Jupyter Notebook server, you need to login to RUPC cluster (use rupc02 server). Next do the following steps to setup and run Jupyter Notebook.
  1. Load Anaconda2 module: 
    module load anaconda2
  2. Generate configuration Jupyter Notebook file: 
    cd ~
jupyter notebook --generate-config

    This command will create a default Jupyter Notebook configuration file: /home/user/.jupyter/jupyter_notebook_config.py, where user should be changed to your username.

    Please assume the change of user to your username in all examples and commands below.

  3. For security purpose, you need to setup a password for your Jupyter Notebook server: 
    python
>>> from notebook.auth import passwd
>>> passwd()
Enter password:<your-password>
Verify password:<your-password>
'sha1:<your-sha1-hash-value>'
>>> Ctrl+Z

    Save the SHA1 hash value for later use, which will look like:

    sha1:44add4a345cc:bdr1fb6343557bff721e65e78fc511c41b13e8be.

  4. Create a self-signed certificate and the matched key: 
    mkdir ~/certs
chmod 700 ~/certs
cd ~/certs
openssl req -x509 -nodes -days 1095 -newkey rsa:2048 -keyout jupyterkey.key -out jupytercert.pem

    The last command will generate a certificate file /home/user/certs/jupytercert.pem and the matching key file /home/user/certs/jupyterkey.key.

    The command output should look like this
  5. Open the default configuration file using text editor, for example vi: 
    vi /home/user/.jupyter/jupyter_notebook_config.py
  6. Find the following lines, uncomment them and modify each of them as shown below. The final result should look like this (Please remember to change user to your username.): 
    c.NotebookApp.certfile = '/home/user/certs/jupytercert.pem'
c.NotebookApp.keyfile = '/home/user/certs/jupyterkey.key'
c.NotebookApp.ip = '10.10.10.2'
c.NotebookApp.open_browser = False
c.NotebookApp.password = 'sha1:<your-sha1-hash-value-from-step-#3>'

    Save and quit the configuration file.

Running Jupyter Notebook

On RUPC

On your computer

Quick Way to Run Jupyter Notebook

This is insecure way to run Jupyter Notebook i.e. no password for the web access to your files accessible in the notebook and also no web encryption. In addition, there is a slight inconvenience to type the --ip address parameter in the command line, every time one launches the notebook.

On RUPC

The rest is the same as described in previous section "On your computer".

Remarks

In order to run Jupyter Notebook for python3 just load corresponsing anaconda version module and procceed in the same way as described above. Take a look at the following output: where one can see commands for loading module anaconda3, starting notebook and after creating new notebook, the jupyter notebook server shutdown using CTRL-C command. Clicking Quit button on Home screen of Jupyter Notebook in your browser is more preferable way to shutdown the Jupiter server.

Jupyter Notebook is also available with standard python installation on rupc02 and rupc07 i.e. one can start it without loading any modules typing the command mentioned about i.e. 
jupyter notebook
provided you configured the Jupyter Notebook on RUPC or 
jupyter notebook  --no-browser --ip 10.10.10.2
if you didn't do the configuration.

Useful links

  1. Running a notebook server
  2. The Jupyter Notebook
  3. Jupyter Notebook Quickstart
  4. Setup Jupyter Notebook server – Centos/Rhel 7

Copyright © 2024, HPC, RUPC, Rutgers, The State University of New Jersey.
Wednesday, 26-Feb-2025 08:21:22 EST