Last modified on 01 Oct 2021.

Installation

Jupyter notebook

# BY PIP
pip install --upgrade pip
pip install --upgrade ipython jupyter

# BY CONDA
conda install ipython jupyter

Or read more in this note.

If you meet error OSError: [Errno 99] Cannot assign requested address, try

jupyter notebook --ip=127.0.0.1 --port=8080
# or
jupyter notebook --ip=127.0.0.1 --port=8080 --allow-root

Setting up a password

# create a juputer notebook config file
# it can be used for other settings
# https://jupyter-notebook.readthedocs.io/en/stable/public_server.html#prerequisite-a-notebook-configuration-file
jupyter notebook --generate-config

# create a new password
# note: sha1 cannot be reverted!!
jupyter notebook password

Inside notebook:

from notebook.auth import passwd
passwd()

With docker

# create a sha1 password
# download file create_sha1.py from https://github.com/dinhanhthi/scripts
# run ./create_sha1.py

# docker-compose.yml
environment:
  - PASSWD='sha1:d03968479249:319e92302e68d601392918f011d6c9334493023f'

# Dockerfile
CMD /bin/bash -c 'jupyter lab --no-browser --allow-root --ip=0.0.0.0 --NotebookApp.password="$PASSWD" "$@"'

R with jupyter notebook

Read more here.

# install jupyter
sudo apt-get install libzmq3-dev libcurl4-openssl-dev libssl-dev jupyter-core jupyter-client

# install R on linux
sudo apt install r-base

# R kernel for Jupyter Notebook
R # enter R environnement
# install R kernel
install.packages(c('repr', 'IRdisplay', 'IRkernel'), type = 'source')
# or
install.packages(c('repr', 'IRkernel'), type = 'source')
# make jupyter see r kernel
IRkernel::installspec() # current user
IRkernel::installspec(user = FALSE) # global

# embedded R
# use by cell magic %%R
pip install rpy2

# in a notebook
%load_ext rpy2.ipython

# then use
%%R
# R's codes

Other tips

  • Running 2 tasks in the same cell TAKE LONGER TIME than running each on different cells.
  • Download a folder in jupyter notebook:

    • Inside notebook, use:

      %%bash
tar -czf archive.tar.gz foldername

    • Or using nbzip (only working on current server).

Check the info

# function's info
?<func-name>

# function's shortcode
??<func-name>

# get the list of current variables
whos

Check where command executed from (in your $path)?

!type python

python is /Users/thi/anaconda/envs/python3.6/bin/python

Multiline commands

# Using '\'
df.columns = df.columns.str.replace('.', ' ')\
                       .str.replace('\s+', ' ')\
                       .str.strip().str.upper()

You CANNOT put # comments at the end of each line break!

Hotkeys / Shortcuts

There are 2 modes: command mode (pres ESC to activate) and edit mode (Enter to activate). Below are the most useful ones (for me).

You can edit / add more shortcuts in Help > Edit Keyboard Shortcuts.

For both modes,

  • Shift + Enter run the current cell, select below.
  • Ctrl + Enter run selected cells.
  • Alt + Enter run the current cell, insert below.
  • Ctrl + S save and checkpoint.

Command modes,

  • Enter take you into edit mode.
  • H show all shortcuts.
  • Up / Down select cell above / below.
  • Shift + Up / Down extend selected cells above / below.
  • A / B insert cell above / below.
  • X cut selected cells.
  • C copy selected cells.
  • V / Shift + V paste cells below / above.
  • D, D (press the key twice) delete selected cells.
  • Z undo cell deletion.
  • S Save and Checkpoint.
  • Y change the cell type to Code.
  • M change the cell type to Markdown.

Edit mode,

  • Esc take you into command mode.
  • Tab code completion or indent.
  • Ctrl + ] indent.
  • Ctrl + [ dedent.
  • Ctrl + A select all.
  • Ctrl + Z undo.
  • Ctrl + Shift + Z or Ctrl + Y redo.

Jupyter notebook on remote server

Open jupyter notebook in local browser but the backend-server is on remote.

  • If jupyter server is already running on remote at http://192.168.0.155:9889, 
    ssh -N -L localhost:9888:192.168.0.155:9899 <username-remote>@<remote-host> -p <port>
# if there is no port, remove `-p <port>`

    Open browser: http://localhost:9888 (type password if needed).

  • If jupyter server is not running on remote yet, 
    # connect to remote
ssh <username-remote>@<remote-host> -p <port>
# if there is no port, remove `-p <port>`

    On remote,

    # run juputer with custom port
jupyter notebook --no-browser --port=9899

# if there is error `OSError: [Errno 99] Cannot assign requested address`
jupyter notebook --ip=0.0.0.0 --no-browser --port=9899

# if there is error `Running as root is not recommended`
jupyter notebook --ip=0.0.0.0 --no-browser --port=9899 --alow-root

    It’s running and there are somethings like that,

    http://127.0.0.1:9889/?token=717d9d276f0537a9...831793df6319ad389accd

    Open another terminal window and type,

    ssh -N -L localhost:9888:localhost:9889 <username-remote>@<remote-host> -p <port>
# if there is no port, remove `-p <port>`
# there is nothing but it's running

    Open browser: http://localhost:9888/?token=717d9d276f0537a9...831793df6319ad389accd.

You can choose any port number you wanna instead of 9888 and 9889 (they can be the same), note that, you need to use a port number GREATER THAN 8000!

Install new python package inside Jupyter Notebook

Using conda[ref],

# Install a conda package in the current Jupyter kernel
import sys
!conda install --yes --prefix {sys.prefix} numpy

# DON'T DO THIS
!conda install --yes numpy

Using pip,

# Install a pip package in the current Jupyter kernel
import sys
!{sys.executable} -m pip install numpy

# DON'T DO THIS
!pip install numpy

Check version and update/upgrade,

!pip show pandas

Display dataframes side-by-side

from IPython.display import display_html
def display_side_by_side(*args):
    html_str=''
    for df in args:
        html_str+=df.to_html()
    display_html(html_str.replace('table','table style="display:inline; margin-right: 5px;"'),raw=True)

display_side_by_side(df1,df2,df1)

Get previous outputs

_ # previous output
__ # second-to-last output
___ # third-to-last output

Display 2 figures side-by-side markdown cell

Put below codes in the markdown cell of Jupyter Notebook.

<tr>
  <td> <img src="Nordic_trails.jpg" alt="Drawing" style="width: 250px;"/> </td>
  <td> <img src="Nordic_trails.jpg" alt="Drawing" style="width: 250px;"/> </td>
</tr>

Magic Functions

  • Check the full list (in examples) here or their docs here.
  • You can define your custom magic functions here.

Auto update the new updated modules (put at the beginning of the notebook)

%load_ext autoreload
%autoreload 2 # Reload all modules every time before executing

%autoreload 0 # disable autoreloader

Check more settings of %autoreload here.

Show the plots inside the notebook:

%matplotlib inline

Get the commands from 1 to 4:

%history -n 1-4 # get commands 1 to 4

Extensions

Table of contents

  1. Install npm and nodejs.
  2. Install this extension.
  3. Enable in jupyter lab view.
  4. Refresh the page.
# errors
# UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 23: ordinal not in range(128)
npm config set unicode false

Debugger

  1. Install xeus-python, jupyterlab

     pip install xeus-python
 pip install jupyterlab
  2. Install this extension.
  3. Refresh the page, you have to choose kernel xpython (instead of Python 3) to use the debugger.