Jupyter Notebook Integration
Jupyter Notebook Integration
What is Jupyter?
Jupyter is an open-source platform that supports interactive computing and provides an environment for creating and sharing documents that contain live code, equations, visualizations, and narrative text.
For our uses within the HPC, we use Jupyter Notebook. This is a web-based application that allows users to create and share documents called Notebooks, which can contain live code, text, equations, visualizations, and interactive widgets. Below are some key features:
- Code Cells: Execute code snippets interactively, which can display output directly beneath the code.
- Markdown Cells: Write rich text for code explanations, documentation, or even LaTeX for mathematical expressions.
- Visualizations: Include charts, graphs, and other visual data representations following the code that generated them.
- Reproducibility: Notebooks can be saved and shared, enabling others to reproduce experiments with the same code and data.
Note: In Jupyter Notebooks, cells are the individual units of code or text that make up the content of a notebook.
What is Jupyter Hub?
While Jupyter Notebooks are perfect for individual users, JupyterHub is the multi-user version of the platform designed to support multiple users, making it ideal for educational environments, research teams, or organizations. JupyterHub allows you to create and manage Jupyter Notebook environments for many users, centralizing resources and making it easier to collaborate.
How to use Jupyter Notebooks on WAVE’s Jupyter Hub
To start an interactive session with Jupyter Hub, follow the steps outlined in the WAVE HPC User Guide - Accessing the HPC.
Note: There is a check box that allows you to launch either a Jupyter Labs or Notebook session. Jupyter Notebooks will launch a singular notebook in which you can edit files through the HPC, while Jupyter Labs will launch multiple notebook instances. For smaller jobs, Jupyter Notebooks is recommended, while Jupyter Labs may be more optimal for larger workflows.
After logging in successfully, the following launcher screen will be displayed:
If you are on Open OnDemand, your launcher screen will just include your file explorer, followed by a “running” tab that includes your active programs.
The launcher screen displays three options:
a. Notebook with Bundles: This option allows you to start a Jupyter Notebook with pre-configured software environments, referred to as "bundles." Bundles are collections of libraries and tools that are grouped together for specific tasks, making it easier for users to begin working without needing to manually install packages. Below are the list of available bundles:
- Python: The standard Python environment, suitable for general-purpose coding, data analysis, and scientific computing.
- Climate Bundle: A set of libraries and tools designed for climate-related simulations and data analysis. It may include packages for handling large climate datasets, climate modeling, and visualization.
- Molecular Modeling: This bundle includes tools and libraries tailored for computational chemistry, bioinformatics, and molecular dynamics simulations, often used for modeling proteins, DNA, or other molecular structures.
- PyTorch: A deep learning framework popular for building neural networks, with support for GPUs and automatic differentiation. This environment comes pre-configured with PyTorch libraries and dependencies.
- TensorFlow: A comprehensive open-source library for machine learning and deep learning. This environment is pre-configured with TensorFlow and its dependencies, enabling you to build and train deep learning models efficiently.
Note: Options b and c are only available within Jupyter Labs, not Jupyter Notebook. If you are planning to use the console and/or terminal along with file editing, please use Jupyter Labs.
b. Console with Bundles: The Console option launches a terminal-like interface that runs Python inside JupyterHub, providing a more traditional command-line experience within your Jupyter environment. Consoles allow you to execute commands, run scripts, and interact with the environment directly, without the need to open a full notebook. The console includes the same set of bundles found in Jupyter Notebooks.
c. Other: This option provides additional tools beyond the standard notebook and console, giving users flexibility in how they interact with the system. Below are the available options:
- Terminal: A full terminal interface where you can execute shell commands, navigate the file system, and run scripts directly on the HPC system. This is useful for managing files, requesting resources, submitting sbatch jobs to the queue, or interacting with system-level tools.
- Text File: A basic text editor within JupyterHub that allows you to create and edit plain text files. This is helpful for writing quick notes, documentation, or configuration files.
- Markdown File: A specialized text file that supports Markdown syntax, allowing you to write formatted text, including headings, bullet points, and links. Markdown files are commonly used for documentation, reports, and explanatory text within JupyterHub.
- Python File: An environment for creating and editing Python .py scripts. This is useful when you want to write longer Python programs outside of the notebook format and then execute, test them from the console or a notebook, or submit them to the queue using an sbatch job.
- Contextual Help: This option provides access to help resources tailored to the specific environment you're working in. It can offer documentation, usage tips, and troubleshooting advice to help guide you through tasks within JupyterHub.
Choose the option that best suits your needs to begin working.
You can open multiple tabs with different notebooks, files, consoles, or tools by clicking “Files” tab in the directory.
next to the launcher screen.
How to use Jupyter Notebook
- Start an interactive session with Jupyter Hub.
- Select the required bundle under the Notebook option in the Launcher screen. Below is the landing screen for a Jupyter Notebook:
- You can switch bundles by clicking on your current bundle on the right-hand side of the screen.
- Below is a list of operations that can be performed on the current cell by clicking these icons on the toolbar:
- Plus (): Creates a new cell in your notebook.
- Edit (✎): Click inside a cell to type in Python code or Markdown text to be executed or edited.
- Play (): Runs the currently selected cell.
- Stop (): Stops the execution of the current cell.
- Replay (): Re-runs the most recently executed cell.
- Fast-Forward (): Restarts the kernel and re-runs all the cells in the notebook. This is useful when you want to reset the environment and re-execute everything from the start.
- The dropdown menu () next to the icons provides three cell type options :
- Code: This option is used for cells that contain executable code (e.g., Python). When set to "Code," the cell allows you to write and run code.
- Markdown: This option is for cells that contain formatted text, such as headings, lists, links, and LaTeX equations. Markdown cells allow you to add explanations, documentation, or other non-executable content. For examples of markdown syntax, see this page.
- Raw: This option is for cells that contain raw text or content that should not be rendered or executed. It is typically used for inserting non-formatted data, text, or for code that needs to be preserved in its raw form without processing.
Below is a sample Jupyter notebook with Python bundle:
The output of the above given Jupyter Notebook:
How to use the Jupyter Console
- Start an interactive session with Jupyter Hub.
- Select the required bundle under the Console option in the Launcher screen. Below is the landing screen for a Jupyter Console:
- Enter the code in the cell below and execute it by either selecting "Run" -> "Run Cell" from the menu bar or using the shortcut key ‘’.
- Below is a sample Console program using the Python bundle:
Additional References: