Getting Started

Getting Started#

Jupyrest is a library that converts Jupyter Notebooks into REST APIs. It turns notebooks into notebook functions.

This guide will demonstrate how to interact with a notebook function and other nice features that Jupyrest provides.

Prerequisites:#

Download the example project:

git clone https://github.com/microsoft/jupyrest.git
cd jupyrest/src/jupyrest_example

Set up the environment:

poetry install
poetry shell
pre-commit install

For here onwards, make sure you have run poetry shell before running anything on the command line. This will ensure you are using the right Python virtual environment.

Starting the Web Server#

Run a Jupyrest HTTP server on port 5051:

python start_http.py 5051

Initializing a Jupyrest Client#

The JupyrestClient is a handy wrapper over the Jupyrest HTTP API. The HTTP API OpenAPI spec can be found at the /docs endpoint.

from jupyrest.client import JupyrestClient
from pprint import pprint
from IPython.display import HTML
import json

# initialize our client
base_url = "http://localhost:5051"
client = JupyrestClient(endpoint=base_url)

Below are some helper functions to better visualize the responses from the JupyrestClient.

from IPython.display import HTML
import json
from fastapi.encoders import jsonable_encoder

# Some helper functions to aid with displaying data
# in this notebook.

def print_json(data):
    print(json.dumps(jsonable_encoder(data), indent=4))

def display_response(response):
    html_out = f"""
    ExecutionId: <a href="{base_url}/api/notebook_executions/{response.execution_id}" target="_blank">{response.execution_id}</a> <br>
    <a href="{base_url}{response.artifacts["html"]}" target="_blank">HTML</a> <br>
    <a href="{base_url}{response.artifacts["html_report"]}" target="_blank">HTML Report</a><br>
    <a href="{base_url}{response.artifacts["ipynb"]}" target="_blank">IPYNB</a><br>
    """
    if response.has_output:
        html_out += f'<a href="{base_url}{response.artifacts["output"]}" target="_blank">OUTPUT</a><br>'
    if response.has_exception:
        html_out += f'<a href="{base_url}{response.artifacts["exception"]}" target="_blank">EXCEPTION</a><br>'
    
    display(HTML(html_out))

Getting the list of available Notebook Functions#

Based on what folder you have your notebooks in, Jupyrest will scan the directory and detect which notebooks in there are eligible notebook functions. This is done by looking for the *.config.json files.

notebooks = await client.get_notebooks()
print_json(notebooks)

Get the details of a Notebook Function#

hello_world_notebook = await client.get_notebook(notebook_id="hello_world")
print_json(hello_world_notebook)

Executing a Notebook Function#

Ok, now to the fun stuff. We can use the execute_notebook_until_complete function to do this.

First, lets try to execute a notebook with parameters that don’t match the expected schema:

await client.execute_notebook_until_complete(
    notebook_id="hello_world",
    parameters={
        "foo": "bar"
    }
)

Let’s execute the notebook with the correct parameters this time:

response = await client.execute_notebook_until_complete(
    notebook_id="hello_world",
    parameters={
        "name": "PyCon 2024"
    }
)

print_json(jsonable_encoder(response))

display_response(response)

Notebook Execution Artifacts#

Any successful execution of a notebook function will have the following artifacts:

output (if any)
- the data passed into the save_output() function in the notebook, if called
html
- an HTML view of the executed notebook
html_report
- an HTML view with the code cells removed
ipynb
- the .ipynb file of the executed notebook

The URLs for these artifacts are present in the response body (shown above). Of course, you can access these artifacts using the client as well:

execution_id = response.execution_id

html = await client.get_execution_html(
    execution_id=execution_id
)
html_report = await client.get_execution_html(
    execution_id=execution_id,
    report_mode=True
)
ipynb = await client.get_execution_ipynb(
    execution_id=execution_id
)
output = await client.get_execution_output(
    execution_id=execution_id
)

Notebook Execution Output#

The most compelling artifact of a notebook execution is the output. This is what makes a notebook into a notebook function.

Our "hello_world" notebook is a notebook function because we can access its output:

print_json(output)

“Erroneous” Notebook Functions#

What happens when the code inside a notebook fails and throws an exception? Pretty much exactly what you’d expect!

We can debug any failed notebook execution by looking at its HTML. No need for any fancy logging or telemetry set ups.

from IPython.display import HTML

response = await client.execute_notebook_until_complete(notebook_id="error", parameters={})
print_json(jsonable_encoder(response))

display_response(response)