{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1cda3fc9",
   "metadata": {},
   "source": [
    "# Getting Started\n",
    "\n",
    "\n",
    "Jupyrest is a library that converts __Jupyter Notebooks__ into __REST APIs__. It turns notebooks into *notebook functions*.\n",
    "\n",
    "This guide will demonstrate how to interact with a notebook function and other nice features that Jupyrest provides.\n",
    "\n",
    "## Prerequisites:\n",
    "\n",
    "- [Git](https://git-scm.com/downloads)\n",
    "- Python 3.9 or later\n",
    "  - [Mac](https://www.python.org/downloads/macos/)\n",
    "  - [Windows (Windows Store version recommended)](https://apps.microsoft.com/detail/9p7qfqmjrfp7?hl=en-us&gl=US)\n",
    "  - [Linux](https://docs.python-guide.org/starting/install3/linux/)\n",
    "- [poetry](https://python-poetry.org/docs/#installation)\n",
    "\n",
    "Download the example project:\n",
    "\n",
    "```bash\n",
    "git clone https://github.com/microsoft/jupyrest.git\n",
    "cd jupyrest/src/jupyrest_example\n",
    "```\n",
    "\n",
    "Set up the environment:\n",
    "\n",
    "```bash\n",
    "poetry install\n",
    "poetry shell\n",
    "pre-commit install\n",
    "```\n",
    "\n",
    "For here onwards, make sure you have run `poetry shell` before running anything on the command line. This will ensure you are using the right Python virtual environment."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae4742d1",
   "metadata": {},
   "source": [
    "## Starting the Web Server\n",
    "\n",
    "Run a Jupyrest HTTP server on port 5051:\n",
    "\n",
    "```bash\n",
    "python start_http.py 5051\n",
    "```\n",
    "\n",
    "## Initializing a Jupyrest Client\n",
    "\n",
    "The `JupyrestClient` is a handy wrapper over the Jupyrest HTTP API. The HTTP API OpenAPI spec can be found at the `/docs` endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99b950da",
   "metadata": {},
   "outputs": [],
   "source": [
    "from jupyrest.client import JupyrestClient\n",
    "from pprint import pprint\n",
    "from IPython.display import HTML\n",
    "import json\n",
    "\n",
    "# initialize our client\n",
    "base_url = \"http://localhost:5051\"\n",
    "client = JupyrestClient(endpoint=base_url)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1fd4b00",
   "metadata": {},
   "source": [
    "Below are some helper functions to better visualize the responses from the `JupyrestClient`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f4a2fd1",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "import json\n",
    "from fastapi.encoders import jsonable_encoder\n",
    "\n",
    "# Some helper functions to aid with displaying data\n",
    "# in this notebook.\n",
    "\n",
    "def print_json(data):\n",
    "    print(json.dumps(jsonable_encoder(data), indent=4))\n",
    "\n",
    "def display_response(response):\n",
    "    html_out = f\"\"\"\n",
    "    ExecutionId: <a href=\"{base_url}/api/notebook_executions/{response.execution_id}\" target=\"_blank\">{response.execution_id}</a> <br>\n",
    "    <a href=\"{base_url}{response.artifacts[\"html\"]}\" target=\"_blank\">HTML</a> <br>\n",
    "    <a href=\"{base_url}{response.artifacts[\"html_report\"]}\" target=\"_blank\">HTML Report</a><br>\n",
    "    <a href=\"{base_url}{response.artifacts[\"ipynb\"]}\" target=\"_blank\">IPYNB</a><br>\n",
    "    \"\"\"\n",
    "    if response.has_output:\n",
    "        html_out += f'<a href=\"{base_url}{response.artifacts[\"output\"]}\" target=\"_blank\">OUTPUT</a><br>'\n",
    "    if response.has_exception:\n",
    "        html_out += f'<a href=\"{base_url}{response.artifacts[\"exception\"]}\" target=\"_blank\">EXCEPTION</a><br>'\n",
    "    \n",
    "    display(HTML(html_out))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78fdeb27",
   "metadata": {},
   "source": [
    "# Getting the list of available Notebook Functions\n",
    "\n",
    "Based on what folder you have your notebooks in, Jupyrest will scan the directory and detect which notebooks in there are eligible notebook functions. This is done by looking for the `*.config.json` files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37d02d17",
   "metadata": {},
   "outputs": [],
   "source": [
    "notebooks = await client.get_notebooks()\n",
    "print_json(notebooks)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5d8f2b8",
   "metadata": {},
   "source": [
    "# Get the details of a Notebook Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "555aefb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "hello_world_notebook = await client.get_notebook(notebook_id=\"hello_world\")\n",
    "print_json(hello_world_notebook)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "887b5355",
   "metadata": {},
   "source": [
    "# Executing a Notebook Function\n",
    "\n",
    "Ok, now to the fun stuff. We can use the `execute_notebook_until_complete` function to do this. \n",
    "\n",
    "First, lets try to execute a notebook with parameters that *don't* match the expected schema:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5dea6471",
   "metadata": {},
   "outputs": [],
   "source": [
    "await client.execute_notebook_until_complete(\n",
    "    notebook_id=\"hello_world\",\n",
    "    parameters={\n",
    "        \"foo\": \"bar\"\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cfb6cc6",
   "metadata": {},
   "source": [
    "Let's execute the notebook with the __correct__ parameters this time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e64ee6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = await client.execute_notebook_until_complete(\n",
    "    notebook_id=\"hello_world\",\n",
    "    parameters={\n",
    "        \"name\": \"PyCon 2024\"\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb573745",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_json(jsonable_encoder(response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fbb4b0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "display_response(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "138f0d80",
   "metadata": {},
   "source": [
    "# Notebook Execution Artifacts\n",
    "\n",
    "Any successful execution of a notebook function will have the following artifacts:\n",
    "* `output` (if any)\n",
    "    * the data passed into the `save_output()` function in the notebook, if called\n",
    "* `html`\n",
    "    * an HTML view of the executed notebook\n",
    "* `html_report`\n",
    "    * an HTML view with the code cells removed\n",
    "* `ipynb`\n",
    "    * the .ipynb file of the executed notebook\n",
    "    \n",
    "The URLs for these artifacts are present in the response body (shown above). Of course, you can access these artifacts using the client as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "494a2f7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "execution_id = response.execution_id\n",
    "\n",
    "html = await client.get_execution_html(\n",
    "    execution_id=execution_id\n",
    ")\n",
    "html_report = await client.get_execution_html(\n",
    "    execution_id=execution_id,\n",
    "    report_mode=True\n",
    ")\n",
    "ipynb = await client.get_execution_ipynb(\n",
    "    execution_id=execution_id\n",
    ")\n",
    "output = await client.get_execution_output(\n",
    "    execution_id=execution_id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d4f596e",
   "metadata": {},
   "source": [
    "## Notebook Execution Output\n",
    "\n",
    "The most compelling artifact of a notebook execution is the `output`. This is what makes a notebook into a notebook function.\n",
    "\n",
    "Our `\"hello_world\"` notebook is a notebook function because we can access its output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2a196ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_json(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da0034dd",
   "metadata": {},
   "source": [
    "# \"Erroneous\" Notebook Functions\n",
    "\n",
    "What happens when the code inside a notebook fails and throws an exception? Pretty much exactly what you'd expect!\n",
    "\n",
    "We can debug any failed notebook execution by looking at its HTML. No need for any fancy logging or telemetry set ups."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b5f89b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "\n",
    "response = await client.execute_notebook_until_complete(notebook_id=\"error\", parameters={})\n",
    "print_json(jsonable_encoder(response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fb44456",
   "metadata": {},
   "outputs": [],
   "source": [
    "display_response(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "800721f3",
   "metadata": {},
   "source": [
    "# Sharing Input/Output Models\n",
    "\n",
    "A good practice in API is to factor commonly used schemas as models and reference them in many API endpoints. Jupyrest lets you do this too!\n",
    "\n",
    "For this example, we will take a Portfolio Analysis notebook adapted from the [Stock Analysis For Quant](https://github.com/LastAncientOne/Stock_Analysis_For_Quant/blob/master/Python_Stock/Portfolio_Analysis.ipynb) Github repository.\n",
    "\n",
    "This notebook takes a `Portfolio` object as input. Rather than define this as raw JSON schema, we can define this as a Python class:\n",
    "\n",
    "The code below is only snippets, the full source code is in the `jupyrest_example/` folder.\n",
    "\n",
    "```python\n",
    "from jupyrest.nbschema import NbSchemaBase\n",
    "from datetime import date\n",
    "from typing import Dict\n",
    "\n",
    "class Portfolio(NbSchemaBase):\n",
    "    start_date: date\n",
    "    end_date: date\n",
    "    holdings: Dict[str, float]\n",
    "```\n",
    "\n",
    "Now we can give this model a name when we create our Jupyrest application:\n",
    "\n",
    "```python\n",
    "deps = InMemoryApplicationBuilder(\n",
    "    notebooks_dir=notebooks_dir,\n",
    "    models={\n",
    "        # we name our model here\n",
    "        \"portfolio\": Portfolio\n",
    "    }\n",
    ").build()\n",
    "```\n",
    "\n",
    "With our API model created and named, we can reference it in our `config.json`:\n",
    "\n",
    "`Portfolio_Analysis.config.json`\n",
    "```json\n",
    "{\n",
    "    \"id\": \"portfolio_analysis\",\n",
    "    \"input\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"portfolio\": {\n",
    "                \"$ref\": \"nbschema://portfolio\"\n",
    "            }\n",
    "        },\n",
    "        \"required\": [\n",
    "            \"portfolio\"\n",
    "        ],\n",
    "        \"additionalProperties\": false\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "The cool thing is that when we use the API to get this notebook's input/output schema, we don't see the `nbschema://`, everything is fully resolved into standard JSON schema:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0591d0da",
   "metadata": {},
   "outputs": [],
   "source": [
    "portfolio_notebook = await client.get_notebook(notebook_id=\"portfolio_analysis\")\n",
    "print_json(portfolio_notebook)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7e9ba6f",
   "metadata": {},
   "source": [
    "Lets now execute our portfolio analysis notebook. We can give it a set of holdings and weights:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7eba8ebe",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = await client.execute_notebook_until_complete(\n",
    "    notebook_id=\"portfolio_analysis\",\n",
    "    parameters={\n",
    "        \"portfolio\":{\n",
    "            \"start_date\": \"2022-04-26\",\n",
    "            \"end_date\": \"2023-04-26\",\n",
    "            \"holdings\": {\n",
    "                \"AAPL\": 0.5,\n",
    "                \"MSFT\": 0.2,\n",
    "                \"AMD\": 0.2,\n",
    "                \"NVDA\": 0.1\n",
    "            }\n",
    "        }\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "326add66",
   "metadata": {},
   "outputs": [],
   "source": [
    "display_response(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7312b1b7",
   "metadata": {},
   "source": [
    "Notice how the parameters have been __converted__ into a Portfolio Python object in the notebook. Isn't that just so cool!\n",
    "\n",
    "The beauty is that our notebook has *no idea* that is being used as a REST API. It is all plain-old Python through and through."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}