Reproducible Prompts

Set Up Ollama

We will be working with Ollama, which lets us run a number of large language models (LLMs) locally or on a cloud machine. Ollama has two parts:

1. The Ollama runtime, which starts a local server.
2. The model files, which are downloaded separately.

This lesson uses llama3.2:1b, a small text-only model. You can use the same code with larger models, but larger models require more memory and compute.

Install Ollama

Ollama is an application-level dependency, not just a Python package. Install it from a terminal so that the ollama command is available to Jupyter and Quarto.

For macOS or Linux, run:

#| eval: false
curl -fsSL https://ollama.com/install.sh | sh

For Windows PowerShell, run:

#| eval: false
irm https://ollama.com/install.ps1 | iex

For Windows Git Bash, run:

#| eval: false
powershell.exe -NoProfile -ExecutionPolicy Bypass -Command "irm https://ollama.com/install.ps1 | iex"

After installing Ollama, open a new terminal or restart this notebook kernel so the updated PATH is available. Check the installation with:

#| eval: false
ollama --version

The Python packages for this lesson are installed from the lesson-specific conda environment, reproducible-prompts.

Start Ollama and Pull a Model

The next cell checks whether Ollama is installed, starts the server if needed, and downloads the model used in this lesson. In GitHub Actions, the same cell can install Ollama on Linux when OLLAMA_AUTO_INSTALL=1 is set.

import subprocess
from pathlib import Path

model = "llama3.2:1b"
setup_script = next(
    path / "scripts/setup_ollama.py"
    for path in [Path.cwd(), *Path.cwd().parents]
    if (path / "scripts/setup_ollama.py").exists()
)

subprocess.run(
    [
        "python",
        str(setup_script),
        "--install-if-missing",
        "--start",
        "--pull",
        model,
    ],
    check=True,
)

>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-amd64.tar.zst
#=#=#                                                                          

                                                                           0.1%
#                                                                          1.8%
###                                                                        4.4%
#####                                                                      7.0%
#####                                                                      7.8%
######                                                                     8.7%
######                                                                     9.6%
#######                                                                   10.4%
########                                                                  11.3%
########                                                                  12.1%
#########                                                                 13.0%
##########                                                                14.7%
###########                                                               15.6%
###########                                                               16.4%
############                                                              17.3%
#############                                                             18.1%
#############                                                             19.0%
##############                                                            19.9%
##############                                                            20.7%
###############                                                           21.6%
################                                                          22.4%
################                                                          23.3%
#################                                                         24.1%
##################                                                        26.2%
####################                                                      28.9%
######################                                                    31.4%
########################                                                  34.2%
#########################                                                 35.9%
###########################                                               37.6%
#############################                                             41.1%
###############################                                           43.7%
#################################                                         46.4%
##################################                                        48.1%
####################################                                      50.8%
######################################                                    53.3%
########################################                                  55.9%
##########################################                                58.5%
###########################################                               60.6%
#############################################                             63.0%
###############################################                           65.9%
#################################################                         69.1%
###################################################                       71.7%
#####################################################                     74.4%
#######################################################                   76.9%
#########################################################                 80.5%
###########################################################               83.1%
#############################################################             85.7%
###############################################################           88.3%
#################################################################         90.8%
###################################################################       93.4%
#####################################################################     96.0%
#######################################################################   98.9%
######################################################################## 100.0%
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.

WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

pulling manifest ⠋ pulling manifest ⠙ pulling manifest ⠹ pulling manifest ⠸ pulling manifest ⠼ pulling manifest ⠴ pulling manifest ⠦ pulling manifest 
pulling 74701a8c35f6:   6% ▕█                 ▏  83 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  13% ▕██                ▏ 175 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  17% ▕███               ▏ 224 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  24% ▕████              ▏ 320 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  31% ▕█████             ▏ 414 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  35% ▕██████            ▏ 464 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  43% ▕███████           ▏ 571 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  51% ▕█████████         ▏ 673 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  54% ▕█████████         ▏ 718 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  62% ▕███████████       ▏ 818 MB/1.3 GB                  pulling manifest 
pulling 74701a8c35f6:  69% ▕████████████      ▏ 907 MB/1.3 GB  834 MB/s      0spulling manifest 
pulling 74701a8c35f6:  73% ▕█████████████     ▏ 969 MB/1.3 GB  834 MB/s      0spulling manifest 
pulling 74701a8c35f6:  81% ▕██████████████    ▏ 1.1 GB/1.3 GB  834 MB/s      0spulling manifest 
pulling 74701a8c35f6:  89% ▕████████████████  ▏ 1.2 GB/1.3 GB  834 MB/s      0spulling manifest 
pulling 74701a8c35f6:  93% ▕████████████████  ▏ 1.2 GB/1.3 GB  834 MB/s      0spulling manifest 
pulling 74701a8c35f6:  98% ▕█████████████████ ▏ 1.3 GB/1.3 GB  834 MB/s      0spulling manifest 
pulling 74701a8c35f6: 100% ▕█████████████████ ▏ 1.3 GB/1.3 GB  834 MB/s      0spulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠋ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠙ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠹ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠸ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠼ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠴ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠦ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠧ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠇ pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest ⠏ 

Installing Ollama using the official Linux installer...
Starting Ollama server at http://127.0.0.1:11434; logging to ollama.log.
Ollama server is ready.
Ensuring Ollama model is available: llama3.2:1b
Ollama setup complete.

pulling manifest 
pulling 74701a8c35f6: 100% ▕██████████████████▏ 1.3 GB                         
pulling 966de95ca8a6: 100% ▕██████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da: 100% ▕██████████████████▏ 7.7 KB                         
pulling a70ff7e570d9: 100% ▕██████████████████▏ 6.0 KB                         
pulling 4f659a1e86d7: 100% ▕██████████████████▏  485 B                         
verifying sha256 digest 
writing manifest 
success 

CompletedProcess(args=['python', '/home/runner/work/esiil-learning-portal/esiil-learning-portal/scripts/setup_ollama.py', '--install-if-missing', '--start', '--pull', 'llama3.2:1b'], returncode=0)

We can check that the model was successfully installed with ollama list.

!ollama list

NAME           ID              SIZE      MODIFIED               
llama3.2:1b    baf6a787fdff    1.3 GB    Less than a second ago    

What is an LLM?

The Ollama setup above lets us work with large language models (LLMs) that run locally on our computer or cloud instance. LLMs are machine learning models with the goal of understanding and generating natural language. These models have a very large number of parameters, often in the billions, and are trained using very large amounts of text.

Jupyter AI Magic

Getting Started

First let’s load the packages we will need for this lesson.

import pandas as pd
from ollama import ChatResponse
from ollama import chat
from pydantic import BaseModel

Now we need to make sure the Jupyter AI magics extension is loaded.

%load_ext jupyter_ai_magics

We can see the LLM providers available through Jupyter AI.

%ai list

Provider	Environment variable	Set?	Models
ai21	AI21_API_KEY	❌
gpt4all	Not applicable.	N/A
huggingface_hub	HUGGINGFACEHUB_API_TOKEN	❌	See https://huggingface.co/models for a list of models. Pass a model’s repository ID as the model ID; for example, huggingface_hub:ExampleOwner/example-model.
ollama	Not applicable.	N/A	See https://www.ollama.com/library for a list of models. Pass a model’s name; for example, deepseek-coder-v2.
qianfan	QIANFAN_AK, QIANFAN_SK	❌
togetherai	TOGETHER_API_KEY	❌

Aliases and custom commands:

Name	Target
gpt2	huggingface_hub:gpt2
gpt3	openai:davinci-002
chatgpt	openai-chat:gpt-3.5-turbo
gpt4	openai-chat:gpt-4
ernie-bot	qianfan:ERNIE-Bot
ernie-bot-4	qianfan:ERNIE-Bot-4
titan	bedrock:amazon.titan-tg1-large
openrouter-claude	openrouter:anthropic/claude-3.5-sonnet:beta

We’ll also set the AiMagics.max_history to 0 so it will not remember past prompt information in the current prompt. This helps make responses more repeatable.

Warning

The docs suggest using %ai reset to clear the chat history. That also clears parameters you have set, so we use max_history here instead.

%config AiMagics.max_history = 0

Now we can ask the model to introduce itself.

%%ai ollama:llama3.2:1b -m {"base_url":"http://localhost:11434"}
Introduce yourself in two sentences.

My Name is [Your Name]

I am an artificial intelligence designed to provide information and assist with tasks, and I’m here to help you with any questions or topics you’d like to discuss.

How I Work

I use natural language processing and machine learning algorithms to understand and respond to your input.

Incorporating a Python Object

We can incorporate a Python object using the {} syntax inside our prompt. First let’s make a list in Python.

random_words = [
    "cat",
    "dog",
    "bird",
    "monkey",
    "elephant",
    "deer",
    "wolf",
    "bison",
    "elk",
]
print(random_words)

['cat', 'dog', 'bird', 'monkey', 'elephant', 'deer', 'wolf', 'bison', 'elk']

Now we can embed this list in our prompt by calling the variable name inside of {}.

%%ai ollama:llama3.2:1b -m {"base_url":"http://localhost:11434"}
Create a bullet list from the text of {random_words} in random order.

• wolf
• deer
• cat
• dog
• elephant
• monkey
• bird
• elk
• bison

We can ask for structured-looking text in a prompt. Later, when we need Python to consume JSON, we will use a schema-backed Python call instead.

%%ai ollama:llama3.2:1b -m {"base_url":"http://localhost:11434","temperature":0}
Create a numbered list from the text of {random_words} in random order.

1. wolf
2. deer
3. monkey
4. bison
5. bird
6. elephant
7. elk
8. cat
9. dog
10. bird

Improving Reproducibility

We can set a seed to try to get more reproducible results. This is an Ollama parameter, so we provide it inside the -m {} arguments. Do not add spaces inside the JSON string, because that can interfere with argument parsing.

%%ai ollama:llama3.2:1b -m {"base_url":"http://localhost:11434","seed":42}
Create a bullet list from the text of {random_words} in random order.

• monkey
• wolf
• bison
• elephant
• deer
• bird
• dog
• elk
• cat

Adjusting Randomness

You can adjust the randomness of responses with temperature. The default is 0.8; higher values are more variable. Setting temperature to 0 can help with reproducibility.

%%ai ollama:llama3.2:1b -m {"base_url":"http://localhost:11434","seed":42,"temperature":1.5}
Create a list of 10 random words.

Quokkas Space Helicopters Nuclear Cheetahs Satellites Lighthouses Knitting Cannons Tigers

%%ai ollama:llama3.2:1b -m {"base_url":"http://localhost:11434","seed":42,"temperature":0}
Create a list of 10 random words.

• Quirky
• Space
• Diamond
• Caramel
• Banjo
• Perfume
• Bicycle
• Harmonica
• Snowflake
• Guitar

Passing Output to Python Code

You may want to pass an LLM output to your Python code. One option is to request a machine-readable JSON response from the AI model, and then use the Jupyter AI -f json option to parse the response. This could give us the response as Python objects if the model returns valid JSON, but in practice models do not always do this consistently.

%%ai ollama:llama3.2:1b -f json -m {"base_url":"http://localhost:11434","seed":42,"temperature":0}
Create a JSON object with exactly ten key-value pairs. Use keys named word1 through word10. Each value should be a random animal name.

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
Cell In[14], line 1
----> 1 get_ipython().run_cell_magic('ai', 'ollama:llama3.2:1b -f json -m {"base_url":"http://localhost:11434","seed":42,"temperature":0}', 'Create a JSON object with exactly ten key-value pairs. Use keys named word1 through word10. Each value should be a random animal name.\n')

File /usr/share/miniconda/envs/reproducible-prompts/lib/python3.11/site-packages/jupyter_ai_magics/magics.py:685, in AiMagics.ai(self, line, cell)
    682 ip = self.shell
    683 prompt = prompt.format_map(FormatDict(ip.user_ns))
--> 685 return self.run_ai_cell(args, prompt)

File /usr/share/miniconda/envs/reproducible-prompts/lib/python3.11/site-packages/jupyter_ai_magics/magics.py:621, in AiMagics.run_ai_cell(self, args, prompt)
    617 self._append_exchange(prompt, output)
    619 md = {"jupyter_ai": {"provider_id": provider_id, "model_id": local_model_id}}
--> 621 return self.display_output(output, args.format, md)

File /usr/share/miniconda/envs/reproducible-prompts/lib/python3.11/site-packages/jupyter_ai_magics/magics.py:494, in AiMagics.display_output(self, output, display_format, md)
    491     return output
    492 if display_format == "json":
    493     # JSON display expects a dict, not a JSON string
--> 494     output = json.loads(output)
    495 output_display = DisplayClass(output, metadata=md)
    497 # finally, display output display

File /usr/share/miniconda/envs/reproducible-prompts/lib/python3.11/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    341     s = s.decode(detect_encoding(s), 'surrogatepass')
    343 if (cls is None and object_hook is None and
    344         parse_int is None and parse_float is None and
    345         parse_constant is None and object_pairs_hook is None and not kw):
--> 346     return _default_decoder.decode(s)
    347 if cls is None:
    348     cls = JSONDecoder

File /usr/share/miniconda/envs/reproducible-prompts/lib/python3.11/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
    332 def decode(self, s, _w=WHITESPACE.match):
    333     """Return the Python representation of ``s`` (a ``str`` instance
    334     containing a JSON document).
    335 
    336     """
--> 337     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338     end = _w(s, end).end()
    339     if end != len(s):

File /usr/share/miniconda/envs/reproducible-prompts/lib/python3.11/json/decoder.py:353, in JSONDecoder.raw_decode(self, s, idx)
    344 """Decode a JSON document from ``s`` (a ``str`` beginning with
    345 a JSON document) and return a 2-tuple of the Python
    346 representation and the index in ``s`` where the document ended.
   (...)    350 
    351 """
    352 try:
--> 353     obj, end = self.scan_once(s, idx)
    354 except StopIteration as err:
    355     raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Expecting ',' delimiter: line 11 column 21 (char 230)

LangChain

Embedding LLM Prompts Directly in Python Code

LangChain and the Ollama Python client let us embed LLM calls into more complex Python code, and receive consistently formatted outputs. We already imported the necessary pandas, ollama, and pydantic libraries.

Using the pandas library, let’s read in the example dataset. This table contains species descriptions. We are going to use an LLM to parse the descriptions and extract trait information.

df = pd.read_csv(
    "https://github.com/cu-esiil-edu/esiil-learning-portal/releases/download/"
    "data-release/reproducible_prompts_Agaricus_descriptiosn.csv"
)
print(df)

    base__id                base_name  description__id  \
0     312386        Agaricus leaianus            36157   
1     125415          Agaricus edulis            50364   
2     125415          Agaricus edulis            50517   
3     170147    Agaricus subfloccosus            30039   
4     125513    Agaricus marasmioides            38151   
5     124122      Agaricus geesterani            33499   
6     124122      Agaricus geesterani            33500   
7     125629     Agaricus gnapholopus            35879   
8     152355         Agaricus petchii            46743   
9     152355         Agaricus petchii            46744   
10    125635      Agaricus zeylanicus            37426   
11    312439    Agaricus clavuliferus            35776   
12    312447     Agaricus microcosmus            35902   
13     78188        Agaricus hirsutus              698   
14     78188        Agaricus hirsutus            44112   
15     78188        Agaricus hirsutus            50422   
16     78188        Agaricus hirsutus            50527   
17    218457   Agaricus angustifolius            50238   
18    218477        Agaricus enudatus            50232   
19    218481     Agaricus subimmundus            50236   
20    218499      Agaricus cuneiforme            50240   
21    218503      Agaricus gigantulus            50233   
22    218518      Agaricus indetritus            50235   
23    218522       Agaricus luridatus            50229   
24    218537  Agaricus convexo-planus            50239   
25    218542   Agaricus subsulphureus            50237   
26    218550    Agaricus illecebrosus            50226   
27    218564  Agaricus sanguineoalbus            50230   
28    218677     Agaricus tenuisporus            50241   
29    218707      Agaricus vepallidus            50231   
30    134478      Agaricus subbalteus            39640   
31    426878      Agaricus allantopus            36105   
32     86276      Agaricus multicolor            50729   
33    426885        Agaricus crinalis            36104   
34     87331       Agaricus cryptarum              768   
35     87331       Agaricus cryptarum             3710   
36     87331       Agaricus cryptarum             7549   
37     87331       Agaricus cryptarum            43383   
38     87331       Agaricus cryptarum            43893   
39    426902          Agaricus lepton            36108   
40    426926      Agaricus mucidolens            36159   
41    426928      Agaricus muculentus            36101   
42    426929      Agaricus nidiformis            36017   
43    314643    Agaricus mesodactylus            39097   
44    314674        Agaricus viscidus            50520   
45    141614  Agaricus porphyrophaeus            20986   
46    141614  Agaricus porphyrophaeus            20987   
47    141614  Agaricus porphyrophaeus            42105   
48    141629       Agaricus popinalis            41559   
49    314916    Agaricus porcellaneus            50396   

                             description_description_  
0   75. Agaricus (Mycena) Leaianus, n. sp.\nPileo ...  
1   XXVIII. AGARICUS CAMPESTRIS. Der gemeine Champ...  
2   9. Agaricus campestris\nAGARICUS stipitatus; p...  
3   Agaricus subfloccosus (J.E. Lange) Hlaváek, My...  
4   508. Agaricus (Naucoria) marasmioides, n. s.\n...  
5   Agaricus geesterani Bas & Heinem., spec. nov. ...  
6   Agaricus geesterani Bas & Heinem., spec. nov. ...  
7   227. Agaricus (Naucoria) gnapholopus, B. & Br....  
8   Psalliota zeylanica.\nPileo primo conico-cylin...  
9   Psalliota zeylanica n. sp.\nPileus at first co...  
10  142. Agaricus (Lepiota) Zeylanicus, n. sp.\nPi...  
11  126. Agaricus (Mycena) clavulifer, B. & Br. \n...  
12  262. Agaricus (Psalliota) microcosmus, B. & Br...  
13  Lenzites betulina L. ex Fries \nCultural chara...  
14  Lenzites betulinus (L.:Fr.) Fr. - Epicr. p. 40...  
15  LXVII. AGARICUS HIRSUTUS. Der haarige Holzschw...  
16  26. Agaricus betulinus\nAGARICUS acaulis coria...  
17  Agaricus augustifolius B. f. 665; Hut 30 breit...  
18  Agaricus enudatus B. f. 576, 627; Hut 40 breit...  
19  Agaricus subimmundus B. f. 483, 574; Hut 100 b...  
20  Agaricus cuneiformis B. f. 491, 497; Hut 70 br...  
21  Agaricus gigantulus B. f. 412; Hut 200 , breit...  
22  Agaricus indetritus B. f. 273; Hut 140 breit, ...  
23  Agaricus luridatus B. f. 490; Hut 90 breit. br...  
24  Agaricus convexoplanus B. f. 666; Hut 40 breit...  
25  Agaricus subsulphureus B. f. 156, 426; Hut 110...  
26  Agaricus illecebrosus B. f. 571; Hut 60 breit,...  
27  Agaricus sanguineoalbus B. f. 662; Hut 130 bre...  
28  Agaricus tenuisporus B. f. 577; Hut 70 breit, ...  
29  Agaricus vepallidus B. f. 419; Hut 90 breit, v...  
30  923. Agaricus (Panaeolus) subbalteatus, n. sp....  
31  27. Agaricus (Pholiota) allantopus, Berk.\nPil...  
32  CXCIII. BOLETUS MULTICOLOR. Der vielfärbige Ba...  
33  24. Agaricus (Mycena) crinalis, n. sp.\nTenerr...  
34  Poria vaillantii (Fries) Cooke\nCultural chara...  
35  Poria vaillantii (DC. ex Fries) Cooke Fig. 103...  
36  PORIA VAILLANTII (DC. ex Fr.) Cke.\nKey Patter...  
37  Poria vaillantii (DC. ex Fr.) Cke., Grevillea ...  
38  Antrodia vaillantii (DC.:Fr.) Ryv. - Norw. J. ...  
39  29. Agaricus (Crepidotus) lepton, n. sp.\nE re...  
40  76. Agaricus (Galera) mucidolens, n. sp.\nOlid...  
41  22. Agaricus (Tricholoma) muculentus, n. sp.\n...  
42  Agaricus (Pleuropus) nidiformis, n. s. praegra...  
43  329. Ag. (Pholiota) mesodactylius, n. s. Pileo...  
44  12. Agaricus viscidus\nAGARICUS stipitatus, pi...  
45  Locis graminosis pratorum juxta Upsaliam nobil...  
46  Entoloma porphyrophaeum (Fr.) Karsten (tav. 44...  
47  Entoloma porphyrophaeum (Fr.) P. Karst., Ryssl...  
48  Rhodocybe popinalis (Fr.: Fr.) Sing. in Lilloa...  
49  XL. AGARICUS PORCELLANEUS. Porcellartartiger M...  

First we create a JSON schema using BaseModel from pydantic.

class Stipe(BaseModel):
    stipe_length: float
    stipe_unit: str

We can call the LLM using the chat function from ollama.

response: ChatResponse = chat(
    messages=[
        {
            "role": "user",
            "content": (
                "Extract the length of the stipe from "
                f"{df.iloc[1, 3]} as stipe_length and the unit of "
                "measurement as stipe_unit."
            ),
        }
    ],
    model="llama3.2:1b",
    format=Stipe.model_json_schema(),
    options={
        "temperature": 0,
        "seed": 42,
        "repeat_last_n": 0,
        "repeat_penalty": 0,
    },
)

The output is saved in a response object. It contains the output inside the message.content property.

response.message.content

'{\n    "stipe_length": 10,\n    "stipe_unit": "cm" }'

This output is a string, but we can use .model_validate_json to convert it into a Stipe object.

stipe_data = Stipe.model_validate_json(response.message.content)
stipe_data

Stipe(stipe_length=10.0, stipe_unit='cm')

Finally, the Stipe object can be directly coerced into a dictionary.

dict(stipe_data)

{'stipe_length': 10.0, 'stipe_unit': 'cm'}

One advantage of embedding LLM calls in Python is that we can use loops. In this example, we go row by row over the table and generate two new columns: stipe_length and stipe_unit.

results = pd.DataFrame()

for _, col in df.head(3).iterrows():
    response: ChatResponse = chat(
        messages=[
            {
                "role": "user",
                "content": (
                    "Extract the length of the stipe from "
                    f"{col.iloc[3]} as stipe_length and the unit of "
                    "measurement as stipe_unit."
                ),
            }
        ],
        model="llama3.2:1b",
        format=Stipe.model_json_schema(),
        options={
            "temperature": 0,
            "seed": 42,
            "repeat_last_n": 0,
            "repeat_penalty": 0,
        },
    )
    stipe = Stipe.model_validate_json(response.message.content)
    results = pd.concat(
        [
            results,
            pd.concat([col, pd.Series(dict(stipe))]).to_frame().T,
        ],
        ignore_index=True,
    )

print(results)

  base__id          base_name description__id  \
0   312386  Agaricus leaianus           36157   
1   125415    Agaricus edulis           50364   
2   125415    Agaricus edulis           50517   

                            description_description_ stipe_length stipe_unit  
0  75. Agaricus (Mycena) Leaianus, n. sp.\nPileo ...          2.5     inches  
1  XXVIII. AGARICUS CAMPESTRIS. Der gemeine Champ...         10.0         cm  
2  9. Agaricus campestris\nAGARICUS stipitatus; p...         10.0         cm