From 39be331d218ec8a816e4772d74830fdcae39db2c Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:07:27 -0700 Subject: [PATCH 1/8] docs(readme): fix issue #118 config.py misdirection in local-models block --- README.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 85a58e8d..be97927e 100644 --- a/README.md +++ b/README.md @@ -454,11 +454,18 @@ The compression system enables deployment with popular open-source models: - **Batch Processing**: Support for multiple concurrent games #### Setup for Local Models -1. Install local model runtime (Ollama, llama.cpp, etc.) -2. Enable compression in `config.py` -3. Configure model endpoint in `config.py` -4. Adjust context window settings for your model -5. Run game normally - compression handles adaptation +NeverEndingQuest ships with two supported local-model runtimes: **LM Studio** and **Ollama**. Both present an OpenAI-compatible HTTP endpoint that the game reaches via the `OPENAI_BASE_URL` environment variable, which the launcher scripts set for you. You do **not** need to edit `config.py` to switch endpoints โ€” `config.py` only holds your OpenAI API key (used for cloud mode; ignored by local runtimes but the file must exist) and the modules directory. + +1. Install one of the supported runtimes: + - **LM Studio:** [lmstudio.ai](https://lmstudio.ai) โ€” see `LMSTUDIO_SETUP.md` + - **Ollama:** [ollama.com](https://ollama.com) โ€” see `OLLAMA_SETUP.md` +2. Load or pull a model. +3. Start the local server (LM Studio's "Start Server" button, or `ollama serve` โ€” auto on macOS/Windows). +4. Launch the game using the runtime's dedicated script: + - LM Studio: `run_with_lmstudio_direct.bat` + - Ollama: `run_with_ollama_direct.bat` (or `.sh` on macOS/Linux) + +Compression is enabled by default in `model_config.py` (`COMPRESSION_ENABLED = True`) and needs no tuning. #### ๐Ÿงช EXPERIMENTAL: LM Studio Integration From fcb6212a4693c684cc05c818995d568cf609ed7f Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:07:52 -0700 Subject: [PATCH 2/8] docs(readme): add Ollama integration section mirroring LM Studio --- README.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/README.md b/README.md index be97927e..c9efb413 100644 --- a/README.md +++ b/README.md @@ -552,6 +552,51 @@ Context Length: 32768 or higher **Support**: LM Studio mode is provided as-is for experimentation. For production gameplay, we recommend using OpenAI's API with the compression system for optimal experience. +#### ๐Ÿงช EXPERIMENTAL: Ollama Integration + +**Status:** EXPERIMENTAL โ€” same prompt-compatibility caveats as the LM Studio integration above. + +NeverEndingQuest supports [Ollama](https://ollama.com) via its OpenAI-compatible endpoint (`http://localhost:11434/v1`). Ollama tends to be simpler to install on headless machines than LM Studio and has a larger model catalog via `ollama pull`. + +**UX is identical to LM Studio:** + +1. Install Ollama from [ollama.com/download](https://ollama.com/download). +2. Pull **one** model (used for every game request): + ``` + ollama pull llama3.1:8b-instruct-q4_K_M + ``` +3. Double-click `run_with_ollama_direct.bat` (Windows) or run `./run_with_ollama_direct.sh` (macOS/Linux). + +The launcher automatically creates the Ollama aliases the game requires on first run. You only ever pick one model โ€” the launcher points both of the game's internal tier names at that single model, just like LM Studio does implicitly. No VRAM thrashing from loading multiple models between requests. + +**To switch models later:** + +```bash +ollama pull +ollama rm gpt-4.1-2025-04-14 gpt-4.1-mini-2025-04-14 +# then re-launch: the launcher will recreate aliases against whatever is pulled +``` + +Or set `OLLAMA_MODEL=` before running the launcher to force re-aliasing. + +**Recommended Models:** +- `llama3.1:8b-instruct-q4_K_M` โ€” 128K context, strong general performance +- `mistral:7b-instruct-q4_K_M` โ€” 32K context, fast and lightweight +- `mistral-nemo:12b-instruct-q4_K_M` โ€” 128K context, strong storytelling + +**Feature gaps:** Ollama has no image-generation or text-to-speech endpoints โ€” NPC/monster portrait generation and spoken narration will 404 in Ollama mode. Disable TTS in the UI and avoid the toolkit image features. Also requires **Ollama 0.4+** for reliable function/tool calling. + +**Known Issues** (shared with LM Studio): +- JSON parsing errors during complex combat +- Inconsistent action detection compared to GPT-4 +- Slower response times on CPU-only systems + +**Documentation:** +- Complete setup guide: `OLLAMA_SETUP.md` +- Quick reference: `OLLAMA_QUICKSTART.txt` + +**Support:** Ollama mode is provided as-is. For production gameplay, we recommend OpenAI's API with the compression system. + ### Performance Metrics Summary | Component | Original Size | Compressed Size | Reduction | From 0ef7e63ef39df115149a098ce3e47f985db2dda8 Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:08:50 -0700 Subject: [PATCH 3/8] feat(ollama): add self-contained launcher with embedded single-model alias setup --- run_with_ollama_direct.bat | 116 +++++++++++++++++++++++++++++++++++++ run_with_ollama_direct.sh | 111 +++++++++++++++++++++++++++++++++++ 2 files changed, 227 insertions(+) create mode 100644 run_with_ollama_direct.bat create mode 100644 run_with_ollama_direct.sh diff --git a/run_with_ollama_direct.bat b/run_with_ollama_direct.bat new file mode 100644 index 00000000..11f76bde --- /dev/null +++ b/run_with_ollama_direct.bat @@ -0,0 +1,116 @@ +@echo off +REM Run NeverEndingQuest with Ollama (Direct Connection - No Proxy) +REM +REM Creates two Ollama aliases on first run pointing at a single user-chosen +REM model, then starts the game. Matches LM Studio's single-model behavior. + +setlocal EnableDelayedExpansion +cd /d %~dp0 + +set "ALIAS_FULL=gpt-4.1-2025-04-14" +set "ALIAS_MINI=gpt-4.1-mini-2025-04-14" + +echo. +echo ======================================================================== +echo NEVERENDINGQUEST - OLLAMA MODE (DIRECT) +echo ======================================================================== +echo. + +REM --- Prerequisite 1: config.py must exist -------------------------------- +if not exist config.py ( + echo [ERROR] config.py not found. Copy config_template.py to config.py first: + echo copy config_template.py config.py + echo Ollama ignores the OPENAI_API_KEY value, but the file must exist. + pause + exit /b 1 +) + +REM --- Prerequisite 2: Ollama daemon reachable ----------------------------- +netstat -an | find "11434" | find "LISTENING" >nul 2>&1 +if %ERRORLEVEL% NEQ 0 ( + echo [ERROR] Ollama daemon is not listening on port 11434. + echo Launch the Ollama app or run 'ollama serve' in another terminal. + pause + exit /b 1 +) + +REM --- Alias check ---------------------------------------------------------- +REM findstr /B does a prefix match, which correctly handles "name" and "name:latest". +set "have_full=0" +set "have_mini=0" +ollama list | findstr /B "%ALIAS_FULL%" >nul 2>&1 && set "have_full=1" +ollama list | findstr /B "%ALIAS_MINI%" >nul 2>&1 && set "have_mini=1" + +if "%have_full%"=="1" if "%have_mini%"=="1" goto :launch + +echo [INFO] Ollama aliases missing. Creating them now... + +REM --- Choose a source model ------------------------------------------------ +set "source_model=" +if defined OLLAMA_MODEL ( + set "source_model=%OLLAMA_MODEL%" + ollama list | findstr /B "!source_model!" >nul 2>&1 + if errorlevel 1 ( + echo [ERROR] OLLAMA_MODEL='!source_model!' is not pulled. + echo Run: ollama pull !source_model! + pause + exit /b 1 + ) +) else ( + REM Build a candidates list: all names in `ollama list`, skipping the + REM header row and the alias names themselves. + set "count=0" + set "first_candidate=" + for /f "skip=1 tokens=1" %%N in ('ollama list') do ( + set "name=%%N" + set "base=!name!" + if /i "!base:~-7!"==":latest" set "base=!base:~0,-7!" + if /i not "!base!"=="%ALIAS_FULL%" if /i not "!base!"=="%ALIAS_MINI%" ( + set /a count+=1 + if not defined first_candidate set "first_candidate=!name!" + set "cand_!count!=!name!" + ) + ) + if !count! EQU 0 ( + echo [ERROR] No Ollama models are pulled. + echo Pull one first, e.g.: ollama pull llama3.1:8b-instruct-q4_K_M + pause + exit /b 1 + ) + if !count! GTR 1 ( + echo [ERROR] Multiple models pulled; can't auto-pick. + echo Set OLLAMA_MODEL to one of the following and re-run: + for /L %%I in (1,1,!count!) do echo !cand_%%I! + pause + exit /b 1 + ) + set "source_model=!first_candidate!" +) + +echo [INFO] Using '!source_model!' for both full and mini tiers. +ollama cp "!source_model!" "%ALIAS_FULL%" +if errorlevel 1 goto :alias_fail +ollama cp "!source_model!" "%ALIAS_MINI%" +if errorlevel 1 goto :alias_fail +echo [INFO] Aliases created. + +:launch +echo [INFO] Redirecting OpenAI SDK to Ollama (localhost:11434)... +set OPENAI_BASE_URL=http://localhost:11434/v1 +set OPENAI_API_KEY=ollama + +REM Prefer `python`, fall back to the Windows launcher `py -3` for installs +REM that lack `python` on PATH but have the official launcher. +where python >nul 2>&1 +if %ERRORLEVEL% EQU 0 ( + python run_web.py +) else ( + py -3 run_web.py +) +pause +exit /b 0 + +:alias_fail +echo [ERROR] ollama cp failed. +pause +exit /b 1 diff --git a/run_with_ollama_direct.sh b/run_with_ollama_direct.sh new file mode 100644 index 00000000..425551d5 --- /dev/null +++ b/run_with_ollama_direct.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# Run NeverEndingQuest with Ollama (Direct Connection - No Proxy) +# +# The game sends hardcoded OpenAI model names in every request. LM Studio +# ignores them; Ollama validates them and 404s unknown tags. This launcher +# creates two Ollama aliases on first run that both point at a single +# user-chosen model -- matching LM Studio's single-model behavior and +# avoiding VRAM thrashing between tiers. + +set -euo pipefail +cd "$(dirname "$0")" + +readonly ALIAS_FULL="gpt-4.1-2025-04-14" +readonly ALIAS_MINI="gpt-4.1-mini-2025-04-14" + +echo +echo "========================================================================" +echo "NEVERENDINGQUEST - OLLAMA MODE (DIRECT)" +echo "========================================================================" +echo + +# --- Prerequisite 1: config.py must exist ------------------------------------ +# Every callsite does OpenAI(api_key=config.OPENAI_API_KEY). Import fails +# before any Ollama request if config.py is missing. +if [ ! -f config.py ]; then + echo "[ERROR] config.py not found. Copy the template first:" + echo " cp config_template.py config.py" + echo "Ollama ignores the OPENAI_API_KEY value, but the file must exist." + exit 1 +fi + +# --- Prerequisite 2: Ollama CLI + daemon -------------------------------------- +if ! command -v ollama >/dev/null 2>&1; then + echo "[ERROR] ollama CLI not found on PATH. Install from https://ollama.com/download" + exit 1 +fi +if ! curl -sf http://localhost:11434/api/tags >/dev/null 2>&1; then + echo "[ERROR] Ollama daemon is not reachable on localhost:11434." + echo "Start it with 'ollama serve' (Linux) or open the Ollama app (macOS)." + exit 1 +fi + +# --- Helper: match a model name in `ollama list` ------------------------------ +# `ollama list` prints names as either "name" or "name:tag". Use awk equality +# (not regex) to avoid metacharacter pitfalls with dots/colons in user tags. +_ollama_has_model() { + local target="$1" + ollama list | awk -v t="$target" ' + NR > 1 { + if ($1 == t) { found = 1; exit } + if ($1 == t ":latest") { found = 1; exit } + if (index($1, t ":") == 1) { found = 1; exit } + } + END { exit !found } + ' +} + +# --- Alias setup (first run only) -------------------------------------------- +if _ollama_has_model "$ALIAS_FULL" && _ollama_has_model "$ALIAS_MINI"; then + echo "[INFO] Aliases already present; skipping setup." +else + echo "[INFO] Ollama aliases missing. Creating them now..." + + # Pick a source model: OLLAMA_MODEL env var, else the single non-alias + # pulled model, else abort asking the user to pick. + if [ -n "${OLLAMA_MODEL:-}" ]; then + source_model="$OLLAMA_MODEL" + if ! _ollama_has_model "$source_model"; then + echo "[ERROR] OLLAMA_MODEL='$source_model' is not pulled." + echo "Run: ollama pull $source_model" + exit 1 + fi + else + # Candidates = column 1 of `ollama list`, excluding header and the + # two alias names themselves (so re-runs don't count them). + mapfile -t candidates < <(ollama list | awk -v a="$ALIAS_FULL" -v b="$ALIAS_MINI" ' + NR > 1 { + name = $1 + # Strip :latest suffix for dedupe comparisons only + base = name + sub(/:latest$/, "", base) + if (base == a || name == a) next + if (base == b || name == b) next + print name + }') + if [ "${#candidates[@]}" -eq 0 ]; then + echo "[ERROR] No Ollama models are pulled." + echo "Pull one first, e.g.: ollama pull llama3.1:8b-instruct-q4_K_M" + exit 1 + elif [ "${#candidates[@]}" -gt 1 ]; then + echo "[ERROR] Multiple models pulled; can't auto-pick." + echo "Set OLLAMA_MODEL to one of the following and re-run:" + printf " %s\n" "${candidates[@]}" + exit 1 + fi + source_model="${candidates[0]}" + fi + + echo "[INFO] Using '$source_model' for both full and mini tiers." + ollama cp "$source_model" "$ALIAS_FULL" + ollama cp "$source_model" "$ALIAS_MINI" + echo "[INFO] Aliases created." +fi + +# --- Launch ------------------------------------------------------------------ +echo "[INFO] Redirecting OpenAI SDK to Ollama (localhost:11434)..." +export OPENAI_BASE_URL="http://localhost:11434/v1" +export OPENAI_API_KEY="ollama" + +PYTHON_BIN="${PYTHON:-python3}" +"$PYTHON_BIN" run_web.py From 26bcc8a9507c5783949b30e0c1d18678a4a945e0 Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:09:35 -0700 Subject: [PATCH 4/8] docs(ollama): add setup guide covering launcher + single-model alias flow --- OLLAMA_SETUP.md | 127 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) create mode 100644 OLLAMA_SETUP.md diff --git a/OLLAMA_SETUP.md b/OLLAMA_SETUP.md new file mode 100644 index 00000000..f0db4870 --- /dev/null +++ b/OLLAMA_SETUP.md @@ -0,0 +1,127 @@ +# Ollama Setup Guide + +Run NeverEndingQuest using your local [Ollama](https://ollama.com) installation instead of OpenAI's API. No API costs, no network dependency after pulling a model, and full offline play. + +**Status: EXPERIMENTAL.** NeverEndingQuest's prompts are tuned for GPT-4/GPT-4.1. Local models may produce malformed JSON or incorrect combat decisions โ€” see *Known Issues* below. + +## How It Works + +``` +NeverEndingQuest -> Ollama (port 11434) +``` + +Direct connection, zero overhead. The launcher sets `OPENAI_BASE_URL` so the OpenAI Python SDK talks to Ollama's OpenAI-compatible endpoint instead of `api.openai.com`. + +The game's source code contains two hardcoded OpenAI model identifiers (`gpt-4.1-2025-04-14` for heavy calls and `gpt-4.1-mini-2025-04-14` for light ones). LM Studio ignores these and serves whatever you've loaded โ€” effectively single-model. Ollama **validates** the identifier, so the launcher creates two Ollama aliases on first run, both pointing at the same user-chosen model. Single model, no VRAM thrashing, LM-Studio-equivalent behavior. + +## Prerequisites + +1. **`config.py` must exist.** Every API callsite in NeverEndingQuest is coded as `OpenAI(api_key=config.OPENAI_API_KEY)`. If the file is missing, the game crashes on import before talking to Ollama. Ollama ignores the key value, but the file is mandatory: + + ```bash + cp config_template.py config.py + ``` + + Leave the placeholder key in place or put any non-empty string โ€” it's sent to Ollama and discarded. + +2. **Ollama 0.4 or newer.** Earlier versions lack reliable function/tool calling on the OpenAI-compatible endpoint, which silently breaks combat and validation. Check with `ollama --version`. + +## 1. Install Ollama + +Download from [ollama.com/download](https://ollama.com/download). On macOS/Windows the installer starts the daemon automatically; on Linux, start it with `ollama serve` (as a systemd unit for persistent setups). + +Verify: +```bash +curl http://localhost:11434/api/tags +``` +Expected: JSON response (possibly `{"models":[]}` if nothing is pulled yet). + +## 2. Pull One Model + +Pick exactly one model โ€” the same model will handle both the game's heavy and light calls. Recommended: + +| Model tag | Notes | +|-----------------------------------------|------------------------------------| +| `llama3.1:8b-instruct-q4_K_M` | 128K context, ~5 GB disk/VRAM | +| `mistral:7b-instruct-q4_K_M` | 32K context, ~4 GB disk/VRAM | +| `mistral-nemo:12b-instruct-q4_K_M` | 128K context, stronger prose | + +Example: +```bash +ollama pull llama3.1:8b-instruct-q4_K_M +``` + +You can pull more than one and pick between them with the `OLLAMA_MODEL` env var (see *Switching models* below). But the **active** game session will always use a single model, just like LM Studio. + +## 3. Launch the Game + +- **Windows:** double-click `run_with_ollama_direct.bat` +- **macOS / Linux:** `./run_with_ollama_direct.sh` + +On **first run** the launcher: +1. Verifies `config.py` and the Ollama daemon +2. Finds your pulled model (or uses `$OLLAMA_MODEL` if set) +3. Creates the aliases `gpt-4.1-2025-04-14` and `gpt-4.1-mini-2025-04-14`, both pointing at your model +4. Starts the web server on `http://localhost:8357` + +On **subsequent runs** the launcher sees the aliases already exist and jumps straight to step 4. + +If you've pulled more than one model, the launcher refuses to guess and prints the list. Pick one: + +```bash +OLLAMA_MODEL=llama3.1:8b-instruct-q4_K_M ./run_with_ollama_direct.sh +``` + +## Switching Models + +```bash +ollama pull +ollama rm gpt-4.1-2025-04-14 gpt-4.1-mini-2025-04-14 +./run_with_ollama_direct.sh # recreates aliases against whatever is pulled +``` + +Or force a specific model without deleting aliases first: + +```bash +OLLAMA_MODEL= ./run_with_ollama_direct.sh +``` + +(Currently aliases are only re-created when they don't exist; force-delete first if you want to re-point them. A future enhancement could add an `--reset` flag.) + +## Verifying It Works + +1. Open `http://localhost:8357` +2. Start a new game and take any action +3. Watch the Ollama log: requests should hit `POST /v1/chat/completions` +4. If you see a 404 for `gpt-4.1-2025-04-14`, the alias didn't land โ€” check `ollama list` and re-run the launcher + +## Known Issues + +**Ollama-specific (not LM Studio):** +- **Image generation is broken.** Ollama has no `/v1/images/generations` endpoint. NPC/monster portrait generation from the toolkit will 404. +- **Text-to-speech is broken.** Ollama has no `/v1/audio/speech` endpoint. Disable TTS in the UI before starting a session. +- **Function/tool calling requires Ollama 0.4+.** Older versions produce free-form text instead of JSON action structures and break combat silently. + +**Shared with LM Studio:** +- JSON parsing errors in combat with smaller models (try `q5_K_M` quantization or a larger model) +- Inconsistent action detection vs GPT-4 +- Slow response on CPU-only systems +- `@TAG` compression notation is sometimes ignored + +## Troubleshooting + +| Symptom | Fix | +|------------------------------------------------------|----------------------------------------------------------| +| `connection refused: localhost:11434` | Start Ollama (`ollama serve` / open the app) | +| `No Ollama models are pulled` | `ollama pull ` and re-run the launcher | +| `Multiple models pulled; can't auto-pick` | Set `OLLAMA_MODEL=` and re-run | +| `OLLAMA_MODEL='X' is not pulled` | Typo or missing pull โ€” check `ollama list` | +| `model 'gpt-4.1-2025-04-14' not found` at runtime | Aliases were deleted externally โ€” re-run the launcher | +| Combat actions fail silently | Try a larger quantization or compare with LM Studio | + +## Configuration Reference + +- **Endpoint:** `http://localhost:11434/v1` (set by launcher via `OPENAI_BASE_URL`) +- **Aliases:** `gpt-4.1-2025-04-14` and `gpt-4.1-mini-2025-04-14` (both point at your chosen model) +- **Override source model:** set `OLLAMA_MODEL` before running the launcher +- **Compression:** enabled by default in `model_config.py` (`COMPRESSION_ENABLED = True`) From 75b174a8840fdc9c1cca65ff6ace88d23a56839f Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:09:48 -0700 Subject: [PATCH 5/8] docs(ollama): add quickstart reference --- OLLAMA_QUICKSTART.txt | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 OLLAMA_QUICKSTART.txt diff --git a/OLLAMA_QUICKSTART.txt b/OLLAMA_QUICKSTART.txt new file mode 100644 index 00000000..6a621b3e --- /dev/null +++ b/OLLAMA_QUICKSTART.txt @@ -0,0 +1,28 @@ +================================================================ +NEVERENDINGQUEST - OLLAMA QUICK START +================================================================ + +1. INSTALL OLLAMA https://ollama.com/download +2. PULL ONE MODEL ollama pull llama3.1:8b-instruct-q4_K_M +3. LAUNCH Windows: run_with_ollama_direct.bat + Mac/Lin: ./run_with_ollama_direct.sh + +That's it. The launcher auto-creates the aliases the game needs on +first run, pointing both of the game's internal model tiers at your +one pulled model (same behavior as LM Studio -- no VRAM thrashing). + +Prerequisite: + config.py must exist. + cp config_template.py config.py + (Ollama ignores the API key value, but the file must be importable.) + +To pick between multiple pulled models: + OLLAMA_MODEL= ./run_with_ollama_direct.sh + +Known gaps: + - Image generation (NPC/monster portraits) and TTS will 404. + Disable TTS in the UI; avoid toolkit image features. + - Requires Ollama >= 0.4.0 for function/tool calling. + +Full guide: OLLAMA_SETUP.md +================================================================ From 36e99adcdc508d66bc19425b3de28cae65844b91 Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:13:35 -0700 Subject: [PATCH 6/8] fix(ollama): replace bash 4+ mapfile with portable while/read loop for macOS --- run_with_ollama_direct.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/run_with_ollama_direct.sh b/run_with_ollama_direct.sh index 425551d5..f4ae4cc1 100644 --- a/run_with_ollama_direct.sh +++ b/run_with_ollama_direct.sh @@ -73,7 +73,11 @@ else else # Candidates = column 1 of `ollama list`, excluding header and the # two alias names themselves (so re-runs don't count them). - mapfile -t candidates < <(ollama list | awk -v a="$ALIAS_FULL" -v b="$ALIAS_MINI" ' + # Portable to bash 3.2 (macOS default) -- no `mapfile`. + candidates=() + while IFS= read -r _name; do + [ -n "$_name" ] && candidates+=("$_name") + done < <(ollama list | awk -v a="$ALIAS_FULL" -v b="$ALIAS_MINI" ' NR > 1 { name = $1 # Strip :latest suffix for dedupe comparisons only From a3aafb2b9dc98bbea47f5c6ab1c864308abcc089 Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:14:38 -0700 Subject: [PATCH 7/8] fix(ollama): honor OLLAMA_MODEL as an override even when aliases exist --- OLLAMA_SETUP.md | 2 +- run_with_ollama_direct.bat | 92 +++++++++++++++++++++----------------- run_with_ollama_direct.sh | 85 +++++++++++++++++++---------------- 3 files changed, 97 insertions(+), 82 deletions(-) diff --git a/OLLAMA_SETUP.md b/OLLAMA_SETUP.md index f0db4870..c0f25f6a 100644 --- a/OLLAMA_SETUP.md +++ b/OLLAMA_SETUP.md @@ -86,7 +86,7 @@ Or force a specific model without deleting aliases first: OLLAMA_MODEL= ./run_with_ollama_direct.sh ``` -(Currently aliases are only re-created when they don't exist; force-delete first if you want to re-point them. A future enhancement could add an `--reset` flag.) +When `OLLAMA_MODEL` is set, the launcher re-points both aliases at the specified model even if they already exist โ€” no `ollama rm` needed. ## Verifying It Works diff --git a/run_with_ollama_direct.bat b/run_with_ollama_direct.bat index 11f76bde..9d93048e 100644 --- a/run_with_ollama_direct.bat +++ b/run_with_ollama_direct.bat @@ -34,58 +34,66 @@ if %ERRORLEVEL% NEQ 0 ( exit /b 1 ) -REM --- Alias check ---------------------------------------------------------- -REM findstr /B does a prefix match, which correctly handles "name" and "name:latest". -set "have_full=0" -set "have_mini=0" -ollama list | findstr /B "%ALIAS_FULL%" >nul 2>&1 && set "have_full=1" -ollama list | findstr /B "%ALIAS_MINI%" >nul 2>&1 && set "have_mini=1" - -if "%have_full%"=="1" if "%have_mini%"=="1" goto :launch - -echo [INFO] Ollama aliases missing. Creating them now... - -REM --- Choose a source model ------------------------------------------------ -set "source_model=" +REM --- Alias setup ---------------------------------------------------------- +REM If OLLAMA_MODEL is set, it is an explicit override -- (re)point the +REM aliases to that model even if they already exist. Otherwise only create +REM aliases when missing. if defined OLLAMA_MODEL ( set "source_model=%OLLAMA_MODEL%" - ollama list | findstr /B "!source_model!" >nul 2>&1 + ollama list | findstr /B /L /C:"!source_model!" >nul 2>&1 if errorlevel 1 ( echo [ERROR] OLLAMA_MODEL='!source_model!' is not pulled. echo Run: ollama pull !source_model! pause exit /b 1 ) -) else ( - REM Build a candidates list: all names in `ollama list`, skipping the - REM header row and the alias names themselves. - set "count=0" - set "first_candidate=" - for /f "skip=1 tokens=1" %%N in ('ollama list') do ( - set "name=%%N" - set "base=!name!" - if /i "!base:~-7!"==":latest" set "base=!base:~0,-7!" - if /i not "!base!"=="%ALIAS_FULL%" if /i not "!base!"=="%ALIAS_MINI%" ( - set /a count+=1 - if not defined first_candidate set "first_candidate=!name!" - set "cand_!count!=!name!" - ) - ) - if !count! EQU 0 ( - echo [ERROR] No Ollama models are pulled. - echo Pull one first, e.g.: ollama pull llama3.1:8b-instruct-q4_K_M - pause - exit /b 1 - ) - if !count! GTR 1 ( - echo [ERROR] Multiple models pulled; can't auto-pick. - echo Set OLLAMA_MODEL to one of the following and re-run: - for /L %%I in (1,1,!count!) do echo !cand_%%I! - pause - exit /b 1 + echo [INFO] OLLAMA_MODEL set; re-pointing aliases at '!source_model!'. + ollama cp "!source_model!" "%ALIAS_FULL%" + if errorlevel 1 goto :alias_fail + ollama cp "!source_model!" "%ALIAS_MINI%" + if errorlevel 1 goto :alias_fail + echo [INFO] Aliases updated. + goto :launch +) + +REM findstr /B /L /C:"..." = literal (non-regex) prefix match; handles "name" and "name:latest". +set "have_full=0" +set "have_mini=0" +ollama list | findstr /B /L /C:"%ALIAS_FULL%" >nul 2>&1 && set "have_full=1" +ollama list | findstr /B /L /C:"%ALIAS_MINI%" >nul 2>&1 && set "have_mini=1" + +if "%have_full%"=="1" if "%have_mini%"=="1" goto :launch + +echo [INFO] Ollama aliases missing. Creating them now... + +REM Build a candidates list: all names in `ollama list`, skipping the +REM header row and the alias names themselves. +set "count=0" +set "first_candidate=" +for /f "skip=1 tokens=1" %%N in ('ollama list') do ( + set "name=%%N" + set "base=!name!" + if /i "!base:~-7!"==":latest" set "base=!base:~0,-7!" + if /i not "!base!"=="%ALIAS_FULL%" if /i not "!base!"=="%ALIAS_MINI%" ( + set /a count+=1 + if not defined first_candidate set "first_candidate=!name!" + set "cand_!count!=!name!" ) - set "source_model=!first_candidate!" ) +if !count! EQU 0 ( + echo [ERROR] No Ollama models are pulled. + echo Pull one first, e.g.: ollama pull llama3.1:8b-instruct-q4_K_M + pause + exit /b 1 +) +if !count! GTR 1 ( + echo [ERROR] Multiple models pulled; can't auto-pick. + echo Set OLLAMA_MODEL to one of the following and re-run: + for /L %%I in (1,1,!count!) do echo !cand_%%I! + pause + exit /b 1 +) +set "source_model=!first_candidate!" echo [INFO] Using '!source_model!' for both full and mini tiers. ollama cp "!source_model!" "%ALIAS_FULL%" diff --git a/run_with_ollama_direct.sh b/run_with_ollama_direct.sh index f4ae4cc1..172220ea 100644 --- a/run_with_ollama_direct.sh +++ b/run_with_ollama_direct.sh @@ -55,50 +55,57 @@ _ollama_has_model() { ' } -# --- Alias setup (first run only) -------------------------------------------- -if _ollama_has_model "$ALIAS_FULL" && _ollama_has_model "$ALIAS_MINI"; then +# --- Alias setup ------------------------------------------------------------- +# Normally: create aliases only if missing. If OLLAMA_MODEL is set, treat it +# as an explicit override and (re)point the aliases to that model even if +# they already exist. +_aliases_present() { + _ollama_has_model "$ALIAS_FULL" && _ollama_has_model "$ALIAS_MINI" +} + +if [ -n "${OLLAMA_MODEL:-}" ]; then + source_model="$OLLAMA_MODEL" + if ! _ollama_has_model "$source_model"; then + echo "[ERROR] OLLAMA_MODEL='$source_model' is not pulled." + echo "Run: ollama pull $source_model" + exit 1 + fi + echo "[INFO] OLLAMA_MODEL set; re-pointing aliases at '$source_model'." + ollama cp "$source_model" "$ALIAS_FULL" + ollama cp "$source_model" "$ALIAS_MINI" + echo "[INFO] Aliases updated." +elif _aliases_present; then echo "[INFO] Aliases already present; skipping setup." else echo "[INFO] Ollama aliases missing. Creating them now..." - # Pick a source model: OLLAMA_MODEL env var, else the single non-alias - # pulled model, else abort asking the user to pick. - if [ -n "${OLLAMA_MODEL:-}" ]; then - source_model="$OLLAMA_MODEL" - if ! _ollama_has_model "$source_model"; then - echo "[ERROR] OLLAMA_MODEL='$source_model' is not pulled." - echo "Run: ollama pull $source_model" - exit 1 - fi - else - # Candidates = column 1 of `ollama list`, excluding header and the - # two alias names themselves (so re-runs don't count them). - # Portable to bash 3.2 (macOS default) -- no `mapfile`. - candidates=() - while IFS= read -r _name; do - [ -n "$_name" ] && candidates+=("$_name") - done < <(ollama list | awk -v a="$ALIAS_FULL" -v b="$ALIAS_MINI" ' - NR > 1 { - name = $1 - # Strip :latest suffix for dedupe comparisons only - base = name - sub(/:latest$/, "", base) - if (base == a || name == a) next - if (base == b || name == b) next - print name - }') - if [ "${#candidates[@]}" -eq 0 ]; then - echo "[ERROR] No Ollama models are pulled." - echo "Pull one first, e.g.: ollama pull llama3.1:8b-instruct-q4_K_M" - exit 1 - elif [ "${#candidates[@]}" -gt 1 ]; then - echo "[ERROR] Multiple models pulled; can't auto-pick." - echo "Set OLLAMA_MODEL to one of the following and re-run:" - printf " %s\n" "${candidates[@]}" - exit 1 - fi - source_model="${candidates[0]}" + # Candidates = column 1 of `ollama list`, excluding header and the + # two alias names themselves (so re-runs don't count them). + # Portable to bash 3.2 (macOS default) -- no `mapfile`. + candidates=() + while IFS= read -r _name; do + [ -n "$_name" ] && candidates+=("$_name") + done < <(ollama list | awk -v a="$ALIAS_FULL" -v b="$ALIAS_MINI" ' + NR > 1 { + name = $1 + # Strip :latest suffix for dedupe comparisons only + base = name + sub(/:latest$/, "", base) + if (base == a || name == a) next + if (base == b || name == b) next + print name + }') + if [ "${#candidates[@]}" -eq 0 ]; then + echo "[ERROR] No Ollama models are pulled." + echo "Pull one first, e.g.: ollama pull llama3.1:8b-instruct-q4_K_M" + exit 1 + elif [ "${#candidates[@]}" -gt 1 ]; then + echo "[ERROR] Multiple models pulled; can't auto-pick." + echo "Set OLLAMA_MODEL to one of the following and re-run:" + printf " %s\n" "${candidates[@]}" + exit 1 fi + source_model="${candidates[0]}" echo "[INFO] Using '$source_model' for both full and mini tiers." ollama cp "$source_model" "$ALIAS_FULL" From 8c98f520a7f53792ae87129a9bcd07ab2378e995 Mon Sep 17 00:00:00 2001 From: Loup Date: Thu, 23 Apr 2026 23:14:47 -0700 Subject: [PATCH 8/8] fix(ollama): add curl connect/total timeouts to daemon precheck --- run_with_ollama_direct.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/run_with_ollama_direct.sh b/run_with_ollama_direct.sh index 172220ea..da621c8e 100644 --- a/run_with_ollama_direct.sh +++ b/run_with_ollama_direct.sh @@ -34,7 +34,7 @@ if ! command -v ollama >/dev/null 2>&1; then echo "[ERROR] ollama CLI not found on PATH. Install from https://ollama.com/download" exit 1 fi -if ! curl -sf http://localhost:11434/api/tags >/dev/null 2>&1; then +if ! curl -sf --connect-timeout 3 --max-time 5 http://localhost:11434/api/tags >/dev/null 2>&1; then echo "[ERROR] Ollama daemon is not reachable on localhost:11434." echo "Start it with 'ollama serve' (Linux) or open the Ollama app (macOS)." exit 1