hyper-ollama

Run a coding-focused open model on an Emory HyPER C3 GPU and use it from a local laptop through an SSH tunnel. The launcher handles the remote Ollama installation, Slurm submission, queue monitoring, model download, readiness notifications, and port forwarding.

The default model is qwen3.6:35b-a3b, served with a 64K context. By default, the launcher races a one-GPU 96 GB RTX PRO request against a two-GPU 48 GB L4 request and uses whichever Slurm allocation starts first.

Requirements

On your local macOS, Linux, or WSL machine:

• bash, curl, ssh, and scp
• working SSH access to Emory HyPER C3

No administrator access or preinstalled software is needed on C3. Each user gets an isolated installation and model cache under ~/.local/share/hyper-ollama on the cluster.

Install

Run this on your local machine:

curl -fsSL https://raw.githubusercontent.com/weinstocklab/hyper-ollama/main/install.sh | bash

If ~/.local/bin is not already in PATH, follow the message printed by the installer and open a new terminal.

Start

Pass the hostname or SSH config alias you normally use for HyPER C3:

hyper-ollama HYPER_SSH_HOST

On the first run, the launcher installs about 2 GB of Ollama software and the Slurm job downloads about 24 GB of model data. Both are cached for later runs.

The launcher reports Slurm queue changes, rings the terminal, and attempts a native macOS/Linux notification when the GPU allocation starts and again when the model is ready. It then exposes:

http://127.0.0.1:11434

Keep the launcher terminal open while using the model. Pressing Ctrl-C closes the tunnel and cancels the Slurm allocation by default so GPU time is not left running accidentally. See all options with:

hyper-ollama --help

GPU queue strategy

The default race strategy submits two pending candidates:

• one 96 GB RTX PRO 6000 GPU on rp6b-1-gm96-c8-m64;
• two 24 GB L4 GPUs on l4-4-gm96-c48-m192.

As soon as either candidate starts, the other is cancelled. The two jobs never consume GPU resources at the same time. To restrict submission to one partition:

hyper-ollama --gpu-preset rtx HYPER_SSH_HOST
hyper-ollama --gpu-preset l4 HYPER_SSH_HOST

Use with OpenCode

With the tunnel running, copy the installed provider configuration into the local project you want OpenCode to edit:

cp "${XDG_DATA_HOME:-$HOME/.local/share}/hyper-ollama/examples/opencode.json" ./opencode.json
opencode --model ollama/qwen3.6:35b-a3b

OpenCode is the recommended first client because it explicitly supports custom OpenAI-compatible providers such as Ollama.

Use with Codex CLI

Current Codex CLI versions expose an open-source-provider mode:

OLLAMA_HOST=http://127.0.0.1:11434 \
  codex --oss --local-provider ollama --model qwen3.6:35b-a3b

Tool calling and agent behavior can differ from the OpenAI models for which Codex is primarily designed, so test the workflow on a non-critical project first.

Direct API test

curl http://127.0.0.1:11434/api/chat -d '{
  "model": "qwen3.6:35b-a3b",
  "messages": [{"role": "user", "content": "Write a Python binary search."}],
  "stream": false
}'

How it works

1. The local installer puts hyper-ollama in ~/.local/bin and small runtime assets in ~/.local/share/hyper-ollama.
2. The launcher connects to C3 and copies the Slurm assets into that remote user's home directory.
3. On the first run, it installs the official Ollama binary without sudo.
4. It races the RTX and L4 Slurm candidates and cancels the loser immediately.
5. It forwards local port 11434 through the C3 login host to the winning node.
6. Ctrl-C tears down the tunnel and winning allocation.

The Ollama API binds only while the Slurm allocation exists. It is not exposed directly to the public internet.

Development

Validate the shell and JSON files with:

bash -n install.sh bin/hyper-ollama remote/*.sh remote/*.sbatch
shellcheck install.sh bin/hyper-ollama remote/*.sh remote/*.sbatch
python -m json.tool examples/opencode.json >/dev/null

License

MIT