OvisOCR2 FastAPI

A production-minded, local-network API for the official ATH-MaaS/OvisOCR2 compact 0.8B document parser. Upload a PDF or image and receive reading-order Markdown, HTML tables, LaTeX formulas, and extracted figure regions. Optionally connect an administrator-configured OpenAI-compatible or native Ollama text model for reliable targeted extraction from that OCR output.

The service prefers an NVIDIA GPU, automatically falls back to CPU, and releases model VRAM after every complete request by default. It has no application-level authentication and is intended to be protected by UFW or an equivalent LAN firewall.

Quick start

Requirements: Docker Engine with Compose v2 on Linux, or Docker Desktop on Windows. NVIDIA GPU use additionally requires a working NVIDIA driver and container runtime. CPU mode remains available but is expected to be much slower.

Linux:

git clone https://github.com/bigsk1/ovis-fastapi.git
cd ovis-fastapi
chmod +x install.sh
./install.sh

Windows PowerShell:

git clone https://github.com/bigsk1/ovis-fastapi.git
Set-Location ovis-fastapi
Set-ExecutionPolicy -Scope Process Bypass
.\install.ps1

The installers detect NVIDIA container support and use the GPU overlay when available. Force CPU mode with ./install.sh --cpu or .\install.ps1 -Cpu.

The API binds to all interfaces on the nonstandard host port 17860. From another LAN machine, replace 127.0.0.1 with the Docker host's LAN address:

• Swagger UI: http://127.0.0.1:17860/docs
• Readiness: http://127.0.0.1:17860/health/ready
• OCR: POST http://127.0.0.1:17860/v1/ocr

The first start downloads approximately 1.7 GB of model weights into the persistent Docker volume ovis-fastapi-model-cache. Watch progress with:

docker compose -f docker-compose.yml -f docker-compose.linux.yml logs -f

On Windows, use docker-compose.windows.yml in place of docker-compose.linux.yml in manual Compose commands.

API

Parse an image or PDF

curl -X POST "http://127.0.0.1:17860/v1/ocr" \
  -F "file=@document.pdf" \
  -F "page_start=1" \
  -F "page_end=10" \
  -F "keep_region_tags=true"

The JSON response contains combined Markdown plus a result for every selected page:

{
  "request_id": "f649e14e-1d1f-491c-b43a-8a032c354e39",
  "filename": "document.pdf",
  "document_type": "pdf",
  "model": "ATH-MaaS/OvisOCR2",
  "pages_processed": 2,
  "total_pages": 2,
  "markdown": "<!-- Page 1 -->\n\n# Document...",
  "pages": [
    {
      "page_number": 1,
      "markdown": "# Document...",
      "regions": [
        {"filename": "bbox_100_200_900_600.jpg", "bbox": [100, 200, 900, 600]}
      ],
      "elapsed_seconds": 3.21
    }
  ],
  "elapsed_seconds": 6.52
}

Set include_region_data=true to include each figure crop as a JPEG data URL. This can make responses large.

Download a complete archive

curl -X POST "http://127.0.0.1:17860/v1/ocr/archive" \
  -F "file=@document.pdf" \
  --output document-ocr.zip

The ZIP contains document.md, result.json, per-page Markdown, and every figure crop referred to by the model's bounding-box tags.

Targeted extraction

POST /v1/generate is a two-stage document-extraction endpoint, not an image-creation endpoint. It first parses every selected page with OvisOCR2's proven fixed Markdown prompt. It then sends page-attributed OCR Markdown and the caller's instruction to an administrator-configured text backend.

This separation is intentional. Testing against the pinned OvisOCR2 revision confirmed that the processor receives different prompt tokens, but the model produced essentially identical OCR Markdown for the standard prompt, Return exactly BANANA and nothing else, and a strict invoice JSON prompt. Arbitrary instruction following is therefore treated as an OvisOCR2 model limitation, not a FastAPI prompt-wiring bug.

Configure the optional backend in the root .env. This is a complete OpenAI configuration; the API key line is intentionally empty in .env.example and must contain your own secret only in the gitignored .env:

OVIS_GENERATE_BACKEND=openai
OVIS_GENERATE_BASE_URL=https://api.openai.com/v1
OVIS_GENERATE_API_KEY=
OVIS_GENERATE_MODEL=gpt-4.1-mini

For native Ollama strict JSON Schema mode on the default Linux host-network deployment:

OVIS_GENERATE_BACKEND=ollama
OVIS_GENERATE_BASE_URL=http://127.0.0.1:11434
OVIS_GENERATE_API_KEY=
OVIS_GENERATE_MODEL=gemma4:latest
OVIS_GENERATE_KEEP_ALIVE_SECONDS=0
OVIS_GENERATE_NUM_CTX=32768

Native Ollama mode sends schemas through Ollama's format field and disables thinking in the returned content. It passes num_ctx and the request's bounded output allowance as Ollama's num_predict option. A keep-alive value of 0 unloads the text model immediately after each request, freeing its VRAM. Linux host networking makes 127.0.0.1 refer to the Docker host; Windows Docker Desktop deployments should use http://host.docker.internal:11434 instead.

The backend URL is accepted only from administrator configuration and can never be supplied in an API request. If the backend URL and model are left empty, /v1/ocr and /v1/ocr/archive continue normally while /v1/generate returns 503 generate_backend_unavailable.

Document-scoped, schema-checked invoice extraction:

curl -X POST "http://127.0.0.1:17860/v1/generate" \
  -F "file=@invoice.png" \
  -F "prompt=Extract the invoice number, total, and due date." \
  -F "scope=document" \
  -F "response_format=json" \
  -F 'json_schema={"type":"object","properties":{"invoice_number":{"type":"string"},"total":{"type":"number"},"due_date":{"type":"string"}},"required":["invoice_number","total","due_date"],"additionalProperties":false}'

scope=document preserves explicit page markers and makes one extraction request over all selected pages. scope=page makes one independent extraction request per page and returns a pages array. Available response formats are text, markdown, and json. JSON responses succeed only after strict parsing and optional JSON Schema validation; malformed or schema-invalid model output returns 502 invalid_model_output instead of being hidden in a successful Markdown response.

The generation response has its own contract:

{
  "request_id": "f649e14e-1d1f-491c-b43a-8a032c354e39",
  "filename": "invoice.png",
  "document_type": "image",
  "scope": "document",
  "response_format": "json",
  "output": "{\"invoice_number\":\"INV-2026-0042\",\"total\":1296,\"due_date\":\"September 9, 2026\"}",
  "parsed_json": {
    "invoice_number": "INV-2026-0042",
    "total": 1296,
    "due_date": "September 9, 2026"
  },
  "pages": [],
  "ocr_model": "ATH-MaaS/OvisOCR2",
  "generation_model": "gpt-4.1-mini",
  "pages_processed": 1,
  "total_pages": 1,
  "elapsed_seconds": 4.12,
  "ocr_elapsed_seconds": 3.21,
  "generation_elapsed_seconds": 0.91
}

Backend timeouts are not automatically retried because the server cannot know whether an ambiguous request completed remotely. They return 504 generate_backend_timeout.

Other routes:

Route	Purpose
GET /health/live	Process liveness for Docker
GET /health/ready	Model readiness
GET /v1/models	Configured, preferred, current, and last inference device state
GET /v1/capabilities	Supported inputs, outputs, and configured limits
POST /v1/ocr	Structured JSON OCR result
POST /v1/ocr/archive	Downloadable Markdown/JSON/crops ZIP
POST /v1/generate	Two-stage targeted document extraction

Accepted files are PDF, PNG, JPEG, WebP, BMP, and TIFF (including multi-page TIFF). Defaults are 100 MB per upload and 50 selectable pages. A combined page/output-token budget, decoded-pixel limits, derived-region limits, generation-context limits, and single-request admission gate protect the local service from resource amplification. A concurrent OCR or generation call receives 503 with Retry-After: 5.

GPU use and VRAM offload

The default OVIS_DEVICE=auto chooses CUDA when the GPU is visible, and CPU otherwise. The GPU image uses the official PyTorch 2.10 CUDA 12.8 wheels, which work with a sufficiently recent NVIDIA driver without installing a CUDA toolkit on the host.

OVIS_GPU_OFFLOAD=after_request keeps weights in system RAM, moves them to the GPU for the whole request, and moves them back after the final PDF page. PyTorch's CUDA cache is then emptied. This releases the model allocations between jobs, although the process can retain a small CUDA runtime context. Available policies are:

Policy	Behavior
after_request	Lowest idle VRAM; pays a CPU-to-GPU transfer on each job
idle	Offloads after OVIS_GPU_IDLE_SECONDS without a new request
never	Keeps the model resident for minimum request latency

If CUDA initialization, allocation, or transfer fails, OVIS_FALLBACK_TO_CPU=true retries on CPU. For repeated large batches, idle or never will be faster than immediate offload.

GET /health/live, GET /health/ready, and GET /v1/models distinguish configured_device, preferred_device, current_device, cuda_available, gpu_offload, inference_active, and last_inference_device. With after_request, an idle model can correctly report current_device=cpu and last_inference_device=cuda; CUDA performed the previous inference and the weights were subsequently offloaded.

To start GPU mode manually:

docker compose -f docker-compose.yml -f docker-compose.linux.yml -f docker-compose.gpu.yml up --build -d

To start the smaller CPU-only image manually:

docker compose -f docker-compose.yml -f docker-compose.linux.yml up --build -d

The Linux overlay uses host networking so traffic reaches UFW's normal INPUT path. The Windows overlay uses Docker Desktop port publishing and Windows Firewall.

Configuration

Copy .env.example to .env and change values before starting. The installers create this real root .env automatically when it is missing. It is deliberately gitignored because it can contain API credentials and host-specific settings. The application also has safe built-in defaults, so OCR can run without .env; Compose uses .env when present and otherwise applies the same defaults.

Variable	Default	Description
OVIS_PORT	17860	Host/LAN TCP port
OVIS_MODEL_ID	ATH-MaaS/OvisOCR2	Hugging Face model ID or mounted path
OVIS_MODEL_REVISION	65c619d374b55d4152e85150fc1b003700bc1f0c	Immutable Hugging Face model revision; clear for a local model path
OVIS_DEVICE	auto	auto, cuda, or cpu
OVIS_FALLBACK_TO_CPU	true	Use CPU when CUDA is unavailable/fails
OVIS_GPU_OFFLOAD	after_request	after_request, idle, or never
OVIS_GPU_IDLE_SECONDS	60	Idle delay when the policy is idle
OVIS_DTYPE	bfloat16	bfloat16 or float32
OVIS_MAX_NEW_TOKENS	16384	Generation ceiling per page
OVIS_MAX_UPLOAD_MB	100	Upload size limit
OVIS_MAX_PAGES	50	Selected-page limit per request
OVIS_MAX_REQUEST_TOKENS	262144	Combined selected-pages × output-token budget
OVIS_MAX_CONCURRENT_REQUESTS	1	OCR requests admitted at once; excess calls get 503
OVIS_MAX_SOURCE_PIXELS	25000000	Maximum decoded pixels per source page
OVIS_MAX_SOURCE_DIMENSION	16384	Maximum decoded width or height
OVIS_MAX_REGIONS_PER_PAGE	100	Maximum model-directed figure crops per page
OVIS_MAX_REGION_PIXELS	25000000	Aggregate crop pixels allowed per page
OVIS_MAX_REGION_MB	25	Aggregate encoded crop data allowed per page
OVIS_MAX_REGIONS_PER_REQUEST	250	Aggregate figure-count limit per document
OVIS_MAX_REGION_REQUEST_PIXELS	100000000	Aggregate crop-pixel limit per document
OVIS_MAX_REGION_REQUEST_MB	100	Aggregate encoded crop-data limit per document
OVIS_PDF_RENDER_SCALE	2.0	PDF rasterization resolution multiplier
OVIS_GENERATE_BACKEND	openai	openai for Chat Completions or ollama for native strict output
OVIS_GENERATE_BASE_URL	empty	Administrator-set backend base URL; required with model
OVIS_GENERATE_API_KEY	empty	Optional bearer token for the generation backend
OVIS_GENERATE_MODEL	empty	Text-instruction model ID; required with base URL
OVIS_GENERATE_TIMEOUT_SECONDS	60	Backend timeout; ambiguous requests are not retried
OVIS_GENERATE_KEEP_ALIVE_SECONDS	0	Native Ollama model residency; 0 unloads after each call
OVIS_GENERATE_NUM_CTX	32768	Native Ollama context window; larger values require more RAM/VRAM
OVIS_MAX_GENERATE_INPUT_CHARS	200000	Maximum page-attributed OCR characters sent to the backend
OVIS_MAX_GENERATE_TOKENS	4096	Maximum backend output tokens per call
OVIS_MAX_GENERATE_REQUEST_TOKENS	16384	Aggregate page-scope output-token budget
OVIS_CORS_ORIGINS	empty	Comma-separated browser origins
OVIS_MEMORY_LIMIT	8g	Container RAM ceiling

After editing .env, repeat the applicable manual Compose command or rerun the installer.

LAN firewall

The container intentionally listens on 0.0.0.0. On Linux, the installer uses host networking so UFW rules govern the service directly; this avoids Docker's normal published-port bypass of UFW's INPUT chain. The following commands derive the subnet attached to the default-route interface, show it for review, and allow only that subnet:

LAN_INTERFACE="$(ip route show default | awk '{print $5; exit}')"
LAN_SUBNET="$(ip -4 route show dev "$LAN_INTERFACE" proto kernel scope link | awk '{print $1; exit}')"
printf 'Allowing Ovis OCR clients from %s on %s\n' "$LAN_SUBNET" "$LAN_INTERFACE"
sudo ufw allow in on "$LAN_INTERFACE" proto tcp from "$LAN_SUBNET" to any port 17860 comment 'Ovis OCR API'
sudo ufw status numbered

Review the printed interface and subnet before running the ufw allow command. Do not forward this port from your router or expose it directly to the public internet. See SECURITY.md for the deployment boundary. Windows users should create the equivalent private-profile, remote-address-scoped Windows Firewall rule.

Local development with uv

The Docker images and local workflow both use uv and a project .venv:

./scripts/setup-local.sh
uv run pytest
uv run uvicorn ovis_fastapi.main:app --host 0.0.0.0 --port 17860 --reload

On Windows, run .\scripts\setup-local.ps1. The local lockfile selects CPU PyTorch; the dedicated GPU Dockerfile replaces it with the CUDA wheel set.

Operations

docker compose -f docker-compose.yml -f docker-compose.linux.yml ps
docker compose -f docker-compose.yml -f docker-compose.linux.yml logs -f
docker compose -f docker-compose.yml -f docker-compose.linux.yml restart
docker compose -f docker-compose.yml -f docker-compose.linux.yml down  # keeps the model cache
docker volume rm ovis-fastapi-model-cache  # explicitly deletes cached weights

Only one Uvicorn worker is used because every worker would load another model copy. Model inference is serialized inside the process; concurrent calls wait instead of duplicating VRAM.

Upstream and license

This is an independent API wrapper, not an official ATH-MaaS repository. OvisOCR2 and its model card are Apache-2.0 licensed. Review the upstream model's disclaimer and verify OCR output before using it for legal, financial, medical, or other high-stakes work.

• OvisOCR2 model and inference guide
• Ovis upstream repository
• OvisOCR2 technical report

This wrapper is licensed under Apache-2.0. See LICENSE.