Test Ollama Models

Test Ollama Models is a desktop benchmarking application for comparing local or remote Ollama-compatible language models through a simple Flet UI.

The app lets you configure a list of models and prompts, run the same question set against multiple models with configurable concurrency, inspect generated answers, and review latency, token, and tokens-per-second metrics collected during each run.

Features

• Run the same prompt set against one or more configured models
• Control request concurrency to simulate parallel usage
• Watch test progress in real time from the main console and per-instance consoles
• Review generated answers in a dedicated tab
• Visualize historical and live metrics for latency, token count, and TPS
• Persist settings and run results in SQLite databases created automatically inside the Data/ directory
• Organize the codebase by UI, Presenter, Application, Domain, Gateway, and Infrastructure layers

Tech Stack

• Python 3.11
• Flet for the desktop UI
• requests for HTTP calls to the Ollama API
• SQLite for configuration and result storage
• dependency-injector for wiring the application
• pytest, black, isort, flake8, mypy, bandit, and pre-commit for development workflows

How It Works

The application starts from main.py, builds the dependency container, and renders a tab-based Flet interface with four main areas:

• Run Tests: select models, set concurrency, and execute benchmark runs
• Answers: inspect collected model responses per instance or across all instances
• Metrics: explore live and historical charts for latency, tokens, and TPS
• Settings: manage API connection parameters, model names, and the shared question list

During a run, the app:

1. Loads the configured models and API endpoint.
2. Performs a health check for each selected model.
3. Sends each question to the Ollama /api/generate endpoint.
4. Measures latency, token count, and tokens per second for every response.
5. Saves raw per-instance results and averaged results into SQLite.
6. Streams logs, answers, and chart data back into the UI through the internal event bus.

Project Structure

.
|-- app/
|   |-- Application/     # Use cases and application services
|   |-- DI/              # Dependency injection container
|   |-- Domain/          # Core entities, scenarios, runner, metrics
|   |-- Gateways/        # Ollama client and response parsing
|   |-- Infrastructure/  # SQLite storage, retry policy, paths, defaults
|   |-- Presenters/      # View models and presenter logic
|   `-- UI/              # Flet views and reusable UI components
|-- tests/               # Unit, UI, and integration tests
|-- main.py              # Application entry point
|-- Makefile             # Developer commands
`-- requirements.txt     # Python dependencies

Getting Started

Prerequisites

• Python 3.11
• An available Ollama-compatible API endpoint
• At least one installed model exposed by that endpoint

Installation

python -m venv .venv

Activate the environment:

# Windows
.venv\Scripts\activate

# macOS / Linux
source .venv/bin/activate

Install dependencies:

pip install -r requirements.txt

If you have make available and want the full development toolchain as well, run:

make install

Running the App

Start the desktop application with:

python main.py

On first launch, the app creates SQLite databases in Data/config.db and Data/results.db.

The default API configuration points to:

• Host: 192.168.194.21
• Port: 11434
• Path: /api/generate

If your Ollama instance uses a different address, update it in the Settings tab before running tests.

Usage

1. Open the Settings tab and verify the API address, available models, and question list.
2. Return to Run Tests and choose one or more models.
3. Set the desired concurrency level.
4. Start the run and monitor the consoles.
5. Open Answers to inspect responses.
6. Open Metrics to compare raw and averaged results, including historical runs.

Data Storage

The project stores data in SQLite:

• Data/config.db: saved settings such as API address, models, and questions
• Data/results.db: benchmark runs, per-question results, and averaged metrics

These files are generated automatically and are not required to exist before the first run.

Development

Useful commands from Makefile:

make format      # black + isort
make lint        # flake8
make mypy        # static type checks
make test        # pytest
make check       # combined code quality checks
make bandit      # security scan
make hooks       # install pre-commit hooks

Testing

The repository includes:

• Unit tests for domain, gateway, presenter, and infrastructure behavior
• UI-oriented tests for view logic
• Integration coverage for the service flow

Run all tests with:

pytest

Notes

• Model discovery is configuration-based: models are managed in the Settings tab rather than fetched dynamically from Ollama.
• Results are stored per run, which makes it possible to reload and compare historical metrics in the charts tab.
• The project already includes a clean separation between business logic and UI code, making it a good base for extending benchmarking scenarios or adding more model providers.