Offline Ollama Assistant

offline-ollama-assistant is a simple tool that uses Ollama models on your computer to help you with coding and other tasks. Everything runs offline on your machine.

What It Can Do

• Find all your Ollama models automatically.
• Use different models for different tasks through simple settings files.
• Read, summarize, and edit files safely on your computer.
• Pull text from PDF and DOCX files (if you install extra packages).
• Save all your settings in easy-to-edit config files.

Prerequisites

Before you start, make sure you have:

• Python 3.10 or newer
• Ollama installed and running locally
• At least one Ollama model pulled on your machine

Example - Get a model:

ollama pull llama3.2:1b

Check that Ollama is running:

ollama list

Setup

Follow these steps to set up the project on your machine.

1. Clone the project

git clone https://github.com/abhinav25232354/CodingAgent.git
cd CodingAgent

2. Create a virtual environment

Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1

macOS/Linux:

python3 -m venv .venv
source .venv/bin/activate

3. Install the project

Basic install:

pip install -e .

Install with document support for PDF and DOCX:

pip install -e .[docs]

4. Check that your model is downloaded

If you don't have a model yet, get one now:

ollama pull llama3.2:1b

Then verify:

ollama list

5. Check your settings

Open settings/settings.json and look at these:

• ollama.base_url: usually stays as http://127.0.0.1:11434
• model_selection.active_model: use a model you already have
• permissions.read_roots: folders the assistant can read
• permissions.write_roots: folders the assistant can edit

Default example:

"model_selection": {
  "mode": "fixed",
  "active_model": "llama3.2:1b",
  "allow_multi_model_fallback": false,
  "default_role": "general"
}

Run The Assistant

You can start the assistant in either of these ways:

python main.py

or:

offline-assistant

Make Your Own Coding Helper

If you want to use this tool as your personal coding assistant, follow these steps:

1. Make a workspace folder

Create a folder where you keep your coding projects.

Example:

DevWorkspace/
|-- CodingAgent/
|-- my-project-1/
|-- my-project-2/

2. Choose what the assistant can see and edit

In settings/settings.json, update:

• read_roots - folders the assistant can read
• write_roots - folders the assistant can edit

If you only want it to work here, use "." for both.

To help with another project, point these settings to that project's folder.

3. Pick a model

Good choices to start:

• For basic tasks: llama3.2:1b
• For better coding: use a larger model if you have one

Set it in:

settings/settings.json -> model_selection.active_model

4. Change prompts and behavior

You can edit:

• settings/system_prompts.json to change how the assistant talks
• settings/settings.json to change what it can do

Use this to:

• Explain code
• Improve code
• Edit files
• Summarize documents
• Get help offline

5. Start the assistant

Turn on your virtual environment, start Ollama, and run the assistant.

First-Time Setup Checklist

Follow this if you're new to the project:

1. Install Python 3.10+
2. Install Ollama
3. Pull at least one model with ollama pull <model-name>
4. Create and activate a virtual environment
5. Run pip install -e .
6. Optionally run pip install -e .[docs]
7. Update settings/settings.json
8. Run python main.py

Project Structure

.
|-- pyproject.toml
|-- README.md
|-- main.py
|-- settings/
|   |-- settings.json
|   `-- system_prompts.json
|-- src/
|   `-- offline_assistant/
|       |-- __init__.py
|       |-- assistant.py
|       |-- cli.py
|       |-- config.py
|       |-- models.py
|       |-- ollama_client.py
|       |-- prompting.py
|       |-- routing.py
|       |-- utils.py
|       `-- tools/
|           |-- __init__.py
|           |-- base.py
|           |-- documents.py
|           |-- files.py
|           |-- registry.py
|           |-- textops.py
|           `-- web.py
`-- examples/
    |-- sample_session.txt
    `-- tasks.md

Important Settings Files

settings/settings.json

Sets:

• How to connect to Ollama
• Which model to use
• Which folders to read and edit
• What tools are on or off
• How the model works

settings/system_prompts.json

Sets how the assistant acts and what it says.

Quick Notes

• Works great with just one model.
• To read PDFs and Word files, run pip install -e .[docs].
• Everything runs on your computer - no internet needed.