Local LLM Agent

A small, tool-augmented local AI agent built around a straightforward agent loop, with a FastAPI backend and a React + TypeScript frontend.

---

🌟 Key Features

• 🤖 Autonomous Function Calling Agent: Integrates with any OpenAI-compatible local model backend (Ollama, LM Studio, vLLM, etc.).
• 🔁 Minimal Agent Loop: Lets the model request a tool, observes the result, and continues until it can produce a final answer—without a heavyweight agent framework.
• 🛠️ Built-in Tool Matrix:
  • 🌐 Web Search (search_web): Locates relevant search results using DuckDuckGo.
  • 📄 Web Scraper (parse_webpage): Downloads and extracts clean readable text from selected webpages.
  • 🐍 Python Sandbox (execute_python_code): Runs Python code locally for calculations, data analysis, and logic verification.
  • 📅 Date & Time Query (get_current_date): Resolves current time context for date-sensitive questions.
• ⚡ Streaming Responses: Streams model output incrementally from the local LLM to the browser over Server-Sent Events (SSE).
• 🔧 Live Tool Activity: Emits structured tool start and completion events so the UI can show which tool is currently running.
• 💻 Two Interfaces:
  • CLI: A lightweight terminal chat in chat.py.
  • Web UI: A classic light conversation interface with Markdown rendering, syntax highlighting, and tool activity indicators.

---

📁 Repository Structure

local-chat-agent/
├── configs/
│   └── llm.yaml          # Model parameters (base_url, model name, system prompt)
├── core/
│   └── agent.py          # Core ChatAgent class with OpenAI function-calling loop
├── tools/
│   ├── definitions.py    # OpenAI function schemas & execution mapping
│   ├── execute_python_code.py
│   ├── get_current_date.py
│   ├── parse_webpage.py
│   └── search_web.py
├── server.py             # FastAPI backend server with SSE streaming endpoints
├── chat.py               # CLI chat interface
├── local-ui-chat/        # React + TypeScript + Vite + Tailwind CSS frontend
├── pyproject.toml        # Python dependencies (managed via uv)
└── README.md

---

🚦 Getting Started

Prerequisites

• Python 3.13+
• uv
• Node.js and npm
• An OpenAI-compatible local LLM server running (e.g. Ollama or LM Studio on http://localhost:11434/v1)

---

1. Backend Setup

1. Install Python Dependencies: Using uv (recommended):

   uv sync

2. Configure Your Model: Edit configs/llm.yaml:

   model: "your-model"                  # Your local model name
   base_url: "http://localhost:11434/v1" # OpenAI-compatible endpoint URL
   api_key: "ollama"                  # API key (dummy string for local Ollama)
   system_prompt: |
     You are a helpful local AI assistant equipped with tools...

   If the LLM server runs on another machine in your LAN, set base_url to that machine's address instead, for example http://192.168.x.x:11434/v1.

3. Start the API Server:

   uv run server.py

   The server will start on http://localhost:8000.

---

2. Frontend Setup

1. Navigate to Frontend Directory:

   cd local-ui-chat

2. Install Dependencies:

   npm install

3. Start Development Server:

   npm run dev

   Open http://localhost:5173 in your browser.

---

3. Alternative: CLI Chat

If you prefer using the terminal directly without launching the frontend:

uv run chat.py

---

📡 API Endpoints

Method	Endpoint	Description
POST	/chat	Send user message ({"message": "string"}) and receive SSE stream (text/event-stream).
DELETE	/chat	Clears current agent conversation memory.