Dilly

A free, grounded AI chat assistant for a personal site. Dilly runs a real LLM on Cloudflare Workers AI, calls it from a Cloudflare Pages Function (so no API key ever reaches the browser), streams the answer token by token, and stays accurate by being grounded in a hand-written facts block instead of a vector database.

This is the assistant that lives on hasithabandara.com. The code here is the real thing, trimmed to the parts that make Dilly work so you can read it or reuse it.

Why it exists

Most "add a chatbot to your site" tutorials either leak an API key into the frontend, cost money per message, or hallucinate about you the moment a visitor asks something specific. Dilly is the opposite:

• 100% free to run. Cloudflare Workers AI has a free daily allocation, and it runs on the same Cloudflare that already hosts the site. No extra account, no separate bill.
• No exposed keys. The model is called server-side through a Pages Function binding. The browser only talks to your own /api/chat route.
• Accurate, not creative. Dilly answers only from a curated KNOWLEDGE block and is told, in plain language, to refuse to invent details and to point people to a real contact instead.

How it works

Browser (Dilly.jsx)
   │  POST /api/chat  { messages: [...] }
   ▼
Cloudflare Pages Function  (functions/api/chat.js)
   │  env.AI.run(model, { messages, stream: true })
   ▼
Cloudflare Workers AI  →  streamed SSE tokens back to the browser

1. functions/api/chat.js — the backend. It builds a system prompt from a fixed persona, a set of rules, and a KNOWLEDGE block of real facts, prepends it to the last few turns of the conversation, and calls the model with stream: true. Workers AI returns an SSE stream, which the function passes straight back to the client.

2. web/Dilly.jsx / web/Dilly.css — the frontend. A floating launcher opens a chat panel, sends the conversation to /api/chat, and reads the SSE stream, appending each response token as it arrives so the reply types itself out.

3. wrangler.toml — declares the AI binding so both local dev and production have access to Workers AI.

Accuracy comes from grounding

There is no vector store here, and for a personal site there does not need to be. The entire corpus (who you are, what you build, how to contact you) fits comfortably in the system prompt. That single KNOWLEDGE string is the only thing Dilly is allowed to answer from:

const SYSTEM = `You are Dilly ...
Rules:
- Answer ONLY using the facts provided below. Never invent details, numbers,
  dates, prices, or features that are not stated. If you do not know something,
  say so plainly and point the visitor to the contact page.
...
Facts you know:
${KNOWLEDGE}`

When your work changes, you edit one string. Real, tested behavior:

• Ask it about a product it knows → accurate, with the right link.
• Ask it a price or a founding year that is not in the facts → it declines instead of guessing.
• Ask it something off-topic → it steers back to what it is here for.

Run it locally

Workers AI bindings only exist inside the Cloudflare runtime, so a plain Vite dev server will not have /api/chat. Use Wrangler, which proxies the binding to the real (free) Workers AI:

npm install -g wrangler
wrangler login
wrangler pages dev            # serves the functions with the AI binding

Then POST a message:

curl -N -X POST http://127.0.0.1:8788/api/chat \
  -H "content-type: application/json" \
  -d '{"messages":[{"role":"user","content":"What is this site about?"}]}'

Deploy

wrangler pages deploy

The functions/ directory is bundled automatically and the [ai] binding in wrangler.toml is attached on deploy.

Make it yours

1. Rewrite the KNOWLEDGE block in functions/api/chat.js with your own facts.
2. Adjust the persona and rules in SYSTEM.
3. Drop Dilly.jsx / Dilly.css into your app and render <Dilly /> once in your layout.
4. Give it a face: the widget loads its avatar from /dilly.gif. Drop in your own (a transparent GIF or WebM works well), or swap the <img> in Dilly.jsx for an inline SVG.

A note on models

Workers AI deprecates model IDs on a schedule. The model is pinned in one constant:

const MODEL = '@cf/meta/llama-4-scout-17b-16e-instruct'

If /api/chat starts returning a "model was deprecated" error, run wrangler ai models, pick a current text-generation model, swap that one line, and redeploy. An earlier llama-3.3-70b-instruct-fp8-fast was dropped here because fp8 quantization occasionally produced degenerate repetition; Llama 4 Scout was stable across repeated runs and is lighter on the free tier.

License

MIT. See LICENSE.