Text2Pen Backend

The Text2Pen Backend — powering AI features for the Text2Pen desktop application.

---

🤖 What is the Text2Pen Backend?

The Text2Pen Backend is a FastAPI-powered service that provides intelligent AI features to the Text2Pen desktop application, including:

• AI Chat — conversational assistance with image analysis support
• Text Processing — grammar correction, rephrasing, tone adjustment, and text generation
• Image Analysis — understanding images via vision models
• Rate Limiting & Telemetry — fair usage enforcement and anonymous usage tracking

The backend is hosted on Render and uses Groq Cloud for AI inference, providing fast, reliable responses with high rate limits on the free tier.

---

✨ Key Features

• AI Chat with Vision — chat with the assistant and upload images for analysis
• Text Actions — correct grammar, shorten text, adjust tone (formal/casual), generate content
• Smart Rate Limiting — per-IP daily limits (10 chat requests, 1000 text actions) reset at midnight UTC
• Queue Management — separate worker threads for text and vision requests to handle load gracefully
• Groq Integration — uses high-capacity Groq API with exponential backoff and automatic retries
• Anonymous Telemetry — optional, GitHub-backed event logging for error tracking (no personal data)
• Multi-format Image Support — PNG, JPEG, GIF, WebP, BMP with automatic compression and resizing

---

🏗️ Architecture

The backend uses a queue-based worker pattern to handle AI requests reliably:

Component	Purpose
FastAPI app	HTTP endpoints for chat, text actions, telemetry, and limits
Large Queue	Worker thread for text-only requests (using Llama 3.3-70B)
Vision Queue	Worker thread for image+text requests (using Llama 4 Scout Vision)
Global Rate Limiter	Throttles Groq API calls (~2s between requests to stay within free tier limits)
Per-IP Rate Limiter	Enforces daily usage limits per client IP with midnight UTC reset
GitHub Integration	Stores anonymous telemetry events in repo via GitHub API

---

🚀 Deployment

Prerequisites

• Python 3.9+
• API keys for:
  • Groq Cloud (GROQ_API_KEY) — for LLM inference
  • GitHub (GITHUB_TOKEN) — for telemetry storage
• A Render account (or compatible hosting)

Environment Variables

GROQ_API_KEY=<your-groq-api-key>
GITHUB_TOKEN=<your-github-token>

Installation

1. Clone the repository:

   git clone https://github.com/pythonIsFast/Text2Pen-Backend.git
   cd Text2Pen-Backend

2. Install dependencies:

   pip install fastapi uvicorn requests pillow python-multipart

3. Set environment variables in your .env file or hosting platform.

4. Run the server:

   uvicorn main:app --host 0.0.0.0 --port 8000

5. The server will start at http://localhost:8000.

Deploying to Render

1. Push your code to GitHub
2. Create a new Web Service on Render
3. Connect your GitHub repository
4. Set the start command:

   uvicorn main:app --host 0.0.0.0 --port $PORT

5. Add environment variables (GROQ_API_KEY, GITHUB_TOKEN)
6. Deploy!

---

📡 API Endpoints

/ping (GET)

Health check endpoint.

Response:

{
  "message": "All systems are up and running!"
}

---

/ai/chat (POST)

Chat with the AI assistant, optionally with image analysis.

Request:

{
  "history": [
    {"role": "assistant", "content": "Hello! How can I help?"},
    {"role": "user", "content": "Hi there"}
  ],
  "message": "Can you help me rephrase this?",
  "image_b64": null
}

Response:

{
  "result": "Of course! I'd be happy to help you rephrase your text...",
  "model": "llama-3.3-70b",
  "queue_position": 1,
  "chat_used": 2,
  "chat_limit": 10
}

Rate Limit: 10 requests per IP per day (resets at midnight UTC)

---

/ai (POST)

Text processing actions (grammar correction, rephrasing, tone adjustment, etc.).

Request:

{
  "action": "correct",
  "text": "i have a dog and a cat",
  "extra": {}
}

Available Actions:

Action	Description
correct	Fix spelling and grammar
shorten	Reduce text length significantly
tone_formal	Rewrite in formal tone
tone_casual	Rewrite in casual/friendly tone
generate	Generate new text on a topic
replace_unknown	Replace special characters with simpler alternatives (requires known_chars in extra)

Response:

{
  "result": "I have a dog and a cat."
}

Rate Limit: 1000 requests per IP per day (resets at midnight UTC)

---

/ai/limits (GET)

Check current rate limits and queue status for your IP.

Response:

{
  "chat_used": 2,
  "chat_limit": 10,
  "text_used": 45,
  "text_limit": 1000,
  "resets_at": "midnight UTC",
  "large_queue_depth": 3,
  "vision_queue_depth": 0
}

---

/telemetry (POST)

Submit anonymous telemetry events (error reports, crash logs, etc.).

Request:

{
  "event": "error",
  "os": "Windows 10",
  "timestamp": "2025-04-11T12:34:56Z",
  "extra": {"error_code": "E001"}
}

Response:

{
  "status": "ok"
}

Rate Limit: 100 telemetry events per IP per hour

---

⚙️ Configuration

Rate Limits

Edit these constants in main.py:

CHAT_LIMIT = 10      # chat requests per day per IP
TEXT_LIMIT = 1000    # text actions per day per IP
_MIN_INTERVAL = 2.0  # seconds between Groq API calls (respect free tier)

AI Models

The backend uses two Groq models:

• llama-3.3-70b-versatile — text-only requests (fast, high-throughput)
• meta-llama/llama-4-scout-17b-16e-instruct — vision + text requests (multimodal)

You can swap these by editing the model constants at the top of main.py.

System Prompt

Customize the AI assistant's behavior by editing SYSTEM_PROMPT in the code. The current prompt instructs the model to:

• Act as Text2Pen AI, integrated into the desktop app
• Provide helpful writing and image analysis assistance
• Respond in the user's language
• Keep responses concise and friendly

---

🔒 Privacy & Security

Telemetry

The backend stores anonymous telemetry in your GitHub repository to help track errors and improve stability.

• Telemetry is never collected without user consent from the frontend
• Only anonymous technical data is stored: error type, OS, timestamp, sanitized error message
• No personal data is ever collected: no typed text, no images, no usernames
• Telemetry events are deduplicated by event type + OS + extra fields (keeping the most recent)

Rate Limiting

Per-IP rate limits prevent abuse:

• Daily limits reset at midnight UTC
• Limits are enforced based on the client's real IP (respects X-Forwarded-For headers for proxied requests)
• Exceeding limits returns a 429 (Too Many Requests) response

API Key Security

• Groq and GitHub API keys are stored as environment variables only (never hardcoded)
• Keys are never logged or exposed in error messages
• Communicate with Groq and GitHub over HTTPS only

---

🛠️ Development

Testing Endpoints Locally

Use curl or Postman to test endpoints:

# Health check
curl http://localhost:8000/ping

# Check limits
curl http://localhost:8000/ai/limits

# Chat endpoint
curl -X POST http://localhost:8000/ai/chat \
  -H "Content-Type: application/json" \
  -d '{"history": [], "message": "Hello!"}'

# Text action
curl -X POST http://localhost:8000/ai \
  -H "Content-Type: application/json" \
  -d '{"action": "correct", "text": "i love python"}'

Common Issues

429 Rate Limit from Groq

• The backend retries with exponential backoff (up to 20 attempts)
• If retries fail, the error is returned to the client
• Check the /ai/limits endpoint to see queue depth

Image Upload Too Large

• Images are automatically resized to 512×512 JPEG (quality 60)
• This reduces token usage on Groq's vision model
• Very large images may still cause timeouts; keep them under 5 MB

Telemetry Not Storing

• Check that GITHUB_TOKEN has repo scope permissions
• Verify the token can write to the repository
• Check GitHub API rate limits (60 requests/hour for unauthenticated, 5000/hour for authenticated)

---

⚠️ Limitations

• Free Tier Limits — Groq free tier has ~30 requests/minute; backend adds 2-second global throttling
• 5-Minute Timeout — requests that don't complete within 5 minutes are canceled
• Image Compression — large/high-quality images are automatically compressed to reduce token usage
• Stateless — the backend does not persist chat history; clients must send full history each request
• No Authentication — the backend relies on IP-based rate limiting (suitable for single-user/small-scale use)

---

📊 Monitoring

Queue Depth

Check the /ai/limits endpoint to see how many requests are pending:

{
  "large_queue_depth": 3,
  "vision_queue_depth": 0
}

• large_queue_depth > 5 indicates the text queue is building up
• vision_queue_depth > 2 indicates slow vision requests (images)

Worker Threads

The backend starts two daemon threads on startup:

1. large-worker — processes text-only requests (Llama 3.3-70B)
2. vision-worker — processes image+text requests (Llama 4 Scout Vision)

Both log their activity to stdout. Check your logs if requests seem stuck.

Errors & Logs

All errors are caught and returned as HTTP 500 responses. Check:

1. Render logs (if hosted on Render)
2. Local console output (if running locally)
3. The telemetry endpoint logs any errors from the frontend app

---

🤝 Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (feature/your-feature)
3. Test your changes locally
4. Submit a pull request with a clear description

Areas for Contribution

• Additional AI models or Groq providers
• Improved rate limiting strategies
• Better error messages and logging
• Performance optimizations
• Additional text actions
• Documentation improvements

---

📜 License

Text2Pen Backend is licensed under the GNU General Public License v3.0 (GPL-3.0).

See the LICENSE file for details.

---

🔗 Related Projects

• Text2Pen Frontend — the Windows desktop application

---

❓ Support

If you encounter issues or have questions:

1. Check the Limitations section above
2. Review the Common Issues section
3. Open a GitHub Issue with:
   • Error message
   • Steps to reproduce
   • Your OS and environment details
4. Contact: Text2Pen.Info@gmail.com