Mc/updates

.env.example

@@ -1,7 +1,265 @@
+# =============================================================================
+# HuMCP - Environment Variables
+# =============================================================================
+# Copy this file to .env: cp .env.example .env
+#
+# HuMCP is standalone — set only the vars for tools you want to use.
+# Tools without their required keys will load but return errors when called.
+# =============================================================================
+
+# -----------------------------------------------------------------------------
+# Server
+# -----------------------------------------------------------------------------
 MCP_SERVER_URL=http://localhost:8080
 
-GOOGLE_OAUTH_CLIENT_ID=your-client-id.apps.googleusercontent.com
-GOOGLE_OAUTH_CLIENT_SECRET=GOCSPX-your-client-secret
-FASTMCP_SERVER_AUTH=fastmcp.server.auth.providers.google.GoogleProvider
+# -----------------------------------------------------------------------------
+# Authentication (optional)
+# -----------------------------------------------------------------------------
+# Google OAuth — enables /login flow and Swagger UI auth
+# GOOGLE_OAUTH_CLIENT_ID=your-client-id.apps.googleusercontent.com
+# GOOGLE_OAUTH_CLIENT_SECRET=GOCSPX-your-client-secret
+# AUTH_ENABLED=false
+
+# JWT verification — for service-to-service (e.g. graphite integration)
+# JWT_SECRET_KEY=your-jwt-secret-key
+
+# API key — restricts all REST endpoints to requests with this key
+# SERVICE_API_KEY=dev-api-key-change-in-production
+
+# -----------------------------------------------------------------------------
+# Storage — S3-compatible (MinIO, AWS S3, GCS)
+# -----------------------------------------------------------------------------
+STORAGE_ENDPOINT=localhost:9000       # AWS S3: s3.amazonaws.com
+STORAGE_ACCESS_KEY=minioadmin         # AWS: AWS_ACCESS_KEY_ID
+STORAGE_SECRET_KEY=minioadmin         # AWS: AWS_SECRET_ACCESS_KEY
+STORAGE_SECURE=false                  # true for AWS S3 / GCS
+# STORAGE_REGION=us-east-1
+# STORAGE_ALLOWED_BUCKETS=my-bucket
+
+# -----------------------------------------------------------------------------
+# Search
+# -----------------------------------------------------------------------------
+# TAVILY_API_KEY=your-tavily-api-key
+# EXA_API_KEY=your-exa-api-key
+# BRAVE_API_KEY=your-brave-api-key
+# SERPAPI_API_KEY=your-serpapi-api-key
+# SERPER_API_KEY=your-serper-api-key
+# JINA_API_KEY=your-jina-api-key
+# LINKUP_API_KEY=your-linkup-api-key
+# SEARXNG_BASE_URL=http://localhost:8888
+# SELTZ_API_KEY=your-seltz-api-key
+# VALYU_API_KEY=your-valyu-api-key
+
+# -----------------------------------------------------------------------------
+# Web Scraping
+# -----------------------------------------------------------------------------
+# FIRECRAWL_API_KEY=your-firecrawl-api-key
+# SPIDER_API_KEY=your-spider-api-key
+# AGENTQL_API_KEY=your-agentql-api-key
+# BROWSERBASE_API_KEY=your-browserbase-api-key
+# BROWSERBASE_PROJECT_ID=your-project-id
+# APIFY_API_TOKEN=your-apify-token
+# SGAI_API_KEY=your-scrapegraph-api-key
+# BRIGHTDATA_API_KEY=your-brightdata-api-key
+# BRIGHTDATA_WEB_UNLOCKER_ZONE=your-zone
+# OXYLABS_USERNAME=your-username
+# OXYLABS_PASSWORD=your-password
+
+# -----------------------------------------------------------------------------
+# LLMs
+# -----------------------------------------------------------------------------
+# OPENAI_API_KEY=your-openai-api-key
+# ANTHROPIC_API_KEY=your-anthropic-api-key
+# GOOGLE_API_KEY=your-google-ai-api-key
+# OLLAMA_HOST=http://localhost:11434
+
+# -----------------------------------------------------------------------------
+# Google Workspace
+# -----------------------------------------------------------------------------
+# GOOGLE_OAUTH_CLIENT_ID=your-client-id.apps.googleusercontent.com
+# GOOGLE_OAUTH_CLIENT_SECRET=GOCSPX-your-client-secret
+# GOOGLE_CLOUD_PROJECT=your-gcp-project-id
+# GOOGLE_MAPS_API_KEY=your-maps-api-key
+
+# -----------------------------------------------------------------------------
+# Messaging
+# -----------------------------------------------------------------------------
+# Slack
+# SLACK_TOKEN=xoxb-your-slack-bot-token
+
+# Discord
+# DISCORD_BOT_TOKEN=your-discord-bot-token
+
+# Telegram
+# TELEGRAM_BOT_TOKEN=your-telegram-bot-token
+
+# WhatsApp (Meta Cloud API)
+# WHATSAPP_PHONE_NUMBER_ID=your-phone-number-id
+# WHATSAPP_TOKEN=your-whatsapp-token
+
+# Email (SMTP)
+# SMTP_HOST=smtp.gmail.com
+# SMTP_PORT=587
+# SMTP_USERNAME=your-email@gmail.com
+# SMTP_PASSWORD=your-app-password
+
+# Resend
+# RESEND_API_KEY=your-resend-api-key
+
+# Twilio (SMS)
+# TWILIO_ACCOUNT_SID=your-account-sid
+# TWILIO_AUTH_TOKEN=your-auth-token
+# TWILIO_FROM_NUMBER=+1234567890
+
+# Webex
+# WEBEX_ACCESS_TOKEN=your-webex-access-token
+
+# -----------------------------------------------------------------------------
+# Social
+# -----------------------------------------------------------------------------
+# X / Twitter
+# X_API_KEY=your-api-key
+# X_API_SECRET=your-api-secret
+# X_ACCESS_TOKEN=your-access-token
+# X_ACCESS_TOKEN_SECRET=your-access-token-secret
+# X_BEARER_TOKEN=your-bearer-token
+
+# Reddit
+# REDDIT_CLIENT_ID=your-client-id
+# REDDIT_CLIENT_SECRET=your-client-secret
+# REDDIT_USER_AGENT=your-app-name/1.0
+
+# YouTube
+# YOUTUBE_API_KEY=your-youtube-api-key
+
+# -----------------------------------------------------------------------------
+# Project Management
+# -----------------------------------------------------------------------------
+# GitHub
+# GITHUB_TOKEN=ghp_your-github-token
+
+# Jira / Confluence
+# JIRA_URL=https://your-org.atlassian.net
+# JIRA_USERNAME=your-email@example.com
+# JIRA_API_TOKEN=your-jira-api-token
+# CONFLUENCE_URL=https://your-org.atlassian.net
+# CONFLUENCE_USERNAME=your-email@example.com
+# CONFLUENCE_API_TOKEN=your-confluence-api-token
+
+# Linear
+# LINEAR_API_KEY=your-linear-api-key
+
+# Notion
+# NOTION_API_KEY=secret_your-notion-key
+
+# Todoist
+# TODOIST_API_KEY=your-todoist-api-key
+
+# Trello
+# TRELLO_API_KEY=your-trello-api-key
+# TRELLO_TOKEN=your-trello-token
+
+# ClickUp
+# CLICKUP_API_KEY=your-clickup-api-key
+
+# Zendesk
+# ZENDESK_SUBDOMAIN=your-subdomain
+# ZENDESK_EMAIL=your-email@example.com
+# ZENDESK_API_TOKEN=your-zendesk-api-token
+
+# Bitbucket
+# BITBUCKET_USERNAME=your-username
+# BITBUCKET_APP_PASSWORD=your-app-password
+
+# -----------------------------------------------------------------------------
+# Database
+# -----------------------------------------------------------------------------
+# PostgreSQL
+# DATABASE_URL=postgresql://user:password@localhost:5432/mydb
+
+# Redshift
+# REDSHIFT_HOST=your-cluster.redshift.amazonaws.com
+# REDSHIFT_DATABASE=your-database
+# REDSHIFT_USER=your-user
+# REDSHIFT_PASSWORD=your-password
+# REDSHIFT_PORT=5439
+
+# Neo4j
+# NEO4J_URI=bolt://localhost:7687
+# NEO4J_USERNAME=neo4j
+# NEO4J_PASSWORD=your-password
+
+# -----------------------------------------------------------------------------
+# Cloud
+# -----------------------------------------------------------------------------
+# AWS
+# AWS_ACCESS_KEY_ID=your-access-key-id
+# AWS_SECRET_ACCESS_KEY=your-secret-access-key
+# AWS_DEFAULT_REGION=us-east-1
+# AWS_SES_FROM_EMAIL=noreply@yourdomain.com
+
+# E2B (code sandboxes)
+# E2B_API_KEY=your-e2b-api-key
+
+# Daytona
+# DAYTONA_API_KEY=your-daytona-api-key
+# DAYTONA_SERVER_URL=https://your-daytona-server
+
+# Airflow
+# AIRFLOW_BASE_URL=http://localhost:8080
+# AIRFLOW_USERNAME=airflow
+# AIRFLOW_PASSWORD=airflow
+
+# -----------------------------------------------------------------------------
+# Finance
+# -----------------------------------------------------------------------------
+# FINANCIAL_DATASETS_API_KEY=your-api-key
+# OPENBB_TOKEN=your-openbb-token
+# EVM_RPC_URL=https://mainnet.infura.io/v3/your-project-id
+
+# -----------------------------------------------------------------------------
+# Media & Image
+# -----------------------------------------------------------------------------
+# REPLICATE_API_TOKEN=your-replicate-token
+# FAL_KEY=your-fal-api-key
+# LUMAAI_API_KEY=your-lumaai-api-key
+# UNSPLASH_ACCESS_KEY=your-unsplash-access-key
+# GIPHY_API_KEY=your-giphy-api-key
+
+# -----------------------------------------------------------------------------
+# Audio
+# -----------------------------------------------------------------------------
+# ELEVEN_LABS_API_KEY=your-elevenlabs-api-key
+# CARTESIA_API_KEY=your-cartesia-api-key
+# SPOTIFY_CLIENT_ID=your-spotify-client-id
+# SPOTIFY_CLIENT_SECRET=your-spotify-client-secret
+# DESI_VOCAL_API_KEY=your-desi-vocal-api-key
+
+# -----------------------------------------------------------------------------
+# Memory
+# -----------------------------------------------------------------------------
+# MEM0_API_KEY=your-mem0-api-key
+# ZEP_API_KEY=your-zep-api-key
+
+# -----------------------------------------------------------------------------
+# Calendar
+# -----------------------------------------------------------------------------
+# CALCOM_API_KEY=your-calcom-api-key
+# CALCOM_BASE_URL=https://api.cal.com/v1
+# ZOOM_ACCOUNT_ID=your-account-id
+# ZOOM_CLIENT_ID=your-client-id
+# ZOOM_CLIENT_SECRET=your-client-secret
+
+# -----------------------------------------------------------------------------
+# E-commerce
+# -----------------------------------------------------------------------------
+# SHOPIFY_STORE_URL=https://your-store.myshopify.com
+# SHOPIFY_ACCESS_TOKEN=your-access-token
+# BRANDFETCH_API_KEY=your-brandfetch-api-key
 
-AUTH_ENABLED=false
+# -----------------------------------------------------------------------------
+# Local Tools
+# -----------------------------------------------------------------------------
+# Allow shell/file tools to use absolute paths (default: false for safety)
+# HUMCP_ALLOW_ABSOLUTE_PATHS=false
+# STORAGE_ALLOW_ABSOLUTE_PATHS=false

CLAUDE.md

@@ -39,21 +39,45 @@ HuMCP exposes tools via both MCP (Model Context Protocol) at `/mcp` and REST at
 
 ### Core Library (`src/humcp/`)
 
-- **`server.py`** - `create_app()` creates FastAPI app, loads tool modules, registers with FastMCP
-- **`decorator.py`** - `@tool(category="...")` marks functions for discovery. Category auto-detects from parent folder if not specified. Tool name = function name (used by FastMCP)
-- **`registry.py`** - `RegisteredTool` NamedTuple wraps FastMCP's `FunctionTool` with category
+- **`server.py`** - `create_app()` creates FastAPI app, loads tool modules, discovers MCP Apps, registers with FastMCP, mounts MCP/REST/OAuth routes
+- **`decorator.py`** - `@tool(category="...", app="...")` marks functions for discovery. Category auto-detects from parent folder, `app` defaults to filename stem. Tool name = function name
+- **`registry.py`** - `RegisteredTool` NamedTuple wraps FastMCP's `FunctionTool` with `category` and `app` fields
 - **`routes.py`** - Generates REST endpoints from registered tools using `FunctionTool.parameters`
 - **`config.py`** - Tool filtering via `config/tools.yaml` (include/exclude with wildcard support)
 - **`skills.py`** - Discovers `SKILL.md` files for category metadata
 - **`schemas.py`** - Pydantic response models for API endpoints
+- **`credentials.py`** - `resolve_credential("KEY_NAME")` resolves credentials from env vars
+- **`storage_path.py`** - `minio://` URL resolution utilities. `is_storage_path()`, `parse_storage_path()`, `resolve_path()` (async context manager that downloads storage objects to temp files)
+- **`auth.py`** - Google OAuth provider via `create_auth_provider()`, `is_auth_enabled()`, `get_current_user_id()` (resolves user from MCP access token or REST ContextVar)
+- **`middleware.py`** - `APIKeyMiddleware` validates `X-API-Key` header for REST requests. Extracts user identity from Bearer JWT into ContextVar. Public paths (`/`, `/docs`, `/playground`, etc.) are exempt
+- **`permissions.py`** - `require_auth()` returns user UUID or raises 401 when `TOOLSET_REQUIRE_AUTH=true`. `check_permission()` is a stub that delegates to `require_auth()` (upgrade to full IAM when available). `STRICT_PERMISSIONS=true` rejects all permission checks
+- **`playground.py`** - `get_playground_html()` returns a self-contained HTML page for interactive tool browsing and execution at `/playground`
+
+### Authentication Modes
+
+The server supports four auth modes, selected by environment variables:
+
+1. **JWT mode** - Set `JWT_SECRET_KEY`: uses `JWTVerifier` for MCP auth (service-to-service). Middleware decodes Bearer JWTs on REST requests
+2. **Google OAuth** - Set `GOOGLE_OAUTH_CLIENT_ID` + `GOOGLE_OAUTH_CLIENT_SECRET` + `AUTH_ENABLED=true`: `GoogleProvider` handles browser login flow with Google scopes (Gmail, Calendar, Drive, Tasks, Docs, Sheets, etc.)
+3. **API key** - Set `SERVICE_API_KEY`: `APIKeyMiddleware` requires `X-API-Key` header on all non-public REST requests. Uses constant-time comparison
+4. **No auth** - Default dev mode when none of the above are configured
 
 ### Tool Discovery Flow
 
 1. `create_app()` loads Python modules from `src/tools/` recursively
 2. Functions with `@tool()` decorator are discovered via `_humcp_tool` attribute
 3. Each tool is registered with FastMCP via `mcp.tool()(func)` which creates a `FunctionTool`
-4. `RegisteredTool(tool=fn_tool, category=...)` pairs the FunctionTool with its category
+4. `RegisteredTool(tool=fn_tool, category=..., app=...)` pairs the FunctionTool with its category and app
 5. REST routes are generated from `FunctionTool.parameters` (JSON Schema)
+6. Tools are filtered by `config/tools.yaml` before route registration
+
+### Tool Categories (24)
+
+api, audio, builder, calendar, cloud, data, database, ecommerce, files, finance, google, image, llm, local, media, memory, messaging, project_management, research, search, social, storage, weather, web_scraping
+
+### App Grouping
+
+The `@tool()` decorator accepts an `app` parameter (defaults to the filename stem). `RegisteredTool` carries this `app` field. REST responses include `apps` grouping so clients can group related tools by their source module.
 
 ### Adding New Tools
 
@@ -62,7 +86,7 @@ Create a `.py` file in `src/tools/<category>/`:
 ```python
 from src.humcp.decorator import tool
 
-@tool()  # category auto-detected from folder name
+@tool()  # category auto-detected from folder name, app from filename
 async def my_tool(param: str) -> dict:
     """Tool description (used by FastMCP and Swagger)."""
     return {"success": True, "data": {"result": param}}
@@ -80,6 +104,26 @@ return {"success": True, "data": {...}}
 return {"success": False, "error": "Error message"}
 ```
 
+### MCP Apps System
+
+Interactive UI panels rendered alongside tool results:
+
+- HTML bundles live in `src/apps/{category}/{tool_name}.html`
+- Registered as `ui://` MCP resources on the FastMCP server
+- Served via `/apps/{tool_name}` REST endpoint and listed at `/apps`
+- Claude.ai renders them as interactive UI panels via iframe + JSON-RPC messaging (`ui/ready`, `ui/initialize`, `ui/toolResult`)
+- The Playground (`/playground`) also renders app HTML for tools that have them
+
+### Builder Subsystem
+
+Custom tool creation at runtime (`src/tools/builder/`):
+
+- `tools.py` - `tool_builder_*` functions for creating, listing, updating, deleting custom tools
+- `sandbox.py` - Sandboxed execution via RestrictedPython
+- `storage.py` - Persistence for custom tool definitions
+- `schemas.py` - Pydantic models for builder requests
+- All builder endpoints require authentication
+
 ### Skills
 
 Add `SKILL.md` in tool category folders with YAML frontmatter:
@@ -105,3 +149,22 @@ exclude:
 ```
 
 Empty config = load all tools.
+
+### Key Environment Variables
+
+| Variable | Purpose |
+|---|---|
+| `MCP_SERVER_URL` | Advertised MCP endpoint URL (default `http://0.0.0.0:8080/mcp`) |
+| `AUTH_ENABLED` | Enable/disable Google OAuth (default `true`) |
+| `JWT_SECRET_KEY` | Enables JWT auth mode for MCP and REST Bearer tokens |
+| `SERVICE_API_KEY` | Enables API key middleware for REST endpoints |
+| `GOOGLE_OAUTH_CLIENT_ID` | Google OAuth client ID |
+| `GOOGLE_OAUTH_CLIENT_SECRET` | Google OAuth client secret |
+| `TOOLSET_REQUIRE_AUTH` | When `true`, `require_auth()` raises 401 for unauthenticated requests |
+| `STRICT_PERMISSIONS` | When `true`, `check_permission()` rejects all calls (no IAM backend) |
+| `STORAGE_ENDPOINT` | MinIO/S3 storage endpoint |
+| `STORAGE_ACCESS_KEY` | MinIO/S3 access key |
+| `STORAGE_SECRET_KEY` | MinIO/S3 secret key |
+| `STORAGE_SECURE` | Use HTTPS for storage connections |
+| `DB_READ_ONLY` | Database read-only mode (default `true`) |
+| `HTTP_ALLOW_PRIVATE` | Allow HTTP requests to private IPs (default `false`) |