Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/telegram-scraper/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ groups, chats or bots.
## 0. Setup (once)

```bash
tgscraper --version || pip install "tgscraper[all] @ git+https://github.com/specialteam/TelegramScraper"
tgscraper --version || pip install "telegram-channel-scraper[all]"
```

If the `telegram-scraper` MCP tools (`get_messages`, `search_messages`, `get_channel_info`,
Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Publish to PyPI

# Push a tag like v2.0.0 (or run manually) to build and upload to https://pypi.org/p/telegram-channel-scraper.
# Uses PyPI Trusted Publishing — no API token stored in GitHub.
on:
push:
tags: ["v*"]
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install build twine
- run: python -m build
- run: twine check dist/*
- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/

publish:
needs: build
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/telegram-channel-scraper
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
name: dist
path: dist/
- uses: pypa/gh-action-pypi-publish@release/v1
27 changes: 14 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
Python library · CLI · MCP server for AI agents · Claude Skill · Web dashboard · Docker

[![Tests](https://github.com/specialteam/TelegramScraper/actions/workflows/python-package.yml/badge.svg)](https://github.com/specialteam/TelegramScraper/actions)
[![PyPI](https://img.shields.io/pypi/v/telegram-channel-scraper)](https://pypi.org/project/telegram-channel-scraper/)
![Python](https://img.shields.io/badge/python-3.9%2B-blue)
![MCP](https://img.shields.io/badge/MCP-server-8A2BE2)
![License](https://img.shields.io/badge/license-MIT-green)
Expand All @@ -16,7 +17,7 @@ Python library · CLI · MCP server for AI agents · Claude Skill · Web dashboa
</div>

```bash
pip install "tgscraper[all] @ git+https://github.com/specialteam/TelegramScraper"
pip install "telegram-channel-scraper[all]"
tgscraper durov
```

Expand Down Expand Up @@ -49,10 +50,10 @@ That's it — the latest posts of `t.me/durov`, in your terminal.

```bash
# everything (CLI + Excel + MCP server + dashboard)
pip install "tgscraper[all] @ git+https://github.com/specialteam/TelegramScraper"
pip install "telegram-channel-scraper[all]"

# or minimal (CLI + library only: httpx + beautifulsoup4)
pip install "git+https://github.com/specialteam/TelegramScraper"
pip install telegram-channel-scraper

# or from a clone
git clone https://github.com/specialteam/TelegramScraper && cd TelegramScraper && pip install -e ".[all]"
Expand Down Expand Up @@ -239,16 +240,16 @@ Plus prompts `summarize_channel` and `track_topic`, and the resource `telegram:/
### Connect it

The only requirement is [uv](https://docs.astral.sh/uv/) (`pip install uv`) — `uvx` downloads and runs the server
on demand. Or `pip install "tgscraper[mcp] @ git+…"` and use `"command": "tgscraper-mcp"` with no args.
on demand. Or `pip install "telegram-channel-scraper[mcp]"` and use `"command": "tgscraper-mcp"` with no args.

<details open>
<summary><b>Claude Code</b></summary>

```bash
claude mcp add telegram-scraper -- uvx --from "tgscraper[mcp] @ git+https://github.com/specialteam/TelegramScraper" tgscraper-mcp
claude mcp add telegram-scraper -- uvx --from "telegram-channel-scraper[mcp]" tgscraper-mcp
```
Inside this repository it is automatic: [`.mcp.json`](.mcp.json) registers the server and
[`.claude/skills/telegram-scraper`](.claude/skills/telegram-scraper/SKILL.md) loads the skill.
Inside this repository it is automatic: [`.mcp.json`](https://github.com/specialteam/TelegramScraper/blob/main/.mcp.json) registers the server and
[`.claude/skills/telegram-scraper`](https://github.com/specialteam/TelegramScraper/blob/main/.claude/skills/telegram-scraper/SKILL.md) loads the skill.
</details>

<details>
Expand All @@ -262,7 +263,7 @@ or `~/.codeium/windsurf/mcp_config.json`:
"mcpServers": {
"telegram-scraper": {
"command": "uvx",
"args": ["--from", "tgscraper[mcp] @ git+https://github.com/specialteam/TelegramScraper", "tgscraper-mcp"],
"args": ["--from", "telegram-channel-scraper[mcp]", "tgscraper-mcp"],
"env": { "TGSCRAPER_PROXY": "" }
}
}
Expand All @@ -280,7 +281,7 @@ or `~/.codeium/windsurf/mcp_config.json`:
"telegram-scraper": {
"type": "stdio",
"command": "uvx",
"args": ["--from", "tgscraper[mcp] @ git+https://github.com/specialteam/TelegramScraper", "tgscraper-mcp"]
"args": ["--from", "telegram-channel-scraper[mcp]", "tgscraper-mcp"]
}
}
}
Expand All @@ -299,7 +300,7 @@ Environment variables: `TGSCRAPER_PROXY` (proxy URL for all requests), `TGSCRAPE

### Claude Skill

[`.claude/skills/telegram-scraper/SKILL.md`](.claude/skills/telegram-scraper/SKILL.md) teaches an agent when and how
[`.claude/skills/telegram-scraper/SKILL.md`](https://github.com/specialteam/TelegramScraper/blob/main/.claude/skills/telegram-scraper/SKILL.md) teaches an agent when and how
to use the CLI (commands, JSON schema, how to cite results). Install it for all your projects:

```bash
Expand All @@ -314,14 +315,14 @@ For claude.ai, zip the `telegram-scraper` folder and upload it under **Settings
- *"Search @xyz for 'airdrop' and give me the dates and links."*
- *"Export the last 1000 posts of t.me/abc to Excel."*

Other agents: [`AGENTS.md`](AGENTS.md) and [`llms.txt`](llms.txt) describe the project for LLMs.
Other agents: [`AGENTS.md`](https://github.com/specialteam/TelegramScraper/blob/main/AGENTS.md) and [`llms.txt`](https://github.com/specialteam/TelegramScraper/blob/main/llms.txt) describe the project for LLMs.

---

## 🖥 Web dashboard

```bash
pip install "tgscraper[dashboard] @ git+https://github.com/specialteam/TelegramScraper"
pip install "telegram-channel-scraper[dashboard]"
tgscraper dashboard # → http://localhost:8501
```

Expand Down Expand Up @@ -400,7 +401,7 @@ rates reasonable. This project is not affiliated with Telegram.
</div>

```bash
pip install "tgscraper[all] @ git+https://github.com/specialteam/TelegramScraper"
pip install "telegram-channel-scraper[all]"
```

<div dir="rtl">
Expand Down
4 changes: 2 additions & 2 deletions llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
> reactions, media, hashtags, links), supports search, date/keyword filters, JSON/CSV/Excel/SQLite export,
> incremental scraping, monitoring with webhooks, media download and analytics.

Install: `pip install "tgscraper[all] @ git+https://github.com/specialteam/TelegramScraper"`
MCP server: `uvx --from "tgscraper[mcp] @ git+https://github.com/specialteam/TelegramScraper" tgscraper-mcp`
Install: `pip install "telegram-channel-scraper[all]"`
MCP server: `uvx --from "telegram-channel-scraper[mcp]" tgscraper-mcp`

## Docs
- [README](README.md): features, CLI, Python API, MCP setup for Claude/Cursor/VS Code, Docker, FAQ
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = ["setuptools>=64"]
build-backend = "setuptools.build_meta"

[project]
name = "tgscraper"
name = "telegram-channel-scraper"
version = "2.0.0"
description = "Scrape public Telegram channels without API keys or login — Python library, CLI, MCP server for AI agents, and dashboard."
readme = "README.md"
Expand All @@ -29,8 +29,8 @@ dependencies = ["httpx[socks]>=0.27", "beautifulsoup4>=4.11"]
excel = ["openpyxl>=3.1"]
mcp = ["mcp>=1.2; python_version>='3.10'"]
dashboard = ["streamlit>=1.35"]
all = ["tgscraper[excel,mcp,dashboard]"]
dev = ["tgscraper[excel,mcp]", "pytest>=7", "pytest-asyncio>=0.23", "respx>=0.21", "flake8"]
all = ["telegram-channel-scraper[excel,mcp,dashboard]"]
dev = ["telegram-channel-scraper[excel,mcp]", "pytest>=7", "pytest-asyncio>=0.23", "respx>=0.21", "flake8"]

[project.scripts]
tgscraper = "tgscraper.cli:main"
Expand Down
4 changes: 2 additions & 2 deletions tgscraper/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ def build_parser() -> argparse.ArgumentParser:
p = sub.add_parser("mcp", help="run the MCP server (stdio) so AI assistants can use tgscraper")
p.add_argument("--transport", default="stdio", choices=["stdio", "sse", "streamable-http"])

sub.add_parser("dashboard", help="open the web dashboard (needs tgscraper[dashboard])")
sub.add_parser("dashboard", help="open the web dashboard (needs telegram-channel-scraper[dashboard])")
return parser


Expand Down Expand Up @@ -255,7 +255,7 @@ def _dashboard() -> None:
try:
import streamlit # noqa: F401
except ImportError:
print("Dashboard needs streamlit: pip install 'tgscraper[dashboard]'", file=sys.stderr)
print("Dashboard needs streamlit: pip install 'telegram-channel-scraper[dashboard]'", file=sys.stderr)
raise SystemExit(1)
app = Path(__file__).with_name("dashboard.py")
raise SystemExit(subprocess.call([sys.executable, "-m", "streamlit", "run", str(app)]))
Expand Down
2 changes: 1 addition & 1 deletion tgscraper/dashboard.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Web dashboard. Run with ``tgscraper dashboard`` (needs ``pip install 'tgscraper[dashboard]'``)."""
"""Web dashboard. Run with ``tgscraper dashboard`` (needs ``pip install 'telegram-channel-scraper[dashboard]'``)."""
from __future__ import annotations

import datetime as dt
Expand Down
2 changes: 1 addition & 1 deletion tgscraper/exporters.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ def export(messages: Iterable[Message], path: Union[str, Path], format: Optional
try:
from openpyxl import Workbook
except ImportError as exc: # pragma: no cover
raise ImportError("Excel export needs openpyxl: pip install 'tgscraper[excel]'") from exc
raise ImportError("Excel export needs openpyxl: pip install 'telegram-channel-scraper[excel]'") from exc
wb = Workbook()
ws = wb.active
ws.title = "messages"
Expand Down
2 changes: 1 addition & 1 deletion tgscraper/mcp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
try: # mcp 1.x
from mcp.server.fastmcp import FastMCP
except ImportError as exc: # pragma: no cover
raise ImportError("The MCP server needs the 'mcp' package: pip install 'tgscraper[mcp]'") from exc
raise ImportError("The MCP server needs the 'mcp' package: pip install 'telegram-channel-scraper[mcp]'") from exc

from .analytics import summarize
from .client import AsyncScraper, ScraperError
Expand Down
Loading