HH Skill Parser

A Python tool for extracting and analyzing technical skills from HH.ru job listings. Scrapes vacancies by search queries, pulls out tech skills, saves results to CSV, and optionally generates a PNG chart.

Features

• Multiple data sources: HH API and HTML fallback (survives ddos-guard blocks)
• Skill filtering: whitelist-based skill extraction with alias groups (kubernetes | k8s)
• Resumable progress: interrupted runs continue from where they stopped
• Chart generation: optional matplotlib PNG bar chart (can be disabled with --no-chart)
• Binary bundling: single-file executable via PyInstaller
• Proxy support: HTTPS_PROXY / --proxy for blocked IPs

Installation

git clone --depth=1 https://github.com/valeksan/hh-skill-parser.git
cd hh-skill-parser
python -m venv .venv && source .venv/bin/activate

# Basic install
pip install -e .

# With chart support
pip install -e ".[chart]"

# Full install (chart + CLI animation)
pip install -e ".[full]"

Or use make:

make install        # basic
make install-full   # everything
make run            # quick start

Usage

Configuration

• queries.txt — search queries, one per line (lines starting with # are ignored)
• skills_whitelist.txt — skills to extract (for --mode description)

Modes (--mode)

Mode	Source	Whitelist	Notes
key-skills	key_skills field only	Not used	Cleaner results, fewer matches
description	Vacancy text	Used	More results, possible false positives
both	Combined, deduplicated	Used	Maximum signal per vacancy

Common commands

# Standard run (API, key-skills)
python parse_skills.py run

# HTML fallback when API is blocked
python parse_skills.py run --source html --mode description

# Combined mode with auto-fallback
python parse_skills.py run --mode both --source auto --html-description-fallback

# Chart only (from existing CSV)
python parse_skills.py chart

# Install as package
pip install -e .
hh-skill-parser run

Makefile targets

make help          # show all commands
make run           # standard run
make run-html      # HTML source
make run-lite      # no chart
make smoke         # local tests
make bundle        # build binary

Binary bundling

Build a single executable with all dependencies:

make install-bundle
make bundle
# Output: dist/hh-skill-parser

Note: queries.txt, skills_whitelist.txt, .env, and output files remain external.

Environment variables

Variable	Description	Default
LOGLEVEL	debug, info, warning, error, critical	info
HH_NO_CHART	Set to 1 to skip chart generation	—
HTTPS_PROXY	Proxy for requests (e.g. http://127.0.0.1:8080)	—

How it works

1. Each line in queries.txt is a standalone search phrase (quoted or unquoted)
2. With --mode key-skills: uses HH's structured key_skills field (most reliable)
3. With --mode description: searches vacancy text against skills_whitelist.txt (more results, possible false positives like c language vs Cyrillic с)
4. Progress is saved to progress.json — resume after interruption
5. Results: top_skills_all_data.csv + optional hh_skills_bar_chart.png

HTTP 403 troubleshooting

If HH.ru blocks your requests:

1. Increase delays: --page-delay-min 5 --page-delay-max 12
2. Use proxy: HTTPS_PROXY='http://proxy:port' ./parse_skills.py run
3. Switch to HTML: --source html --mode description
4. Combine modes: --source auto --html-description-fallback

The tool includes exponential backoff retry, browser-like headers, and automatic API-to-HTML fallback.

License

MIT — see LICENSE

Contributors

• tonakihan — code fixes and improvements

Author

Vitaly (@valeksan)