hashdog is a research fork of hashcat focused on pushing GPU-accelerated password recovery throughput beyond the current state of the art. This project conducts structured performance analysis of the hashcat execution pipeline and implements measurable optimizations across GPU kernel execution, host-device data transfer, candidate generation, and work scheduling.
For attacks using --slow-candidates (dictionary, combinator, or mask), hashdog achieves up to +70% throughput over upstream hashcat by overlapping CPU candidate generation with GPU kernel execution and vectorizing the rule engine case-conversion / class-replace functions.
| Hash Mode | hashcat v7.1.2 | hashdog | Improvement |
|---|---|---|---|
| 0 MD5 | 12.07 MH/s | 12.29 MH/s | +1.8% |
| 1400 SHA256 | 12.06 MH/s | 11.85 MH/s | -1.7% (variance) |
| 1700 SHA512 | 11.60 MH/s | 12.26 MH/s | +5.7% |
| 400 phpass | 878.2 kH/s | 1003.3 kH/s | +14.2% |
| 500 md5crypt | 533.9 kH/s | 557.1 kH/s | +4.3% |
| 7400 sha256crypt | 115.9 kH/s | 197.2 kH/s | +70.1% |
| 1800 sha512crypt | 78.59 kH/s | 108.2 kH/s | +37.7% |
| 3200 bcrypt | 16.15 kH/s | 19.38 kH/s | +20.0% |
Workload: 128K-word dictionary × 66 rules = 8.65M candidates per pass, RTX 3090, runtime=25s, median of 3 runs, autotune cache cleared between runs.
| Hash Mode | Mask | hashcat | hashdog | Improvement |
|---|---|---|---|---|
| 1800 sha512crypt | ?l?l?l?l?l?l?d?d |
133.7 kH/s | 192.2 kH/s | +43.7% |
| Hash Mode | hashcat | hashdog | Improvement |
|---|---|---|---|
| 1800 sha512crypt | 104.0 kH/s | 107.5 kH/s | +3.4% |
The improvement scales with the GPU/CPU ratio: slow hashes (sha256crypt, sha512crypt, bcrypt) where the GPU dominates execution time benefit most because the CPU candidate generation phase is fully hidden behind GPU computation. Fast hashes show smaller gains because the GPU finishes before the CPU can stage the next batch — but the SSE2 rule engine still contributes a few percent.
The pipeline extends to all three --slow-candidates dispatch paths: STRAIGHT (dictionary + rules), COMBI (combinator), and BF (mask). All three paths share the same persistent GPU worker thread infrastructure.
For brute-force attacks (-b benchmark mode), hashdog matches upstream hashcat across all 108 default hash modes (the modes hashcat tests by default with -b). These workloads generate candidates on the GPU, so the pipeline parallelism does not apply — but importantly, there are also no regressions.
Summary statistics (median of 3 runs, RTX 3090, autotune cache cleared between runs, 6s runtime each):
- Mean delta: +0.16%
- Median delta: 0.00% (perfect parity)
- 75% of modes: within ±1% of upstream (parity)
- 17% of modes: improved by 1-16%
- 8% of modes: small regressions (-1% to -3.8%, all within measurement variance — no systematic issue)
| Range | # modes | % |
|---|---|---|
| +10% or more | 1 | 0.9% |
| +5% to +10% | 1 | 0.9% |
| +1% to +5% | 16 | 14.8% |
| ±1% (parity) | 81 | 75.0% |
| -5% to -1% | 8 | 7.4% |
| -10% to -5% | 0 | 0.0% |
| -10% or worse | 1 | 0.9% (re-tested as parity — single-run noise) |
| Mode | Algorithm | hashcat | hashdog | Δ |
|---|---|---|---|---|
| 33700 | Microsoft Online Account (PBKDF2-HMAC-SHA256 + AES) | 320.89 kH/s | 372.68 kH/s | +16.1% |
| 15700 | Ethereum Wallet, SCRYPT | 16 H/s | 17 H/s | +6.2% |
| 17200 | PKZIP (Compressed) | 578.50 MH/s | 595.75 MH/s | +3.0% |
| 1000 | NTLM | 119.31 GH/s | 122.64 GH/s | +2.8% |
| 22921 | RSA/DSA/EC/OpenSSH Private Keys ( |
6.23 GH/s | 6.40 GH/s | +2.8% |
| 11700 | GOST R 34.11-2012 (Streebog) 256-bit | 201.68 MH/s | 206.55 MH/s | +2.4% |
| 1500 | descrypt, DES (Unix) | 2.53 GH/s | 2.58 GH/s | +1.9% |
| 29421 | VeraCrypt SHA512 + XTS 512 bit | 2.66 kH/s | 2.71 kH/s | +1.7% |
| 10510 | PDF 1.3-1.6 w/ RC4-40 | 76.39 MH/s | 77.59 MH/s | +1.6% |
| 17400 | SHA3-256 | 2.02 GH/s | 2.05 GH/s | +1.4% |
The full 108-mode comparison table is in research/results/full_robust_comparison.md. All raw data CSVs are under research/results/full_robust_20260409_121441/.
Phase 4: Advanced Optimizations — COMPLETE
- SSE2 rule engine — Nine mangle functions vectorized to process 16 bytes at a time using SSE2 intrinsics:
mangle_lrest,mangle_urest,mangle_trest(case conversion),mangle_replace(literal char replace),mangle_replace_class_l/u/d/lh/uh(class-based replace for lower, upper, digit, lower-hex, upper-hex). Provides additional speedup on top of pipeline parallelism for rules that touch large fractions of the password. - Pipeline extension to COMBI and BF/mask paths — The persistent GPU worker thread now drives all three
--slow-candidatesdispatch paths (STRAIGHT, COMBI, BF). Pipeline helper functions (pipeline_gpu_start/stop/join_prev/submit) are shared across paths to avoid duplication.
Phase 3: Pipeline Parallelism — COMPLETE
- Persistent GPU worker thread — While
run_crackerblocks on the GPU kernel, the main dispatch thread generates the next batch of candidates into alternate buffers. Buffer pointers are swapped after each batch completes. POSIX semaphores for low-overhead signaling. - Double-buffered candidate buffers —
pws_comp,pws_idx,pws_base_buf(host) andcuda_d_pws_comp_buf,cuda_d_pws_idx(device) all have alternate copies. CUDA stream + transfer event allocated for future async H2D extension. - Conditional allocation — Alternate buffers only allocated when
--slow-candidatesis enabled (the only path that benefits), avoiding GPU memory pressure on memory-heavy modes like 9400 (MS Office) or 22000 (WPA). - Graceful fallback — If alternate buffer allocation fails, the dispatch loop reverts to the sequential path. No regression for memory-constrained scenarios.
Phase 2: Low-Risk Optimizations — COMPLETE
- Autotune caching — Persistent disk cache at
~/.hashcat/hashcat.autotuneeliminates 10-30s startup cost per hash mode per device on subsequent runs. Cache key captures device identity, algorithm, and tuning parameter bounds. - Pinned host memory — Candidate password buffers use page-locked memory on CUDA/HIP backends for faster DMA-based H2D transfers, bypassing the kernel staging copy. Falls back gracefully on OpenCL/Metal.
- Rule engine allocation fix — Replaced per-candidate
hcmalloc/hcfreein_old_apply_rule()with a stack buffer, eliminating malloc overhead in the hot path for dictionary+rules attacks.
Phase 1: Architectural Analysis — COMPLETE
A full source-level decomposition of the hashcat execution pipeline identified six bottleneck domains, ranked by estimated impact:
| Priority | Domain | Key Finding | Status |
|---|---|---|---|
| 1 | Pipeline stalls | GPU idles during candidate generation and H2D transfer | Solved (Phase 3) |
| 2 | Autotune startup | 10-30s per hash mode, results not cached across sessions | Solved (Phase 2) |
| 3 | Rule engine | CPU-side, not vectorized, per-candidate malloc overhead | Solved (malloc + SSE2 case ops in Phase 4) |
| 4 | Work scheduling | Mutex-serialized allocation, static proportional balancing | Phase 4 |
| 5 | Memory transfer | Non-pinned host memory, no async overlap with compute | Solved (Phase 2 — pinned memory) |
| 6 | Wordlist I/O | Core uses fread buffering | Deferred (not a bottleneck) |
Full analysis: research/thesis.md
# Build hashdog
make clean && make
# Run a slow_candidates dictionary+rules benchmark
./hashcat -m 1800 hash.txt wordlist.txt -r rules/best66.rule \
--slow-candidates --runtime=40 --potfile-disableThe pipeline activates automatically when --slow-candidates is enabled and there is enough GPU memory for double-buffered candidate buffers. Use -D HASHDOG_PERF to compile with per-stage instrumentation that prints CPU/GPU/copy timing breakdowns to stderr.
hashcat is the world's fastest and most advanced password recovery utility, supporting five unique modes of attack for over 300 highly-optimized hashing algorithms. hashcat currently supports CPUs, GPUs, and other hardware accelerators on Linux, Windows, and macOS, and has facilities to help enable distributed password cracking.
hashcat is licensed under the MIT license. Refer to docs/license.txt for more information.
Download the latest release and unpack it in the desired location. Please remember to use 7z x when unpacking the archive from the command line to ensure full file paths remain intact.
Your platform may also provide packages.
Please refer to the Hashcat Wiki and the output of --help for usage information and general help. A list of frequently asked questions may also be found here. The Hashcat Forum also contains a plethora of information. If you still think you need help by a real human come to Discord.
Refer to BUILD.md for instructions on how to build hashcat from source.
Tests:
| Travis | Coverity | GitHub Actions |
|---|---|---|
 |
 |
 |
Contributions are welcome and encouraged, provided your code is of sufficient quality. Before submitting a pull request, please ensure your code adheres to the following requirements:
- Licensed under MIT license, or dedicated to the public domain (BSD, GPL, etc. code is incompatible)
- Adheres to gnu99 standard
- Compiles cleanly with no warnings when compiled with
-W -Wall -std=gnu99 - Uses Allman-style code blocks & indentation
- Uses 2-spaces as the indentation or a tab if it's required (for example: Makefiles)
- Uses lower-case function and variable names
- Avoids the use of
!and uses positive conditionals wherever possible (e.g.,if (foo == 0)instead ofif (!foo), andif (foo)instead ofif (foo != 0)) - Use code like array[index + 0] if you also need to do array[index + 1], to keep it aligned
You can use GNU Indent to help assist you with the style requirements:
indent -st -bad -bap -sc -bl -bli0 -ncdw -nce -cli0 -cbi0 -pcs -cs -npsl -bs -nbc -bls -blf -lp -i2 -ts2 -nut -l1024 -nbbo -fca -lc1024 -fc1
Your pull request should fully describe the functionality you are adding/removing or the problem you are solving. Regardless of whether your patch modifies one line or one thousand lines, you must describe what has prompted and/or motivated the change.
Solve only one problem in each pull request. If you're fixing a bug and adding a new feature, you need to make two separate pull requests. If you're fixing three bugs, you need to make three separate pull requests. If you're adding four new features, you need to make four separate pull requests. So on, and so forth.
If your patch fixes a bug, please be sure there is an issue open for the bug before submitting a pull request. If your patch aims to improve performance or optimize an algorithm, be sure to quantify your optimizations and document the trade-offs, and back up your claims with benchmarks and metrics.
In order to maintain the quality and integrity of the hashcat source tree, all pull requests must be reviewed and signed off by at least two board members before being merged. The project lead has the ultimate authority in deciding whether to accept or reject a pull request. Do not be discouraged if your pull request is rejected!