A skill bundle is a set of instructions, so the only way to judge one is to read the instructions — and then count what it says about itself. Claude-BugHunter (12,519 stars, 1,590 forks, created 2026-05-05) is a good candidate because it makes its value proposition numeric: 83 skills, 15 slash commands, 681 disclosed-report patterns. Numbers can be checked.
So I checked them. I cloned the repo at main, counted citations in every pattern library, ran all seven of its own scripts and the pytest suite, installed it into a throwaway $HOME, and drove the scope matcher with 22 adversarial hostnames. The bundle is more real than most: 91 tests pass in 3.9 seconds, the scope gate held every trick I threw at it, and all 433 cited report URLs that exist really do exist.
The defect is narrower and more interesting than “the tool is fake.” The bundle’s own rule for its per-class counts — “grep-auditable: the number always equals what you can count in the file” — is false in 8 of the 36 files that have a library, and the two largest claims in the headline (225 + 174 = 399, i.e. 58.6% of it) live in the two places where there is nothing to grep at all.
What the bundle claims, in five numbers
The word “report” carries at least three different meanings in this repo, and the numbers only look contradictory until you find the sentence that redefines each one.
| Number | Where it appears | What it is defined as |
|---|---|---|
| 83 | README hero, docs/skills.md, disk | SKILL.md directories — confirmed: 83 on disk, 83 after install |
| 82 | GitHub “About” description | Stale by one; the About box was never re-counted |
| 681 | README hero, GitHub About, docs/architecture.md | “disclosed-report patterns” — and in two other sentences, “disclosed HackerOne reports” |
| 574+ | docs/credits.md table footer | Sum of per-skill tags (the table’s 15 rows actually add up to 565) |
| 433 | README hero, docs/disclosed-reports/README.md, CHANGELOG | “individually cited & auditable” — distinct hackerone.com/reports/NNNN URLs |
| 1,122 | nowhere | What I get by summing report_count: across the 70 skills that declare one |
That last row is the one that matters, and it is my number, not theirs. report_count is frontmatter, it ships to the model with every skill, and 70 of the 83 skills declare it. Add them up:
# count declared report_count across all skills
grep -h '^report_count:' skills/*/SKILL.md | awk '{s+=$2} END {print s}'
# 1122
# count distinct cited report URLs in the pattern libraries
grep -rhoE 'hackerone\.com/reports/[0-9]+' docs/disclosed-reports/*.md | sort -u | wc -l
# 433
1,122 declared, 433 real. A ratio of 2.59 declared units per clickable citation. And the gap is not uniformly distributed — it is concentrated exactly where the marketing is loudest.
The convention, and the eight files that break it
docs/disclosed-reports/README.md is unusually explicit. It defines the convention, promises it is machine-checkable, and says why that matters:
A skill’s
report_count:frontmatter = the number of distinct disclosed-report URLs cited in its pattern library (grep -oE 'reports/[0-9]+'→ unique). It is grep-auditable: the number always equals what you can count in the file. Nothing is asserted that a reader can’t click.
That is a testable sentence. So I tested it: for each of the 36 libraries, count the unique report URLs, then read the matching skill’s report_count and compare.
| Pattern library | Declared report_count | Distinct URLs actually in the file | Delta |
|---|---|---|---|
hunt-xss | 174 | 0 | −174 |
hunt-open-redirect | 28 | 0 | −28 |
hunt-deserialization | 22 | 0 | −22 |
hunt-cors | 19 | 0 | −19 |
hunt-session | 18 | 0 | −18 |
hunt-host-header | 16 | 0 | −16 |
hunt-nosqli | 14 | 0 | −14 |
hunt-auth-bypass | 12 | 5 | −7 |
| 28 other libraries | 432 | 428 | −4 (shared IDs) |
Eight of 36 disagree with their own file. Across those eight, 303 reports are declared and 5 are clickable. For the other 28, the convention holds — which is exactly what makes the failures diagnostic rather than cynical: someone did enforce this rule on most of the repo, then shipped the rest anyway.
The cause is visible in the files themselves. hunt-cors.md opens with “Pattern library built from 19 public bug bounty reports,” then contains 19 patterns with zero report URLs — every request and response genericized to target.com. hunt-xss.md goes further and argues against its own frontmatter:
Source: Shopify’s public bug-bounty program (HackerOne
shopify) has disclosed many XSS reports… Cite the program/class, not a specific report number.
That is a defensible editorial position for a technique guide. It is not compatible with a report_count: 174 that claims to be grep-auditable, and it is certainly not compatible with docs/credits.md listing hunt-xss | 174 H1 reports curated under a column titled “H1 reports curated.”
58.6% of the headline has nothing to click
Two numbers dominate the 681. hunt-misc is 225 — a third of the headline on its own — and it has no pattern library file at all. There is no docs/disclosed-reports/hunt-misc.md. The skill exists, is 20 KB, declares report_count: 225, and its description says “Built from 225 public bug bounty reports.” Nothing behind it.
hunt-xss is 174 — the second-largest — and its library exists but cites zero reports.
Put them together: 399 of 681 = 58.6% of the headline number rests on the two skills with no verifiable citation trail. Strip them and the remaining 34 classes declare 456 with 433 real, which is close enough to the truth that the 433 figure is clearly a genuine reconciliation — of the part that was reconciled.
Worth saying plainly: the 433 number checks out exactly. I counted 433 distinct URLs across 36 files, the same number the README, the CHANGELOG and the library README all claim. When a project gets one number right to the digit, the numbers it missed are more likely to be unfinished work than invention — and the CHANGELOG says as much, in a line worth reading closely:
Some counts reconciled down where a prior number was asserted without a backing corpus (e.g.
hunt-source-leak31→7) — honesty, not a coverage loss.
So the project knows this failure mode exists, has a documented policy for it, applied it to one skill, and left eight.
Why no gate caught it
The repo has eight CI workflows, including doc-count-check.yml, skill-lint.yml and leak-guard.yml. It is not a repo that neglects gates. It is a repo whose gates were scoped to the wrong numbers.
scripts/check_doc_counts.py runs clean:
Ground truth: 83 skills (58 hunt-*), 15 slash commands.
Checked 21 doc assertions + 2 structural sums: 0 error(s).
Twenty-one assertions, and the docstring says exactly what they cover:
It deliberately does NOT try to reconcile every document’s bespoke sub-categorization… only the three ground-truth totals: all skills, hunt-* skills, and slash commands.
report_count is not one of the three. Meanwhile scripts/lint_skills.py does see the field — 83 skills, 0 errors, 11 warnings — but its only rule about it is type-checking:
| Gate | What it checks about citations | Result on main |
|---|---|---|
lint_skills.py | report_count is a non-negative integer | pass (0 errors) |
check_doc_counts.py | 83 skills / 58 hunt-* / 15 commands | pass (21 assertions) |
scan_identifiers.py | no client identifiers leak into prose | pass (336 files, 0 matches) |
research/reports/verify_citations.py | fetches each cited report, re-runs the class router | not run in CI — needs network; claim: 433 reports, 0 mismatch |
| (nothing) | declared count == count in file | does not exist |
The one gate that comes closest, verify_citations.py, verifies that 433 cited reports actually support the class they are cited under. It has nothing to say about the 302 declared-but-uncited — a report that is not cited cannot be mismatched.
I did not re-run verify_citations.py: it fetches 433 HackerOne pages and re-classifies them, and I was not going to hang a claim on a rate-limited crawl. So “0 mismatch, 0 weak” is the one assertion in this audit I am leaving unverified, and I am labelling it that way rather than implying I checked it. My count of distinct URLs matching their count is the part I can back.
The part that loads whether you use it or not
Every audit of a skill bundle eventually arrives at the same question: what does it cost before it does anything? Claude Code’s documentation is unusually precise about the mechanism:
In a regular session, skill descriptions are loaded into context so Claude knows what’s available, but full skill content only loads when invoked.
So the always-on cost is the description set. In this bundle:
| Layer | Bytes | Approx. tokens (÷4) |
|---|---|---|
| All 83 SKILL.md bodies | 1,675,925 | ~418,981 |
| The 83 descriptions (always loaded) | 49,398 | ~12,350 |
Largest single body (osint-methodology) | 94,368 | ~23,592 |
Largest description (hunt-ato) | 1,017 chars | ~254 |
12,350 tokens of descriptions before a single skill fires. Whether that is a problem depends on the listing budget, which the docs describe like this:
The budget scales at 1% of the model’s context window. When the listing overflows, Claude Code drops descriptions starting with the skills you invoke least, so the skills you use most keep their full text.
Run the arithmetic: 1% of a 200,000-token window is 2,000 tokens, and this bundle’s descriptions are 6.2× that. Even on a 1,000,000-token window, 1% is 10,000 tokens — still 1.2× smaller than the 12,350 this bundle emits. Some descriptions will be dropped on any current window.
That would be a footnote if the bundle supported invocation by name. It does the opposite. The README’s first promise is:
All triggered automatically by topic — describe what you’re testing in plain English and the relevant skill loads. No invocation by name.
The docs also say what dropping a description removes: “the keywords Claude needs to match your request.” So the failure mode is structural, not theoretical: the least-invoked skills lose their descriptions first, and for this bundle a lost description is a skill that can no longer be reached in plain English — only by typing its slash name, which the bundle explicitly tells users not to do. The one-line diagnostic exists (/context reports the post-budget listing size; --debug logs an overflow warning; /skill-doctor flags never-invoked skills), but nothing in the bundle’s own documentation mentions the budget at all.
The bodies carry their own surprise. The docs are blunt that “once a skill loads, its content stays in context across turns, so every line is a recurring token cost,” and after auto-compaction “Claude Code re-attaches the most recent invocation of each skill… keeping the first 5,000 tokens of each.” Against that rule:
| Skill | Body tokens | Survives re-attach (5,000 cap) |
|---|---|---|
osint-methodology (1,640 lines) | ~23,592 | 21% |
bug-bounty (1,599 lines) | ~19,272 | 26% |
bb-local-toolkit (1,553 lines) | ~18,375 | 27% |
bb-methodology (508 lines) | ~7,402 | 68% |
The four biggest skills — the methodology layer you would want loaded all engagement — are the four that get cut hardest after a long session compacts. This is also what the repo’s own linter sees: it warns on exactly 11 skills for exceeding a 500-line body guideline, and those warnings never fail the build.
Two skills, one text
While measuring bodies I diffed them. bug-bounty (77,087 bytes) and bb-local-toolkit (73,500 bytes) are the same document:
# compare substantive lines, ignoring frontmatter
diff <(sed -n '/^---$/,/^---$/!p' skills/bug-bounty/SKILL.md) \
<(sed -n '/^---$/,/^---$/!p' skills/bb-local-toolkit/SKILL.md) | grep -c '^[<>]'
# 10
795 of bug-bounty’s 826 substantive lines appear verbatim in bb-local-toolkit — 96.2%. Both are titled # Bug Bounty Master Workflow. Both descriptions end with the same Chinese trigger list (中文触发词:漏洞赏金、安全测试、渗透测试、漏洞挖掘…), and both descriptions claim the same coverage — the only distinction is that one says “For local tool and wordlist paths, use bb-local-toolkit” and the other says “For general workflow routing without local tooling, use bug-bounty.” A mutual cross-reference is a routing instruction, not a disambiguation: for a prompt containing 漏洞賞金, both descriptions match.
If both trigger, that is ~37,647 tokens of near-identical text in one session — 1.5× the 25,000-token budget the docs give for re-attaching skills after compaction.
There is a defensible reading: bb-local-toolkit is documented in docs/credits.md as “Personal customization of the master bug-bounty workflow with author’s pipeline preferences,” i.e. a fork kept for the author’s own environment. Fine. But it is shipped in the same 83 as the original, installed by the same script, with a description engineered to trigger on the same words, and it accounts for 18,375 of the 418,981 tokens.
The measurement the project published against itself
Here is what makes this bundle different from most audited repos: it committed a negative result about its own headline claim.
eval/BASELINE.md documents a skills-on / skills-off ablation on OWASP Juice Shop, model held constant, three challenges, self-graded oracle:
| Condition | Solved | Solve-rate | Median turns | Median $ |
|---|---|---|---|---|
| skills-off | 3/3 | 100% | 2 | 0.178 |
| skills-on | 3/3 | 100% | 3 | 0.254 |
| delta | +0 | +0% | +1 | +0.076 |
And the note underneath, unsoftened:
Ceiling effect — as predicted. Both conditions solved all three; on memorized, curl-trivial Juice Shop challenges the hunt-* skill context is pure overhead (skills-on took more turns/cost, not fewer), so v0 measures pipeline soundness + autonomy + cost, NOT skill value.
The file then explains why the stronger tier (PortSwigger Academy labs, “less memorized”) is not in the repo: it needs an account, a manual JS/CSRF-gated lab launch, and Playwright credentials that “are not” present. So the honest reading of this bundle’s published evidence is: on the only measured target, the skills made the agent slower and 43% more expensive per run and solved nothing more.
That is not proof the skills are useless — a 3-challenge set of curl-trivial Juice Shop bugs has no headroom to demonstrate value, exactly as the author says. It is proof that the README’s “battle-tested” and “Claude Code stops being a chatbot and starts behaving like a senior bug-hunting researcher” rest on zero published measurement outside that file. And the failure mode the author identified himself — “the eval harness measured the wrong thing” — is the one that survives in the marketing copy.
ENGAGEMENTS.md calibrates the same claim from the other direction, and deserves credit for it. It lists exactly two authorized engagements (both 2026 Q2: one WAPT with findings 3C/0H/2M/6L, one external red team with 2C/4H/5M/3L = 27 findings total), redacted to quarter and continent, with the caveat:
Future engagements will be added as authorizations permit; some prior engagements are not listed because the SoW / NDA does not permit even abstracted disclosure.
So “battle-tested across authorized red-team and bug-hunting engagements” = two engagements, 27 findings, abstracted. The bundle is more honest about this than its own hero line suggests.
What held up under the same scrutiny
An audit that only reports failures is a hit piece, so here is what survived, measured the same way.
The tests are real. 91 tests and 33 subtests pass in 3.98 seconds:
$ python3 -m pytest tests engine -q
91 passed, 33 subtests passed in 3.98s
That covers scope parsing, ledger rotation, redirect handling, MCP config emission, and the eval harness’s own failure paths — not just smoke tests.
The scope gate is the best thing in the repo. engine/scope.py enforces scope “in CODE, not by trusting an LLM,” deny-wins, default-deny. I drove it with the hostnames that break naive endswith checks:
| Input | Host extracted | In scope? |
|---|---|---|
https://evilacme.com/ | evilacme.com | denied |
https://acme.com.evil.com/ | acme.com.evil.com | denied |
https://acme.com@evil.com/ | evil.com | denied |
https://evil.com/acme.com | evil.com | denied |
https://ACME.COM/ | acme.com | allowed |
https://acme.com./ | acme.com | allowed |
https://acme.com:8443/ | acme.com | allowed |
https://billing.acme.com/ | billing.acme.com | denied (deny wins) |
http://10.1.2.3/ | 10.1.2.3 | denied (deny wins) |
http://11.0.0.1/ | 11.0.0.1 | denied (outside CIDR) |
not a url at all | not a url at all | denied |
22 inputs, 22 correct decisions, including the userinfo trick that defeats most hand-rolled matchers. The apex-vs-wildcard distinction is also right: bare acme.com matches subdomains, *.acme.com explicitly does not match the bare apex — a distinction that trips many scope parsers. The one documented limit is honest rather than hidden: IPv4 CIDR only, so http://[2001:db8::1]/ falls through to default-deny.
Frontmatter is compliant, which is not universal. All 83 SKILL.md files have a parseable frontmatter block, a name matching the directory, and a description. The largest description is 1,017 characters — under both the bundle’s own 1,024 linter cap and the 1,536-character truncation point documented for the skill listing today. Compare that with the claude-red bundle where 28 of 79 skills had no frontmatter at all: this bundle got the mechanical part right.
Install works. bash scripts/install.sh --hermes --no-shell into a throwaway $HOME produced 83 skills, 2.5 MB, with report_count preserved. Manifest-driven, idempotent, uninstall-aware.
The 433 is real. Every one of the 433 distinct report URLs exists in 36 library files, exactly as claimed — 437 occurrences, 4 report IDs cited under two classes each.
| Check | Result |
|---|---|
| SKILL.md on disk | 83 |
hunt-* skills | 58 (matches README) |
| Slash commands | 15 (matches README) |
| Pattern libraries | 36 (matches docs/architecture.md) |
| Distinct cited report URLs | 433 (matches README) |
Declared report_count sum | 1,122 |
| Skills declaring a count with no library file | 34 |
| Tests | 91 passed, 33 subtests |
The numbers that were never reconciled with each other
Beyond the citation gap, the metadata layer has its own drift — the kind that accumulates when a number lives in prose instead of a test.
| Claim | Where | Reality |
|---|---|---|
| “82 skills” | GitHub About | 83 on disk |
| “83 skills” | README, plugin manifest | 83 ✓ |
| 83 skills in the plugin | plugin.json, marketplace.json | source ./ → main tree, so 83 ✓ — but version: "2.1.0" unchanged across 71 → 83 skills |
| 71 skills | tag v2.1, the last release (2026-06-05, 168 days before this audit) | 12 skills were added after the last tag |
| “43 original skills + 8 vendored” | README docs table | docs/credits.md: 55 original + 20 community + 8 vendored |
| “574+” curated reports | docs/credits.md table footer | the 15 rows sum to 565 |
hunt-rce “Built from 67 public bug bounty reports” | its own description | report_count: 87 in the same frontmatter block |
That last one is worth pausing on, because the description is the only part of a skill that enters context before it triggers. Ten skills tell the model one number in the description and carry another in report_count:
| Skill | Description says | report_count says |
|---|---|---|
hunt-rce | 67 | 87 |
hunt-sqli | 12 | 29 |
hunt-ssrf | 15 | 34 |
hunt-idor | 26 | 39 |
hunt-graphql | 12 | 26 |
hunt-business-logic | 12 | 25 |
hunt-oauth | 19 | 22 |
hunt-csrf | 15 | 18 |
hunt-cache-poison | 10 | 8 |
hunt-race-condition | 12 | 10 |
Both numbers are wrong in the same way — neither matches the file — but they are wrong differently, in the same YAML block, in a repo with a CI job whose entire purpose is doc-count consistency. Nobody re-read the description when the count was reconciled.
If you still want to install it
The bundle is not without value: the OSINT and enterprise platform skills are long, specific, and clearly derived from real work, and engine/scope.py is a piece of code I would copy before I copied anything else. But adopt it with the measurement habit the project itself demonstrates in eval/BASELINE.md and does not carry into its README.
Concrete steps, in order:
- Install, then immediately measure the listing. Run
claude --debug, then/context— the Skills row reports the post-budget listing size, and--debuglogs a warning if the listing overflowed. Then/skill-doctorfor per-skill cost and never-invoked flags. If the listing is over budget, the fix isskillOverrides— set the 40 skills you will never use to"off", not to"name-only", since a name without a description cannot trigger. - Validate the frontmatter before you trust the triggers.
claude plugin validate ~/.claude/skillsfinds files whose frontmatter doesn’t parse. All 83 pass today; the point is that it is one command and it catches the silent failure where a skill loads with no description and therefore never self-triggers. - Do the grep the README promises. Ten seconds, no network:Treat
for f in docs/disclosed-reports/hunt-*.md; do printf '%-34s declared=%s cited=%s\n' "$(basename $f .md)" \ "$(grep -m1 '^report_count:' skills/$(basename $f .md)/SKILL.md | awk '{print $2}')" \ "$(grep -oE 'reports/[0-9]+' $f | sort -u | wc -l)" donecitedas the number you can rely on. It is the only one with a file behind it. - Pick two classes and read the library, not the skill. The library is where the technique is genericized and the citations are; the skill is the operator-facing summary. If a class has
report_countbut no library file, you are reading one person’s summary of a corpus you cannot see. - Plan for the compaction cut. Skills over ~20,000 tokens keep only their first 5,000 after compaction. If your workflow depends on a 1,600-line methodology body, put the parts you need in a reference file the skill loads on demand, or restate them in your own handoff notes.
- If you port it to claude.ai, the Skills API, or
package_skill.py, strip two keys first.sources:andreport_count:are Claude-Code-only fields; outside Claude Code, “if you include any field the spec doesn’t allow, packaging or upload fails with a hard error.” The bundle already ships the fix —install.sh --normalize-frontmatterstrips exactly those two — which is itself the clearest statement that its provenance metadata is not portable.
FAQ
Is the bundle dishonest about its numbers?
I would say “unfinished,” not “invented.” The 433 is exact, the reconciliation policy is documented, and one skill was explicitly corrected downward (31→7) for having no backing corpus. What is missing is the same pass applied to eight more libraries and to hunt-misc. The problem is that the fix stopped where the number was largest.
Does report_count affect whether a skill works?
No. It is inert metadata — Claude Code ignores fields it does not recognize. That is precisely why it went un-reconciled: nothing breaks when it is wrong. It matters because it is the number buyers use to compare bundles, and because the same frontmatter is what breaks a claude.ai upload.
Are 83 skills too many for one session? The mechanics say the listing will overflow: 12,350 tokens of descriptions against a 1%-of-window budget that is 2,000 on a 200K window and 10,000 on a 1M window. The bundle’s trigger model is description-only. So the practical ceiling here is not “can 83 install” (they can) but “how many descriptions survive the budget,” and that number is smaller than 83.
Is the eval result evidence the skills don’t help? No — three Juice Shop challenges both conditions solve in 2–3 turns cannot distinguish skill value from noise, and the author says so in the file. It is evidence that no target in the repo has yet produced a positive delta, which is a different and weaker claim than the README makes.
What is actually unique here? The engine. 2,510 lines of Python with 658 lines of tests, deny-wins scope enforcement, a coverage ledger, and a deliberate refusal to trust the model’s judgment about scope. If you take one thing from this repo, take the scope matcher and its tests, then read the skills as a well-organized reading list rather than a checklist.
Should I use it on a live engagement with an unverified target? The authorization posture is more careful than most: gates, out-of-scope rebuttals, redaction rules, and an explicit statement that Opus 5 falls back to an older model on some cyber requests. But scope still comes from a file you write. The code enforces what you tell it; it cannot tell you whether you are allowed.
What this bundle is
Claude-BugHunter is a large, mostly well-built knowledge artifact with a marketing layer that outran its data layer. The 83 skills exist, all 83 parse, 91 tests pass, the scope gate is genuinely good, and 433 cited HackerOne reports are real and countable. Against that: 1,122 declared report units against 433 citations, 8 of 36 libraries contradicting their own stated invariant, 58.6% of the headline living in the two skills with no citation trail, a 96.2%-duplicate skill pair, an always-on listing that exceeds the documented budget on every current context window, and a single committed ablation where the skills-on condition cost more and solved nothing extra.
None of those is a reason to call the bundle a scam. Together they are a reason to do what its own eval/BASELINE.md does — measure the thing before repeating the number. The README’s best sentence is the one it applied to one skill and then abandoned: honesty, not a coverage loss.