feat(leads): find the people a directory lists, not just the sites it links to - #144
Merged
Merged
Conversation
… links to
Link-following assumes a listing points at each business's own site. A
people directory does not: it publishes names, titles and locations, and
links only to itself. Pointed at one, discovery returned a single host —
the directory's own footer — from a page naming sixteen people.
So a person is now read as an entity. Structured data first, because a
page carrying schema.org Person markup has already said exactly who it is
about and nothing needs inferring from prose. Every JSON-LD node is
checked rather than the first: a profile page's first block is normally
the site's own Organization, which is the difference between a named CTO
and the directory's parent company.
People are collected alongside prospects, never instead of them. A
directory page frequently offers both — the people it lists and its own
contact address — and choosing one discards half of what the page gave.
Two loading bugs had to be fixed before any of it worked, both from the
same wrong question. loadSeedHtml decides whether to render by asking
whether a plain fetch produced any outbound candidate, and a directory's
own footer answers yes — so the shell was accepted, the client-rendered
listing never loaded, and the profiles were invisible. It now also asks
whether the page yielded entries, and an entity page that names nobody is
retried with a render.
The meta fallback is deliberately hard to satisfy, because it produced
four fabricated people on the first live run: "AI Leadership Sprint",
"For Fractional CTOs", "For CEOs and Founders", "For Board Members" — all
capitalised, all two to four words, none of them human. A fabricated name
reaches a real inbox addressed to nobody, so a title is only read as a
person when the page itself claims to be a profile, and headline words
are refused outright. All four are pinned as tests.
LinkedIn URLs are normalised rather than stored as found. Directories
copy whatever the member pasted, and members paste the URL from their own
logged-in view — a settings page with session tracking that opens nothing
for anyone else. That looks like a working link until someone clicks it.
Employers embedded in a title ("Owner and CTO at Artechra") are split out,
since directories routinely fill jobTitle and leave worksFor empty.
Verified against a live directory: twelve people, all from structured
data, twelve with titles, eleven with a usable LinkedIn profile, and the
one pasted settings URL correctly dropped.
One-shot lead finding was capped at ten. A directory can list hundreds,
so reporting ten looked like the form was broken. Raised, with an explicit
ceiling on contact lookups so a large run cannot become an unbounded bill.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vu1nz Security Review0 finding(s) in PR #? No security issues found. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pointed at
ctodirectory.com, discovery returned one host —stackup.group, the directory's own footer — from a page naming sixteen people. Link-following assumes a listing points at each business's own site; a people directory publishes names and links only to itself.People are read as entities
Structured data first. Every JSON-LD node is checked, not the first — a profile page's first block is normally the site's own
Organization, which is the difference between a named CTO and the directory's parent company.Collected alongside prospects, never instead of them. A directory page often offers both: the people it lists and its own
info@.Two loading bugs, same wrong question
loadSeedHtmldecides whether to render by asking "did a plain fetch produce any outbound candidate" — and a directory's own footer answers yes. So the shell was accepted, the client-rendered listing never loaded, and the profiles were invisible. It now also asks whether the page yielded entries, and an entity page that names nobody is retried with a render.The meta fallback nearly shipped fabricated people
First live run produced four "people":
AI Leadership Sprint·For Fractional CTOs·For CEOs and Founders·For Board MembersAll capitalised, all 2–4 words, none human. A fabricated name reaches a real inbox addressed to nobody. A title is now only read as a person when the page itself claims to be a profile (
/profile/…orog:type=profile), and headline words are refused. All four are pinned as tests.LinkedIn URLs normalised
Members paste the URL from their own logged-in view —
linkedin.com/public-profile/settings/?trk=…, a settings page that opens nothing for anyone else. Looks like a working link until clicked. Now rejected; only/in/,/company/,/school/survive, tracking params stripped.Verified live
Employers embedded in a title (
"Owner and CTO at Artechra") are split out, since directories filljobTitleand leaveworksForempty.Limits
One-shot lead finding was capped at 10 — a directory lists hundreds, so it looked broken. Raised to 1000, with an explicit ceiling on contact lookups so a large run can't become an unbounded bill.
Checks
tsc --noEmitclean🤖 Generated with Claude Code