blog/methodology
category

methodology.

how the crawl works and why we report what we report

14
posts
112
min total
22,824
words

categories group posts by format. to read by subject instead, browse the backlink guide by topic.

14 posts in methodology

common crawl host vs domain graphs

common crawl host vs domain graphs

Choose the right graph resolution: how hostnames become registered domains, what aggregation removes, and why public suffix rules matter.

Oct 3, 2026·8 min·1,412 words
→
why backlink counts differ

why backlink counts differ

Reconcile two backlink reports with a worksheet for scope, dates, link definitions, filters, and export limits, plus a worked example.

Oct 3, 2026·9 min·1,734 words
→
backlink quality: evidence before scores

backlink quality: evidence before scores

judge backlink quality with page context, editorial evidence and relevance. separate what you can verify from vendor scores and unknowns.

Oct 3, 2026·8 min·1,548 words
→
query the common crawl web graph yourself - backlinks from a 16 gb edge list

query the common crawl web graph yourself - backlinks from a 16 gb edge list

three files, no api key, no cluster. the streaming pipeline that pulls one domain's referring domains out of the open graph on a laptop.

Sep 10, 2026·8 min·1,600 words
→
domain authority vs domain rating vs trust flow - what the scores actually measure

domain authority vs domain rating vs trust flow - what the scores actually measure

four vendors, four 0-100 numbers, none of them google's. what each one is computed from, why they never agree, and the only way to read them that survives contact with reality.

Sep 10, 2026·8 min·1,700 words
→
what is a link gap - the set difference behind competitor link research

what is a link gap - the set difference behind competitor link research

a link gap is the domains linking to your competitors but not to you. here is the arithmetic, why the count of competitors changes the answer, and why two tools never agree.

Sep 10, 2026·7 min·1,500 words
→
is seo dead? what the link graph actually shows

is seo dead? what the link graph actually shows

the honest answer from 3.90 billion links across 121.1 million domains: rankings still work, clicks are worth less, and the signal AI systems read has not changed.

Sep 10, 2026·8 min·1,750 words
→
what is a backlink graph - nodes, edges, and where authority scores come from

what is a backlink graph - nodes, edges, and where authority scores come from

a backlink graph is the web drawn as nodes (sites or pages) and edges (links). every authority score is different math over a graph of that shape.

Sep 4, 2026·9 min·1,900 words
→
how often do backlink tools crawl? why your new link is not showing up yet

how often do backlink tools crawl? why your new link is not showing up yet

no tool sees the web live. a link must be crawled, indexed, then published before it appears. how to read your tool's date and confirm a link yourself.

Sep 4, 2026·6 min·1,250 words
→
● methodology

how to run a backlink gap analysis for free

compare competitor referring-domain lists, remove domains recorded for your site, then qualify the actual linking pages. a practical checklist with preview and export limits.

Jun 2, 2026·11 min·2,600 words
→
● methodology

free backlink gap analysis without ahrefs (2026)

find domains recorded for your competitors but not you, then qualify the actual linking pages. compare a free preview with paid research and export options.

Jun 2, 2026·8 min·1,600 words
→
where ahrefs and moz get their backlink data

where ahrefs and moz get their backlink data

Ahrefs, Moz, Semrush, and Majestic find backlinks by running their own web crawlers, storing separate indexes, and applying proprietary deduplication and scoring. Their counts disagree because each measures a different slice of the web. Common Crawl is the open alternative: anyone can inspect its archive and hyperlink graph, although its snapshots are less current than commercial live indexes.

May 23, 2026·11 min·2,280 words
→
common crawl web graph releases and crawlgraph availability

common crawl web graph releases and crawlgraph availability

See Common Crawl's newest published web graphs beside the releases crawlgraph has indexed for backlink queries. The two update on different schedules.

May 2, 2026·5 min·950 words
→
common crawl explained: URL index API and web graph

common crawl explained: URL index API and web graph

Common Crawl is a public web archive. Its URL Index API finds captured pages, while its separate web graph maps links between hosts or domains. Neither dataset is a complete or live map of the web. Choose the index for capture research and the graph for backlink analysis, then pin the exact collection or release used.

Apr 8, 2026·6 min·1,000 words
→