a backlink graph is the web drawn as a mathematical graph: every site or page is a node, and every hyperlink is a directed edge pointing from the source to the target. a backlink is one edge. a referring domain is one node with at least one edge into you. every authority score in the industry, from domain authority to our own cg authority, is a different function computed over a graph of this shape, which is why no two of them ever agree. for the practical, tool-by-tool version rather than the theory, the hub guide on how to find backlinks for free covers that.
a backlink graph, defined
a graph is a set of nodes and a set of edges connecting them. a backlink graph - also called a link graph, a hyperlink graph, or a web graph - is the case where the nodes are web pages or websites and the edges are hyperlinks. the edges are directed: an edge from news.org to shop.com is a different fact from an edge the other way. that direction is the whole point, because a link is a one-way vote.
everything else in backlink analysis is a query over that structure. who links to me is the set of edges pointing at my node. who links to my competitor but not to me is a set difference over two nodes' in-edges, which is all a gap analysis is. an authority score is a centrality measure over the same nodes and edges.
a backlink graph is a directed graph in which each node is a website or a page and each edge is a hyperlink from one to another. backlink metrics are functions over that graph: link counts are edge counts, referring domains are distinct source nodes, and authority scores are centrality measures.
nodes and edges, worked out
four hosts, five links. here is the whole graph, the roll-up step written out, and the two numbers everyone quotes falling out of the result:
nodes: 4 hosts shop.com bloga.com forum.net news.org edges: 5 hyperlinks, page level 1 bloga.com/review-1 -> shop.com/product 2 bloga.com/review-2 -> shop.com/product 3 forum.net/thread-88 -> shop.com/ 4 news.org/article-12 -> shop.com/pricing 5 news.org/article-12 -> bloga.com/review-1 roll-up step: replace each url with its host, then merge duplicate pairs 1 bloga.com -> shop.com (same pair as 2) 2 bloga.com -> shop.com (same pair as 1) => one edge, weight 2 3 forum.net -> shop.com (no twin) => weight 1 4 news.org -> shop.com (no twin) => weight 1 5 news.org -> bloga.com (no twin) => weight 1 host graph: 5 page edges became 4 host edges bloga.com -> shop.com weight 2 forum.net -> shop.com weight 1 news.org -> shop.com weight 1 news.org -> bloga.com weight 1 for shop.com backlinks = 4 page edges targeting it: 1, 2, 3, 4 referring domains = 3 host edges targeting it: bloga, forum, news for bloga.com backlinks = 1 page edge 5 referring domains = 1 news.org
the roll-up is the only step with any subtlety, and it is still just deduplication. rewrite each endpoint as its host, then merge pairs that are now identical. edges 1 and 2 both become bloga.com -> shop.com, so they merge into a single host edge of weight 2. edges 3, 4, and 5 have no twin, so they carry weight 1. five page edges in, four host edges out.
now read the counts off the two graphs. four page edges point at shop.com, so it has 4 backlinks. three host edges point at it, so it has 3 referring domains. bloga.com is both a source and a target in the same graph: it sends 2 edges to shop.com and receives 1 from news.org, so it has 1 backlink and 1 referring domain. no judgment call anywhere. that is what people mean when they say backlink data is objective: at this layer it is arithmetic on an edge list.
host-level vs page-level
the same web builds two very different graphs depending on what you call a node.
- page-level. one node per URL. billions of nodes, tens of billions of edges. this is the resolution you need to answer which page links to which page, what the anchor text said, and whether the link was nofollowed.
- host-level or domain-level. one node per registrable domain, with every page on it collapsed into that single node. parallel edges collapse into one edge carrying a weight. the result is orders of magnitude smaller and far more stable between crawls.
vendors publish scores at both resolutions - moz ships page authority next to domain authority, for instance - so this is not a case where the page graph goes unscored. what is true is that the site-level scores are the ones people quote, and the reason those exist at all is manipulation resistance. on the page graph, anyone who can generate a million pages can generate a million edges. collapsing to one node per site turns that million-page footprint back into one vote. a page-level score has to fight that problem some other way.
why referring domains and backlinks are different numbers
the distinction is not a definitional quibble. it falls straight out of the two resolutions above:
- backlinks to X = the number of edges whose target is X, in the page-level graph.
- referring domains to X = the in-degree of X in the host-level graph, meaning the number of distinct source nodes.
a sitewide footer link on a 40,000-page site adds 40,000 to the first number and 1 to the second. the ratio between them, links per referring domain, is itself a profile signal: 200,000 backlinks from 300 referring domains describes a very different site from 200,000 backlinks from 40,000. the longer treatment is in referring domain vs backlink.
centrality, and why it is the honest basis for authority
an authority score answers one question: how important is this node in the graph? graph theory has had answers for decades, and they are called centrality measures.
- degree centrality. count the in-edges, or the distinct in-neighbors. cheap to compute and the easiest to inflate.
- pagerank. model a surfer who follows a random link at each step. a node's score is the long-run probability the surfer is standing on it. it is recursive: a link from a high-scoring node with few out-edges is worth more than one from a low-scoring node with thousands.
- harmonic centrality. for a node v, sum
1 / d(u, v)over every other node u, where d is the shortest-path distance. nodes reachable in one or two hops from many places score high. unreachable pairs contribute 1/infinity = 0, which is why harmonic centrality behaves on a disconnected graph where plain closeness centrality breaks. the web is extremely disconnected, so this matters.
common crawl publishes both a harmonic centrality and a pagerank value for every vertex in each release, so the expensive part is already done. crawlgraph reads that ranks file directly and normalizes the harmonic rank onto a 0-100 scale:
cg_authority = round(100 * (1 - log10(harmonic_rank) / log10(total_domains))), then clamped to 0-100 and stored as an integer.
the rounding is part of the definition, not a display choice, so recomputing it from the same release file reproduces the same integer. the rank-1 domain out of the roughly 121 million in the current release scores 100 and the last one scores 0. note what this is not: it is not a percentile. a percentile would put the median domain at 50, whereas this puts it near 4, because the log compresses the enormous tail. the log is there because centrality is heavy-tailed: the gap between rank 10 and rank 100 means much more than the gap between rank 1,000,000 and rank 1,000,090.
centrality is the honest basis for an authority score because it is a property of the graph rather than an opinion about the graph. hand two people the same edge list and the same definition and they get the same numbers. the moment a score blends in crawl freshness, a spam classifier, or a model fitted to observed rankings, it stops being reproducible and starts being a product.
how the open hyperlink graph gets built and published
a backlink crawler is a bot that fetches pages, extracts every <a href>, and records the source-target pair. that list of pairs is the graph. common crawl's bot is ccbot. the commercial indexes each run their own: ahrefsbot for ahrefs, and for moz's link index dotbot, which is the crawler behind domain authority and is a different bot from rogerbot, the moz pro crawler that runs site crawl on a campaign you own. which of them reach your site is a question your own access log answers; grep it for the user agent strings rather than trusting anyone's ranking of who crawls hardest.
common crawl publishes the extracted link structure as a standalone hyperlink graph, separate from the page archive, in a host-level and a domain-level version. each release ships three gzipped, tab-separated files:
# ...-domain-vertices.txt.gz ~200 MB id rev_domain num_hosts 1042 com.example 3 # ...-domain-edges.txt.gz ~16 GB compressed from_id to_id 1042 880134 # ...-domain-ranks.txt.gz ~2.4 GB compressed harmonicc_pos harmonicc_val pr_pos pr_val host_rev n_hosts
domain names are stored reversed, com.example rather than example.com, so a plain sort groups every domain under a TLD together and keeps a site and its subdomains in one contiguous block. the vertices file is the node list, the edges file is the edge list as integer id pairs, and the ranks file is the precomputed centrality per node.
the release we currently index, cc-main-2026-apr-may-jun, holds about 121 million domain nodes and 3.9 billion domain-to-domain edges. treat those as the size of this release rather than a fixed property of the dataset: common crawl runs a new main crawl on a roughly monthly rhythm upstream, we ingest and republish a combined release on a roughly quarterly one (hence ids that span three months), and the node and edge totals move every time. older write-ups, ours included, quote the counts from whichever release was current when they were published. more on the dataset and where it falls short is in common crawl explained. what does not change is that anyone can download the files: no contract, no API key.
common crawl dropped the header line from the domain vertices and edges files starting with cc-main-2026-apr-may-jun. the ranks file kept its header. a parser that assumes a header on all three will silently eat your first data row.
why every authority metric disagrees
once you see each score as a function over a graph, the disagreements stop being mysterious. two scores differ if either input differs: a different graph, or a different function over it. usually both.
the graph half of that is covered in depth in where ahrefs and moz get their backlink data, which walks through the crawl-frontier, dedup, and rendering choices that make two vendors report different counts for the same site. this section is about the other half: why the scores still differ even when the counts happen to line up.
moz publishes domain authority and page authority, ahrefs publishes domain rating, majestic publishes trust flow. each vendor documents its own metric, and those descriptions are the only authority on what the metric currently means, so this table deliberately compares the kind of object each score is rather than restating anyone's definition:
| vendor scores (da, dr, trust flow) | cg authority | |
|---|---|---|
| graph it reads | an index built by that vendor and held internally | common crawl domain graph, published as a public download |
| who defines the function | the vendor, described in its own documentation | the formula in the section above |
| can an outsider recompute it | no, the underlying index is not distributed | yes, from the release files |
| meaningful against another tool score | ✗ no | ✗ no |
that last row is the one people ignore. a site can carry a high score in one tool and a middling one in another because the two crawlers reached different sets of linking sites, or because one function is a log-scaled sum over weighted in-links while the other is fitted to predict search results. neither is lying. they answer slightly different questions on slightly different data.
the practical rule: never compare a score across tools, only within one. a 60 in one tool and a cg authority of 60 are not the same claim about the same web. what does travel between tools is the graph fact underneath, the list of referring domains, because that is closer to an observation than to an interpretation.
a link graph is not a knowledge graph
these get conflated constantly, including by people searching for “moz knowledge graph.” a link graph's nodes are documents or sites and its edges are hyperlinks. a knowledge graph's nodes are entities - people, companies, places - and its edges are typed relations such as founded-by or located-in. google's knowledge graph is the second kind. a backlink index, moz's included, is the first kind. nothing in a hyperlink graph knows what any node is about; it only knows what points where.
look at your own node
the fastest way to make any of this concrete is to look up your own domain against the release we index. the free check needs no signup and returns two things: your node's in-degree, meaning the total count of referring domains pointing at you, and the first 5 of those domains by name. the full edge list, with the cg authority of each linking domain attached, is what the $99 one-time unlock and the API return.
if you would rather work with the edge list directly, the API returns the same rows as JSON, and the free tier is 15 backlink lookups a month with no card - request a key on the API docs page. or skip us entirely and pull the common crawl files yourself. that is the point of building on an open graph: our copy has no privileged access to it, and you can check our arithmetic.
writes the queries we run internally. ships one tactical post a week.
plus one when a new common crawl release lands. that is all.