how often do backlink tools crawl? why your new link is not showing up yet

no backlink tool sees the web live. a link has to be crawled, folded into an index, and then published to you, and every one of those stages adds delay. that is why a link that went up last Tuesday can be real, followed, and already found by google while your backlink tool shows nothing. rather than rank vendors by a speed nobody outside those companies can measure, this page explains the two shapes a backlink index comes in, states our own cadence plainly, and shows you how to get the real answer for whichever tool you actually use.

why your new link is not showing up yet

a link clears three separate hurdles before any tool will show it. a crawler has to fetch the page carrying it. the vendor has to fold that fetch into an index. the vendor has to publish that index to you. a delay at any one of the three looks identical from the outside, which is why “my link is not showing up” is the most common question in every backlink tool's support queue. if you are still assembling a toolkit, the free lineup is in how to find backlinks for free.

before blaming any of that, rule out the boring causes. the page carrying your link may not be linked from anywhere a crawler will find it. the link may be injected by javascript that the crawler does not execute. the site may block that specific crawler in robots.txt, which large publishers do routinely. and the page may simply be low priority: crawl budget is allocated by importance, so a new post on a small blog gets fetched far later than the same link on a busy domain.

two ways a backlink index gets built

whatever the logo on the dashboard, the index behind it has one of two shapes, and the shape is the actual answer to “how often does this thing crawl”.

• a continuously recrawling commercial index. the vendor runs its own crawler, revisits urls on a priority schedule it sets internally, and folds each fetch into the index as it arrives. there is no edition and no date: the index is a moving target, and what you see is whatever had been folded in by the moment you ran the query.
• a periodically published open snapshot. a crawl runs, the whole dataset is processed, and the result is published as a dated release. between releases the data does not change at all. crawlgraph is this shape, because it reads the common crawl link graph.

that difference explains why the question has no tidy numeric answer. a continuous index has no single interval to quote, because every url is on its own priority-weighted schedule and a headline figure describes the fast end of a very wide distribution. a snapshot has no interval either, but it has something better: a date you can read, which tells you exactly which links it cannot possibly know about. the crawler names and the methods behind each index are broken down in where ahrefs and moz get their backlink data.

what each model is good and bad at

neither column is the better product. they answer different questions, and most of the frustration in this topic comes from asking one of them the other's question.

our cadence, stated plainly

common crawl publishes new crawls upstream on a roughly monthly rhythm. crawlgraph ingests those and republishes a combined link graph on a roughly quarterly rhythm, which is why our release ids are three-month spans. the current release is cc-main-2026-apr-may-jun, labeled Apr-Jun 2026, covering 121.1M domains and 3.90B domain-level links.

so, said without hedging: a link that appeared last week is not in our data. a link that appeared last month is probably not either. it will show up when the next release lands and we rebuild, and not before. if that is a dealbreaker for your job, it is a dealbreaker, and you should use something else for that job.

how to find the answer for your own tool

do not take a cadence from a blog post, including this one. published intervals change, and third-party comparisons go stale quietly. check the source instead.

• read the date your tool prints. any index that shows a build, release, or last-crawled date is telling you precisely which links it cannot know about: everything placed after it. treat that date as the boundary and stop refreshing until it moves.
• if it prints no date, read the vendor's own documentation. vendors publish their crawler names and how their index is built. that page is current; a comparison table written a year ago is not.
• check search console for google's view. the links report is free and it is google's own record rather than a vendor's reconstruction. it runs on a lag of its own, so a few quiet days there is normal rather than a failure.
• check whether the crawler is allowed in at all. if the linking site's robots.txt blocks a given crawler, that tool will never show the link, and no cadence explains it.

the fastest way to confirm a link went live

the quickest check is not a backlink tool at all. fetch the page and look.

# 1. is the link actually on the page?
curl -s https://linking-site.com/the-post | grep -o "yoursite.com"

# 2. can a crawler reach that page at all?
curl -s -o /dev/null -w "%{http_code}\n" https://linking-site.com/the-post
curl -s https://linking-site.com/robots.txt | head -20

if the first command prints your domain, the link exists and every index will catch up on its own schedule. if the second returns anything other than 200, or robots.txt blocks the crawlers, no index will ever show that link and the cadence question is moot.

which clock your question runs on

• did my new link go live? fetch the page, then search console. not a snapshot dataset, and honestly not a backlink tool at all.
• did I lose a link this month? a continuously recrawled index you subscribe to. a snapshot cannot answer a question about a recent event.
• who links to my competitors and not to me? a periodic snapshot answers this as well as a live index does.
• what number goes in the client deck? whichever tool the client already recognizes, with the index date printed next to it.

that third one is worth spelling out, because it is the case where lag costs you almost nothing. a link gap is a question about a set of domains, not a recent event. the sites worth pitching are the ones that have linked to two or three of your competitors for months or years: industry directories, roundup posts, review sites, association pages. all of those are in a snapshot that closed a quarter ago. run the same query against a live index and you get essentially the same prospecting list, minus a handful of placements you were not going to pitch first anyway.

if a snapshot fits the job you actually have, the check above returns the top 5 referring domains for any domain. a free api key gets you 15 backlink lookups a month without a card, and a free gap analysis run takes an email address, compares up to four competitors, and shows you the first 5 gaps. the full list and the export are $99 once. and if your job is monitoring a placement you made this week, keep the subscription that does it: the split is not which index is better, it is which clock your question runs on.