We are our own first case study.
Measured in the open.

Round three · first re-measurement, Aug 23

Seven days after the content shipped:
too early to call — and two things moved anyway.

On Aug 16 we shipped our on-site content pack — the FAQ, the comparison page, two mechanics explainers, and this page — then froze the site and touched nothing for a week. On Aug 23 we re-ran the exact same measurement: same frozen questions, same pinned model versions, same controls. The pass rule was written down before the round ran: a change only counts if it beats twice the control-brand noise band on at least two engines in the same direction.

The pre-registered verdict: no layer passed. In the three category layers — the questions where buyers don’t know our name yet — we are still at exactly zero, on every engine. And this zero is a harder zero than the baseline’s: before the round ran, we confirmed all five new pages were indexed in both Google and Bing. So “the engines haven’t seen it” is off the table. The honest reading is visible but not yet chosen — the pages are in the index; they have not been picked up into answers.

We are not calling that failure, and we are also not spinning it. Retrieval pools take days to weeks to absorb new pages; a seven-day window reads an early signal, not a full effect. The call that matters — works or doesn’t — needs the next rounds. What we can publish now is what moved and what didn’t, with the noise band attached.

What moved: the OpenAI stack started recognizing us

The clearest movement in the round is entity resolution on the OpenAI stack. Asked brand-direct questions, it resolved “ClawWorld” to the actual us in 0 of 8 answers in round one, 1 of 8 in round two — and 6 of 8 in round three, with answers describing our pre-pivot company falling from 4 to 1.

“Resolved to us” share of brand-direct answers, per engine. The teal line is the story; the gray lines are context. Gemini’s wobble is discussed below; Claude has yet to resolve us correctly once.

The same round produced a second first: the OpenAI stack cited claw-world.app in 7 of 48 answers — after two straight rounds of never citing us at all. Gemini’s citation rate has drifted up within a two-point band since round one; Claude has never cited us.

Answers citing claw-world.app, all question layers, per engine. Note the axis tops out at 20% — these are early, small movements, drawn to their own scale rather than inflated.

Both trends are monotonic across three rounds, which is what makes them worth reporting. They are still descriptive: entity resolution has no pre-registered noise band, the sample is 8 answers per round, and neither figure is a verdict. If the next round holds them, they become evidence.

What we are not claiming: Gemini’s “resolved to us” share went 9 → 6 → 10 out of 20 across the three rounds. Reading the last step as improvement would mean picking the dip as the starting point — that is the before/after trick this page exists to reject. It is inside its own wobble, and it stays unclaimed.

Correction: the 53-point swing was our bug, not the engine

Until Sep 26 this section told a different story. It said that in the problem-phrased layer Claude’s answers had stopped triggering web search this round, that the control brands’ mention rates had collapsed with it by up to 53 points, and that the controls had caught engine behavior which would otherwise have been reported as a result. The collapse was not engine behavior. 25 of the 30 answers in that cell were an API error, “529 Overloaded”, which the Claude command-line tool printed where an answer should be and our pipeline stored as one.

With the round’s 38 error records removed, that cell holds 5 real answers. None of them searched, so that part stands; the control brands moved 3.3 points against the previous round, not 53. The tables above use the corrected round, which is why this page now counts 38 fewer answers than it did before Sep 26. The round’s verdict did not change: no layer passed before the correction and none passes after it. What happened and what we changed in the collector is written up in Our AI visibility monitor counted 38 API errors as answers.

What this data cannot tell you

The limits, stated up front.

The two rounds are hours apart, not weeks. The second round ran as soon as both search indexes confirmed they had re-crawled our new site. So the difference between them measures instrument stability, not how long an identity change takes to propagate. We cannot claim a propagation timeline from this data, and we will not.

ChatGPT and Perplexity are sampled by hand, not automated. Automating consumer chat interfaces violates their terms, so the largest-traffic engine is covered by a small manual sample. Conclusions extrapolated to ChatGPT are inference, not measurement.

Five samples per question is enough to catch a step, not a slope. This design detects a brand going from zero to visible; it cannot resolve a one- or two-point drift, and it says so rather than reporting noise as signal.

We broke our own data twice, and both times are on the record. During one collection run the API returned a rate-limit notice that our script stored as if it were an engine answer, quietly turning 75 records into false negatives. We caught it because a control brand’s number moved in a way reality does not allow, then re-collected the affected records and added a detector. In round three a different server error, “529 Overloaded”, slipped past that detector and turned 38 Claude records into false negatives. We found it three days later reading raw transcripts; this page carried the wrong figures until Sep 26 (see the correction above). Both incidents are written up in full in our measurement log, the second also on our blog — a measurement practice that has never found a bug in itself is a measurement practice nobody is checking.