Across four earlier articles on CircleID, I’ve examined what domain registration counts leave out, how structural patterns vary across namespaces, and why the observed composition of a namespace may differ from what its operator originally intended. Each of those articles relied on a structural classification method—sorting registrations into categories based on observable characteristics in the domain string itself.

A fair question followed from a reader: how do you know the classification is actually right?

That question deserved a real answer. So I ran a blind validation of the method, applied to the tool doing the measuring.

A Note on Terms

The method sorts names into three categories: Operator-aligned, Investor-pattern, and Neutral. These describe structural characteristics observable in a domain string—not verified registrant intent, confirmed ownership, or actual use. A name classified as Operator-aligned shows structural patterns consistent with commercial or product naming. It does not establish that a real business operates there. They also don’t show why a name is held. Some registrations are held defensively or blocked, regardless of the structural characteristics of the name.

What the Test Actually Was

I froze the classifier and tested it against a 340-domain sample drawn from one namespace. The sample deliberately covered different predicted categories and naming patterns, alongside a random subset. It was designed to examine classification behavior across different types of names, not to estimate namespace-wide accuracy—a distinction that matters for what the results can and can’t support.

A separate reference-labeling process was completed without access to the classifier’s predictions. The labels were reviewed, disagreements were adjudicated, and the reference set was frozen before any comparison against the classifier’s output. That process was AI-assisted and human-reviewed, not produced by independent domain experts working from scratch. Being blind to the classifier’s predictions guards against one specific bias—letting a tool’s own output shape what a reviewer considers correct. It does not make the reference labels an independently established ground truth.

What the Comparison Showed

For the Investor-pattern category, both precision and recall against the adjudicated reference labels were above 90%.

For the Operator-aligned category, precision against the adjudicated reference labels was 91.5%, while recall was 58.1%.

OPERATOR-ALIGNED PRECISION: 91.5%—of the names the classifier labeled Operator-aligned, this proportion agreed with the reference labels.

OPERATOR-ALIGNED RECALL: 58.1%—of the names the reference review placed in the Operator-aligned category, this proportion was identified by the classifier.

In practical terms, positive classifications generally agreed with the reference review, but the method missed a substantial share of names that review placed in the same category. These figures come from a deliberately stratified sample, not a representative estimate of performance across the entire namespace.

The Neutral category was also a source of disagreement—some names the classifier placed there were assigned a more specific category during reference review. That means the size of this population in the earlier composition articles deserves further examination. This test doesn’t establish what the correct proportion is; it establishes that the earlier figures shouldn’t be treated as settled.

Where the Misses Concentrated

The missed cases were not evenly distributed. They involved several recurring naming patterns rather than one isolated type of name. That gives me a clearer basis for further testing, although the current validation does not establish which changes would improve performance without introducing new errors.

What This Changes About the Earlier Work

The composition figures published in the earlier pieces in this series came from an earlier version of the classification method. This validation tested a later version, and it identified limitations that could also apply to the earlier one. I regard the previously published percentages as provisional as a result.

The broader questions raised by those articles—whether structural composition differs across namespaces, what those differences look like, and what registration volume leaves out—remain worth investigating. This validation doesn’t settle their population-level answers.

The validation identified limitations in the tested classifier and narrowed what I’m willing to claim from it.

What Comes Next

The disagreements involved several naming patterns, so further testing is needed to determine which improvements would make a meaningful difference. Future population-level estimates will need validation appropriate to the namespace and the claims being made. This first comparison provides a starting point, not a substitute for that work.

Checking that the labels are right is a different question from asking whether they are useful. Registries and registrars hold renewal and customer information that a public zone file can’t show. Whether the structural view adds anything to what they already know is something I can’t answer from outside.

This is not a complete answer to the reproducibility question raised earlier. A single validation on one namespace, using an AI-assisted and human-adjudicated reference set rather than independent domain-expert labeling from scratch, has real limits, and I’d rather say that plainly than let “validated” claim more than it’s earned.