'bars_mutation passes 63%' is not what it looks like. What backtest search-method tables actually tell you.

The setup

In the flagship post I published this table:

The story I was originally telling: random search wastes compute; mutating existing winners finds edges 15,000× more efficiently.

That story is wrong in a specific way that I want to explain because the correction is more interesting than the original. Every time you see a "search methods compared" table in a crypto research post, you should apply the same reading.

What the numbers actually mean

bars_mutation's 1,528 trials come from exactly 4 distinct parent strategies. Each parent gets swept 382 ways (382 different maxBars values). That's not a search method. That's a local parameter sweep around 4 already-passing strategies. Sweep hold-length around a strategy that already works, and it usually still works — because you haven't changed anything meaningful about it.

Reporting bars_mutation's 63% pass rate as if it were comparable to random search is like reporting "wiggling every joint on a working robot doesn't break it" as evidence that wiggling joints is a good way to design robots. It's not a search method's hit rate. It's a fitness-landscape smoothness measurement around 4 known points.

Same story with tp_mutation (85% pass, 4 parents) and adx_mutation (100%, 4 parents). Both are local sweeps around a tiny number of already-winning parents. All three are in the "we already found something that works, let's poke it" category, not the "let's search the space" category.

cross_pollinate at 48.5% is actually informative. 1,108 trials from 55 distinct parent pairings. That IS closer to a real search method — hybridizing pairs of graduated strategies to produce novel offspring. Its pass rate holds up as a real signal about how well the fitness landscape blends between known winners.

"Random" isn't random either. The 89,656 random configs are drawn from a pre-filtered region: 5-minute timeframe, specific session hours, adxMax caps, top-N coin basket. It's undirected sampling inside a region we already know is edge-friendly, and it still finds a passing strategy 3 times in 89,656 tries.

That is a comment on how sparsely tradeable configurations are distributed even inside a region we've curated to be friendly. It is not evidence that "random search doesn't work as a general technique." Redraw random search across the full 20-dimensional space and the pass rate probably goes from 0.003% to 0.0000% — the pre-filter is doing enormous lifting.

The one honest cross-method comparison

Given all that, is there ANYTHING in the table that supports a genuine claim about search-strategy efficiency?

Yes: cross_pollinate at 48.5% across 55 parents. That's a real search method operating over a real number of starting points. The fact that ~half of hybrid offspring survive the same gates the parents survived tells you the fitness landscape is smooth enough to reward hybridization — which is a useful design principle for the search loop.

Everything else in the table is a local sensitivity result, a null-signal detector, or an in-region sampling result. Not directly comparable to each other. Not evidence of one search method beating another.

What survivor count actually means

The flagship reported 1,676 survivors. Reading the parent-lineage column tells you the deeper story: across the entire 1,676, only 35 distinct parent strategies produced any passing descendant. Almost every "survivor" is a variant of one of those 35 parents.

So the corpus doesn't contain 1,676 discoveries. It contains 35 discoveries and their siblings.

If tomorrow's market regime shifts against those 35 lineages, essentially nothing in the survivor pool remains. The pass-rate optics say "we have a diverse library." The parent-lineage math says "we have a small set of ideas that variants can be spun off from."

The pattern to look for elsewhere

Any time you see a research-lab table like this:

• Ask "how many distinct parents did each row come from?" A method with 1,528 trials from 4 parents is not comparable to a method with 1,108 trials from 55 parents. If the "distinct parents" column isn't there, treat pass rates as measurements of local landscape smoothness, not search efficiency.
• Ask "is 'random' actually random, or random-within-constraints?" If it's constrained, the pass rate says more about the constraint quality than the search method.
• Ask "what fraction of survivors come from the same lineages?" If it's most of them, the count overstates diversity.

None of these are hard to compute. They're not shown in any retail backtesting tool I know of. They're not shown in most academic ML papers either, honestly.

What I would do differently now

For the lab: I'd stop calling bars_mutation, tp_mutation, adx_mutation "search methods" in the schema. They're local variant generators applied to graduated parents. Different category. The pass rates for local variant generators tell you fitness-landscape smoothness around the seed points; they don't tell you anything about global search.

For posts: any table that combines local variants and global search methods needs the "distinct parents" column right next to the pass-rate column, or the table is misleading by construction.

For readers of other people's research: look for the same. If it's not there, ask.

Why publish the correction

The flagship post was drafted with the naive reading. Before publishing, I ran a hostile-persona audit on it — a HN critic, an r/algotrading practitioner, and a data journalist read it cold and tried to find what would embarrass the post. All three landed on the same finding about the mutation-type table. The correction landed before publication, but it landed because someone was actively looking for the flaw.

Publishing this as a standalone post makes the correction as visible as the original claim would have been. The flagship post carries the fixed version. This post is here so the pattern is available if someone else wants to check their own tables.

The corpus is on /research. A sanitized full-CSV with parent_id preserved is planned for Kaggle publication when reconciliation gaps we caught in the last self-audit are closed — the current lab-only export is not ready.

Comments and corrections at support@stratproof.com or @stratproof on X. If you spot another category error in the corpus I'll write about that next.