We replicated a viral “61% from Trump tweets” trading claim. It didn’t survive.

A project called trump-code went around claiming a 61.3% hit rate at predicting S&P 500 direction from Donald Trump’s Truth Social posts — 564 verified predictions, open-source, reproducible. That is the kind of claim worth taking seriously precisely because it is checkable. So we checked it.

We ran it as a pre-registered replication: decide what would count as success before looking at the results, write it down, then follow the plan whether we like where it lands or not. It’s the discipline that stops you quietly moving the goalposts once the numbers come in.

The number is real. The edge isn’t.

First finding: the 61.3% reproduces. Our independent calculation came out at 61.35% — a match to two decimal places. Their schema is readable, their maths checks out. Credit where it’s due.

The problem is what the number means. About a quarter of that hit rate (26% under our worst-case test) is lag-contaminated — the model is often reporting a move that has already happened rather than predicting one that hasn’t. A coin that tells you which way the coin already landed looks impressive and pays nothing.

So we tested the thing that actually matters: does the signal translate into tradeable edge on a real venue? We built 100 pre-committed strategies — five signal rules, ten Polymarket political markets across five categories, two direction conventions — and put every one through seven survival gates: enough samples, a real hit rate, margin over the fee, net edge, a multiple-comparisons correction, a reverse-direction check, and stability over time.

Zero of a hundred

None of the hundred cleared all seven gates. The two binding constraints — needing at least fifteen samples and a hit rate of at least 55% — turned out to be mutually unsatisfiable on this dataset: the cells with enough data never hit hard enough, and the cells that hit hard enough never had enough data.

Verdict: KILL. One rule was underpowered rather than falsified — not enough data to condemn it — so it’s flagged as the single candidate worth revisiting if anyone picks the work up. Everything else is a clean negative.

Why we publish the ones that fail

A negative result that took nine logical phases, a dozen commits and about two days of wall-clock work isn’t a failure — it’s the answer. Most “we found an edge” claims never survive contact with fees, lag, and a proper multiple-comparisons correction. Being able to tell the difference quickly, and being honest when the difference goes against you, is the actual skill.

The whole thing — the pre-registration, the gates, the counter-memo we wrote against our own conclusion before finalising it — is public. This is the kind of rigour we bring to the AI systems we build for clients: decide what “working” means up front, then let the evidence rule.