Reverse Turing Test

1. Base models train on the training split.
2. The meta-classifier is fitted on the validation split, using the three models' output probabilities as its only inputs.
3. Every number above is measured on the test split.

The meta-classifier is deliberately fitted on data the base models never trained on. Fitting it on the training split instead would let it learn from their overconfidence on rows they had memorised, and the ensemble would look better than it is.

Transformer (DistilBERT)

Fine-tuned on the training split, reading up to 256 tokens. Catches phrasing and semantics that statistics miss, but degrades on text unlike anything it trained on.

Stylometric (logistic regression)

Five features: average sentence length, sentence-length variance, average word length, vocabulary richness, punctuation density. Cheap, interpretable, and surprisingly hard to beat — but it needs enough text for the statistics to stabilise.

Visual CNN

Renders each text as a 224×224 heatmap — one row per sentence, one column per measurement — and classifies the resulting texture. It sees structural rhythm rather than words.

There is no inference server. Both neural models were converted to int8 ONNX — about four times smaller than the originals — and run through ONNX Runtime Web via WebAssembly, while the stylometric and stacking models are a few lines of arithmetic. The text you paste never leaves your device.

The JavaScript port reproduces the Python pipeline exactly, down to float32 rounding in the heatmap encoding and OpenCV's colour-map table. A fixture of Python-generated values is replayed against it in web/parity.html, so a port that drifted would fail loudly rather than quietly degrade predictions.

• Measured against a baseline. Every model is compared to the majority-class baseline. A classifier that cannot clear it has learned nothing, whatever its raw accuracy suggests.
• Scored on unseen data. The meta-classifier is fitted on validation and evaluated on test, so the ensemble is never scored on data used to tune it.
• Reproducible end to end. Every figure here is regenerated by src/train_meta.py.

• Short text is unreliable. Below roughly 50 words the stylometric features are dominated by noise, and the ensemble leans almost entirely on the transformer.
• Domain shift hurts. These numbers are in-distribution for RAID, whose splits skew toward one of its eight domains. Text from a different domain — or a newer generator — should be expected to score worse.
• Editing defeats it. Generated text that a human has revised carries a mix of both signatures, which is exactly the region where the models disagree most.
• This is not proof of authorship. Every output is a probability. Do not use it to accuse anyone of anything.