Validation study
The in-house build test

Elaiia AI twins vs. a standard LLM – benchmark against the European Social Survey

Same 15 European Social Survey questions, same scales, same public ground truth. Elaiia twins: 1.5% mean deviation. A standard LLM role-playing 100 respondents: 14.0%.

Elaiia calibrated twins

1.5%

standard model, 100 personas

14.0%

standard model, direct estimate

11.0%

Mean deviation from the real Swiss population across all 15 variables.

Accuracy holds only where the answer has already been published

Line chart: mean deviation from the European Social Survey (Switzerland, Round 11). Standard LLM 6.5% on 9 items with a published statistic, 25.2% on 6 personal-value items; Elaiia calibrated twins 1.5% on both. — Split the benchmark by whether a published statistic for the item plausibly exists.

The ranking insight teams act on does not survive

Chart: value ranking of the Swiss population, ESS vs. Elaiia twins vs. a standard LLM. — On these scales a lower mean means the value matters more to the population.

Every objection we could raise against ourselves, granted in full

Chart: five hostile variants of the benchmark with contested items removed, Elaiia twins vs. a standard LLM. — Five hostile variants of the same benchmark, each with contested items removed.

Value hierarchy

Dot chart: mean position on the 1–6 importance scale for six values — ESS 11 real Swiss respondents, Elaiia AI twins and standard GPT personas; span of the value hierarchy ESS 2.50, twins 2.43, GPT 1.98. — From the validation deck, p. 8

Headline result

Validation deck page 4: mean deviation from ESS — Elaiia AI twins 1.5%, GPT 100 personas 14.0%, GPT direct estimate 11.0%; twins’ accuracy advantage 9.4×. — From the validation deck, p. 4

At a glance

StudyValidation study

BenchmarkEuropean Social Survey, Switzerland (Round 11)

Ground truthn = 1,384

Questions15, same scales

Elaiia twins1.5% mean deviation

Standard LLM, 100 personas14.0%

Standard LLM, direct estimate11.0%

Book a demo

Full validation report available on request.

ELAIIA

Always current

Stay close to what we're learning.

New case studies, research papers, validation, training content and Forum insights from across Elaiia Academy.

Book a demo