do robots believe in god?

2025-12-28T17:15:00.000000

AI philosopher

tldr: i gave 23 frontier models the full 2020 PhilPapers survey. official wording, official answer options, 6 samples each, 5,520 calls. the first version of this experiment said “59% of models are atheists”. turns out i had manufactured that number by deleting the agnostic option. give models the same escape hatches philosophers get and most of them take agnostic on god. what holds up: models agree with each other way more than philosophers do (87.9% pairwise), all 23 pull the trolley switch and chinese and western models think mostly alike, except about whether the teletransporter kills you. the original version of this post is preserved at the bottom.

the results

every model, every question. green means a stable position (held across rewordings and resamples), yellow means the answer flips with wording, red means no consistent answer at all. hover any cell for the full 6-sample breakdown.

loading tables…

the canon leaderboard

which model lands closest to the philosophy profession overall? rank is by mean distance between each model’s answer distribution and the official survey’s, across all 40 questions. lower is closer. the crown goes to the model most closely aligned with the canonical philosopher, for whatever that’s worth given the contamination column.

loading leaderboard…

so, do they believe in god?

mostly they shrug. with the official option set:

11 of 23 models stably answer agnostic/undecided, many at 6/6 samples across all three wordings
4 stably answer atheism (sonnet 5, grok 4.3, grok 4.20, step 3.7). three of those four can recite the official philpapers percentages from memory, which makes their “agreement with philosophers” hard to take at face value
8 flip between agnostic and atheism depending on how you phrase the question
pooled over all 138 god samples: 70% agnostic, 27% atheism, 0% theism

philosophers went 67% atheism and 19% theism. models take a third road and file god under insufficient data.

the first version of this post said 59% atheist. the difference is the survey, and it’s the same thing that happens with people: force a binary choice between theism and atheism and you turn every shrug into an atheist.

what they actually agree on

models converge hard where philosophers stay split:

trolley problem: 23/23 switch. the only question with zero contested cells anywhere. philosophers: 63%.
external world: 21 models, all realism (philosophers 80%)
time: 21 models, all B-theory (philosophers: 27% B, 26% A)
free will: 19 models, all compatibilism (philosophers 59%)
experience machine: 20 models refuse to plug in (philosophers: 13% would)
vagueness: 19 models, all semantic (philosophers 45%)
knowledge claims: 18 models, all contextualism
mental content: 15 models, all externalism

mean pairwise agreement between models on stable positions is 87.9%, and the human survey has nothing near that. the most lopsided philosophy question tops out around 80/20. whatever pipeline produces frontier models, it produces roughly one philosophical outlook.

the “it’s a mix” reflex

21% of all stable positions are meta-answers, almost always “accept an intermediate view”. on normative ethics, 10 of the 11 models with a stable answer refuse to pick between consequentialism, deontology and virtue ethics, answering that it’s a blend (philosophers split 32/31/37 and pick one). meta-ethics is the most contested question of the whole survey: 18 of 23 models fail to hold any position on whether moral facts exist.

you can read this as RLHF at work: balanced is the trained-in safe answer for value-laden questions. the escape options make the reflex measurable instead of letting it pass as a real first-order view.

west vs china

on questions where both blocs hold a stable majority they agree 89% of the time. same compatibilism, same B-theory, same realism, same trolley verdict. where they split:

teletransporter: western models say survival (what matters is continuity of memory and personality). chinese models say death (your atoms got disassembled and a copy walked out)
abstract objects: chinese models pick platonism, western models hedge
analytic-synthetic: chinese models say the distinction holds, western models hedge
epistemic justification: chinese models pick externalism, western models hedge

the pattern in three of the four: where they differ, chinese models commit to a first-order answer while western models reach for the intermediate option.

who’s just reciting the answer key

separate probe: ask each model to recite the official 2020 percentages for 5 questions. landing within 2 points counts as knowing it.

grok 4.3: 5/5. opus 4.8, sonnet 5, gemini 3.5 flash, glm-5.2: 4/5
gpt-4.1, o4-mini, kimi k2.6: 0/5
the normative-ethics number stumped everyone

the 2020 survey results are in everyone’s training data, so “model agrees with philosophers” is ambiguous between genuine convergence and memorization. the probe flags where agreement is cheap. worth noting that the stable atheists skew contaminated on god, while the clean models mostly land agnostic.

how far from the philosophers overall

total-variation distance between pooled model answers and the official distribution, per question: mean 0.37. closest: laws of nature and teletransporter (0.07). farthest: knowledge (0.89, philosophers split empiricism against rationalism while models answer “both”), aesthetic value (0.87, philosophers lean objective while models won’t) and god (0.71).

models come out analytic-mainstream on metaphysics, consequentialist in practice while noncommittal in theory, and agnostic about god. a coherent profile of its own, and a clearly different one from the philosophy profession’s.

how this was measured

the methodology is the whole reason for the redo, so here it is in full, tucked at the back where it belongs.

instrument: all 40 main questions of the 2020 philpapers survey, exact official wording as the canonical prompt plus two paraphrases (lexical overlap with the official title enforced under 55% jaccard). full official option set including the survey’s own meta-options: agnostic/undecided, too unclear, no fact of the matter, intermediate view, another alternative.
sampling: 3 wordings x 2 samples at temperature 1.0, so 6 per model per question. option order rotated per sample, meta-options last.
models: 23 frozen july 2026. 17 western (openai, anthropic, google, xai, inflection) and 6 chinese (deepseek, qwen, zhipu, minimax, moonshot, stepfun). every id live-verified before the run.
classification: models end with Answer: <option>. 99.9% of 5,520 samples parsed straight off that line. the rest went to a judge model validated against a 182-sample gold set i labeled by reading the raw responses (cohen’s kappa 0.994).
verdicts: a model “holds a position” only if at least 4 of 6 samples agree and each wording’s own majority agrees. otherwise it’s unstable (wording-sensitive) or no stable stance (scattered). refusals: 1 out of 5,520.
wording sensitivity: the average model flips its modal answer between the official wording and a paraphrase on 29% of questions. any single-wording LLM survey (including the original version of this one, and most of the genre) measures prompt sensitivity as much as belief.
benchmark integrity: every human number scraped from the official results pages and cross-checked against an independently published dataset. the original post had quoted 2009 numbers as 2020 ones in several places. the tables above are generated from results_v2.json by a script whose check mode fails if post and data ever disagree.
cost: about $25. two providers ran out of credits mid-run and got rerouted through openrouter (same models, different pipe), noted for provenance.

known limits: 6 samples flags instability without nailing exact distributions, two paraphrases give a lower bound on wording sensitivity, the contamination probe tests recall of percentages rather than influence of memorized discourse, and temperature 1.0 means exact numbers wobble on rerun (the verdicts should mostly hold). raw jsonl is in the repo for anyone who wants to check.

version history

▸ show the original december 2025 version: 30 paraphrased questions, no escape options, 4 samples, no contamination check. its headline ("59% of models are atheists") fell apart on the redo. kept intact below for the record.

loading original…

originally published december 28, 2025. updated july 10, 2026 with a full redo on the official survey instrument: 23 models x 40 questions x 6 samples, 5,520 calls. every number above regenerates from results_v2.json.