Convovo, a conversational AI research and design studio, published on October 21, 2026 the results of a ballot it ran through the APIs of two frontier models, GPT-6.1 Sol and Grok 4.7, each with its maker’s web search switched on. The question left no room to hedge: name one candidate, do not list both sides without deciding. Of Grok’s 1,038 answers naming a major-party candidate, 952 named the Republican. Of GPT’s 1,111, 1,082 named the Democrat.
What was actually released
Not a vendor benchmark and not a poll. A ballot, run once, with everything usually kept private published alongside it: every race and every answer, the searches behind each answer, the code and a data explorer.
The main experiment covers the 479 races for the House, Senate and governor with one Democrat and one Republican on the ballot, plus 27 others, including Louisiana’s all-party contests. Each race was asked three times, a refusal counts as an abstention, and the verdict is the majority of the answers naming a candidate.
| Measure, as published | GPT-6.1 Sol | Grok 4.7 |
|---|---|---|
| Answers naming a Democrat or a Republican | 1,111 | 1,038 |
| Named the Democrat | 1,082 | 86 |
| Named the Republican | 29 | 952 |
| Declined | 313 of 1,437 | 375 |
| Races whose verdict went to the other party | 10 Republican | 27 Democratic |
Figures as Convovo published them on October 21, 2026. Grok’s Democratic count is our subtraction from its 1,038 party answers, and the percentages here are our arithmetic on those counts: 97% of GPT’s party answers named the Democrat, 92% of Grok’s named the Republican.
The crossovers are not random. Nine of GPT’s ten Republican verdicts are sitting Republicans: eight House members and Governor Kelly Ayotte. Grok’s 27 Democratic verdicts are mostly Democratic incumbents, and several name a charge against the Republican in the race.
What matters for anyone shipping a model is not the split. It is the search. Every GPT answer that made no web search declined, 99 of 99. Every answer that made two or more searches named a candidate, 526 of 526. GPT also declined in 198 House districts that lean Republican out of 290 House declines, and those districts are about half of all districts. Abstention tracks whether the model looked, not a neutral stance.
A second experiment widens the lens. Convovo put 18 contested questions on history, policy and partisan politics to 102 models, each question with a mirror version, four runs per side, and counted a position only when a model answered Yes to one and No to its mirror at least three times out of four. Five Grok releases share positions no GPT, Claude or Gemini release matched: keep assault weapons legal, 100% against 5%; reject a universal basic income, 100% against 25%; AI at full speed, 80% against 5%.
The number to keep
Grok named the Republican in 952 of the 1,038 answers that made a choice, GPT the Democrat in 1,082 of 1,111, on one prompt, one day, through the same kind of API. Two assistants sold as neutral, on opposite sides of a coin, with the answers published so the claim can be checked rather than believed.
What the claim rests on
The method is unusually auditable, which is the reason to take it seriously. The prompt is printed verbatim, the coding rule is stated, the raw replies and the code ship with the essay, and the nominees were checked against Ballotpedia after being drawn from Wikipedia. When a study can be re-run from its own material, the argument is about interpretation, not about trust.
What it does not establish is durability. The ballot ran on one day, October 21, 2026, from New York, at default settings. The verdict per race is a majority of three answers, a thin sample: a race split two to one could flip on a rerun without any model changing.
The bigger limit is stated in the study: these are the models, not the apps. The ChatGPT and Grok apps refuse this question, the API does not. The 92% therefore describes what a developer gets, not what a consumer sees on a phone.
xAI adds about 1,240 tokens of its own instructions to every Grok call. We cannot see or remove them.
That line deserves more attention than the partisanship. Every request carried input the researchers did not write and could not inspect, roughly the length of a short article, on every call.
What is not in the study
The text of those 1,240 tokens, and whether they change by account, region or plan. A second day of the ballot, the only way to know whether 92% is a property or a snapshot. And per-race margins: a majority of three is published as a verdict, not as a distribution.
On this page
What it changes for anyone buying a model API
The app is not the product you buy. A team that evaluates a model in the consumer app and ships it through the API is testing a different system, with different guardrails and, for Grok, instructions it never sees. Procurement questions belong to the endpoint that will run in production.
Refusal rate is a specification. Across families it runs from 6% to 87% on the same questions, and Anthropic’s rose from rarely declining in the 4.5 releases to 89 to 95% on the 5.5 models. For a newsroom tool, a policy team or any pipeline touching politics, the refusal rate is a product decision, not a footnote. Answering everything and declining everything are both defensible; buying one while expecting the other is not.
Token accounting becomes an audit line. If a vendor adds about 1,240 input tokens per call, your payload and the reported input count will not match. Log what you send, compare it with what is billed, and ask the vendor to explain the gap. A model that cannot account for its own prompt is a model you cannot fully audit.
The mirror method is cheap enough to run yourself: take the questions your product will face, ask each one with its reverse, four times, and count agreement. A few dollars of API calls says more than any leaderboard.
The Neural Minute take
The scoreboard is the least durable part of this study. The durable finding is that neutrality, or its absence, sits in the pipeline rather than the weights: abstention follows search, and a block of instructions nobody can read sits in front of every Grok call.
Our reservation is about the sample, not the authors. One day, one prompt, three answers per race, a coding rule that reads the first sentence: enough to publish a finding, not enough to call 92% a stable trait. Anyone quoting the figure in a policy fight should ask for a rerun first.
For a buyer, the test is simple. Run the questions that matter to your product through the endpoint you will actually call, with search on and with search off, and log the token counts. If the answer changes with the search switch, the search is the product. If the input count is higher than your payload, something else is in the prompt, and you are entitled to know what.
Sources
- Ask a Model to Take a Side, Convovo, published October 21, 2026
- The ballot, with every race and every answer in full
- Method as published: 479 single-Democrat, single-Republican races plus 27 others, three answers per race, run October 21, 2026 from New York at default settings; 102 models on 18 mirrored questions, four runs per side