I tested GPT for political, gender, and racial bias across 8 datasets. Full data is inside
I run a small AI ethics nonprofit, and over the past few months, I've tested the world's frontier models (GPT-5.4, Claude Sonnet 4.6, Claude Opus 4.7, Gemini Pro, and Gemini Flash) for bias using around 20,600 examples from many different datasets, revolving around political, gender, and racial bias. Datasets include WinoBias, BBQ, SeeGULL, OpinionsQA, cajcodes, Political Compass, and a custom evidence-refusal pilot I built myself.
Every single frontier model, GPT, Claude Opus, and Gemini leaned left in every single political dataset. Models classified things as more left-leaning than professional humans did, and they rated themselves as very left leaning as well.
However, these models diverged in interesting places. GPT refused to answer race-related questions 20.3% of the time, even when the scenario presented disambiguating context where race was supposed to be mentioned. Claude Sonnet refused to answer these questions just 5% of the time, showing a vast difference.
When testing on the WinoBias dataset for racial bias, every single model answered questions more accurately when the sentence abided by stereotypes. GPT 5.4 showed a 15.4% accuracy gap between stereotype-aligned and anti-stereotype questions. The full breakdown with graphs and all models can be found here:
https://www.civicsparklearning.org/ai-nonprofit-dashboard