The BaseResearcharXiv
BDBBQ: A Human-AI Collaborative Benchmark for Bengali Social Bias in LLMs
According to the abstract, Provat Saha and colleagues build BDBBQ, a bilingual benchmark of 61,312 question-answering instances across 12 social categories to measure bias in language models in Bengali and English. Eight models were evaluated; the abstract reports lower accuracy in Bengali for all of them.
Why it matters
The same thing, for different readers.
What it means for you
Saha and colleagues report that the eight language models they tested were less accurate in Bengali than in English on questions about social bias.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A bilingual extension of BanglaBBQ with 61,312 question-answering instances across 12 social categories; validation by structural checks, 220 statistical tests and expert annotation of 150 instances; eight LLMs evaluated on 490,496 instances.
- What it found
- Disambiguated contexts raise accuracy by 36.59 percentage points in Bengali versus 17.71 in English; all models are less accurate in Bengali; Mistral-Nemo shows the smallest gap, 5.12 points.
- What it does not show
- Not stated in the excerpt available.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- How is AI changing childhood, health and daily life?
- People
- Provat Saha (Wrote it) · Ashika Habib Khan (Wrote it) · Tahseen Tanha (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of November 10, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “BDBBQ: A Human-AI Collaborative Benchmark for Bengali Social Bias in LLMs”, November 9, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-11-09-bdbbq-bengali-social-bias
Our notesLight lane: abstract only, cut off after two of five stated findings.