The BaseResearcharXiv

Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models

According to the abstract, Xingqi Wang, Xiaoyuan Yi, Xing Xie and Jia Jia present SPY-Bench, with 32,700 samples, to test whether vision-language models respect privacy limits that users specify in words. Across 21 models, most performed poorly; the best-performing o4-mini reached about 24% compliance accuracy.

Open the source ↗

Why it matters

The same thing, for different readers.

What it means for you

Telling an AI that sees your photos what to keep private often does not work, according to this benchmark.

For researchers

SPY-Bench offers 32,700 image-question samples across 67 categories and 24 scenarios, with metrics for user specification and context awareness.

How the finding was reached

In our words, from the source. Weigh the finding by how it was made.

What was done
A benchmark of 32,700 image-question pairs with personalized privacy instructions across 67 categories and 24 real-world scenarios, applied to 21 vision-language models.
What it found
Most models, even the top-performing o4-mini, performed poorly, with about 24% compliance accuracy, and showed limited contextual privacy understanding.
What it does not show
Not given in the excerpt; the authors also describe initial improvement methods that were cut off.
Status
A preprint: not yet peer reviewed

The record

Questions
Are AI companies putting stronger safeguards into practice? · How is AI changing childhood, health and daily life?
People
Xingqi Wang (Wrote it) · Xiaoyuan Yi (Wrote it) · Xing Xie (Wrote it) · Jia Jia (Wrote it)
What we read
We have read only the description the publisher gives. No quotations until someone reads the source itself.
Found by
The Radar, run of November 11, 2026, via arxiv-ai-safety-query (tier 1: the source’s own channel or an official record)
Cite this
arXiv, “Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models”, November 10, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-11-10-spy-bench-personalized-privacy-lvlms
Our notesLight lane: triage excerpt only, cut off where the authors describe their improvement attempts.