Google's Agile Deliberation Teaches AI to Classify Fuzzy Subjective Images

Image classifiers handle visually concrete categories such as stop signs more easily than concepts such as inspiring art, healthy food, or clickbait. Google Research’s Agile Deliberation framework helps people define those subjective categories through repeated interaction with a vision-language model. The team released the open-source code and published the CVPR 2026 paper .

Subjective labels often conceal unresolved policy choices. A person may call one smoothie healthy and reject another based on added sugar, processing, portion size, or ingredients without having written down those criteria. Agile Deliberation surfaces such edge cases, asks the user to resolve them, and incorporates the answers into the prompt that drives the classifier.

Why fuzzy labels drift

Conventional labeling workflows assume annotators begin with a clear and stable category definition. That assumption works for many concrete objects. Categories shaped by culture, personal judgment, editorial standards, or changing policy develop as people encounter examples. Unwritten exceptions then produce inconsistent labels, leaving the classifier to learn contradictions that its creators never intended.

Agile Deliberation turns category design into a structured feedback loop. The human defines the policy, while the vision-language model searches for examples near its current boundary. Those examples reveal missing rules early enough for the user to revise the concept specification and classifier prompt.

Two passes expose the boundary

A “Healthy Food” specification might include salads, grain bowls, and lean proteins while excluding heavily processed foods. A fruit smoothie then tests the boundary: the user must decide how raw fruit, added sugar, processing, and portion size affect the label. Each decision makes the policy more explicit.

The iteration stage draws on active learning, a technique in which a model selects informative examples for human review. Agile Deliberation selects examples for their ability to expose ambiguity in the user’s definition. The workflow updates the prompt used by the vision-language classifier as that definition develops.

An evaluation built around consistency

The researchers evaluated the framework in 18 live sessions lasting 90 minutes each. Participants worked in two domains: “Healthy Food” and “Pay-to-Play,” a moderation category covering misleading clickbait in gaming and entertainment. Because neither domain has a universal answer key, the study measured alignment with each participant’s evolving definition and consistency over time.

The study compared Agile Deliberation with three workflows:

1. Zero-shot classification: users wrote one prompt without supplying task-specific examples.
2. Modeling Collaborator: an AI agent decomposed the concept without user involvement.
3. Manual deliberation: users managed examples in spreadsheets and repeatedly rewrote prompts.

The authors report that structured deliberation produced more consistent labels over time and classifiers that outperformed the three baselines. The largest gains appeared on highly subjective images, where generic model assumptions were less likely to match a participant’s stated intent.

Expert participants also used the system diagnostically. Borderline examples exposed omissions and contradictions in their initial definitions, allowing them to revise the policy before those gaps spread through a larger labeled dataset.

Guided search reduces cognitive load

Manual prompt engineering required participants to invent search terms that might uncover useful edge cases. They also worried that fixing one classification error could degrade behavior elsewhere. Participants using Agile Deliberation reported less stress, frustration, fatigue, and mental effort, and every participant preferred it to the manual workflow.

From notebook to prototype

The repository includes an interactive Colab notebook based on the “Healthy Food” example. Developers can adapt the workflow to another visual domain through a five-step process:

1. Write an initial description of the target concept.
2. Divide it into included subcategories and explicit exclusions.
3. Use the VLM to surface images near the current semantic boundary.
4. Resolve those examples and feed the decisions into prompt optimization.
5. Evaluate the revised classifier against held-out examples and known policy cases.

Promising applications include content moderation filters with detailed policy rules, curated collections shaped by editorial judgment, scientific image taxonomies still under development, and personalized classifiers for niche concepts such as domain-specific clickbait.

Where the evidence stops

The evaluation covers two concepts, 18 sessions, and short-term interactions. Production-scale behavior, long-term policy maintenance, and disagreements among multiple policy owners remain outside that scope. Consistency with one person’s definition also cannot establish fairness, safety, or agreement across a broader community.

Production teams still need representative test sets, regression checks after prompt or model changes, versioned policy definitions, and monitoring for distribution drift. The underlying VLM can also bring its own biases and semantic blind spots into the examples it selects.

Agile Deliberation assigns the model two concrete jobs: propose cases near the current boundary and translate the user’s resolutions into a revised classifier prompt. The human retains responsibility for the category definition. For teams dealing with underspecified policies, the framework offers a structured path from a vague concept to a testable visual classifier.