[ Visual normative reasoning / NeurIPS 2026 ]

NoRAEvaluating Grounded Reasonableness in
Visual First-person Normative Action Reasoning

Sichao Li · Sai Ma · Daniel Kilov · Secil Yanik Guyot · Zhuang Li · Seth Lazar

Read the paper Dataset Code

A plausible action is only part of the answer.
NoRA asks whether it follows from the right facts and reasons.

01 / A decision in context

What should I do next, and why?

First-person video · lawn mowing

Five consecutive first-person frames: a person walks across grass while holding the handle of a lawn mower with a rear collection bag. — Pre-action observationFive frames from one clip · EgoNormia source media

FVisible facts

F3 The mower has a rear collection bag attached.

F7 I am already mid-pass with the lawnmower rather than setting it up or putting it away.

RSupporting reason

The fact that the mower has a collection bag and mowing is ongoing gives me reason to continue mowing in an orderly pass, because that supports completing the yard work efficiently.

R3 · supported by F3, F7

AReasonable action

Continue pushing the mower forward while keeping a straight, controlled line across the grass.

A1 · supported by R3

One support path at a time, excerpted verbatim from the reviewed annotation. Other facts, reasons, and links are omitted here for clarity. These are candidate next actions, not a prediction of what the wearer actually did.

The idea

Beyond choosing
a plausible action.

In everyday situations, an agent is not handed a menu of possible actions. It has to notice what matters, identify reasonable next steps, and explain why those steps fit the situation.

NoRA turns this into an open-ended evaluation task. Models propose actions from first-person visual context, linking each action to supporting reasons and each reason to scene facts. Multiple actions can be reasonable; the benchmark evaluates both the candidate set and its support.

Read the full formulation in the paper

02 / The benchmark

Everyday scenes.
Inspectable support.

Built on EgoNormia clips, with separate human-annotated evaluation and model-annotated training splits.

HumanGold Test

190

Human-annotated clips

The primary evaluation reference. Each example connects descriptive facts, normative reasons, and candidate actions.

LLMSilver Train

1,230

Model-annotated clips

A larger, separately identified annotation tier. It complements the benchmark without replacing human review.

Resources. Dataset annotations are hosted on Hugging Face; evaluation code and usage guides are maintained on GitHub.

03 / Evaluation

Three questions.
One grounded assessment.

A good action, the relevant facts, and the right support must work together.

Action alignment

Does the model recover reasonable candidate next actions?

action_f1

Factual grounding

Does it identify the relevant scene facts in the reference support graphs?

fact_f1

Support binding

Are facts and reasons connected to the appropriate actions?

reasoning_f1

R = 3√A · F · S

Grounded reasonableness combines the available components with a geometric mean per clip. Reported scores average across valid clips, using soft or one-to-one Hungarian alignment.

What this score means. Agreement with reviewed references, not universal moral correctness or a safety guarantee. Invalid and missing predictions are reported separately. Model comparisons use the same valid clips and must include coverage.

From the paper

Seeing the scene
is not the whole task.

Actions need their own support.

A relevant fact or plausible action is not enough when the reason is attached to the wrong next step.

More structure is not a universal fix.

Deliberate and structured prompting show model-specific trade-offs, rather than one consistently winning recipe.

The candidate set matters.

Selecting one defensible action and recovering the broader space of reasonable actions are distinct evaluation targets.

See the paper's experiments and results

04 / Use NoRA

Bring your model.

Use pre-action frames or video, connect a model callback or compatible API, or evaluate saved predictions.

Score annotation-format predictions

uv sync --locked --extra scorer
uv run nora validate predictions.jsonl
uv run --extra scorer nora evaluate \
  --predictions predictions.jsonl \
  --output runs/my-model

Start with the provided visual inputs and prompts, generate model responses, then reconstruct and score them. The package includes test references.

Prediction-to-score walkthrough

Validation and scoring do not make paid API calls. Inference and text reconstruction may incur provider fees.

Reference

Cite NoRA

@inproceedings{li2026nora,
  title = {NoRA: Evaluating Grounded Reasonableness in Visual
           First-person Normative Action Reasoning},
  author = {Li, Sichao and Ma, Sai and Kilov, Daniel and
            Yanik Guyot, Secil and Li, Zhuang and Lazar, Seth},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2026},
  note = {Accepted to the NeurIPS 2026 Evaluations and Datasets Track},
  url = {https://arxiv.org/abs/2606.04806}
}