From Passive Perception to Active Interaction

§ 01 · Abstract

One representation, three roles

Embodied AI aims to develop agents capable of perceiving, reasoning, and acting in physical environments. Beyond recognizing objects and scenes, these agents must identify possible interactions, determine which are relevant to the current task and embodiment, and translate them into executable behavior. Affordance learning supports this process by grounding perception in action possibilities and connecting it with task reasoning and control. Existing surveys, however, have largely adopted either a perception-centered or robotics-centered perspective, leaving how affordance perception, reasoning, and action jointly contribute to embodied intelligence underexplored.

To bridge this gap, this survey presents a unified framework that organizes the literature around three complementary roles: affordance perception identifies what and where interaction is possible; affordance reasoning selects opportunities relevant to the task, context, and agent state; and affordance-guided action uses affordance representations to support planning and policy learning. Within this framework, we develop fine-grained taxonomies, systematically compare representative methods, and trace the field's progression from passive visual recognition toward action-aware embodied learning. We further organize representative datasets and evaluation protocols, and outline open challenges and future directions.

I

Affordance Perception

“What — and where — is interaction possible?”

Detects, segments, and grounds action possibilities as masks, heatmaps, keypoints, and 3D regions.

78 papers → II

Affordance Reasoning

“Which affordance matters for this task, context, and agent state?”

Selects task-relevant opportunities through relations, language, and agentic feasibility checks.

54 papers → III

Affordance-Guided Action

“How does affordance become executable behavior?”

Turns grounded affordances into primitives, plans, and policies — explicit, implicit, or as reward.

61 papers →

§ 02 · Unified framework

What, where, and how

The survey formalizes affordance learning as one mapping whose outputs answer the three questions an embodied agent must ask before it acts.

𝓕 = f_θ( o, l, c ) = ( a, r, η ) o observation · l language / task · c conditioning evidence

a

What

the action that is possible — grasp, open, pour, push

r

Where

the grounded region — part, mask, heatmap, keypoint, 3D location

η

How

the execution specification — pose, direction, score, post-contact trajectory

Overview taxonomy figure: affordance perception, affordance reasoning, and affordance-guided action, each expanded into its families and leaf categories. — Fig. 01 · Plate I — Taxonomy overview Three roles, eighteen leaves The literature organized around perception, reasoning, and action — from identifying actionable possibilities to reasoning about task relevance and translating them into embodied behavior.

§ 03 · Evolution

Five paradigm shifts

Affordance learning has moved from supervised visual prediction of action regions to action-grounded embodied learning. Five shifts carry that arc.

01dense labelsdiverse evidence
Image-level labels, sparse points, human–object interactions, model priors, and physical simulation complement dense masks; reliability depends on evidence quality and simulation fidelity.
022D image plane3D metric space
From view-dependent heatmaps to metric grounding on point clouds and Gaussian splats.
03static regionsactionable dynamics
Beyond masks: contact points, motion directions, poses, and post-contact trajectories.
04closed-set recognitionopen-set generalization
Fixed vocabularies dissolve into language queries and foundation-model grounding.
05modular pipelinesintegrated policies
Affordance moves inside diffusion and VLA policies — as condition, representation, or reward.

Timeline of representative advances in affordance learning from 2016 to 2026, grouped into perception, reasoning, and action families. — Fig. 02 · Plate II — Temporal evolution Representative advances, 2016 → 2026 Temporal evolution of representative advances in affordance learning over the past decade (2016–2026).

Research trends through July 2026: 2D and 3D affordance grounding, open-set methods, and cumulative publications across perception, reasoning, simulation execution, and real-robot execution. — Fig. 03 · Plate III — Research trends Perception, reasoning, and execution through July 2026 Research trends in affordance learning through July 2026. (a) and (b) analyze affordance grounding across perception and reasoning, showing the transition from 2D to 3D and the growth of open-set methods, respectively. (c) shows the cumulative publications and their distribution across perception, reasoning, simulation execution, and real-robot execution.

§ 04 · Atlas index

Explore the corpus

Every method and dataset in the survey, indexed by the taxonomy. Filter by role, leaf, year, venue, or modality — press / to search. Each entry comes with a one-line takeaway.

The interactive index needs JavaScript. The same corpus lives in the Awesome-Affordance-Learning list.

§ 05 · Comparisons

Method & dataset tables

All four comparison tables from the paper, sortable in place. Dots mark input modalities and output forms; venues follow the paper source.

§ 06 · Open problems

Challenges & future directions

Five research directions toward scalable, reliable, and actionable affordance learning for open-world embodied interaction.

7.1
Affordance representation and action integration

Explicit masks, contacts, and trajectories offer interpretable action interfaces; latent features, auxiliary objectives, and optimization signals support other forms of policy integration. Controlled comparisons should match observations, training data, policy backbones, and action interfaces, test no-affordance, predicted-affordance, and oracle-affordance conditions, and use interventions to establish whether affordance information improves execution.
7.2
Scalable learning and open-world generalization

Human interactions, demonstrations, model-generated labels, simulation, and robot experience broaden supervision, but each introduces uncertainty and bias. Future data pipelines should control quality while varying objects, tasks, states, and environments. Affordance-guided data generation and predictive models can scale interaction examples while preserving transferable functional relations.
7.3
Reliable and autonomous affordance grounding

Multi-stage grounding can propagate early errors and depend heavily on prompts, thresholds, tool choices, and fallback rules. Future systems should select evidence and tools adaptively, track provenance and uncertainty, verify semantic and spatial consistency, and retry when needed. Evaluation should report human corrections, intermediate failures, tool calls, latency, cost, and sensitivity to component replacement.
7.4
Closed-loop affordance modeling and planning

Affordances change as actions alter object states, visibility, and scene configuration. Agents should maintain and update affordance beliefs through observation, action, and execution feedback. Active sensing and low-risk exploration can resolve uncertainty; long-horizon planning should model affordance transitions, track progress, revise later choices, and recover from unexpected outcomes.
7.5
Embodied evaluation, safety, and trustworthiness

Perceptual agreement does not establish whether an interaction is reachable, stable, or compatible with a particular robot. Benchmarks should measure embodiment-specific feasibility, execution success, and uncertainty calibration, using validated simulation alongside robot trials. Evaluation should also test complementary sensory evidence, unsafe and negative affordances, and whether actions respect human intent and social norms.

§ 07 · Cite

BibTeX

Preprint v1 posted on Preprints.org, 7 October 2026 — the entry will be updated on publication.

@article{li2026affordance,
  title   = {From Passive Perception to Active Interaction: A Survey
             of Affordance Learning for Embodied AI},
  author  = {Li, Gen and Wang, Hanqing and Li, Jingliang and Han, Yifan
             and Jia, Jindou and Lin, Tao and Liu, Yuanzhe and Wang, Yutong
             and Zhao, Bo
             and Ding, Fangqiang and Nguyen, Anh and Sevilla-Lara, Laura
             and Xu, Huazhe and Chirikjian, Gregory S. and Pollefeys, Marc
             and Mees, Oier and Xiong, Hui and Yang, Jianfei},
  journal = {Preprints},
  year    = {2026},
  doi     = {10.20944/preprints202610.0392.v1},
  url     = {https://www.preprints.org/manuscript/202610.0392}
}