Overview
Robots operating in domestic and office environments must understand object ownership to correctly execute instructions such as “Bring me my cup.” However, ownership cannot be reliably inferred from visual features alone. To address this gap, we propose Active Ownership Learning (ActOwL), a framework that enables robots to actively generate and ask ownership-related questions to users. ActOwL employs a probabilistic generative model to select questions that maximize information gain, thereby acquiring ownership knowledge to improve learning efficiency. Additionally, by leveraging commonsense knowledge from Large Language Models (LLM), objects are pre-classified as either shared or owned, and only owned objects are targeted for questioning. Through experiments in a simulated home environment and a real-world laboratory setting, ActOwL achieved significantly higher ownership clustering accuracy with fewer questions than baseline methods.
Before actively asking questions, the robot estimates whether each object is likely shared or individually owned using commonsense knowledge from a Large Language Model (LLM). This step reduces unnecessary questions by excluding obviously shared objects.
You are a household assistant robot.
Given the following object name, determine whether the object is
more likely to be a shared object or an individually owned object
in a typical household.
Object: "red mug"
Answer with one of the following labels:
- Shared
- Owned
Provide only the label.
The classification result is treated as a prior (pseudo-answer) in the probabilistic ownership model. Objects classified as shared are excluded from active questioning.
Ownership is modeled as a latent variable inferred from object attributes, spatial information, and user responses. The model maintains uncertainty and is updated online through interaction.
The probabilistic generative model integrates:
After each interaction, the posterior distribution over ownership is updated using Bayesian inference, enabling incremental ownership learning.
To efficiently reduce ownership uncertainty, the robot selects an object that maximizes information gain (IG) and generates a natural-language question using an LLM.
You are a service robot interacting with a user.
Your goal is to determine the owner of an object.
Generate a polite and natural question to ask the user
about the ownership of the following object.
Object: "red mug"
Constraints:
- The question should be short and clear.
- Ask only one question.
- Do not include explanations.
The LLM is used only for language generation, while the selection of the object to ask about is determined by the probabilistic model based on information gain.
To correctly update ownership beliefs, the robot must identify which user is responding to the question. User recognition conditions the probabilistic update of ownership.
You are a robot interacting with multiple users.
Based on the following information, identify the user
who is currently speaking.
Input:
- Detected face ID: User_2
- Voice direction: front
- Current interaction context: ownership question
Output the user ID only.
The identified user ID is used to update the ownership distribution, enabling multi-user ownership learning in shared environments.
@article{hashimoto2026actowl,
author = {Hashimoto, Saki and Hasegawa, Shoichi and Ishikawa, Tomochika and Taniguchi, Akira and Hagiwara, Yoshinobu and El Hafi, Lotfi and Taniguchi, Tadahiro},
title = {Toward Ownership Understanding of Objects: Active Question Generation with Large Language Model and Probabilistic Generative Model},
journal = {Artificial Life and Robotics},
year = {2026},
note = {under review}
}
This work was supported by JSPS KAKENHI Grants-in-Aid for Scientific Research (Grant Numbers JP23K16975, JP25K15292) and JST Moonshot Research & Development Program (Grant Number JPMJMS2011).