Chuanyang Jin

We introduce ThoughtTrace, the first large-scale dataset that pairs real-world multi-turn human-AI conversations with users' self-reported thoughts. Our experiments show that these thoughts are hard for frontier LLMs to infer, and provide actionable signals for user-behavior prediction and model alignment, opening new directions for user modeling, model training, and evaluation.

We introduce AutoInteract, which learns user models from real interaction logs and turns verified tasks into realistic multi-turn interactions with clarifications, changing requirements, and feedback, while keeping each task’s executable test. Reinforcement learning on this data substantially improves coding agents, especially in multi-turn settings.

We posit that to achieve continual model improvement and multifaceted alignment, future models must learn from natural human interaction. We introduce Reinforcement Learning from Human Interaction (RLHI), a paradigm that learns directly from in-the-wild user conversations, leveraging organic replies and long-term history as learning signals. Trained on WildChat, RLHI outperforms RLHF in personalization and instruction-following, and similar feedback enhances performance on reasoning benchmarks.

SPICE is a reinforcement learning framework where a single model improves itself by playing two roles: a Challenger that creates tasks based on corpora, and a Reasoner that solves them. By grounding this self-play in corpora, SPICE addresses hallucination and lack of diversity issues, significantly outperforming standard (ungrounded) self-play across reasoning benchmarks.

We introduce MindZero, a self-supervised reinforcement learning framework that trains multimodal LLMs for efficient and robust online Theory-of-Mind reasoning. During training, the model is rewarded for generating mental state hypotheses that best explain observed human actions. Across four experimental settings, MindZero significantly outperforms LLMs and model-based methods in accuracy and efficiency.

AutoToM is an automated agent modeling method for scalable, robust, and interpretable mental inference. Given a ToM problem, AutoToM proposes a minimal agent model, performs Bayesian inverse planning based on the model, and iteratively refines the model until it can confidently infer the target mental state. It achieves SOTA on five benchmarks, produces human-like confidence estimates, and supports embodied decision-making.

We introduce OS-Genesis, a manual-free data pipeline for synthesizing GUI agent trajectory. It enables agents to actively explore web and mobile environments through stepwise interactions, then derive meaningful low- and high-level task instructions from observed interactions and state changes.

MuMA-ToM evaluates Theory of Mind reasoning in embodied multi-agent interactions, revealing that current multimodal LLMs significantly lag behind human performance. To bridge this gap, we propose LIMP, a method that combines language models with inverse multi-agent planning to achieve superior results.

Can machines understand people's minds from multimodal inputs? We introduce a comprehensive benchmark, MMToM-QA, and highlight key limitations in current multimodal LLMs. We then propose a novel method that combines the flexibility of LLMs with the robustness of Bayesian inverse planning, achieving promising results.