Jiayuan Mao 茅佳源

Jiayuan Mao is a PhD candidate at MIT EECS, advised by Prof. Josh Tenenbaum and Prof. Leslie Kaelbling. Previously, she obtained her Bachelor's degree from YaoClass, Tsinghua University.

I am on the job market 2024-2025. Links: CV | Research Statement

News
Click to show older news ...
Research Highlights

My long-term research goal is to build machines that can continually learn concepts (e.g., properties, relations, skills, rules and algorithms) from their experiences and apply them for reasoning and planning in the physical world. The central theme of my research is to decompose the learning problem into learning a vocabulary of neuro-symbolic concepts. The symbolic part describes their structures and how different concepts can be composed; the neural part handles grounding in perception and physics. I leverage structures to make learning more data-efficient, more compositionally generalizable, and also inference and planning faster.

Neuro-Symbolic Concepts: Representations

How should we represent various types of concepts?

How to capture the programmatic structures underlying these concepts (The Theory-Theory of Concepts)?

Show/Hide Work on Representations

Learning, Reasoning and Planning Theories and Algorithms

How can we efficiently learn these concepts from natural supervisions (e.g., language, videos)?

How can we leverage the structures of these concepts to make inference and planning faster?

Show/Hide Work on Algorithms

Publications ( show selected / show all by date / show all by topic )

Topics: Concept Learning and Language Acquisition / Reasoning and Planning / Scene and Activity Understanding
Past topics: Object Detection / Structured NLP (*/†: indicates equal contribution.)

Keypoint Abstraction using Large Models for Object-Relative Imitation Learning
Xiaolin Fang*, Bo-Ruei Huang*, Jiayuan Mao*, Jasmine Shone, Joshua B. Tenenbaum, Tomás Lozano-Pérez, Leslie Pack Kaelbling

ICRA 2025 Paper / project page / Code
CoRL 2024 Workshop on Language and Robot Learning (Best Paper)

BLADE: Learning Compositional Behaviors from Demonstration and Language
Weiyu Liu*, Neil Nie*, Ruohan Zhang, Jiayuan Mao†, Jiajun Wu

CoRL 2024 Paper / Project Page
CoRL 2024 Workshop on Learning Effective Abstractions for Planning (Oral)

Embodied Agent Interface: A Single Line to Evaluate LLMs for Embodied Decision Making
Manling Li*, Shiyu Zhao*, Qineng Wang*, Kangrui Wang*, Yu Zhou*, Sanjana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, Jiajun Wu

NeurIPS 2024 Datasets and Benchmarks Track (Oral)
SoCal NLP 2024 (Best Paper)
Paper / Project Page / Code / Data

Hybrid Declarative-Imperative Representations for Hybrid Discrete-Continuous Decision-Making
Jiayuan Mao, Joshua B. Tenenbaum, Tomás Lozano-Pérez, Leslie Pack Kaelbling

WAFR 2024 Paper Preprint / Slides

Finding Structure in Logographic Writing with Library Learning
Guangyuan Jiang, Matthias Hofer, Jiayuan Mao, Lio Wong, Joshua B. Tenenbaum, Roger P. Levy

CogSci 2024 (Best Undergraduate Student Paper) Paper

Grounding Language Plans in Demonstrations through Counter-factual Perturbations
Yanwei Wang, Tsun-Hsuan Wang, Jiayuan Mao, Michael Hagenow, Julie Shah

ICLR 2024 (Spotlight) Paper / Project Page / Code / MIT News / TechCrunch

Learning Adaptive Planning Representations with Natural Language Guidance
Lio Wong*, Jiayuan Mao*, Pratyusha Sharma*, Zachary S. Siegel, Jiahai Feng, Noa Korneev, Joshua B. Tenenbaum, Jacob Andreas

ICLR 2024 Paper / Project Page / Code

What Planning Problem Can A Relational Neural Network Solve
Jiayuan Mao, Tomás Lozano-Pérez, Joshua B. Tenenbaum, Leslie Pack Kaelbling

NeurIPS 2023 (Spotlight) Paper / Project Page / Code

What’s Left? Concept Grounding with Logic-Enhanced Foundation Models
Joy Hsu*, Jiayuan Mao*, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2023 Paper / Project Page / Code

Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
Renhao Wang*, Jiayuan Mao*, Joy Hsu, Hang Zhao, Jiajun Wu, Yang Gao

ICLR 2023 (Notable Top 25%) Paper / Project Page

Sparse and Local Hypergraph Reasoning Networks
Guangxuan Xiao, Leslie Pack Kaelbling, Jiajun Wu, Jiayuan Mao

LoG 2022 Paper / Project Page / Code

PDSketch: Integrated Domain Programming, Learning, and Planning
Jiayuan Mao, Tomás Lozano-Pérez, Joshua B. Tenenbaum, Leslie Pack Kaelbling

NeurIPS 2022 Paper / Project Page / Code

Grammar-Based Grounded Lexicon Learning
Jiayuan Mao, Haoyue Shi, Jiajun Wu, Roger P. Levy, Joshua B. Tenenbaum

NeurIPS 2021 Paper / Project Page / Code

Temporal and Object Quantification Networks
Jiayuan Mao*, Zhezheng Luo*, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu, Leslie Pack Kaelbling, Tomer D. Ullman

IJCAI 2021 Paper / Project Page / Code

(First two authors contributed equally; order determined by coin toss.)
Multi-Plane Program Induction with 3D Box Priors
Yikai Li*, Jiayuan Mao*, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu

NeurIPS 2020 Paper / Video / Project Page

Visually Grounded Neural Syntax Acquisition
Haoyue Shi*, Jiayuan Mao*, Kevin Gimpel, Karen Livescu

ACL 2019 (Best Paper Nomination) Paper / Project Page / Code

The Neuro-Symbolic Concept Learner:
Interpreting Scenes, Words, and Sentences From Natural Supervision
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 (Oral) Paper / Project Page / Code / MIT News / MIT Technology Review

Neural Logic Machines
Honghua Dong*, Jiayuan Mao*, Tian Lin, Chong Wang, Lihong Li, Dengyong Zhou

ICLR 2019 Paper / Project Page / Code

Acquisition of Localization Confidence for Accurate Object Detection
Borui Jiang*, Ruixuan Luo*, Jiayuan Mao*, Tete Xiao, Yuning Jiang

ECCV 2018 (Oral) Paper / Code

Universal Agent for Disentangling Environments and Tasks
Jiayuan Mao, Honghua Dong, Joseph J. Lim

ICLR 2018 Paper / Project Page