Jiayuan Mao 茅佳源

I am a third-year PhD student at MIT EECS, advised by Prof. Josh Tenenbaum and Prof. Leslie Kaelbling. Previously, I obtained my bachelor's degree from YaoClass, Tsinghua University.

News
Research Highlights

My long-term research goal is to build machines that can continually learn new knowledge from their experiences and apply the knowledge in their reasoning and planning in the physical world.

  • Concept Learning & Language Acquisition: neuro-symbolic concept learning, grammar-based language learning, pragmatics.
  • Reasoning & Planning: relational and temporal models for reasoning, skill learning and planning.
  • Scene & Activity Understanding: programmatic scene understanding, activity understanding.
Publications ( show selected / show all by date / show all by topic )

Topics: Concept Learning and Language Acquisition / Reasoning and Planning / Scene and Activity Understanding / Others: Vision / Others: NLP (*: indicates equal contribution.)

Grammar-Based Grounded Lexicon Learning
Jiayuan Mao, Haoyue Shi, Jiajun Wu, Roger P. Levy, Joshua B. Tenenbaum

NeurIPS 2021 Paper

Temporal and Object Quantification Networks
Jiayuan Mao*, Zhezheng Luo*, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu, Leslie Pack Kaelbling, Tomer D. Ullman

IJCAI 2021 Paper / Project Page / Code

(First two authors contributed equally; order determined by coin toss.)
Language-Mediated, Object-Centric Representation Learning
Ruocheng Wang*, Jiayuan Mao*, Samuel J. Gershman, Jiajun Wu

ACL 2021 (Findings) Paper / Talk / Project Page

Hierarchical Motion Understanding via Motion Programs
Sumith Kulal*, Jiayuan Mao*, Alex Aiken, Jiajun Wu

CVPR 2021 Paper / Talk / Project Page

Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning
Zhenfang Chen, Jiayuan Mao, Jiajun Wu, Kwan-Yee K. Wong, Joshua B. Tenenbaum, Chuang Gan

ICLR 2021 Paper / Project Page

Object-Centric Diagnosis of Visual Reasoning
Jianwei Yang, Jiayuan Mao, Jiajun Wu, Devi Parikh, David D. Cox, Joshua B. Tenenbaum, Chuang Gan

ArXiv 2020 Paper

Multi-Plane Program Induction with 3D Box Priors
Yikai Li*, Jiayuan Mao*, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu

NeurIPS 2020 Paper / Video / Project Page

Perspective Plane Program Induction from a Single Image
Yikai Li*, Jiayuan Mao*, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

CVPR 2020 Paper / Project Page

Visual Concept-Metaconcept Learning
Chi Han*, Jiayuan Mao*, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2019 Paper / Project Page / Code

Program-Guided Image Manipulators
Jiayuan Mao*, Xiuming Zhang*, Yikai Li, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICCV 2019 Paper / Project Page

(First two authors contributed equally; order determined by coin toss.)
Visually Grounded Neural Syntax Acquisition
Haoyue Shi*, Jiayuan Mao*, Kevin Gimpel, Karen Livescu

ACL 2019 (Best Paper Nomination) Paper / Project Page / Code

Neurally-Guided Structure Inference
Sidi Lu*, Jiayuan Mao*, Joshua B. Tenenbaum, Jiajun Wu

ICML 2019 Paper / Project Page / Code

Unified Visual-Semantic Embeddings:
Bridging Vision and Language with Structured Meaning Representations
Hao Wu*, Jiayuan Mao*, Yufeng Zhang, Weiwei Sun, Yuning Jiang, Lei Li, Wei-Ying Ma

CVPR 2019 (Oral) Paper

The Neuro-Symbolic Concept Learner:
Interpreting Scenes, Words, and Sentences From Natural Supervision
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 (Oral) Paper / Project Page / Code / MIT News / MIT Technology Review

Neural Logic Machines
Honghua Dong*, Jiayuan Mao*, Tian Lin, Chong Wang, Lihong Li, Dengyong Zhou

ICLR 2019 Paper / Project Page / Code

Neural Phrase-to-Phrase Machine Translation
Jiangtao Feng, Lingpeng Kong, Po-Sen Huang, Chong Wang, Da Huang, Jiayuan Mao, Kan Qiao, Dengyong Zhou

ArXiv Preprint Paper

Acquisition of Localization Confidence for Accurate Object Detection
Borui Jiang*, Ruixuan Luo*, Jiayuan Mao*, Tete Xiao, Yuning Jiang

ECCV 2018 (Oral) Paper / Code

Learning Visually-Grounded Sementics from Contrastive Adversarial Samples
Haoyue Shi*, Jiayuan Mao*, Tete Xiao*, Yuning Jiang, Jian Sun

COLING 2018 Paper / Code

Universal Agent for Disentangling Environments and Tasks
Jiayuan Mao, Honghua Dong, Joseph J. Lim

ICLR 2018 Paper / Project Page

What Can Help Pedestrian Detection?
Jiayuan Mao*, Tete Xiao*, Yuning Jiang, Zhimin Cao

CVPR 2017 Paper