Haotong Yang

Haotong Yang

Fourth-year Ph.D student in Peking University

Haotong Yang （杨昊桐）

Welcome to my personal homepage!

Email: haotongyang “at” pku “dot” edu “dot” cn

Biography

Haotong is a fourth-year Ph.D. student in the School of Intelligence Science and Technology and the Institute for Artifical Intelligence at Peking University. He is a member of ZeroLab, led by Prof. Zhouchen Lin, and GraphPKU, led by Prof. Muhan Zhang. His research focuses on Large Language Models (LLMs) particularly on enhancing their reasoning ability of LLMs, explanation and interpretation their mechanisms, and exploring multi-modal LLMs. Prior to his graduate studies, Haotong receive his bachelor degree from the School of Mathematical Science at Peking University, where he worked with Prof. Zhanxing Zhu.

杨昊桐，北京大学智能学院和人工智能研究院四年级在读博士研究生，师从林宙辰教授和张牧涵助理教授，目前主要从事和大语言模型推理、可解释性和多模态大模型相关的研究。在ICLR、ICML、NeurIPS等学术会议上发表多篇文章，并多次担任审稿人。其中一作文章 Rethinking knowledge graph evaluation under the open-world assumption 获评NeurIPS 2022口头报告。本科毕业于北京大学数学科学学院。曾多次获得北京大学三好学生和校级奖学金，并于2021年荣获北京市优秀毕业生。在学术生活之外，他还热心学生工作和社会服务，研究生期间两次担任本科带班辅导员，曾在建国70周年和建党百年纪念活动中担任志愿者。

Research Interest

Reasoning of LLMs My research focuses on understanding and improving the reasoning mechanisms of Large Language Models (LLMs). Specifically, I investigate how LLMs acquire world knowledge (or basic knowledge), learn concepts and task solutions, and solve complex language reasoning tasks. This involves exploring (1) the mechanisms of attention, positional embedding, tokenization, and their relationship with various reasoning tasks, (2) different types of chain-of-thought reasoning, and (3) the analysis of typical reasoning tasks.

Multi-modal Reasoning I am also interested in extending the reasoning capabilities of LLMs to multi-modal scenarios. This includes integrating graph learning (via GNNs) with LLMs’ powerful factual knowledge and reasoning abilities, as well as exploring vision-language models’ reasoning pipelines, and so on.

Publications

  1. [ICLR-25] H. Yang, Y. Hu, S. Kang, Z. Lin, and M. Zhang: Number Cookbook: Number Understanding of Language Models and How to Improve It. The Thirteenth International Conference on Learning Representations (ICLR-2025), 2024. (PDF)(Presentation in English)(Presentation in Chinese)(WeChat article)(GitHub Page)
  2. [ICML-24] Y. Hu, X. Tang, Y. Yang, and M. Zhang: Case-based or rule-based: how do transformers do the math? The Forty-First International Conference on Machine Learning (ICML-2024), 2024. (PDF)
  3. [ICLR-24] X. Wang, H.Yang, Z. Lin, and M. Zhang: Neural common neighbor with completion for link prediction. The Twelfth International Conference on Learning Representations (ICLR-2024), 2023. (pdf)
  4. [NeurIPS-22] H. Yang, Z. Lin, and M. Zhang: Rethinking knowledge graph evaluation under the open-world assumption. The Thirty-Sixth Annual Conference on Neural Information Processing Systems (NeurIPS-2022), 2022. (PDF)(report) oral presentation (1.7% acceptance rate)

Preprint or Under Review

  1. H. Yang*, Q. Zheng*, Y. Gao*, Y. Yang, Y. He, Z. Lin, and M. Zhang: VACT: A Video Automatic Causal Testing System and a Benchmark (arxiv)
  2. Y. Hu*, S. Kang*, H. Yang, H. Xu, and M. Zhang: Training Large Language Models to be Better Rule Followers (arxiv)
  3. H. Yang*, Xiyuan Wang*, Q. Tao, S. Hu, Z. Lin, M. Zhang: GL-Fusion: Rethinking the Combination of Graph Neural Network and Large Language model (arxiv)
  4. H. Yang, F. Meng, Z. Lin, and M. Zhang: Parrot Mind: Towards Explaining the Complex Task Reasoning of Pretrained Large Language Models with Template-Content Structure (arxiv)
  5. Y. Hu, H. Yang, Z. Lin and M. Zhang: Code Prompting: a Neural Symbolic Method for Complex Reasoning in Large Language Models (arxiv)