Yuhang Yao | PhD@CMU

Building Super-Intelligent Agents

I am an Agent Research Engineer at Zoom, working on enterprise virtual agents with multi-step planning, NL2SQL, and orchestration. My work focuses on agent systems, post-training, model routing, and efficient inference.

Previously, I was a founding research scientist at TensorOpera AI, where I helped build a production LLM agent system from 0 to 1M DAU and worked on Router, ScaleLLM, and Fox-1. I earned my PhD in Electrical and Computer Engineering from Carnegie Mellon University in 2024, advised by Prof. Carlee Joe-Wong, and completed my BS at Shanghai Jiao Tong University in 2019.

Open to collaboration on reliable agents, post-training loops, eval-driven iteration, and efficient model systems. For research, product, or hiring conversations, feel free to reach out via LinkedIn or email.

Recent highlights

2026 - Agent systems in production

At Zoom, I work on enterprise agents with multi-step planning, NL2SQL, and orchestration for real customer workflows in Zoom Virtual Agent.

2024-2025 - 0 to 1M DAU at TensorOpera

Built production LLM agent system CoCo (0 to 1M DAU), designed hybrid routing for super agents (paper), and developed multi-model serving components (ScaleLLM, Router) plus on-device models (Fox-1-1.6B, paper).

2024-2026 - Research impact and recognition

Google Scholar: 982 citations, h-index 11, i10-index 13. Top-cited papers include LLM multi-agent systems: Challenges and open problems (269), FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System (163), and FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks (126).

News

Nov 10, 2025 Graph based Agent Planning, GAP are accepted by NeurIPS workshop Nora.
Nov 01, 2025 FedGraph and FedLink are accepted by NeurIPS workshop NPGML.
Aug 04, 2025 Gave an 1-min talk at Berkeley Agentic AI Summit! Finished 10K (6.2 miles) at San Francisco Marathon!
May 09, 2025 Got the A.G. Milnes Award 2025 (Best PhD Thesis Award at CMU ECE Department)!!
May 07, 2025 Check my interview from MIT Technology Review and DeepTech Wechat Blog.

Selected Publications

  1. super_agent.png
    Toward super agent system with hybrid ai routers
    Yuhang Yao, Haixin Wang , Yibo Chen , and 5 more authors
    arXiv preprint arXiv:2504.10519, 2025
    PDF Video Blog Slides Website
  2. router.png
    Tensoropera router: A multi-model router for efficient llm inference
    Dimitris Stripelis , Zijian Hu , Jipeng Zhang , and 6 more authors
    EMNLP (Industry Track), 2024
    PDF Slides
  3. scalellm.png
    Scalellm: A resource-frugal LLM serving framework by optimizing end-to-end efficiency
    Yuhang Yao, Han Jin , Alay Dilipbhai Shah , and 7 more authors
    EMNLP (Industry Track), 2024
    PDF Slides
  4. fox.png
    Fox-1: Open Small Language Model for Cloud and Edge
    Zijian Hu , Jipeng Zhang , Rui Pan , and 8 more authors
    arXiv preprint arXiv:2411.05281, 2024
    PDF Blog Website
  5. fedgraph.png
    FedGraph: A Research Library and Benchmark for Federated Graph Learning
    Yuhang Yao, Yuan Li , Xinyi Fan , and 7 more authors
    arXiv preprint arXiv:2410.06340, 2024
    PDF Code
  6. fedgat.png
    FedGAT: A Privacy-Preserving Federated Approximation Algorithm for Graph Attention Networks
    Siddharth Ambekar , Yuhang Yao, Ryan Li , and 1 more author
    arXiv preprint arXiv:2412.16144, 2024
    PDF Code
  7. fedllm.png
    Federated large language models: Current progress and future directions
    Yuhang Yao, Jianyi Zhang , Junda Wu , and 8 more authors
    arXiv preprint arXiv:2409.15723, 2024
    PDF
  8. fedgcn.png
    FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks
    Yuhang Yao, Weizhao Jin , Srivatsan Ravi , and 1 more author
    Advances in Neural Information Processing Systems, 2024
    PDF Code Slides
  9. wyzerule.png
    Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking
    Mohammad Mahdi Kamani , Yuhang Yao, Hanjia Lyu , and 5 more authors
    Advances in Neural Information Processing Systems, 2024
    PDF Code Slides Website
  10. fedsecurity.png
    Fedmlsecurity: A benchmark for attacks and defenses in federated learning and llms
    Shanshan Han , Baturalp Buyukates , Zijian Hu , and 8 more authors
    Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024
    PDF Code
  11. multi_agent.png
    LLM multi-agent systems: Challenges and open problems
    Shanshan Han , Qifan Zhang , Yuhang Yao, and 2 more authors
    arXiv preprint arXiv:2402.03578, 2024
    PDF
  12. thesis.png
    Efficient Federated Graph Learning: Formulation, Algorithms and Applications
    Yuhang Yao
    Carnegie Mellon University , 2024
    🏆 A.G. Milnes Award (Best Thesis Award)
    PDF
  13. fedrule.png
    FedRule: Federated Rule Recommendation System with Graph Neural Networks
    Yuhang Yao, Mohammad Mahdi Kamani , Zhongwei Cheng , and 3 more authors
    In Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation , 2023
    PDF Code Slides
  14. fedhe.png
    FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System
    Weizhao Jin , Yuhang Yao, Shanshan Han , and 4 more authors
    FL-NeurIPS Workshop, 2023
    PDF Code
  15. neuralmab.png
    A neural-based bandit approach to mobile crowdsourcing
    Shouxu Lin , Yuhang Yao, Pei Zhang , and 2 more authors
    In Proceedings of the 23rd Annual International Workshop on Mobile Computing Systems and Applications , 2022
    PDF
  16. gcnse.png
    Gcn-se: Attention as explainability for node classification in dynamic graphs
    Yucai Fan , Yuhang Yao, and Carlee Joe-Wong
    In 2021 IEEE International Conference on Data Mining (ICDM) , 2021
    PDF Code
  17. rnngcn.png
    Interpretable clustering on dynamic graphs with recurrent graph neural networks
    Yuhang Yao, and Carlee Joe-Wong
    In Proceedings of the AAAI Conference on Artificial Intelligence , 2021
    PDF Code