Xinjie Liu

Ph.D. Student · UT Austin · Robot Learning

xinjie.jpeg

Hello and welcome! I am a fourth-year Ph.D. student in Electrical and Computer Engineering at The University of Texas at Austin. I am co-advised by Ufuk Topcu and David Fridovich-Keil. I obtained a master’s degree in Robotics with the highest distinction (cum laude) from the Delft University of Technology, the Netherlands, where I was advised by Javier Alonso-Mora. Before that, I received a bachelor’s degree in Automotive Engineering from Tongji University, Shanghai.

I develop algorithms and theory for scalable, sample-efficient, and trustworthy robot learning. Examples of my work include:

  • Multi-fidelity reinforcement learning methods that learn stably from scarce real-robot data, complemented by abundant but imperfect simulation data.

  • Learning behavior models that predict how agents act in interaction-rich, uncertain scenarios.

My research draws on reinforcement learning, game theory, and differentiable optimization, and I ground it in real-world systems such as robotic manipulators and mobile robots.

I am looking for Summer 2027 research internships in reinforcement learning and robot learning. Please feel free to reach out!

publications

  1. Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement Learning
    Xinjie Liu, Ruihan Zhao, Anirban Chaudhuri, Cyrus Neary, Ufuk Topcu, David Fridovich-Keil
    Under review, 2026
    arXiv website
  2. Horizon-Scalable Q-Learning
    Haoran Xu, Pragya Srivastava, Xinjie Liu, Siddhant Agarwal, Amy Zhang
    Under review, 2026
  3. A Multi-Fidelity Control Variate Approach for Policy Gradient Estimation
    Xinjie Liu*, Cyrus Neary*, Kushagra Gupta, Wesley A. Suttle, Christian Ellis, Ufuk Topcu, David Fridovich-Keil
    Transactions on Machine Learning Research (TMLR), 2026
    Awarded J2C Certification (selected to present at NeurIPS 2026)
    OpenReview arXiv website code video
  4. Linear-Quadratic Gaussian Games with Distributed Sparse Estimation
    Tianyu Qiu, Filippos Fotiadis, Xinjie Liu, Christian Ellis, Jesse Milzman, Wesley A. Suttle, Ufuk Topcu, David Fridovich-Keil
    IEEE Control Systems Letters (L-CSS), 2026
    paper arXiv
  5. Bayesian Inverse Games with High-Dimensional Multi-Modal Observations
    Yash Jain*, Xinjie Liu*, Lasse Peters*, David Fridovich-Keil, Ufuk Topcu
    Under review at The International Journal of Robotics Research (IJRR), 2025
    arXiv
  6. Approximate Feedback Nash Equilibria with Sparse Inter-Agent Dependencies
    Xinjie Liu, Jingqi Li, Filippos Fotiadis, Mustafa O. Karabag, Jesse Milzman, David Fridovich-Keil, Ufuk Topcu
    Conditionally accepted at IEEE Transactions on Automatic Control (T-AC), 2025
    arXiv website
  7. Second-Order Algorithms for Finding Local Nash Equilibria in Zero-Sum Games
    Kushagra Gupta, Xinjie Liu, Ross Allen, Ufuk Topcu, David Fridovich-Keil
    Under review, 2025
    arXiv
  8. Inferring Foresightedness in Dynamic Noncooperative Games
    Cade Armstrong*, Ryan Park*, Xinjie Liu, Kushagra Gupta, David Fridovich-Keil
    IEEE Robotics and Automation Letters (RA-L), 2025
    paper arXiv
  9. Policies with Sparse Inter-Agent Dependencies in Dynamic Games: A Dynamic Programming Approach
    Xinjie Liu, Jingqi Li, Filippos Fotiadis, Mustafa O. Karabag, Jesse Milzman, David Fridovich-Keil, Ufuk Topcu
    Autonomous Agents and Multi-Agent Systems (AAMAS), Extended Abstract, 2025
    paper website
  10. Auto-Encoding Bayesian Inverse Games
    Xinjie Liu*, Lasse Peters*, Javier Alonso-Mora, Ufuk Topcu, David Fridovich-Keil
    Algorithmic Foundations of Robotics XVI (WAFR), 2024
    paper arXiv website code
  11. Learning to Play Trajectory Games against Opponents with Unknown Objectives
    Xinjie Liu*, Lasse Peters*, Javier Alonso-Mora
    IEEE Robotics and Automation Letters (RA-L), 2023
    paper arXiv website code

* Equal contribution

news

Oct 13, 2026 I will present our TMLR paper “A Multi-Fidelity Control Variate Approach for Policy Gradient Estimation” at the Atlanta venue of NeurIPS 2026 through the Journal-to-Conference (J2C) track!
Oct 1, 2026 Our preprint “Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement Learning” is out! Our method, MFPG-PPO, learns stably on a physical Franka arm using only 4 real-robot episodes per update.
Jun 15, 2026 Our paper “Linear-Quadratic Gaussian Games with Distributed Sparse Estimation” has been published in the IEEE Control Systems Letters (L-CSS)!
Mar 3, 2026 I gave a talk, “Boosting Reinforcement Learning with Imperfect Data: A Multi-Fidelity Policy Gradient Framework,” at the Texas Symposium on Machine Learning, Responsible AI & Robotics, UT Austin [video]!
Feb 11, 2026 Our paper “A Multi-Fidelity Control Variate Approach for Policy Gradient Estimation” has been accepted to the Transactions on Machine Learning Research (TMLR) and awarded a J2C Certification, selected for presentation at NeurIPS/ICML/ICLR!
Dec 27, 2025 We submitted Bayesian Inverse Games with High-Dimensional Multi-Modal Observations to The International Journal of Robotics Research (IJRR)!
Oct 15, 2025 Our paper “Inferring Foresightedness in Dynamic Noncooperative Games” has been accepted at the IEEE Robotics and Automation Letters (RA-L) and will be presented at ICRA 2026!
Sep 29, 2025 We submitted A Multi-Fidelity Control Variate Approach for Policy Gradient Estimation!
May 23, 2025 I presented Policies with Sparse Inter-Agent Dependencies in Dynamic Games: A Dynamic Programming Approach at AAMAS 2025, Detroit! We submitted an extended version of this work to IEEE Transactions on Automatic Control (T-AC).
Apr 11, 2025 I’m honored to have received a travel grant from Texas Robotics!
Oct 16, 2024 We submitted Policies with Sparse Inter-Agent Dependencies in Dynamic Games: A Dynamic Programming Approach!
Oct 9, 2024 I’m honored to have received a travel grant from WAFR 2024 and to have presented our work Auto-Encoding Bayesian Inverse Games. It was a lot of fun in Chicago!
Aug 17, 2024 Our paper Auto-Encoding Bayesian Inverse Games was accepted to WAFR 2024. See you in Chicago!
May 29, 2024 An open-source project on implementation of numerical optimization algorithms.
May 22, 2024 We submitted Second-Order Algorithms for Finding Local Nash Equilibria in Zero-Sum Games!
Feb 2, 2024 We submitted Auto-Encoding Bayesian Inverse Games!
Jul 10, 2023 Graduated from TU Delft! I successfully defended my thesis and obtained my MSc degree with Cum Laude distinction in Robotics [slides]!
Jul 5, 2023 I gave a talk at the Control and Learning for Autonomous Robotics (CLeAR) Lab, UT Austin!
Apr 17, 2023 Our paper “Learning to Play Trajectory Games Against Opponents with Unknown Objectives” has been accepted at the IEEE Robotics and Automation Letters (RA-L) and will be presented at IROS 2023. See you in Detroit!
Jan 24, 2023 I gave an invited talk at the Safe Robotics Laboratory, Princeton [slides]. Thanks for the invitation!
Dec 1, 2022 Our work “Learning to Play Trajectory Games Against Opponents with Unknown Objectives” has been submitted and is now available! Please check out the preprint version here.
Jun 30, 2022 We won the Hackathon challenge at the European Robotics Forum 2022! (Post from the sponsor Franka Emika)