Tackling Unbounded State Spaces in Continuing Task Reinforcement Learning
[arxiv]Brahma S. Pavse, Yudong Chen, Qiaomin Xie, Josiah P. Hanna
Welcome!
I am a second-year Computer Science PhD candidate at the University of Wisconsin-Madison. I am broadly interested in representation learning questions in the context of reinforcement learning. In particular, I study how AI agents can learn suitable representations about the world that may be useful in continuing, long-horizon, and off-policy evaluation tasks.
I am fortunate to be advised by Josiah Hanna, and I also closely collaborate with Qiaomin Xie and Yudong Chen. I have also interned at Sony AI.
Previously, I completed my BS and MS in Computer Science from the University of Texas at Austin, where I was fortunate to be advised by Peter Stone. I also worked as a software engineer at Salesforce and SAS Institute.
Feel free to shoot me an email if you want to chat!
News
- September 2023: Our paper, State-Action Similarity-Based Representations for Off-Policy Evaluation, was accepted at NeurIPS 2023!
- May 2023: I will be working on the RL team at Sony AI this summer!
- November 2022: Our paper, Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction, was accepted at AAAI 2023!
- May 2022: I received the CS Summer Research Fellowship!
- January 2022: I joined UW-Madison to start my PhD!
Publications
Pre-prints
2023
Brahma S. Pavse, Yudong Chen, Qiaomin Xie, Josiah P. Hanna
Conference Papers
2023
State-Action Similarity-Based Representations for Off-Policy Evaluation
[pdf] [bibtex]Brahma S. Pavse, Josiah P. Hanna
Proceedings of the 36th Neural Information Processing Systems (NeurIPS), December 2023.
Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction (Oral Presentation)
[pdf] [bibtex]Brahma S. Pavse, Josiah P. Hanna
Proceedings of the 37th Association for the Advancement of Artificial Intelligence (AAAI), February 2023.
An earlier version appeared at the Offline RL Workshop: Offline RL as a "Launchpad" at NeurIPS 2022.
2020
Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration
[pdf] [bibtex] Brahma S. Pavse*, Faraz Torabi*, Josiah Hanna, Garrett Warnell, Peter Stone*Equal contribution.
Contains material from my undergraduate honors thesis.
IEEE Robotics and Automation Letters, July 2020.
Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020), October 2020.
An earlier version appeared in the Imitation, Intent, and Interaction (I3) workshop at ICML 2019.
Reducing Sampling Error in Batch Temporal Difference Learning
[pdf] [bibtex] Brahma S. Pavse, Ishan Durugkar, Josiah Hanna, Peter StoneMaster's thesis.
Proceedings of the 37th International Conference on Machine Learning (ICML 2020), July 2020.
Journal Articles
2020
Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration
[pdf] [bibtex] Brahma S. Pavse*, Faraz Torabi*, Josiah Hanna, Garrett Warnell, Peter Stone*Equal contribution.
Contains material from my undergraduate honors thesis.
IEEE Robotics and Automation Letters, July 2020.
Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020), October 2020.
An earlier version appeared in the Imitation, Intent, and Interaction (I3) workshop at ICML 2019.
Theses
Reducing Sampling Error in Batch Temporal Difference Learning
[pdf] [bibtex] Brahma S. Pavse, advised by Peter Stone and Josiah HannaMS Thesis, University of Texas at Austin, 2020.
Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration
[pdf] [bibtex] Brahma S. Pavse, advised by Peter StoneBS Honors Thesis, University of Texas at Austin, 2019.
Workshops and Symposia
2020
On Sampling Error in Batch Action-Value Prediction Algorithms
[pdf] [bibtex] Brahma S. Pavse, Josiah Hanna, Ishan Durugkar, Peter StoneWorkshop on Offline Reinforcement Learning, Neural Information Processing Systems (NeurIPS), December 2020.
2019
UT Austin Villa: RoboCup 2019 3D Simulation League Competition and Technical Challenge Champions
[pdf] [bibtex] Patrick MacAlpine, Faraz Torabi, Brahma S. Pavse, Peter StoneRoboCup International Symposium 2019
2018
UT Austin Villa: RoboCup 2018 3D Simulation League Champions
[pdf] [bibtex] Patrick MacAlpine, Faraz Torabi, Brahma S. Pavse, John Sigmon, Peter StoneRoboCup International Symposium 2018
Awards and Honors
- UW Madison CS Summer Research Fellowship (2022).
- UW Madison CS Graduate Scholarship (2022).
- UT Austin University Honors (2020).
- UT Austin CS Special Departmental Honors (Research) (2020).
- RoboCup 3D Simulation League World Champions (2019).
- Eva Stevenson Woods Endowed Presidential Scholarship (2019).
- National Instruments Endowed Scholarship (2019).
- RoboCup 3D Simulation League World Champions (2018).
Service
- Program Committee Member, Association for the Advancement of Artificial Intelligence (AAAI) (2023).
- Graduate Student Mentor, Wisconsin Science and Computing Emerging Research Stars (WISCERS) (2022).
- Reviewer, Neural Information Processing Systems (NeurIPS) (2022).
- Reviewer, International Conference on Learning Representations (ICLR) (2022).
- Reviewer, International Conference on Robotics and Automation (ICRA) (2021).
- Reviewer, UT Austin Computer Science Dept. MS Admissions Committee (2020).