RankCLIP: Ranking-Consistent Language-Image Pretraining
Yiming Zhang*, Zhuokai Zhao*, Zhaorun Chen, Zhili Feng, Zenghui Ding, and Yining Sun
In submission, 2024
Zhuokai Zhao
PhD Candidate in Computer Science
University of Chicago
biography
I am currently a final-year PhD Candidate in Computer Science at the
University
of Chicago.
My research focus is to develop novel algorithms in both training and inference stages to
improve trustworthiness, robustness, and data efficiency for multimodal deep learning models (mostly large
language models and vision-language models).
During my PhD, I am fortunate to have been working with Prof. Yuxin Chen (advisor), Prof. Bo Li (committee member), and Prof. Michael Maire (committee member). I obtained my Master's degree in Computer Science as part of my PhD in 2021, advised by Prof. Gordon Kindlmann.
Previously, I received my Master's degree in Robotics from the Johns Hopkins University, under the supervision of Prof. Nassir Navab in 2018, and my Bachelor's degree with Honors in Electrical Engineering from the University of Illinois at Urbana-Champaign in 2016, where I was advised by Prof. Seth Hutchinson.
During my PhD, I am fortunate to have been working with Prof. Yuxin Chen (advisor), Prof. Bo Li (committee member), and Prof. Michael Maire (committee member). I obtained my Master's degree in Computer Science as part of my PhD in 2021, advised by Prof. Gordon Kindlmann.
Previously, I received my Master's degree in Robotics from the Johns Hopkins University, under the supervision of Prof. Nassir Navab in 2018, and my Bachelor's degree with Honors in Electrical Engineering from the University of Illinois at Urbana-Champaign in 2016, where I was advised by Prof. Seth Hutchinson.
experience
-
Applied Scientist Intern, Twitch (Amazon Science)
06/2023 - 08/2023
-
Part-time Student Researcher, Meta AI
08/2023 - 11/2023
-
PhD Machine Learning Engineer Intern, Meta AI
06/2023 - 08/2023
-
Research & Development Intern, Kitware, Inc.
06/2019 - 09/2019
-
Research Intern, JD.com
05/2018 - 09/2018
-
Research Intern, Siemens Corporate Research
05/2017 - 08/2017
selected publications
(* indicates equal contributions)
PANDORA: Detailed LLM Jailbreaking via Collaborated Phishing Agents with Decomposed Reasoning
Zhaorun Chen*, Zhuokai Zhao*, Wenjie Qu, Zichen Wen, Zhiguang Han, Zhihong Zhu, Jiaheng Zhang, and Huaxiu Yao
Preliminary version appeared in ICLR Workshop on Secure and Trustworthy Large Language Models, 2024
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding
International Conference on Machine Learning (ICML), 2024
Preliminary version appeared in ICLR Workshop on Reliable and Responsible Foundation Models, 2024
Preliminary version appeared in ICLR Workshop on Reliable and Responsible Foundation Models, 2024
AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition
Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024
Preliminary version appeared in ICLR Workshop on Reliable and Responsible Foundation Models, 2024
Preliminary version appeared in ICLR Workshop on Reliable and Responsible Foundation Models, 2024
Direct Acquisition Optimization for Low-Budget Active Learning
Zhuokai Zhao, Yibo Jiang, and Yuxin Chen
In submission, 2024
Multi-Modality Guidance Network For Missing Modality Inference
Zhuokai Zhao, Harish Palani, Tianyi Liu, Lena Evans, and Ruth Toner
IEEE International Conference on Multimedia and Expo (ICME) Industry Track, 2023
Evaluating Machine Learning Models with NERO: Non-Equivariance Revealed on Orbits
In submission, 2023
Breaking the Curse of Quality Saturation with User-Centric Ranking
29th Conference on Knowledge Discovery and Data Mining (KDD), 2023
System and Method for Assisted Patient Positioning
Zhuokai Zhao, Yao-Jen Chang, Ruhan Sa, Kai Ma, Jiangping Wang, Vivek Kumar Singh, Terrence Chen, Andreas Wimmer, and Birgi Tamersoy
U.S. Patent No. 10,783,655, 2020
other publications
EscIRL: Evolving Self-Contrastive IRL for Trajectory Prediction in Autonomous Driving
Zhaorun Chen, Zhuokai Zhao, Dandan Liang, Zhihong Zhu, Liang Gong, Chengliang Liu, Chenglin Miao, and Lu Su
In submission, 2024
Safe Reinforcement Learning via Hierarchical Adaptive Chance-Constraint Safeguards
In submission, 2024
RELAX: Reinforcement Learning Enabled 2D-LiDAR Autonomous System for Parsimonious UAVs
Guanlin Wu, Zhuokai Zhao, and Yutao He
In submission, 2023
Early Feasibility Studies of Augmented Reality Navigation for Lateral Skull Base Surgery
Otology & Neurotology 41(7):p 883-888, 2020
dissertations
Robust Data Utilization for Efficient and Trustworthy Deep Learning
Zhuokai Zhao
Ph.D. in Computer Science, 2024
Utilizing Both Past and Future: Multi-Frame Memory Based Network in Solving Particle Image Velocimetry
Zhuokai Zhao
MS in Computer Science, 2021
other projects
OpenChemistry/Stempy: Stable Version
Patrick Avery, Christopher J. Harris, Peter Ercius, Alessandro Genova, Marcus Hanwell, and Zhuokai Zhao
Zenodo, 2024
Trajectory Planning and Control for Nonholonomic Robot Among Onstacles
Zhuokai Zhao, Mengdi Xu, and Changxin Yan
Nonlinear Control and Planning in Robotics, 2018
Head-Mounted Display Integration for Orthopedic Surgery
Advanced Computer-Integrated Surgery, 2017
Service
Conference Reviewer
-
Neural Information Processing Systems (NeurIPS)
'23
-
International Conference on Learning Representations (ICLR)
'23, '24
-
SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)
'24
-
Conference on Empirical Methods in Natural Language Processing (EMNLP)
'23
-
ACM Multimedia (ACM MM)
'24
-
IEEE Visualization Conference (IEEE VIS)
'23
-
International Conference on Robotics and Automation (ICRA)
'24
-
IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
'23, '24
-
SIAM International Conference on Data Mining (SDM)
'24
-
The International Conference on Information Systems (ICIS)
'23
-
IEEE Transactions on Visualization and Computer Graphics (TVCG)
'23
-
Transactions on Machine Learning Research (TMLR)
'23
