Publications

(* represents equal contribution)

2025

• Preference Poisoning Attacks on Reward Model Learning

  Junlin Wu, Jiongxiao Wang, Chaowei Xiao, Chenguang Wang, Ning Zhang, Yevgeniy Vorobeychik.

  IEEE S&P 2025. [pdf]

• ProteinDT: A Text-guided Protein Design Framework

  Shengchao Liu, Yanjing Li, Zhuoxinran Li, Anthony Gitter, Yutao Zhu, Jiarui Lu, Zhao Xu, Weili Nie, Arvind Ramanathan, Chaowei Xiao*, Jian Tang*, Hongyu Guo*, Anima Anandkumar*

  Nature Machine Intelligence 2025 [pdf]

• AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs

  Xiaogeng Liu*, Peiran Li*, G. Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, Chaowei Xiao.

  ICLR 2025 (spotlight). [pdf]

• Can Watermarks be Used to Detect LLM IP Infringement For Free?

  Zhengyue Zhao, Xiaogeng Liu, Somesh Jha, Patrick McDaniel, Bo Li, Chaowei Xiao.

  ICLR 2025. [pdf]

• Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset

  Yingzi Ma*, Jiongxiao Wang*, Fei Wang, Siyuan Ma, Jiazhao Li, Jinsheng Pan, Xiujun Li, Furong Huang, Lichao Sun, Bo Li, Yejin Choi, Muhao Chen, Chaowei Xiao.

  ICLR 2025. [pdf]

• CVE-Bench: Benchmarking LLM-based Software Engineering Agent’s Ability to Repair Real-World CVE Vulnerabilities

  Peiran Wang, Xiaogeng Liu, Chaowei Xiao.

  NAACL 2025. [pdf]

• RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process

  Peiran Wang, Xiaogeng Liu, Chaowei Xiao.

  NAACL 2025. [pdf]

• Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations

  Wenjie Jacky Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Hadi Askari, Chaowei Xiao, Muhao Chen.

  NAACL 2025. [pdf]

• T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching

  Zizheng Pan, Bohan Zhuang, De-An Huang, Weili Nie, Zhiding Yu, Chaowei Xiao, Jianfei Cai, Anima Anandkumar.

  ICLR 2025. [pdf]

• LeanAgent: Lifelong Learning for Formal Theorem Proving.

  Adarsh Kumarappan, Mo Tiwari, Peiyang Song, Robert Joseph George, Chaowei Xiao, Anima Anandkumar.

  ICLR 2025. [pdf]

• MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding

  Fei Wang, Xingyu Fu, James Y. Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, Tianyi Lorena Yan, Wenjie Jacky Mo, Hsiang-Hui Liu, Pan Lu, Chunyuan Li, Chaowei Xiao, Kai-Wei Chang, Dan Roth, Sheng Zhang, Hoifung Poon, Muhao Chen.

  ICLR 2025. [pdf]

• DataGen: Unified Synthetic Dataset Generation via Large Language Models

  Yue Huang, Siyuan Wu, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Chaowei Xiao, Jianfeng Gao, Xiangliang Zhang, Lichao Sun.

  ICLR 2025. [pdf]

• EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage

  Zeyi Liao, Lingbo Mo, Chejian Xu, Mintong Kang, Jiawei Zhang, Chaowei Xiao, Yuan Tian, Bo Li, Huan Sun.

  ICLR 2025. [pdf]

2024

• MProt-DPO: Breaking the ExaFLOPS Barrier for Multimodal Protein Design Workflows with Direct Preference Optimization.

  MProt-DPO Team

  SC 24. International Conference for High Performance Computing.2024 (Gorden Bell Prize Finalist). [pdf]

• Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment

  Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Yixuan Li, Patrick McDaniel, Muhao Chen, Bo Li, Chaowei Xiao

  NeurIPS 2024. [pdf]

• HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection

  Xuefeng Du, Chaowei Xiao, Yixuan Li.

  NeurIPS 2024 (Spotlight). [pdf]

• Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustness

  Yiquan Li, Zhongzhu Chen, Kunjin, Jiongxiao Wang, Bo Li, Chaowei Xiao.

  NeurIPS 2024. [pdf]

• AgentPoison: Red-teaming LLM Agents via Memory or Knowledge Base Backdoor Poisoning

  Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, Bo Li. .

  NeurIPS 2024. [pdf]

• Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks

  Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, Chaowei Xiao.

  COLM 2024. [pdf]

• Dolphins: Multi- modal Language Model for Driving.

  Yingzi Ma, Yulong Cao, Jiachen Sun, Marco Pavone, Chaowei Xiao.

  ECCV 2024. [pdf]

• AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting.

  Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, Chaowei Xiao.

  ECCV 2024. [pdf]

• RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios.

  Wenhao Ding, Yulong Cao, Ding Zhao, Chaowei Xiao, Marco Pavone.

  ECCV 2024. [pdf]

• Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensation.

  Haizhong Zheng, Jiachen Sun, Shutong Wu,Bhavya Kaikhura, Zhuoqing Mao, Chaowei Xiao, Atul Prakash.

  ECCV 2024. [pdf]

• On the exploitability of reinforcement learning with human feedback for large language models.

  Jiongxiao Wang, Junlin Wu, Muhao Chen, Yevgeniy Vorobeychik, Chaowei Xiao.

  ACL 2024. [pdf]

• Position Paper: TrustLLM: Trustworthiness in Large Language Models.

  TrustLLM Team.

  ICML 2024. [pdf]

• PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees.

  Chulin Xie, De-An Huang, Wenda Chu, Daguang Xu, Chaowei Xiao*, Bo Li*, and Anima Anandkumar*.

  CVPR 2024. [pdf]

• Do Not Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models.

  Zhiyuan Yu, Xiaogeng Liu, Shuning Liang, Zach Cameron,Chaowei Xiao, Ning Zhang.

  USENIX Security 2024. [pdf]

• Reinforcement Learning with Human Feedback for Realistic Traffic Simulation.

  Yulong Cao, Boris Ivanovic, Chaowei Xiao, Marco Pavone.

  ICRA 2024. [pdf]

• AutoDan: Generating stealthy jailbreak prompts on aligned large language models.

  Xiaogeng Liu, Nan Xu, Muhao Chen, Chaowei Xiao

  ICLR 2024. [pdf]

• ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback

  Shengchao Liu, Jiongxiao Wang, Yijin Yang, Chengpeng Wang, Ling Liu, Hongyu Guo, Chaowei Xiao

  ICLR 2024. [pdf]

• CALICO: Self-Supervised Camera-LiDAR Contrastive Pre-training for BEV Perception.

  Jiachen Sun, Haizhong Zheng, Qingzhao Zhang, Atul Prakash, Z. Morley Mao, and Chaowei Xiao

  ICLR 2024. [pdf]

• ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Trigger.

  Jiazhao Li, Yijin Yang, Zhuofeng Wu, V.G.Vinod Vydiswaran, Chaowei Xiao.

  NAACL 2024. [pdf]

• Instructional Fingerprinting of Large Language Models.

  Jiashu Xu, Fei Wang, Mingyu Derek Ma, Pang Wei Koh, Chaowei Xiao, Muhao Chen.

  NAACL 2024. [pdf]

• Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models.

  Jiashu Xu, Fei Wang, Mingyu Derek Ma, Pang Wei Koh, Chaowei Xiao, Muhao Chen.

  NAACL 2024. [pdf]

• From Shortcuts to Triggers: Backdoor Defense with Denoised PoE.

  Qin Liu, Fei Wang, Chaowei Xiao, Muhao Chen.

  NAACL 2024. [pdf]

• Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking.

  Nan Xu, Fei Wang, Ben Zhou, Bangzheng Li, Chaowei Xiao, Muhao Chen.

  NAACL 2024 (findings). [pdf]

• Prismer: A Vision-Language Model with Multi-Task Experts

  Shikun Liu, Linxi Fan, Edward Johns, Zhiding Yu, Chaowei Xiao, Anima Anandkumar.

  TLMR 2024. [pdf]

• Voyager: An open-ended embodied agent with large language models

  Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar

  TLMR 2024. [pdf]

• Multi-modal molecule structure–text model for text-based retrieval and editing

  Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang*, Chaowei Xiao*, Animashree Anandkumar*.

  Natural Machine Intelligence. [pdf]

2023

• Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study

  Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro

  EMNLP 2023. [pdf]

• Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning

  Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Anand Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu, Shiyi Lan, Bo Li, Mohammad Shoeybi, Ming-Yu Liu, Yuke Zhu, Bryan Catanzaro, Chaowei Xiao*, Anima Anandkumar*

  EMNLP 2023. [pdf]

• HiCL: Hierarchical Contrastive Learning of Unsupervised Sentence Embeddings

  Zhuofeng Wu, Chaowei Xiao, V.G.Vinod Vydiswaran

  EMNLP 2023. [pdf]

• On the Exploitability of Instruction Tuning

  Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao*, Tom Goldstein*

  NeurIPS 2023. [pdf]

• [LLM Safety]Defending against Insertion-based Textual Backdoor Attacks via Attribution.

  Jiazhao Li, Zhuofeng Wu, Wei Ping, Chaowei Xiao, V.G. Vinod Vydiswaran.

  ACL 2023

• CodeIPPrompt: Intellectual Property Infringement Assessment of Code Language Models

  Zhiyuan Yu, Yuhao Wu, Ning Zhang, Chenguang Wang, Yevgeniy Vorobeychik, Chaowei Xiao

  ICML 2023. [pdf]

• A Critical Revisit of Adversarial Robustness in 3D Point Cloud Recognition with Diffusion-Driven Purification.

  Jiachen Sun, Jiongxiao Wang, Weili Nie, Zhiding Yu, Zhuoqing Mao, Chaowei Xiao

  ICML 2023. [pdf]

• VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion.

  Yiming Li, Zhiding Yu, Chris Choy, Chaowei Xiao, Jose M. Alvarez, Sanja Fidler, Chen Feng, Anima Anandkumar

  CVPR 2023 (Highlight). [pdf]

• Expunging Clean-label Invisible Poisons via Pre-trained Diffusion Models.

  Xiaogeng Liu, Minghui Li, Haoyu Wang, Shengshan Hu, Dengpan Ye, Hai Jin, Libing Wu,Chaowei Xiao.

  CVPR 2023. [pdf]

• DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothing.

  Jiawei Zhang, , Zhongzhu Chen, Huan Zhang, Chaowei Xiao, Bo Li.

  USENIX Security 2023. [pdf]

• DensePure: Understanding Diffusion Models towards Adversarial Robustness.

  Chaowei Xiao*, Zhongzhu Chen*, Kun Jin*, Jiongxiao Wang*, Weili Nie, Mingyan Liu, Anima Anandkumar, Bo Li, Dawn Song

  ICLR 2023. [pdf]

• Defending against Adversarial Audio via Diffusion Model

  Shutong Wu, Jiongxiao Wang, Wei Ping, Weili Nie,Chaowei Xiao

  ICLR 2023. [pdf]

• Retrieval-based Controllable Molecule Generation

  Zichao Wang, Weili Nie, Zhuoran Qiao, Chaowei Xiao , Richard Baraniuk, Anima Anandkumar

  ICLR 2023 (spotlight). [pdf]

• Semantically Meaningful Adversarial Audio Attack

  Zhiyuan Yu, Yuanhaur Chang, Ning Zhang, Chaowei Xiao

  USENIX Security 2023. [pdf]

• GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics.

  Maxim Zvyagin*, Alexander Brace*, Kyle Hippe*, Yuntian Deng*, Bin Zhang, Cindy Orozco Bohorquez, Austin Clyde, Bharat Kale, Danilo Perez-Rivera, Heng Ma, Carla M. Mann, Michael Irvin, J. Gregory Pauloski, Logan Ward, Valerie Hayot, Murali Emani, Sam Foreman, Zhen Xie, Diangen Lin, Maulik Shukla, Weili Nie, Josh Romero, Christian Dallago, Arash Vahdat, Chaowei Xiao, Thomas Gibbs, Ian Foster, James J. Davis, Michael E. Papka, Thomas Brettin, Rick Stevens, Anima Anandkumar, Venkatram Vishwanath, Arvind Ramanathan.

  ACM Gordon Bell Special Covid Prize[pdf]

2022

• Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models

  Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, Chaowei Xiao

  NeurIPS 2022. [pdf]

• Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models

  Boxin Wang, Wei Ping, Chaowei Xiao, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bo Li, Anima Anandkumar, Bryan Catanzaro.

  NeurIPS 2022[pdf]

• Robust Trajectory Prediction against Adversarial Attacks

  Yulong Cao, Danfei Xu, Xinshuo Weng, Z. Morley Mao, Anima Anandkuma, Chaowei Xiao, Marco Pavone

  CORL 2022 (Oral)[pdf]

• Diffusion Models for Adversarial Purification

  Weili Nie, Brandon Guo, Yujia Huang,Chaowei Xiao, Arash Vahdat, Anima Anandkumar.

  ICML 2022[pdf]

• Understanding the robustness in vision transformers.

  Daquan Zhou, Zhiding Yu, Enze Xie,Chaowei Xiao, Anima Anandkumar, Jiashi Feng, Jose M Alvarez.

  ICML 2022[pdf]

• AdvDO: Realistic Adversarial Attacks for Trajectory Prediction

  Yulong Cao,Chaowei Xiao, Anima Anandkumar, Danfei Xu, Marco Pavone.

  ECCV 2022[pdf]

• SecretGen: Privacy Recovery on Pre-trained Models

  Zhuowen Yuan, Fan Wu, Yunhui Long,Chaowei Xiao, Bo Li

  ECCV 2022[pdf]

• Taxonomy of Machine Learning Safety: A Survey and Primer

  Sina Mohseni, Zhiding Yu, Chaowei Xiao Jay Yadawa, Haotao Wang and Zhangyang Wang

  ACM Computing Survey 2022[pdf]

• RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning

  Xiaojian Ma, Weili Nie, Zhiding Yu, Huaizu Jiang,Chaowei Xiao, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar

  ICLR 2022[pdf]

• Characterizing Attacks on Deep Reinforcement Learning.

  Xinlei Pan*,Chaowei Xiao*, Warren He, Shuang Yang, Peng Jian, Mingjie Sun, Mingyan Liu, Bo Li, Dawn Song.

  AAMAS 2022 [pdf]

2021

• Adversarially Robust 3D Point Cloud Recognition Using Self-Supervisions.

  Jiachen Sun, Yulong Cao, Christopher Choy, Zhiding Yu, Anima Anandkumar, Z. Morley Mao, Chaowei Xiao.

  NeurIPS 2021

• AugMax: Adversarial Composition of RandomAugmentations for Robust Training.

  Haotao Wang,Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Animashree Anandkumar, Zhangyang Wang.

  NeurIPS 2021

• Efficient Transformers for Language and Vision.

  Chen Zhu, Wei Ping,Chaowei Xiao, Mohammad Shoeybi, Tom Goldstein, Anima Anandkumar, Bryan Catanzaro.

  NeurIPS 2021 (84.1% Top-1 accuracy solely trained on ImageNet-1K)

• Can Shape Structure Features Improve Model Robustness?

  Mingjie Sun*, Zichao Li*,Chaowei Xiao*, Haonan Qiu, Bhavya Kailkhura, MingyanLiu, Bo Li.

  ICCV 2021

• Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks.

  Yulong Cao*, Ningfei Wang*,Chaowei Xiao*, Dawei Yang*, Jin Fang, RuigangYang, Qi Alfred Chen, Mingyan Liu, Bo Li.

  IEEE Symposium on Security and Privacy (Oakland) 2021

• Application-Driven Privacy-Preserving Data Publishing with Correlated Attributes.

  Aria Rezaei,Chaowei Xiao, Jie Gao, Bo Li, Sirajum Munir.

  Embedded Wireless Systems and Networks (EWSN 2021)Best Paper Award

• Behavior Privacy Perserving in RF Sensing.

  Jianwie Liu,Chaowei Xiao, Kaiyan Cui, Jinsong Han, Xian Xu, Kui Ren.

  IEEE Transcations on Dependable and Secure Computing. [pdf]

2020

• SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing

  Haonan Qiu*, Chaowei Xiao*, Lei Yang*, Xinchen Yan, Honglak Lee, Bo Li.

  ECCV 2020

• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations

  Huan Zhang*, Hongge Chen*, Chaowei Xiao, Bo Li, Mingyan Liu, Duane Boning, Cho-Jui Hsieh.

  NeurIPS 2020 (Spotlight).

• Towards Stable and Efficient Training of Verifiably Robust Neural Networks

  Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, Cho-Jui Hsieh.

  ICLR 2020.

2019

• AdvIT: Adversarial Frames Identifier Based on Temporal Consistency In Videos

  Chaowei Xiao, Ruizhi Deng, Bo Li, Taesung Lee, Jinfeng Yi, Ian Molloy, Mingyan Liu, Dawn Song

  In International Conference on Computer Vision (ICCV) 2019.

• MeshAdv: Adversarial Meshes for Visual Recognition

  Chaowei Xiao*, Dawei Yang*, Bo Li, Jia Deng, Mingyan Liu

  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019. (Oral presentation) [pdf]

• Performing Co-Membership Attack Against Deep Generative Model

  Kin Sum Liu, Chaowie Xiao , Bo Li, Jie Gao

  In IEEE International Conference on Data Mining (ICDM) 2019

• Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved Features

  Liang Tong, Bo Li, Chen Hajaj, Chaowei Xiao, Ning Zhang, Yevgeniy Vorobeychik.

  In USENIX Security 2019.

• Adversarial Sensor Attack on LIDAR-based Perception in Autonomous Driving

  Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, Z. Morely Map

  In the 26th ACM Conference on Computer and Communications Security (CCS'19) [pdf]

2018

• Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation

  Chaowei Xiao, Duizhi Deng, Bo Li, Fisher Yu, Mingyan Liu and Dawn Song

  In European Conference on Computer Vision (ECCV), 2018 [pdf][supplementary material]

• Spatially Transformed Adversarial Examples

  Chaowei Xiao*, Jun-Yan Zhu*, Bo Li, Warren He, Mingyan Liu and Dawn Song

  In International Conference on Learning Representations (ICLR), 2018 [pdf]

• Generating Adversarial Examples with Adversarial Networks

  Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu and Dawn Song

  In International Joint Conference on Artificial Intelligence (IJCAI), 2018. [pdf]

• From Behavior Similarity to Symptom Similarity: Using Community Detection for Early Discovery of Software Exploits

  Chaowei Xiao, Armin Sarabi,Yang Liu , Bo Li, Mingyan Liu and Tudor Dumitras

  In USENIX Security Symposium (USENIX Security), 2018 [pdf]

• Robust Physical-World Attacks on Machine Learning Models

  Kevin Eykholt*, Ivan Evtimov*, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno and Dawn Song

  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018 [pdf]

Before 2018

• Static Power of Mobile Devices: Self-updating Radio Maps for Wireless Indoor Localization

  Chenshu Wu, Zheng Yang, Chaowei Xiao, Chaofan Yang, Yunhao Liu and Mingyan Liu

  In IEEE International Conference on Computer Communications (INFOCOM), 2015

• Tagoram: Real-time Tracking of Mobile RFID Tags to High Precision Using COTS Devices.

  Lei Yang, Yekui Chen, Xiangyang Li, Chaowei Xiao, Mo Li and Yunhao Liu

  In International Conference on Mobile Computing and Networking (MobiCom), 2014. Best Paper Award