Quantifying privacy risks in LLMs is an important research question. We take a step towards answering this question by defining a real-world threat model wherein an entity seeks to augment a foundation model LLM with private data they possess, via methods such as fine-tuning or in-context learning. The entity also seeks to improve the quality of their LLM outputs over time by learning from human feedback. We examine attacks on this system in the form of poisoning attacks, and defenses in the form of systems that provide differential privacy guarantees.

Zoom: https://princeton.zoom.us/my/panda

Adviser: Prateek Mittal