This research introduces "persona vectors" to control and monitor character traits in language models. The authors show that undesirable personality changes in LLMs, induced by finetuning or prompts, are strongly correlated with shifts along persona vectors, and propose methods for predicting and mitigating these shifts. They also introduce a novel steering method to prevent or reduce these shifts, and show how to proactively flag problematic training data before finetuning.

The authors declare affiliations with multiple institutions that are working on LLM alignment and safety, including Anthropic, Truthful AI, Constellation, and UC Berkeley.

This research introduces a novel and systematic approach to controlling and monitoring character traits in LLMs. The automated pipeline for extracting persona vectors is highly valuable, along with its applications in controlling and mitigating persona shifts during finetuning and pre-finetuning. The study also investigates the potential of steering for addressing undesirable persona shifts. Despite some limitations, such as the computational cost of data filtering and the limited model and trait coverage, the overall methodology and findings are significant.