Robert Pratten This week saw the publication of Stanford’s report “Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations”.
Clearly the potential and threat of AI (decision support, decryption etc) has been discussed for a long time but the report’s value is that it focuses specifically on Influence Operations.
The main area of interest for me is how generative language models have the potential to change all three aspects of influence operations: actors, behaviours, and content:
Actors: Language models will drive down the cost of running influence operations, making them more accessible to new actors. This means that even small, low-resourced actors will be able to run large-scale influence operations. Additionally, propagandists-for-hire that automate the production of text may gain new competitive advantages, which will lead to an increase in the number of actors participating in influence operations.
Behaviours: Influence operations with language models will become easier to scale, and tactics that are currently expensive such as generating personalized content will become cheaper. Language models may also enable new tactics to emerge, such as real-time content generation in chatbots. This will lead to a significant increase in the number of influence operations taking place, and will make them more difficult to detect.
Content: Text creation tools powered by language models may generate more impactful or persuasive messaging compared to traditional propagandists, especially those who lack the necessary linguistic or cultural knowledge of their target audience. As with behaviours, they will make influence operations less discoverable, as they can repeatedly create new content without resorting to copy-pasting or other noticeable time-saving behaviours.
At Conducttr we’re already building AI assistance into our crisis exercise software to help clients generate bespoke scenarios in a fraction of the time it would previously and being able to generating target audience behaviours in even more realistic ways. They’ll be a client-only webinar in the coming weeks to educate & train clients on this exciting and frightening new world! Here’s an example of our AI use from just before Christmas
