Prompt Injections for Defense
Researchers from Tracebit found that embedding forbidden prompt injections alongside AWS secrets can effectively halt AI hacking agents, as the large language model shuts down upon encountering commands that violate its guardrails—a technique they call "context bombing."
Researchers from Tracebit have found that placing prompt injections alongside passwords and secrets on Amazon Web Services can effectively shut down AI hacking agents. The technique, named "context bombing," directs attacking large language models (LLMs) to perform actions forbidden by their guardrails, such as developing inhalable Anthrax spores or referencing the Tank Man from Tiananmen Square. Once the LLM encounters these forbidden commands, it ceases to follow its existing instructions, halting the attack. The approach leverages the AI's own safety restrictions as a defense mechanism.