We wrapped a phishing payload in JSON and asked for "test data." The system generated working mal...

Summary

Security researchers successfully exploited a large language model by wrapping phishing payloads in JSON, requesting 'test data,' and encoding forbidden instructions in Base64 to bypass safety filters. The researchers chained multiple obfuscation techniques together, demonstrating that the LLM would generate working malicious code and execute encoded instructions when framed as legitimate requests. This research highlights critical vulnerabilities in current LLM safety mechanisms against prompt injection and instruction injection attacks.

Entities