The AI Attack Surface Map v1.0 AI 攻击⾯地图 v1.0 The AI Attack Surface AI 攻击⾯ Introduction 介绍 This resource is a first thrust at a framework for thinking about how to attack AI systems. 该资源是思考如何攻击 AI 系统的框 架的第⼀个推动⼒。 At the time of writing, GPT-4 has only been out for a couple of months, and ChatGPT for only 6 months. So things are very early. There has been, of course, much content on attacking pre-ChatGPT AI systems, namely how to attack machine learning implementations. 在撰写本⽂时， GPT-4 只发布了⼏个⽉，⽽ ChatGPT 只有 6 个⽉。所以事情还 很早。当然，有很多关于攻击 ChatGPT AI 系统的内容，即如何攻击 机器学习实 现。 It’ll take time, but we’ve never seen a technology be used in real-world applications as fast as post-ChatGPT-AI. 这需要时间，但我们从未见过⼀种技术像后 ChatGPT-AI 那样快速 ⽤于现实世界的 应⽤程序。 But as of May of 2023 there has not been much content on attacking full systems built with AI as part of multiple components. This is largely due to the fact that integration technologies like Langchain only rose to prominence in the last 2 months. So it will take time for people to build out products and services using this tooling. 但截⾄ 2023 年 5 ⽉，关于攻击使 ⽤ AI 作为多个组件的⼀部分构建的完整系统的 内容并不多。这主要是由于像 Langchain 这样的集成技术在过去 2 个⽉才崭露头 ⾓。因此，⼈们使⽤此⼯具构建产品和服务需要时间。 Natural language is the go-to language for attacking AI systems. ⾃然语⾔是攻击 AI 系统的⾸选语 ⾔。 Once those AI-powered products and services start to appear we’re going to have an entirely new species of vulnerability to deal with. We hope with this resource to bring some clarity to that landscape. ⼀旦这些⼈⼯智能驱动的产品和服务开始出现，我们将⾯临⼀个全新的脆弱性。 我们希望利⽤这⼀资源使这⼀情况更加清晰。 Purpose ⽬的 The purpose of the this resource is to give the general public, and offensive security practitioners specifically, a way to think about the various attack surfaces within an AI system. 此资源的⽬的是为公众，特别是攻击性安全从业者提供⼀种思考 AI 系统中各种攻 击⾯的⽅法。 The goal is to have someone consume this page and its diagrams and realize that AI attack surface includes more than just models.⽬标是让某⼈使⽤此页⾯及其图表，并意识到 AI 攻击⾯不仅仅包括模型。 We want anyone interested to see that natural language is the primary means of attack for LLM-powered AI systems, and that it can be used to attack components of AI-powered systems throughout the stack. 我们希望任何感兴趣的⼈都能看到⾃然语⾔是 LLM 驱动的 AI 系统的主要攻击⼿ 段，并且它可⽤于攻击整个堆栈中 AI 驱动的系统的组件。 Components 组件 Click images to expand 点击图⽚展开 We see a few primary components for AI attack surface, which can also be seen in the graphics above. Langchain calls these Components. 我们看到 AI 攻击⾯的⼀些主要组 件，也可以在上图中看到。 Langchain 称这些组 件为 Components 。How Langchain breaks things down Langchain 如何分解事物 Prompts are another component in Langchain but we see those as the attack path rather than a component. 提⽰是 Langchain 中的另⼀个组 件，但我们将提⽰视为攻击路径⽽不是组件。 AI Assistants ⼈⼯智能助⼿ Agents 代理 Tools ⼯具 Models 模型 Storage 存储 AI Assistants ⼈⼯智能助⼿ We’ve so far always chosen to trade privacy for functionality, and AI will be the ultimate form of this. 到⽬前为⽌，我们⼀直选择⽤隐私换取功能，⽽⼈⼯智能将是其中的最终形式。 AI Assistants are the agents that will soon manage our lives. They will manipulate our surroundings according to our preferences, which will be nice, but in order to do that they will need extraordinary amounts of data about us.⼈⼯智能助⼿是很快将管理我们⽣活的代理⼈。他们会根据我们的喜好操纵我们 的周围环境，这很好，但为了做到这⼀点，他们将需要⼤量关于我们的数据。 Which we will happily exchange for the functionality they provide. 我们很乐意交换它们提供的功能。 AI Assist ants combine knowledge and access, making them like a digital soul. ⼈⼯智能助⼿将知识和访问相结合，使它们像⼀个数字灵魂。 Attacking people’s AI Assist ants will have high impact. 攻击⼈们的 AI 助⼿将产⽣很⼤的影 响。 For AI Assist ants to be useful they must be empowered, meaning they need 1) to know massive amounts about you, including very personal and sensitive information for the highest efficacy, and 2) they need to be able to behave as you. 为了使 AI 助⼿有⽤，它们必须被授 权，这意味着他们需要 1 ）了解⼤量关于你的信 息，包括⾮常私⼈和敏感信息以获得最⾼效率， 2 ）他们需要能够像你⼀样⾏事。 Which means sending money, posting on social media, writing content, sending messages, etc. An attacker who gains this knowledge and access will have significant leverage over the target. 这意味着汇款、在社交媒体上发帖、撰写内容、发送消息等。获得此知识和访问 权限的攻击者将对⽬标具有重⼤