A group of researchers has published a study that raises alarm bells regarding \u00a0privacy\u00a0 when using \u00a0artificial intelligence (AI)\u00a0. This study reveals that it is possible to know the exact prompt that a user used when asking a chatbot something. Such revelations place AI companies in a delicate position, as they can know more about us than ever before.<\/p>\n
<\/p>\n
The title of the study, “Linguistic models are injective and, therefore, invertible<\/strong>,” might sound like jargon to many, but it highlights an alarming reality. Conducted by European researchers, the study explains that large language models (\u00a0LLM\u00a0) possess a significant \u00a0privacy issue\u00a0. This stems from the transformer architecture, which is designed such that each unique prompt corresponds to a different “embedding” in the model’s latent space. This structure makes it feasible for models to backtrack and connect responses to specific inputs.<\/p>\n <\/p>\n During the development of their theory, the researchers devised an algorithm named \u00a0SIPIT\u00a0 (Sequential Inverse Prompt via ITerative updates). This innovative algorithm reconstructs the exact input text from hidden activations or states, boasting the capability to do so in linear time. In less complex terms, this means the model can be \u00a0“snapped”\u00a0 back into shape easily and rapidly, raising significant privacy concerns for users.<\/p>\n <\/p>\n The implications of this research are substantial. When you receive a response from an AI model, it can inadvertently reveal the precise nature of your prompt. It turns out that the answer itself isn’t the giveaway\u2014rather, it’s the hidden states or embeddings the AI utilizes to generate its responses. This reality poses a significant risk, especially since AI companies often keep these states concealed, thereby enabling them to ascertain user prompts with remarkable accuracy.<\/p>\n <\/p>\n While it\u2019s true that many companies already save the prompts inputted by users, the concept of “injectivity” introduces an additional layer of privacy risk. Many \u00a0embeddings\u00a0 or internal states are retained for various reasons, such as caching, monitoring, diagnosis, or customization. If a company merely deletes the visible text of the conversation but retains the embeddings, it becomes possible to recover the original prompts from that file. This study reveals that any system that stores hidden states effectively handles the input text itself, posing an imminent danger to user privacy.<\/p>\n <\/p>\n A significant legal component arises from these findings. Until now, companies and regulators debated whether internal states constituted “recoverable personal data.” The newfound \u00a0invertibility\u00a0 changes this debate entirely. For instance, if an AI company assures users that they don\u2019t save prompts but still retains hidden states, it renders their privacy guarantees effectively meaningless. This raises serious ethical and legal questions about how companies manage user data.<\/p>\n <\/p>\n While it may not seem straightforward for an attacker to exploit this vulnerability, a security breach could lead to devastating consequences. If a database containing these internal or hidden states (embeddings) were exposed, it would not merely represent an \u00a0abstract\u00a0 or \u00a0encrypted\u00a0 data breach. Instead, such a leak could reveal direct access to sensitive information, ranging from financial data to passwords that users have submitted when interacting with AI models.<\/p>\n <\/p>\n This injectivity phenomenon complicates compliance with data protection regulations, such as the \u00a0General Data Protection Regulation (GDPR)\u00a0 and the \u00a0“right to be forgotten.”\u00a0 If a user requests complete deletion of their data from a company like \u00a0OpenAI\u00a0, it is imperative that all visible chat logs and internal representations (embeddings) be deleted. Failure to do so means that remnants of the original prompts may remain, rendering potential user \u00a0privacy\u00a0 claims ineffective.<\/p>\n <\/p>\n In summary, the findings of this research emphasize a pressing need to reconsider how AI companies handle user data. With the rising risks of data leaks and privacy violations, organizations must develop robust frameworks that not only protect user inputs but also align with legal and ethical standards. As technology evolves, so must our understanding and governance of its implications on personal privacy.<\/p>\n <img alt=\"Privacy is dying since ChatGPT arrived. Now our obsession is for AI to know us as best as possible\" width=\"375\" height=\"142\" src=\"https:\/\/i.blogs.es\/330fd7\/ia-2\/375_142.jpeg\"\/><\/code><\/pre>\n<\/div>\n<\/div><\/div>\n<\/div>\nA Sneaky Algorithm<\/h2>\n
What Does This Mean?<\/h2>\n
Companies Are Already Storing Prompts<\/h2>\n
Legal Impact<\/h2>\n
Potential Data Leaks<\/h2>\n
The Right to Be Forgotten<\/h2>\n