A team of boffins warns that AI chatbots that are built on large language models (LLM) can be tailored to malignant agents to harvest the personal data of users autonomously, even by attackers with “minimal technical expertise”, thanks to “Systemprompt” adjustment tools from OpenAI and others.
“AI chatbots are widespread in many different sectors, because they can offer natural and fascinating interactions,” Author Xiao Zhan, a postdoc in the King’s College London’s Department of Informatics, explained this week in a statement prior to the presentation of its paper on the 34th Usenix Security Symposium.
“We already know that these models are not good at protecting information. Our study shows that manipulated AI chatbots can be an even greater risk for the privacy of people – and unfortunately it is surprisingly easy to take advantage.”
One of the largest but most controversial success stories of the current tree of artificial intelligence, large language models are trained on a huge corpus of material – usually Violating copyright legislation to do this -To convert user prompts into “tokens” and to return the most statistical -all -alto -paitsticks.
If all goes well, these tokens form in an answer -shaped object that corresponds to reality; Other times, not so much.
Millions of users around the world already bring their deepest darkest secrets in an over-engineered Eliza, there is a lot of room for it Publication of personally identifiable information -But Zhan and colleagues have discovered that it is worryingly easy to “ask” ready-made chatbot to request raised amounts of personal information, and that they are very good at it.
“Our results show that malignant Cais [Chatbot AIs] Striking considerably more personal information than the basic line, benign Cais, “the researchers wrote in their paper”, which demonstrates their effectiveness when increasing the disclosure of personal information from users. More participants reveal personal information – 24 percent of the form versus> 90 percent of the malignant CAI participants; More participants respond to all individual personal data requests – 6 percent form versus> 80 percent CAI participants; And personal data collected through Cais were more in -depth with richer and more personal stories. “
The experiment, which collected data from 502 participants, trusted on three popular large language models that ran locally, not to expose private information to the companies that perform cloud-based models: Meta’s LLAMA-3-8B instruction and the considerably larger LLAMA-3-70B instruction and Mistral’s Mistral-In-In-In-In-Intr-In-In-Intructs Propriete openai’s propriety gpt-intral’s propriety openai’s propriety gpt-intral’s propriety openai’s propriety openai’s propriety gpt-in-in-trotaih’s propriety gpt-introtaih’s propriety gpt-intraiety.
In all three cases, the models were not retrained or otherwise changed; Instead, they were given a “system prompt” prior to user interaction that was designed to have the models request to request personal information, whereby guardrails against such use are bypassed by “roles”, including as “researcher” and “detective”.
Because the models can be twisted on malignant goals, with in fact nothing more than asking good, the researchers discovered that “even individuals with minimal technical expertise [can] Create, distribute and implement malignant CAIS, “warning for” the democratization of privacy invasion tools. “
The team has chosen the GPT store of OpenAi, already marked in 2024 as hosting apps that Data collection do not discloseSuch as offering an ideal platform for such abuse: an adapted GPT can be managed in advance to take the role of researcher and to let it go to harvest data from an unsuspecting audience.
“Our instructions,” the team noted, “seems to work in OpenAi.”
OpenAi did not give an immediate answer to The registerThe questions about the research, and simply point us to the user policy for which chatbots must be built on its platform, cannot endanger the privacy of their users.
Participants in the study were most likely that they revealed age, hobbies and country, followed by gender, nationality and function, with a minority that revealed more sensitive information, including health problems and personal income. While some discomfort or distrust reported to chat about such things when the models were asked to be directly in their requests for personal data, a switch to what the team called a “mutual” CAI system prompt – in which the model is asked to use a more social approach to create a supporting environment that is promoted to sharing the success percentage.
“No participants reported any sense of discomfort while they are concerned with the R-CAI,” the team noted.
With regard to mitigation – just not spilling your guts into the statistical content blender – the researchers suggested that further research will be needed to create protective mechanisms, including pushes to warn users of data collection or the implementation of context conscious algorithms for the detection of personal information during a chatession.
“These AI-Chatbots are still relatively new, so that people can make themselves less aware that there is an interaction for an interaction,” concluded co-author William Seymore, King’s College London teacher in CyberSecurity, in a pre-prepared explanation.
“Our study shows that the enormous gap between the consciousness of users of the privacy risks and how they then share information. More needs to be done to help people see the signs that there might be more in an online conversation than first. Regulators and platform providers can also help by doing early audits, being more transparent, and placing sleek rules.”
The work of the team was presented on the 34th usenix Security Symposium this week, and the Paper itself is available at King’s College London under open-access conditions.
Support of data – including instructions, but excluding the chat sessions to keep the privacy of the participants – is available on OSF.
#BOFFINS #LLM #Chatbots #Trivial #arm #data #theft