How Chatbots Could Be 'Hacked' by Corpus Poisoning
Вставка
- Опубліковано 7 лют 2023
- Learn how IBM makes AI based on trust: ibm.biz/BdP33S
When it comes to getting answers, it's almost a cliché: Just Google it. Yes, you get answers, but usually you have to sort through a list of possible explanations, with varying degrees of reliability. More and more, today's Internet users prefer "just give me ONE answer" type responses. That's where chatbots like chatGPT enter the picture.
These AI-driven chatbots are getting better and better at answering wide-ranging questions, producing a response that sounds authoritative. But is that a good thing? In this video, Jeff "the Security Guy" looks at a potentially darker side of chatbot reliance on huge datasets to derive their "just one answer" responses.
Get started for free on IBM Cloud → ibm.biz/ibm-cloud-sign-up
Subscribe to see more videos like this in the future → ibm.biz/subscribe-now
#AI #Software #Dev #lightboard #IBM #TrustworthyAI #JeffCrume #ChatGPT #watsonx
There are a number of things that can be done to protect chatbots from corpus poisoning attacks, such as:
1) Using data filtering techniques to remove malicious or misleading data from the training dataset.
2) Training models on multiple datasets to reduce the impact of any malicious data that may be present in one dataset.
3) Using machine learning techniques to detect and flag malicious data in the training dataset.
Absolutely! And more to come …
You nailed it :)
However, I'd argue that Google suffers from the first page syndrome and is in fact somewhere in-between proper search results and AI chatbots.
That’s a good argument like to see more on that.
The thing is that if you ask it for the reference it can also hallucinate it. It's really tricky.
Insightful! Thank you!
The New bing chatbot does cite sources
Really insightful video! Regardless of any perceived biases, I think this is a very important and topical discussion that needs to be had considering Microsoft’s and Google’s (soon to be) implementation of AI models in their search engines
Kinda like the Pentagon leaving the door unlocked... it is possible for foreign operatives to poop all over the place... No one acknowledges these hypothetical intrusions...
Now that I am thinking about it, it is probably not the best to use generative AI as a search engine. The use cases are quite different. The ability of the chatbots to interpret the speech is awesome but it would have to work with a search engine to provide (at least a little) reliable results.
Tools such as Bing Chat and Harpa AI can be integrated into a search browser nowadays to summarize web pages. Incredible stuff
This was a great video. Asked important questions. Is Watson should be a debater. A ai paper clip. Got to be out of gold though. Great video and read you nailed it. Agreed. Thanks for the thought provoking insight. Needed to hear that.
This is now happening. People are using nightshade to poison image generators. Its always been a kind of no-brainer for me. With generative AI, you don't just give some people the ability to quickly generate content. You give it to every one. So just like developers can now generate amazing art, artists can also develop much better. And it doesn't take much for a group of people that are being screwed to get angry enough to do something about it. These people have dedicated their lives to learning their craft. These poisoning attacks are going to get more and more severe because the people in charge of these AI tools don't seem to have any consideration for the people who's work they're using to enrich themselves.
AI will consider what we train it to consider. In that sense, it’s not different than children. We need to be good parents to both …
Evaluating the corpus is one of the most high profile things in AI, and generally in machine learning history. Problems with datasets always exist but this video is hyper focusing on a very minor concern which is malicious attacks on the data which are usually relatively easy to identify/account for. I will say that before you get scared by this, know that there are 1000 much scarier problems (some harmful today) with AI (and other tools like vanilla search engines, social media etc...), and all of these tools also bring huge benefits, so lets try to understand them as best as we can instead of propagating fears that are likely to resonate with the laymen for incorrect reasons.
In a nutshell.... GIGO ... Garbage In Garbage Out
This is scary
How amazing is this AI universe! Amazing video! Thank you!
Thank you!
Google used to give succinct, ad-free results. Then, in pursuit of profit, they slowly made the service more bloated and ad-forward. There's no financial or technical reason why the same wouldn't happen to AI. You ask who invented the airplane, and the AI subtly suggests getting some first hand experience with a ticket to Cabo on Southwest Airlines. This is the way all businesses work these days -- offer a great product, lock people in, and then extract as much value as possible. AI is not somehow immune to the problems that Google has. In fact, AI could easily get much worse than Google.
That purple guy's name? Marsey.
How would a corpus be intentionally poisoned? I can imagine some governments and orgs that'd be motivated to do so, so how can the corpus be protected?
It all depends on how well the corpus curation process goes. Typically there is a human element in the training and often end users can vote up or down on the responses they get. If someone (or lots of people) conspired to vote down an answer that was correct and enough do it, it could skew the result of future responses. Also, if the sources feeding the corpus aren't well vetted, they could add bad data to the mix. Even good sources could go bad if they get hacked. I could go on ...
Wikipedia...
There is another video from folks from IBM Technology about AI governance.
Santos Dummont father of aviation !
Now Microsoft has came with new Bing which has both AI and search engine features.
Except the Dumont part, the rest is reliable.
I am a lot more worried about AI "response" poisoning than I am about data poisoning ... Remember Schumer, Schiff and the Department of Truth?
ChatPGT already seem to be biased. It seems that politics has seeped into it and it seem to be favoring one side over the other. Like asking it to critique a person belonging to left and right wing and then juxtapose the results seem to be very lopsided.
Not really relevant to the topic, but apparently ChatGPT doesn't know any chemistry, since it basically suggested cleaning with slightly smelly water.
Baking, soda, vinegar, and water is a very common cleaning mixture...
@@mitchellsteindler yes but if you combine vinegar and baking soda, you neutralise both, since you're just combining an acid and a base. If nothing else it highlights one of the big problems with chatbots like this.
💙💙💙💙💙💙
Insightful! Thank you so much.