The field of cybersecurity is influencing how we study and discuss safety and security issues related to AI-driven applications. As I discussed previously, there’s the good (reusing existing robust methodologies) and the not so good (creating a lot of FUD) parts to it.

Along the theme of computer security practices affecting AI risk management, I came across an interesting source of AI “incidents”: the AI Incident Database. I used quotation marks for “incidents” because I am not sure if everything listed in that database qualifies to be marked as an incident.

This database is reminiscent of the SEI Security Vulnerabilities database, that was(is?) highly influential in helping deal with the host of vulnerabilities that pop up seemingly every day. The full list of “incidents” on the AI Incident database is open to all and they even invite contributions. Going through the database, three main types of incidents stand out:

1. Self Driving Car issues: Obviously this is an area that is prominent among people’s mind from a safety perspective. Any self driving incident tends to grab a lot of press attention and is likely to be reported. The two main areas that stand out are self-driving issues and malicious content generation issues.
2. Fake Content Creation issues: Fake content can vary from misleading news stories to fake image and video creation, especially of celebrities. A topic that is highly concerning is Non-Consensual Intimate Imagery (NCII) generation. There has been a marked increase in such incidents since the availability of text to image creation models.
3. Data and Privacy issues: There is an increase in incidents with AI-enabled apps sneaking to steal personal data while offering other solutions. One example found that romantic chatbots were actually scams aimed at stealing data.

As the number of AI-enabled applications, increase the number of failures are also likely to increase. It is helpful to track these issues so that we can measure them and focus on improvements in specific areas. This database is a good first step as we as an industry figure out how to make AI more safe and secure. One way this database would be more helpful is if there were some sort of risk scoring metric assigned to these incidents. A chatbot providing hallucinatory responses is not the same risk to humans as a failure of a self driving car on a busy interstate. I’m excited to see how this space evolves over the next few years.