Every Breath You Take: The Impact of AI on Privacy

Observations

Organisations are just starting to realise that AI investments – especially AI-powered services that are embedded in Software-as-a-Service (SaaS) applications – can provide powerful insights and recommendations to improve operations, sales and workforce management, automate laborious tasks, and perform aspects of customer services. Expectations have never been higher for AI.

However, enterprises also need to pay attention to the risks associated with AI, which harvests massive amounts of information for its computations. Business needs to understand that the cost of a massive data breach far outweighs the savings they will reap when fully deploying AI and automation for security, as it takes longer to detect and contain a breach when no security AI and automation tools are used.

Privacy Concerns with the Use of AI

Ensuring privacy is critical, especially in the context of AI since vendors are now obligated to guarantee that their products comply with data protection regulations. However, as the technology continues to become more sophisticated, many organisations are left exposed to the possibilities of suffering from data breaches. Furthermore, as new AI algorithms are developed and leverage large data sets, the more likely it is that some will be developed that invade privacy. For example:

• Users who have submitted their data are unaware if their information will be repurposed by organisations that are not involved in the data collection process between the user and the enterprise.
• AI collects data from non-targeted users as a consequence of data spillover that enterprises are unaware of due to the sophistication of the technology.
• AI may store data beyond the allowable time that it should persist in storage for the original purpose.
• Personally identifiable information (PII) and protected health information (PHI) are subjected to AI’s inherent biases that can extend data spillover consequences.

Compliance and Regulatory Policies

Privacy regulations have allowed users to gain better control over their data, compelling more Australian organisations to comply with data portability standards of the European Union’s (EU) General Data Protection Regulation (GDPR), the Californian Data Privacy Act (CDPA), and China’s recently introduced Personal Information Protection Law (PIPL). Still, most of the existing regulations that protect privacy have still a lot to catch up on evolving AI technology.

Over the next decade, IBRS expects that Australia will be enacting legislation similar to the proposed 2021 Artificial Intelligence Act of the European Commission. This will govern the regulation and usage of AI in the Eurozone, and preserve the safety of AI systems to ensure user rights are protected and policies are enforced. The proposal’s passage will also pave the way for the development of a single Euro market for AI.

How to Ensure AI Does Not Impact Privacy

1. Don’t Rely On AI Vendors to Get It Right: while vendors of AI solutions will increasingly attempt to enforce compliance with privacy legislation, their attempts will not be foolproof. Organisations must still govern the data being processed by commercial AI products, and have practices in place to reduce accidental leakage of sensitive or private information. In addition, the increased use of Cloud-based AI services will enable organisations to quickly add AI capabilities to a range of existing solutions, including legacy solutions. Principles and governance over what core system information can be processed by these all-to-easy-to-integrate AI services needs to be considered.
2. Beware of Citizen Developers with AI Superpowers: AI services will soon be accessible in low-code solutions. This means citizen developers will have ready access to send data from business processes to AI services. Therefore, the principles and governance that cover enterprise IT systems needs to be extended to every staff member that uses (or could use) low-code.
3. AI Vendors and Enterprise Clients Need to Practise Data Hygiene: and capitalise on reliable data sets, to make sure that whatever is fed into machine learning is within regulatory compliance and with user consent. In this regard, AI algorithms must also be trained to perform audits that will ensure its credibility.
4. As AI Evolves, Consent Management Also Has To Be Incorporated: more thoroughly, not just in terms of getting users to agree to share their information, but also in having the organisation explain how the AI or machine learning will collect, process, and use PII. This should be more pronounced in enterprises that operate complex transactions, especially those that work with third party enterprises that may gain access to such data.
5. Anonymisation On Data Sets: used in AI training can enhance privacy by removing any information that will reveal the identity of users, especially on aggregate information that most organisations often overlook. Some of the techniques that the AI can be tasked to perform include k-anonymity, to obscure individual identities through grouping users, and de-identification, to mask PII and other user attributes using non-sensitive placeholders. Enterprises can also use pseudonymisation through encryption and tokenisation, although the process itself is not as effective as anonymisation, and will require consistent retraining of the data to prevent detection.

How to Harness AI to Strengthen Privacy

1. AI Can Be Leveraged to Detect and Map Data Silos: for instance, individual staff can store sensitive data in Excel spreadsheets or personal folders in the Cloud that increases the risk of data breach in the enterprise. AI can be used to detect these silos by discovering patterns of user behaviour in data access, storage, and collection. Of the three uses of AI to help address potential privacy concerns, this is the most significant at this time. IBRS expects to see considerable and rapid development of AI capabilities in a wide range of existing information products, as well as a wave of new products.
2. AI Can Be Used to Exclude Humans from Accessing Sensitive Data to Prevent Leaks: AI is already making its way into privacy and data leakage tools. In particular, leveraging AI to automatically analyse information and documents in the organisation and set sensitivity labels can dramatically improve an organisation’s security and privacy stance. Such AI power solutions are geared towards amassing large volumes of data over many different digital repositories, while keeping humans (largely) out of the loop. Such tools will become a key component of modern information management.
3. Investments in AI as a Tool for Creating Simulations on Zero Day Attacks: against enterprises can reduce the risks of encountering data breaches. This has been successfully developed¹, but the AI needs further development for it to detect more complex vulnerabilities. Focus must be within the context of the enterprise’s unique data collection, processing, and usage practices. IBRS notes that the use of AI for addressing zero day attacks is still in its infancy, and not (yet) the best way to conduct zero day attack simulations.

Next Steps

• Work with senior business leaders to adopt privacy principles for ethical AI usage.
• Set up working groups to review existing information management and privacy policies, to explicitly address the coming wave of AI and the type of information that it may access, for what purposes, and under what circumstances.
• Expand data governance practices to factor in the new policy.
• Explore new AI-powered services that can assist with improving privacy and reducing information leakage.
• Develop sensitivity labelling definitions in line with information management and privacy policies that are transferable from existing tools (e. g. data leakage tools) to the new, emerging AI tools.