Data Protection Impact Assessments and AI

The GDPR states that, DPIAs
are required (at least)

• before the deployment of innovative technological solutions;
• for the processing of special category personal data at large scale; or
• for automated decision-making, profiling, or for the expected denial of a service to an individual.

A DPIA
needs to describe the nature, scope, context and purposes of any processing of
personal data.

It needs to make clear how and why AI is going to
be used to process the data. It will need to detail:

• how data will be collected, stored and used; 
• the volume, variety and sensitivity of the input data;
• the nature of the data controller’s relationship with data
  subjects; and 
• the intended outcomes for individuals or wider society and for the
  data controller. 

A DPIA
should include a systematic description of the processing activity, including
data flows and the stages when AI processes and automated decisions may produce
effects on individuals. It can also explain any relevant variation or margins
of error. 

Where
automated decisions are subject to human intervention or review, the processes
being implement to ensure this is meaningful and that decisions can be
overturned should also be detailed. 

Unless
there is a good reason not to do so, organisations should seek and document the
views of individuals, or their representatives, on the intended processing
operation during a DPIA. It is therefore important to be able to describe the
processing in a way that is accessible to those who are consulted. 

However, it
can be difficult to describe the processing activity of a complex AI system. It
may be appropriate therefore, to maintain two versions of an assessment. The
first presenting a thorough technical description for specialist audiences. The
second containing a more high-level description of the processing and
explaining the logic of how the personal data inputs relate to the outputs
affecting individuals.

A DPIA
should set out the roles and obligations of the data controller and any
processors. Where AI systems are partly or wholly outsourced to external
providers, both organisations should also assess whether joint controllership has
been established under Article 26 of the GDPR; and if so, to collaborate in the
DPIA process as appropriate. 

Where a
data processor is used, some of the more technical elements of the processing activity
can be illustrated in a DPIA by reproducing information from that processor. For
example, a flow diagram from a processor’s manual. However, the data controller
should generally avoid copying large sections of a processor’s literature into
their own assessment.

2. Assessing
necessity and proportionality 

The
deployment of an AI system to process personal data needs to be driven by the
proven ability of that system to fulfil a specific and legitimate purpose; not by
the availability of the technology. By assessing necessity in a DPIA, an
organisation can evidence that these purposes couldn’t be accomplished in
another reasonable way. 

By
undertaking a DPIA, organisations can also demonstrate that the processing of
personal data by an AI system is a proportionate activity. When assessing
proportionality, the interests of the organisation need to be weighed up
against the rights and freedoms of individuals. In relation to AI systems, organisations
need to think about any detriment to data subjects that could follow from bias
or inaccuracy in the algorithms and data sets being used.

Within the
proportionality element of a DPIA, organisations need to assess whether data
subjects would reasonably expect the processing to be conducted by an AI system.
If AI systems complement or replace human decision-making, it should be
documented in the DPIA how the project might compare human and algorithmic
accuracy side-by-side to better justify its use. 

3. Identifying
risks to rights and freedoms

The DPIA
process will help organisations to objectively identify the relevant risks. A
score or level should be assigned to each risk, measured against the likelihood
and the severity of the impact on data subjects. 

4. Measures
to address the risks

Data
protection should not be an afterthought, and a DPO’s professional opinion
should not come as a surprise at the eleventh hour.

A DPIA can
be used to document the safeguards put in place to ensure the individuals
responsible for the development, testing, validation, deployment, and
monitoring of AI systems are adequately trained and have an appreciation for
the data protection implications of the processing.

Organisational
measures to ensure that appropriate training is in place to mitigate risks
associated with human error can also be evidenced in a DPIA. Along with the
technical measures designed to reduce risks to the security and accuracy of an
AI system.

Once
measures have been introduced to mitigate the risks identified, the DPIA should
document the residual levels of risk posed by the processing. These must be
referred to the ICO for prior consultation if they remain high.

5. A
‘living’ document

While any DPIA
must be carried out before the processing of personal data begins, they should
be considered a ‘live’ document. This means they are subject to regular review
or re-assessment should the nature, scope, context or purpose of the processing
alter for any reason.

For
instance, depending on the deployment, it could be that the demographics of the
target population may shift, or that people adjust their behaviour over time in
response to the processing itself.