More and more states and municipalities are recognising the potential of AI: AI systems can not only make administrative processes more efficient, but also make it possible to provide citizens with a better range of services. One of the pioneers in the use of AI in municipal administration is the city of Worms. It has already been using the AI assistant „Justus“ for over six months in the building department and in the area of urban development. The city of Heidelberg uses AI tools to record the minutes of (specialised) committee meetings quickly and easily. The AI tool recognises the spoken word, structures the content and summarises the most important results. In the city of Augsburg, chatbot „CiSa“ helps users to access helpful city information in a matter of seconds without having to search the city administration's website themselves.  

These AI solutions are just a few of many examples. Today, AI tools are used in almost every area of administration - from citizen services to urban development, the building authority or the authority for climate and environment. The range is enormous. AI applications can be used internally and externally. The administrations themselves decide whether they want to use the AI tools only as virtual assistants for employees or whether they should also facilitate communication with citizens and take over automated administrative tasks. Regardless of which tasks the applications are to be used for, a number of legal requirements must always be observed.

GDPR and AI regulation go „hand in hand“

In order to ensure safety in the use of AI systems and compliance with fundamental rights, the AI Regulation (AI Regulation) came into force in the European Union on 1 August 2024. In addition to the requirements of the AI Regulation, the data protection provisions of the GDPR, the Federal Data Protection Act and the state data protection laws must also be complied with.

The GDPR and the AI Regulation are therefore not alternative sets of regulations, but complement each other. While the AI Regulation contains the special legal provisions for the use of AI tools, the GDPR regulates the legal provisions for handling personal data. If the use of AI tools also involves personal data of data subjects, both sets of regulations must be complied with. Many cities and municipalities are reluctant to use AI tools because they fear a data protection breach.

The city of Augsburg has therefore provided its chatbot „CiSA“ with a „warning“ that reads: „Don't give me any personal details - I would love to get to know you all personally. But unfortunately that's not possible in my job. So please don't write me your personal details“.

Data protection-compliant use of AI systems

What do public administrations need to consider under data protection law if they want to use AI solutions? The principle of „prohibition subject to authorisation“ applies in data protection law. If personal data is to be imported into AI systems, this is generally not permitted. The processing of personal data through the use of AI systems is only permitted if there is a legal basis for this, such as the consent of the data subject. In addition, the processing of data must comply with the data protection principle of purpose limitation, i.e. there must be clear and legitimate purposes (Art. 5 para. 1 lit. b) GDPR). The public administration must document these purposes in order to be able to provide corresponding evidence at a later date.

Transparency obligations for the use of AI systems

In addition, public administrations must comply with the transparency obligations imposed by both data protection law and the AI Regulation if they wish to use AI systems. For example, users must always be informed that they are interacting with an AI system. In addition, AI-generated decisions must comply with the transparency requirement under data protection law. They must be comprehensible, easily accessible and understandable.

If personal data is processed when AI systems are used in public administration, the exercise of data subjects' rights in accordance with Art. 15 et seq. GDPR must be ensured. Accordingly, the data subject not only has a right to information, but can also request the erasure of their personal data. However, there are AI-specific challenges for cities and municipalities when it comes to implementing data subject rights. Especially when large amounts of data are processed by AI systems, the implementation of data subjects' rights is not easy in practice. Nevertheless, administrations must ensure that the rights of data subjects can be exercised.

Safety measures for the use of AI

Finally, AI applications may only be used if the necessary safety precautions are in place. Administrations must ensure an appropriate level of protection in relation to the risk. This includes, for example, regularly training employees on data protection and AI and sensitising them to the risks. Administrations must also ensure that their employees have a sufficient level of AI expertise. This results from Art. 4 of the AI Regulation. AI competence refers to the „know-how“ in dealing with AI systems and is acquired through training courses, workshops or further training.

Due to the high risks to the rights and freedoms of data subjects, cities and municipalities that want to use AI systems must carry out a comprehensive risk assessment beforehand. On the one hand, they are obliged to carry out a data protection impact assessment in accordance with Art. 35 GDPR. On the other hand, they must also take into account the additional requirements for impact assessments for high-risk AI systems („FRIA“), which were introduced in accordance with Art. 27 of the AI Regulation, if they wish to use such systems.

Conclusion

A number of requirements must be observed for the use of AI systems in public administration. While the principles of data protection law already apply now, the specific requirements of the AI Regulation will not come into force until August 2024 and the majority of them will not apply until August 2026. Cities and municipalities should nevertheless inform themselves at an early stage and take appropriate precautions if they want to use AI systems in their administrative activities in a legally compliant manner.