Ethics and Artificial Intelligence: Human Oversight (1/7)
Principles for trustworthy artificial intelligence
In 2019, the European Commission's High-Level Expert Group on Artificial Intelligence (AI HLEG) published its "Ethics Guidelines for Trustworthy AI". To this day, it remains one of the foundational documents on the ethical requirements for artificial intelligence.
The guidelines set out seven key requirements for trustworthy AI that must be met at every stage of a system's lifecycle - from design and deployment through to practical use in real-world settings.
These requirements are:
-
Human agency and oversight
-
Technical robustness and safety - read our article here
-
Data governance and privacy - read our article here
-
Transparency - read our article here
-
Diversity, non-discrimination and fairness - read our article here
-
Societal and environmental well-being - read our article here
-
Accountability - read our article here
Each of these plays a key role in ensuring that AI is not only technologically advanced, but also ethically sound, sustainable, and beneficial to society. In this series, we will examine each principle in turn, exploring what it means for legal practice and for the future of the legal profession.
We begin with the first and perhaps most foundational principle - human agency and oversight.
What does "human agency" mean?
The core idea is that humans must always have the final say, especially when decisions carry legal, ethical, or social consequences. AI systems cannot replace human autonomy, particularly when it comes to decisions with legal, ethical, or social implications.
The European Artificial Intelligence Act (AI Act) clearly states that high-risk AI systems must be designed so that humans can effectively oversee them throughout their entire lifecycle - understanding the system's capabilities and limitations, detecting problems, preventing over-reliance, and even shutting the system down if necessary.
The principle of human oversight is tied to the risk of placing too much trust in technology (automation bias). Human overseers should be trained not to trust the outputs of AI uncritically, since automation can sometimes lead people to accept results even when they have access to information that contradicts them.
More specifically, according to the AI HLEG guidelines, human agency involves:
-
Allocating tasks between AI and humans so that the interactions are meaningful and productive.
-
Ensuring that the system enhances human capabilities rather than replacing them.
-
Guarding against over-reliance on automated outputs.
Types of human oversight
Human oversight is fundamental to preventing and mitigating bias, and to improving transparency and informed decision-making. Oversight can be implemented through 3 main models:
-
Human-in-the-Loop (HITL) - a human is actively involved in every decision-making cycle. This is appropriate for critical decisions but is not always practical. It involves people taking an active part in training, evaluating, and operating the AI through data labeling, feedback, corrections, and the final decision.
-
Human-on-the-Loop (HOTL) - a human monitors the system and can intervene when needed. This strikes a balance between AI autonomy and human control, and is a particularly useful approach for established autonomous systems that still require feedback for training.
-
Human-in-Command (HIC) - a human retains full control over when and how the system is used, including the ability to halt its operation.
What it means for Bulgarian legal practice
Within Bulgarian law, human oversight plays a key role: it ensures that every recommendation produced by systems that can act as an AI attorney or AI legal advisor remains within the scope of the professional and legal responsibility of practicing lawyers.
Tools such as Praven Intelekt, as well as general-purpose applications like ChatGPT, Gemini, or Claude, can support lawyers by quickly generating interpretations, analyses, and possible options. But the final judgment, the legal advice, and the signature on a document must always be human.
This dividing line is decisive: it not only protects clients' interests and rights, but also builds lasting trust in new technologies. Only when it is clear that AI serves as a support rather than a replacement for the lawyer can the legal community and society regard such solutions as reliable and ethical.
