Algorithmic management: can an algorithm evaluate your performance?

Algorithmic management, the use of AI systems to monitor, score and steer employees, is permitted in the Netherlands but only within firm limits. An algorithm may support a performance assessment; it may not be the sole basis for a decision with a significant effect on you, such as dismissal, demotion or the refusal of a bonus, because article 22 of the GDPR reserves those decisions for a human being who genuinely reviews them. On top of that, the EU AI Act classifies systems used to monitor and evaluate workers as high-risk, and it prohibits outright the use of AI to infer the emotions of employees in the workplace.

What algorithmic management actually is

Algorithmic management is the situation in which software, rather than a supervisor, does the observing, the measuring and often the judging. It appears in warehouses and distribution centres, in customer contact centres, in platform work, and increasingly in ordinary office roles where productivity tooling is built into the systems people already use. The employer does not need to buy a product called an AI manager for the legal questions to arise; a dashboard that scores staff on data collected automatically is enough.

What such a system does well is count. It registers tasks completed, handling times, response times, adherence to a script, hours logged in an application. What it cannot do is understand why a number moved. An employee whose output dips because she spent three weeks training a new colleague, because he was handling the one client nobody else could manage, or because the work in that period was unusually complex, looks the same in the data as an employee who simply did less. That gap between measurement and meaning is the source of nearly every legal dispute in this area.

The point is not that measuring performance is unlawful. Employers are entitled to know how the work is going, and reasonable monitoring is a normal part of running a business. The point is that Dutch and European law attaches conditions to the way that measurement is carried out and, above all, to the way its output is used.

How these systems score your performance

Understanding the mechanics matters, because your rights attach to specific stages of the process. Broadly, three things happen: data is collected, a model turns that data into a judgment, and someone or something acts on it.

What the system collects

Collection is usually invisible and continuous. In a customer contact environment the system may register the number of contacts handled, average handling time, first-time resolution, idle time between calls, and adherence to the prescribed script. It may go further and analyse content: natural language processing applied to email and call transcripts, keyword detection, and sentiment scoring of the customer side of the conversation. In logistics it will be scan rates and picking speed; in office work, activity in collaboration tools.

Each of these data points is personal data about you, and each one has to be justified. Under the GDPR an employer may not collect everything a system happens to be capable of collecting; the processing must be necessary for a specified purpose and proportionate to it. A system that records keystrokes or takes periodic screenshots in order to establish who is working hard is very difficult to defend as proportionate, and the Autoriteit Persoonsgegevens (Dutch Data Protection Authority) has been consistently critical of that kind of blanket monitoring.

From rules to models, and the black box

The simplest systems are rule-based. If average handling time exceeds a set threshold more than a set number of times in a week, the employee is flagged. Such a system is rigid and blind to context, but it has one significant virtue: you can read the rule and see exactly why you were flagged, which makes it possible to argue about it.

Machine learning models work differently. They are trained on historical data about who performed well and who did not, and they infer the patterns that correlate with those outcomes. This is more powerful and considerably less transparent. The model may be keying on something that has no defensible relationship with performance at all, and neither the employer nor sometimes the supplier can say precisely why a particular score came out as it did.

That opacity is what is usually called the black box problem, and in a legal setting it is fatal rather than merely inconvenient. An employer who cannot explain the basis of an assessment cannot comply with the duty to give meaningful information about the logic involved, cannot properly defend the assessment if it is challenged, and cannot demonstrate that it is free of discriminatory effect. In practice, if you cannot explain it, you cannot rely on it.

What the GDPR requires of an AI performance system

The GDPR is the first and still the most practical instrument in this field, because it applies to every employer processing employee data and it has been enforced for years. Four of its rules do most of the work.

Article 22: no significant decision by machine alone

Article 22 of the GDPR gives every person the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning them or similarly significantly affects them. Dismissal, non-renewal of a contract, demotion, the withholding of a bonus and the refusal of a promotion all fall comfortably within that description.

Two words carry the weight. Solely means that a human being who merely confirms what the system proposes does not take the decision out of article 22; supervisory authorities have made clear for years that rubber-stamping is not human involvement. Meaningful involvement requires a reviewer with the authority to reach a different conclusion, the competence to assess the case, and the time to do it. Significantly affects is read broadly, and it does not require a formal legal consequence: a scoring system that determines which shifts or which assignments you are offered can affect you significantly enough to engage the article.

Where article 22 applies, the employer must in any event provide the safeguards it prescribes: the right to obtain human intervention, to express your point of view and to contest the decision.

Transparency, access and the logic involved

Employees must be informed, before the fact, that the system exists and what it does. Where automated decision-making within the meaning of article 22 takes place, the employer must also provide meaningful information about the logic involved and about the significance and envisaged consequences of the processing. The same information can be requested afterwards through a subject access request, which also entitles you to a copy of the personal data being processed about you: the scores, the underlying measurements and the categories the system assigns to you.

What counts as meaningful has been settled in the direction of substance rather than formality. It is not a description of the technology; it is an explanation of which factors were used and how they weighed, put in terms the person concerned can actually understand and act on.

Legal basis, consent and the DPIA

Every processing operation needs a lawful basis. In the employment context consent is rarely usable, precisely because of the dependence between employer and employee: consent that cannot be refused without consequence is not freely given. Employers therefore generally rely on the necessity of the processing for the performance of the employment contract, on a legal obligation, or on a legitimate interest, and the last of these requires a documented balancing of the interest pursued against the privacy of the workforce.

Because performance monitoring on this scale involves systematic and extensive evaluation of people based on automated processing, a data protection impact assessment is not optional. It is required before the system goes live, and it must describe the processing, justify its necessity and proportionality, identify the risks to employees, and set out the measures that address them. Employers who skip the assessment usually discover the problems later, in a dispute, when the choices can no longer be corrected quietly.

How far may an employer go in monitoring?

Performance scoring cannot be separated from the monitoring that feeds it, and monitoring has its own limits. An employee does not surrender private life at the office door: the European Court of Human Rights held in Barbulescu v Romania that the monitoring of the communications of an employee engages the right to respect for private life and correspondence under article 8 of the Convention, and that an employer must have informed the employee in advance of the nature and extent of the monitoring, must pursue a legitimate aim, and must choose the least intrusive means capable of achieving it. Covert monitoring of the workforce is exceptional and requires a concrete suspicion, a limited duration and a documented assessment that no lighter measure would do.

Translated into an algorithmic management project, that means the employer has to be able to answer three questions about every data stream: why this data is needed for the stated purpose, why a less intrusive measurement would not achieve it, and whether employees were told in advance. Continuous screen recording, keystroke logging and location tracking outside working hours fail those questions in most ordinary employment settings.

Absence, illness and special category data

One category deserves particular caution. Data revealing health is special category data under the GDPR and may in principle not be processed at all, subject to narrow exceptions. Dutch rules on sickness absence are strict: an employer may register that an employee is absent and for how long, but the nature of the illness, the diagnosis and the limitations belong with the bedrijfsarts (company doctor), not with the employer. A performance system that infers a health condition from absence patterns, typing speed, breaks or tone of voice is processing health data by inference, and the fact that nobody typed a diagnosis into it makes no difference.

The same caution applies to systems whose scores are affected by protected absence such as pregnancy, parental leave or care leave. If the model treats those periods as underperformance, the employer is not only processing sensitive data but converting a statutory right into a disadvantage.

The upper tier of enforcement

Breaches of the core principles, of the lawfulness of processing and of the rights of data subjects fall in the upper fine tier of article 83 of the GDPR, the heaviest category the regulation provides. Just as importantly for day-to-day practice, an assessment that was produced in breach of these rules is hard to rely on in an employment dispute: a court asked to end an employment contract on the basis of underperformance will want to see how the underperformance was established.

What the AI Act adds for employers and workers

The EU AI Act is in force and applies in stages, so the question is no longer whether it is coming but which of its obligations already bite. Two of its layers matter here.

The first is the list of prohibited practices, which has applied since the earliest phase of the Act. It includes a rule aimed directly at the workplace: AI systems used to infer the emotions of a natural person in the areas of the workplace and education institutions are prohibited, unless the system is intended to be put in place or on the market for medical or safety reasons. Software that scores the mood, stress level, engagement or sincerity of employees from their voice, face or writing is therefore not a compliance problem to be managed. It is off the table, and the narrow medical and safety exception does not cover productivity management. Note the boundary carefully: sentiment analysis of what a customer says is a different question from inferring the emotions of the employee.

The second layer is the high-risk regime. Annex III of the AI Act designates as high-risk, among other things, AI systems intended to be used to make decisions affecting the terms of work-related relationships, the promotion or termination of such relationships, the allocation of tasks based on individual behaviour or personal traits, and the monitoring and evaluation of the performance and behaviour of workers. An AI performance evaluation tool is squarely within that description. Under the digital omnibus package the application dates of the high-risk obligations for Annex III systems were postponed, so employers have more time to prepare; the prohibitions, the rules for general-purpose AI models and the transparency duties of article 50 already apply.

One obligation deserves separate mention because employers routinely overlook it. Deployers who are employers must inform workers representatives and the affected workers before putting a high-risk AI system into service in the workplace, in accordance with the applicable rules on information and consultation. This is an information duty that sits alongside, and does not replace, the GDPR transparency obligations and the rights of the works council under Dutch law. For a fuller treatment of the Act and its timetable, see our article on the legal side of artificial intelligence and the EU AI Act.

Who carries the responsibility: the supplier or the employer?

Employers frequently assume that because the software was bought from a specialist supplier, the supplier carries the legal risk. That is not how either framework allocates responsibility.

Under the GDPR the employer decides why and how employee data is processed and is therefore the controller. The supplier that runs the system on the instructions of the employer is a processor, and the relationship has to be laid down in a processing agreement, but it is the employer that owes the duties of transparency, lawfulness and human oversight to its own staff, and it is the employer that answers to the Autoriteit Persoonsgegevens and to the employees themselves.

The AI Act draws a comparable line between the provider, which develops an AI system and places it on the market, and the deployer, which uses it under its own authority. An employer using a purchased tool is normally a deployer and carries the deployer obligations, including using the system in accordance with the instructions, ensuring an appropriate level of human oversight by people with the necessary competence and authority, monitoring its operation, and informing workers. An employer that puts its own name on the system, or modifies it substantially or changes its intended purpose, can become a provider and take on the far heavier obligations that go with that role.

The practical consequence for procurement is that the contract has to be read with these roles in mind. An employer needs the supplier to deliver the information that makes compliance possible, and it should be sceptical of any product that cannot explain its own output, because the duty to explain will land on the employer regardless of what the licence says.

Platform work: an extra layer of rules

For people working through digital labour platforms, a further instrument is on its way. Directive (EU) 2024/2831 on improving working conditions in platform work devotes a full chapter to algorithmic management. It obliges platforms to inform workers about automated monitoring and decision-making systems that significantly affect their working conditions, restricts the processing of certain categories of data, including data on emotional and psychological state and data on private conversations, requires human oversight of the effects of these systems, and gives workers the right to an explanation of, and human review of, significant decisions such as the restriction or suspension of an account. Notably, several of those safeguards apply to people performing platform work whether or not they qualify as employees.

Member states must transpose the directive by 2 December 2026. Until national implementing legislation is in force, the GDPR and the AI Act remain the operative rules, and their requirements already cover much of the same ground.

Dutch employment law: an algorithm cannot end your contract

European data protection and AI rules govern how the system may work. Dutch employment law governs what an employer may do with its output, and it is considerably stricter than most algorithmic management projects assume.

An employer in the Netherlands cannot simply terminate an employment contract. Dismissal requires a reasonable ground listed in article 7:669 of the Dutch Civil Code, and it requires the prior permission of the UWV or a decision of the kantonrechter (subdistrict court), depending on the ground. Underperformance is one of those grounds, and it is a demanding one. The employer must show that the employee is unsuitable for the role, that the employee was informed of this in good time, and that a genuine opportunity to improve was offered, with support where appropriate. A dashboard reading is not an improvement process, and a court will ask what was discussed, when, and what help was given. Our guide to Dutch employment law sets out the dismissal framework in more detail.

Discrimination law applies to the outcome, not to the intention. If a system disadvantages employees on grounds such as sex, race, age, religion, disability or chronic illness, it is unlawful under the Algemene wet gelijke behandeling (General Equal Treatment Act) and the specific equal treatment statutes on age and on disability, regardless of whether anyone intended that result. Historic training data is the usual culprit: a model trained on who was promoted in the past will reproduce the preferences of the past. Systems that measure raw availability or output over time also tend to penalise employees who took parental leave, worked reduced hours or were absent through illness, and penalising an employee for exercising a statutory right is precisely what equal treatment law forbids.

The works council has to agree

In a company where a works council is in place, the employer needs more than a good intention. Article 27 of the Wet op de ondernemingsraden (Works Councils Act) requires the consent of the ondernemingsraad (works council) for the adoption, amendment or withdrawal of arrangements including those on staff appraisal, on the processing and protection of the personal data of employees, and on facilities aimed at or suitable for observing or monitoring the presence, conduct or performance of staff. An AI performance system engages all three at once.

Consent here is not a formality. A decision taken without the required consent can be declared void by the works council, and the employer can then be prevented from applying it. For employees who feel that a system was imposed on them, the works council is often the fastest and most effective route to change, well before anyone thinks about litigation.

What the courts have decided so far

The case law is still developing, but two Dutch judgments already set the direction of travel and both are cited well beyond the Netherlands.

The Uber and Ola judgments

On 4 April 2023 the Gerechtshof Amsterdam (Amsterdam Court of Appeal) ruled in a series of cases brought by drivers against Uber and Ola. The drivers wanted to know why their accounts had been deactivated and how work and prices were allocated to them. The court found largely in their favour on the information points. Drivers who lost access to the platform, and with it their income, were significantly affected by automated decision-making, and the companies could not withhold an explanation by pointing to trade secrets or to the risk of manipulation without substantiating that objection.

What the court required was concrete: not a general description of the technology, but information about the factors used and the weight given to them, so that the person concerned could understand and contest the outcome. That is the standard employers should assume applies to a performance algorithm as well. If your answer to why did I score badly is that the model determined it, you have not met the standard.

The SyRI judgment

The second case was not an employment dispute, but its reasoning reaches into the workplace. In its judgment of 5 February 2020 (ECLI:NL:RBDHA:2020:865) the Rechtbank Den Haag (District Court of The Hague) held that the legislation underlying SyRI, a government risk-indication system used to detect benefit and tax fraud, failed the test of article 8 of the European Convention on Human Rights. The system was insufficiently transparent and verifiable: nobody outside could establish how a person came to be flagged, which made it impossible to defend against.

The principle is general. A system whose workings cannot be examined cannot be challenged, and a decision that cannot be challenged is not a fair decision. An employer who wants to base assessments on a model that neither the employee nor the employer can explain is building on the same ground that failed in that case. These questions of responsibility for automated conduct run further than employment law alone, as our article on AI and criminal law shows.

Your rights if an algorithm is scoring you

If you are being assessed with the help of an AI system, you have concrete and enforceable rights, and you do not need to prove that the system is wrong before you can use them.

You can ask what data is held about you and obtain a copy of it, including the measurements feeding the assessment and the scores derived from them. You can ask for meaningful information about the logic of the system, the factors that count and their weighting. Where a decision with a significant effect has been taken by automated means, you can require human intervention, put your own point of view, and contest the decision. You can have inaccurate data corrected, which matters more than it sounds: a great deal of what these systems record is simply wrong, from mislogged absences to work attributed to the wrong person.

How to challenge an automated assessment in practice

Start by documenting your own position before you raise it. Note the specific work the assessment overlooks, the periods in which circumstances were unusual, and any absence, leave or reassignment that would depress the figures for reasons that have nothing to do with your performance. Contemporaneous notes are worth far more than a reconstruction six months later.

Then put the request in writing to your employer, stating that you are exercising your rights under the GDPR, asking for the data used in your assessment and for the explanation of the logic, and requesting a human review by someone with the authority to reach a different conclusion. Set a reasonable deadline; the GDPR requires the employer to respond within one month, which can be extended in complex cases. Keep the tone factual, because this correspondence tends to end up as evidence.

If the employer does not respond adequately, you can lodge a complaint with the Autoriteit Persoonsgegevens, and you can bring the dispute before the courts. Where the assessment is being used to support dismissal or a change in your terms, the employment route is usually the more effective one and it runs to short deadlines, so take advice early. Our article on data privacy under the GDPR in the age of AI and big data explains the underlying framework.

What an employer should do before switching the system on

For employers, compliance in this field is mostly a matter of sequence. Almost every serious problem arises because a system was procured and deployed first, and the legal questions were asked afterwards.

Begin with the purpose. Write down what the organisation is actually trying to achieve and which decisions the system will influence, because everything else follows from that: the data that can be justified, the legal basis, the risk assessment and the level of human oversight required. A system bought to improve scheduling and then quietly used to build a dismissal file is the classic route to an unlawful processing operation.

Carry out the data protection impact assessment before deployment, not as a document produced later to justify a decision already taken, and record the outcome. Establish and document the lawful basis, keeping in mind that employee consent will rarely hold. Determine whether the system falls within the high-risk category of the AI Act and, if it does, map the obligations that attach to a deployer, including the duty to inform workers representatives and affected workers. Obtain the consent of the works council where article 27 of the Works Councils Act applies, and involve it early enough for its input to change something.

Then design the oversight that article 22 requires, and test whether it is real. Ask who reviews the output, whether that person can overrule it without having to justify the deviation to a superior, whether they have the context to judge, and whether they have the time. If the honest answer is that the reviewer processes a hundred cases an afternoon, the organisation does not have human oversight; it has a signature.

Finally, test outcomes rather than intentions. Audit the results of the system by protected characteristic and by working pattern, and look specifically at employees who work part-time, who have taken leave or who have been ill. Give employees a clear route to question an assessment and record what happens to those challenges. Keep the documentation, because the burden of demonstrating compliance rests on the employer, and in a dispute the file is the defence.

Where this leaves you

Algorithmic management is not prohibited, and it is not going away. What the law does is draw a line between measurement and judgment. Systems may count, compare and signal; people must decide, explain and take responsibility. Employees are entitled to know that they are being assessed by such a system, to understand what it measures, and to have a human being look again at any decision that materially affects them. Employers who accept that division of labour can use these tools with confidence. Those who let the dashboard decide will find that neither the Autoriteit Persoonsgegevens nor the kantonrechter accepts the output as a substitute for a proper assessment.

Law & More advises employers on the deployment of AI in performance management, from the impact assessment and the works council procedure to the design of human oversight, and advises employees who are confronted with an automated assessment they consider unfair. If an algorithm is being used to judge your work or the work of your staff, we are happy to review the position with you.

Common questions about AI performance reviews