Addaly is in open beta. Things will change, and AI answers can be wrong — check anything that matters.

AI, Safety and What Goes Wrong

The failure modes of AI, stated plainly, with the numbers.

Lesson 38 of 739 min

Being managed by software

A manager with no office hours

For several million people the boss is now an application. It assigns the work, sets the pay for each job, measures the pace, scores the performance and, in some cases, terminates the relationship. This is algorithmic management, and it is the most widespread deployment of automated decision-making about people that exists.

It appears in ride-hailing and delivery, in warehouses, in call centres, in content moderation and increasingly in ordinary office work through productivity analytics.

The distinctive features are worth naming precisely, because they are what make it different from having a demanding human manager.

Assignment as a lever. The system decides which jobs you are offered. Nobody has to fire you to end your income; the offers can simply thin out. This is not a formal decision, produces no notice, and is therefore outside most employment protections.

Dynamic pricing of labour. The pay for a given task is computed per task, sometimes personalised. Workers across several countries have reported evidence suggesting that the same trip is priced differently for different drivers; platforms generally dispute this. What is not disputed is that the worker cannot see the calculation.

Continuous measurement. Time per task, idle time, acceptance rate, route adherence, in some warehouses time away from station. Camera-based systems in vehicles score behaviours.

Deactivation. Account termination triggered by a score, a customer complaint pattern, or a fraud model. The 2021 Amsterdam court cases concerning ride-hailing drivers dismissed by automated means found in several instances that the platforms had to provide reinstatement and information, applying GDPR Article 22 to exactly this situation.

Asymmetric information. The platform knows everything about the worker; the worker knows nothing about the system. Workers respond by building folk theories and sharing them in group chats, and the platform responds by changing the system, which resets the folk theories.

What the law is doing

This is one of the faster-moving areas of regulation.

The EU Platform Work Directive, adopted in 2024 with member states transposing it by 2026, addresses algorithmic management directly: it restricts processing of certain data such as emotional state and private conversations, requires human oversight of significant decisions, gives workers a right to an explanation and to human review of decisions such as suspension or termination, and creates a presumption of employment where control indicators are present.

The GDPR's Article 22 already applies where a decision is solely automated and has legal or similarly significant effects — which deactivation plainly does. The Amsterdam rulings are the practical demonstration.

Spain's "rider law" (2021) requires works councils to be informed of the parameters and rules of algorithms affecting working conditions. California, New York and others have introduced warehouse quota transparency laws requiring that quotas be disclosed in writing and that they not prevent legally required breaks.

India's position is different: the vast majority of platform workers are classified as independent contractors, the Code on Social Security 2020 recognises gig and platform workers as a category with a welfare fund mechanism, and implementation has been slow. Rajasthan's 2023 platform-worker legislation was an early state-level attempt at registration and welfare deductions. There is at present no general right to an explanation of an automated deactivation.

If this is happening to you

Request your data. In jurisdictions with an access right, a subject access request to the platform can produce the ratings, scores and decision records they hold about you. It is free, and there is a statutory deadline.

Ask specifically whether the decision was solely automated, and request human review. Under Article 22 those are distinct rights and both must be requested clearly. Put it in writing and keep a copy.

Keep your own records. Screenshots of offers, earnings and messages. Platforms revise their own interfaces and history; your screenshots do not change.

Find the collective route. Worker collectives and unions have obtained more disclosure through coordinated data requests and litigation than individuals have through complaints, because a hundred simultaneous access requests reveal the system in a way one cannot.

If you are designing this

The test is simple and uncomfortable: could a worker read a plain-language description of how the system evaluates them and predict its output for their own week? If not, they cannot improve, cannot contest, and cannot plan — and a management system that cannot be understood by the managed is not management. It is weather.

The one thing to keep

Algorithmic management controls income through assignment rather than formal decisions, which keeps it outside most employment protections — so the practical routes are data access requests, an explicit demand for human review, and acting collectively.

Before you move on

A delivery rider's account is suspended automatically after a fraud model flags their trip pattern. Which action is most likely to produce an actual reconsideration in a jurisdiction with GDPR-style rights?

Pick the one you would defend. Nobody sees your answer.

No ads. No data sale. No public scores on people. Ever.

© 2026 Addaly