When the human perspective become a technical requirement

A couple of weeks ago, I published an essay about how we can make a long series of decisions, each of which appears rational, innovative and perhaps even right, and still end up in a digital landscape that none of us would reasonably have designed if we had been allowed to begin with the question of what kind of society we actually want to live in.

I wrote about children exposed to grooming in environments designed for play and connection, about women whose bodies can be recreated and sexualised using generative AI, and about a democratic conversation that increasingly takes place on privately owned platforms whose business models reward whatever keeps us engaged, regardless of what it does to us. Above all, I wrote about how responsibility for the harmful consequences of digitalisation has, for far too long, been placed on those with the least ability to influence the systems themselves. The child was expected to block the groomer, the parent to manage screen time, the woman to report the deepfake and the citizen to see through industrially produced information manipulation. My conclusion was fairly simple: responsibility must follow the power to act.

On 17 September, the European Commission presented its proposal for the KIDS Act. Most of the attention will probably focus on the age limits. Children under thirteen would not be allowed to have their own accounts on the social platforms covered by the legislation, thirteen- and fourteen-year-olds would only be able to use restricted accounts under the supervision of a parent or guardian, and only from the age of fifteen would a young person be able to open an independent account.

That is a major change. But if the discussion stops at whether fifteen is the right age, we risk missing the most interesting part of the proposal. The KIDS Act does not merely seek to determine when children may use digital services. It also reaches into how those services are designed.

For minors, platforms would no longer be allowed to use streaks that penalise absence, notifications designed to pull children back in, or infinite feeds without natural stopping points. Recommendation systems would have to be optimised for safety, quality and mental health rather than maximum engagement. Children would not be contactable by strangers without consent, and AI companions could not be designed in ways that create emotional dependency. For the largest platforms, the burden of proof would also be reversed. It would no longer be up to the child, the parent or the public authority to demonstrate, after the fact, that a digital environment had caused harm. The platform would have to demonstrate in advance that it had been designed to be safe.

It is an attempt to rewrite the technical requirements.

For a long time, we have had a fairly shared understanding of what it means for a digital service to work. The system should be fast, stable and scalable. The film should start playing, the payment should go through, and the algorithm should deliver content that the user is likely to keep watching. If the code does what it was built to do, we are inclined to regard the solution as a technical success.

But if a perfectly functioning recommendation system leads a child further and further into content that harms them, is it still functioning well? If a notification does exactly what it was designed to do and wakes a thirteen-year-old in the middle of the night because they risk losing their streak, is that good technology? If a platform succeeds in maximising engagement by exploiting what we know about the human need for validation, the fear of being left out and our difficulty walking away from a reward that might arrive with the next swipe, what exactly have we optimised?

The KIDS Act offers a political answer to those questions. A service is not well designed simply because it works technically and delivers commercially. How it affects a child’s sleep, safety, relationships, mental health and ability to resist manipulation is also part of its quality.

Who gets to define good technology?

When human behaviour, relationships and vulnerability become part of the technical requirements, another question arises: what kinds of knowledge need to be in the room when technology is built?

We need developers, AI specialists, systems architects and cybersecurity experts. But we also need people who understand how grooming works, why a child responds to a stranger, how peer pressure operates, how girls and boys can be exposed to harm in different ways, and what a relationship with an AI companion may feel like to someone who is lonely. We need law, education, psychology, gender equality expertise, children’s rights and the lived experience of the people who actually live with the consequences of technology.

The question is not simply whether these perspectives are invited in, but when they enter the process and how much power they have when they do. Someone who is consulted after the system has already been procured, the model trained and the business logic established may be able to suggest a warning message, a training initiative or yet another button for reporting harm. Someone who is there when the problem itself is being defined can influence the architecture. Bring the knowledge in early and we may be able to change an assumption. Bring it in late and we may have to change an entire system.

There is an old Swedish concept for the power that resides precisely here: problemformuleringsprivilegiet, the privilege of defining the problem.

The way we describe a problem influences which causes we see, where we place responsibility and which solutions we are even capable of imagining. If the problem is framed as children lacking self-control, the answer becomes screen-time tools, information campaigns and more conversations around the kitchen table. If the problem is instead framed as digital services being designed to turn children’s self-control into a source of revenue, very different solutions become possible. Algorithms, default settings, reward systems and business models move into focus.

This is the shift the KIDS Act is attempting to make. The problem definition moves from the child’s behaviour to the design of the system, and with that shift, some of the power and responsibility moves too.

At the same time, the proposal challenges another assumption that still shapes many organisations: the division between the “hard” questions of technology and the supposedly “soft” questions concerning people, relationships, communication and culture. When a child’s right to sleep means that the notification system must be designed differently, when knowledge about dependency changes the architecture of the feed, and when an understanding of grooming is expected to shape contact functions and permissions, the human perspective is no longer sitting alongside the technology.

It has become a technical requirement.

That should reasonably also affect how we value the expertise required to understand people and society. Programming, engineering and cybersecurity are readily described as forms of expertise developed through education, training and experience. Yet someone who can read a room, build trust between people with fundamentally different interests, or enable several professions to solve a problem they cannot even agree on how to define is easily described as “good with people”, “a strong communicator” or someone who “gets people on board”.

When technical skill is described as competence while relational skill is described as personality, we have already begun to value them differently. Status then becomes salary, budget, mandate, job titles, representation in the executive team and, ultimately, the ability to define the problem before someone else starts designing the solution.

The European Commission’s proposal has not yet become law, and much may change as the European Parliament and the Member States begin their negotiations. Age verification raises difficult questions about privacy, and no regulation on its own can create a safe digital childhood. But the proposal nevertheless marks an important shift: society is no longer content simply to ask technology companies to clean up after the harms their systems contribute to. It is attempting to influence what gets built, what objectives systems are allowed to optimise for, and which values must be present from the very beginning of the design process. It is an attempt to change the order in which we ask the questions.

Not first: what can technology do, and how should children, families and society adapt to it?

But rather: what kind of digital society do we want children to grow up in, and what technology will help us build it?

Once we ask that question, it does not merely change the answer. It changes which forms of knowledge we need to give status to, which people we need to give authority to, and which perspectives must be allowed to shape the problem before the solution begins to take form.

Technology is power. But so is the power to decide whose knowledge counts when the future is being built.