Safe speech by design

Proponents of a duty of care for online platforms have long dwelt on the theme of safety by design. It has come to the fore again recently with the government’s publication of a draft Statement of Strategic Priorities (SSP) for Ofcom under the Online Safety Act.  Safety by Design is named as one of five key areas.

Ofcom is required to have regard to the final version of the SSP in carrying out its functions under the Act. Given Ofcom’s regulatory independence the government can go only so far in suggesting how Ofcom should do its job. But, in effect, the SSP gives Ofcom a heavy hint about the various directions in which the government would like it to go.

So what does safety by design really mean? How might it fit in with platform (U2U) and search engine duties under the Online Safety Act (OSA)?

Before delving into this, it is worth emphasising that although formulations of online platform safety by design can range very widely [1] [2], for the purposes of the OSA safety by design has to be viewed through the lens of the specific duties imposed by the Act.

This piece focuses on the Act’s U2U illegality duties. Three of the substantive duties concern design or operation of the service:

• Preventing users encountering priority illegal content by means of the service (S. 10(2)(a))
• Mitigating and managing the risk of the service being used for the commission or facilitation of a priority offence (as identified in the most recent illegal content risk assessment of the service) (S. 10(2)(b))
• Mitigating and managing the risks of physical or psychological harm to individuals (again as identified in the most recent illegal content risk assessment) (S. 10(2)(c))

Two further substantive illegality duties are operational, relating to:

• Minimising the length of time for which priority illegal content is present on the service (S. 10(3)(a))
• Swiftly taking down illegal content where the service provider is alerted to or otherwise becomes aware of its presence. (S. 10(3)(b))

S.10(4) of the Act provides examples of the areas of design, operation and use of a service to which the duties apply and, if proportionate, require the service provider to take or use measures. Those include “design of functionalities, algorithms and other features.”

Safety by design in the Online Safety Act

When applied to online speech, the notion of safety by design prompts some immediate questions: What is safety? What is harm?

The OSA is less than helpful about this. It does not define safety, or safety by design. It defines harm as physical or psychological harm, but that term appears in only one of the five substantive illegality duties outlined above. Harm has more a pronounced, but not exclusive, place in the prior illegal content risk assessment that a platform is required to undertake.

Safety by design gained particular prominence with a last-minute House of Lords addition to the Bill: an introductory ‘purpose’ clause. This amendment was the result of cross-party collaboration between the then Conservative government and the Labour Opposition.

What is now Section 1 proclaims (among other things) that the Act provides for a new regulatory framework which has the:

“general purpose of making the use of internet services regulated by this Act safer for individuals in the United Kingdom.”

It goes on that to achieve that purpose, the Act (among other things):

“imposes duties which, in broad terms, require providers of services regulated by this Act to identify, mitigate and manage the risks of harm (including risks which particularly affect individuals with a certain characteristic) from

(i) illegal content and activity, and

(ii) content and activity that is harmful to children, …”

Finally, and most relevantly, it adds that:

“Duties imposed on providers by this Act seek to secure (among other things) that services regulated by this Act are … safe by design…”.

A purpose clause is intended to assist in the interpretation of the legislation by setting out the purposes for Parliament intended to legislate, rather than leaving the courts to infer them from the statutory language.

Whether such clauses in fact tend to help or hinder is a matter of lawyerly debate. This clause is especially confusing in its use of terms that are not defined by the Act and do not have a clear and obvious ordinary meaning (“safe” and “safe by design”), mixed up with terms that are specifically defined in the legislation (“harm”, meaning physical or psychological harm).

One thought might be that “safe” means safe from physical or psychological harm, and that “safe by design” should be understood accordingly. However, that seems unlikely since four of the five substantive illegality duties on service providers relate to illegal content and activity per se, irrespective of whether they might involve a risk of physical or psychological harm to individuals.

S.235 defines Ofcom’s “online safety functions” in terms of all its functions under the Act. In contrast, the transitional provisions for Video Service Providers define “safety duties” in terms focused on platform duties in respect of illegality and harm to children.

Similarly, in the earlier part of the Act only those two sets of duties are described (albeit merely in the section headings) as “safety duties”. “Safe by design” may possibly refer to those duties alone.   

The concept of safety by design tends to embody some or all of a number of elements: risk-creating features; prevention and reduction of harm; achieving those by appropriate design of a risk-creating feature, or by adding technical safeguards.

The most general aspect of safety by design concerns timing: that safety should be designed in from the outset rather than thought about afterwards.

Prevention itself has a temporal aspect, but that may relate as much to the kind of measure as to the stage of development at which it should be considered. Thus the Minister’s introduction to the Statement of Strategic Priorities says that it:

“includes ensuring safety is baked into platforms from the start so more harm is caught before it occurs”.

This could refer to the point at which a safety measure intervenes in the user’s activity, as opposed to (or as well as) the stage at which the designers consider it.

Later in the Statement, safety by design is expressly said to include deploying technology in content moderation processes. Providers would be expected to:

“…embed proportionate safety by design principles to mitigate the [risk of their service being used to facilitate illegal activity]. This should include steps such as … where proportionate, deploying technology to improve the scale and effectiveness of content moderation, considering factors including providers’ capacity and users’ freedom of expression and privacy rights.”

An analogy with product safety could suggest that safety by design is about identifying risk-creating features at the design stage and either designing those features in the safest way or incorporating safeguards. That aspect is emphasised by Professor Lorna Woods in a recent paper [3]:

“The objective of ‘safety by design’ is – like product safety – to reduce the tendency of a given feature or service to create or exacerbate such issues.”

Applied to products like cars that would mean that you should consider at the outset where safely to position the fuel tank, not unthinkingly place it somewhere dangerous and try to remedy the problem down the line, or after an accident has happened. Or, if a piece of machinery has a sharp cutting blade, consider at the outset how to add a guard into the design. A culture of safety by design should help to ensure that potential safety risks are considered and not overlooked.  

However, a focus on risk-creating features gives rise to particular difficulties when safety by design is translated to online platforms.

The underlying duty of care reasons for this have been rehearsed on previous occasions (here and here). In short, speech is not a tripping hazard, nor is it a piece of machinery. A cutting machine that presents a risk of physical injury to its operator is nothing like a space in which independent, sentient human beings can converse with each other and choose what to say and do.

Professor Woods [3] suggests that ‘by design’ seeks to ensure that products respect the law (my emphasis). If that is right, then by the same token it could be said that safety by design when applied to online platforms seeks to ensure that in their communications with each other users respect the law (or boundaries of harm set by the legislation). That is a materially different exercise, for which analogies with product safety can be taken only so far.

The June 2021 DCMS/DSIT paper Principles of safer online platform design opened with the statement that:

“Online harms can happen when features and functions on an online platform create a risk to users’ safety.”

For the illegality duties imposed by the OSA, when we set about identifying concrete features and functionalities that are said to create or increase risk of illegality, we run into problems when we move beyond positive platform conduct such as recommender and curation algorithms.

The example of recommender and curation algorithms has the merit of focusing on a feature that the provider has designed and which can causally affect which user content is provided to other users.

But the OSA duties of care – and thus safety by design - go well beyond algorithmic social media curation, extending to (for instance) platforms that do no more than enable users to post to a plain vanilla discussion forum.

Consider the OSA safety duties concerning priority illegal content and priority offences.  What kind of feature would create or increase a risk of, for example, an online user deciding to offer boat trips across the Channel to aspiring illegal immigrants?

The further we move away from positive content-related functionality, the more difficult it becomes to envisage how safety by design grounded in the notion of specific risk-creating features and functions might map on to real-world technical features of online platforms.

The draft SSP confirms that under the OSA safety by design is intended to be about more than algorithms:

“When we discuss safety by design, we mean that regulated providers should look at all areas of their services and business models, including algorithms and functionalities, when considering how to protect all users online. They should focus not only on managing risks but embedding safety outcomes throughout the design and development of new features and functionalities, and consider how to make existing features safer.”

Ofcom faced the question of risk-creating features when preparing the risk profiles that the Act requires it to provide for different kinds of in-scope service. For the U2U illegality risk profile it has to:

“carry out risk assessments to identify and assess the following risks of harm presented by [user to user] services of different kinds—

(a) the risks of harm to individuals in the United Kingdom presented by illegal content present on regulated user-to-user services and by the use of such services for the commission or facilitation of priority offences; …”

The risks that Ofcom has to identify and assess, it should be noted, are not the bare risk of illegal content or illegal activity, but the risk of harm (meaning physical or psychological harm) to individuals presented by such content or activity.

Ofcom is required to identify characteristics of different kinds of services that are relevant to those risks of harm, and to assess the impact of those kinds of characteristics on such risks. “Characteristics” of a service include its functionalities, user base, business model, governance and other systems and processes.

Although a platform has to carry out its own illegal content risk assessment, taking account of Ofcom’s risk profile, the illegality risks that the platform has to assess also include bare (non-harm-related) illegality.

Ofcom recognises that functionalities are not necessarily risk-creating:

“Functionalities in general are not inherently positive nor negative. They facilitate communication at scale and reduce frictions in user-to-user interactions, making it possible to disseminate both positive and harmful content. For example, users can engage with one another through direct messaging and livestreaming, develop relationships and reduce social isolation. In contrast, functionalities can also enable the sharing of illegal material such as livestreams of terrorist atrocities or messages sent with the intent of grooming children.” [6W.16]

Ofcom overcomes this issue in its proposed risk profiles by going beyond characteristics that of themselves create or increase risks of illegality. This is most clearly expressed in Volume 2 of its Illegal Harms Consultation:

“We recognise that not all characteristics are inherently harmful; we therefore use the term ‘risk factor’ to describe a characteristic for which there is evidence of a risk of harm to individuals. For example, a functionality like livestreaming is not inherently risky but evidence has shown that it can be abused by perpetrators; when considering specific offences such as terrorism or CSEA, a functionality like livestreaming can give rise to risk of harm or the commission or facilitation of an offence.” [5.26]

General purpose functionality and features of online communication can thus be designated as risk factors, on the basis that there is evidence that wrongdoers make use of them or, in some instances, certain combinations of features.

Since measures focused on general purpose features are likely to be vulnerable to objections of disproportionate interference with freedom of expression, for such features the focus of preventing or mitigating the identified risk is more likely to be on other aspects of the platform’s design, on user options and controls in relation to that feature (e.g. an option to disable the feature), or on measures such as content moderation.

Ofcom implicitly recognises this in the context of livestreaming:

“6.11 We acknowledge that some of the risk factors, which the evidence has demonstrated are linked to a particular kind of illegal harm, can also be beneficial to users. This can be in terms of the communication that they facilitate, or in some cases fulfilling other objectives, such as protecting user privacy. …

6.13 While livestreaming can be a risk factor for several kinds of illegal harm as it can allow the real-time sharing of illegal content, it also allows for real-time updates in news, providing crucial information to a wide-range of individuals.

6.14 These considerations are a key part of the analysis underpinning our Codes measures.”

The result is that while the illegality risk profiles that Ofcom has proposed include as risk factors a range of platform features that could be viewed as general purpose, they tend not to translate into recommended measures aimed at inhibiting that feature.

Here is a selection of features included in the proposed illegality risk profile:

Service feature	Risk (likelihood of increased risk of harm related to offences involving):
Ability to create user profiles	Grooming, harassment, stalking, threats, abuse, drugs and psychoactive substances, unlawful immigration, human trafficking, sexual exploitation of adults; and for the risk of fake profiles: Grooming, harassment, stalking, threats, abuse, controlling or coercive behaviour, proceeds of crime, fraud and financial services, foreign interference offences.
Users can form user groups	Grooming, encouraging or assisting suicide or serious self-harm, drugs and psychoactive substances, unlawful immigration, human trafficking.
Livestreaming	Terrorism, grooming, image-based CSAM, encouraging or assisting suicide or serious self-harm, harassment, stalking, threats, abuse.
Direct messaging	Grooming and CSAM, hate, harassment, stalking, threats, abuse, controlling or coercive behaviour, intimate image abuse, fraud and financial services offences.
Encrypted messaging	Terrorism, grooming, CSAM, drugs and psychoactive substances, sexual exploitation of adults, foreign interference, fraud and financial services offences.
Ability to comment on content	Terrorism, grooming, encouraging or assisting suicide or serious self-harm, hate, harassment, stalking, threats, abuse.
Ability to post images or videos	Terrorism, image-based CSAM, encouraging or assisting suicide or serious self-harm, controlling or coercive behaviour, drugs and psychoactive substances, extreme pornography, intimate image abuse.
Ability to repost or forward content	Encouraging or assisting suicide or serious self-harm, harassment, stalking, threats, abuse, intimate image abuse, foreign interference.
Ability to search for user generated content	Drugs and psychoactive substances, firearms and other weapons, extreme pornography, fraud and financial services offences.
Hyperlinks	Terrorism, CSAM URLs, foreign interference offences.

Other functionality risk factors include anonymity, user connections (such as friending and following), group messaging, and ability to post or send location information.

Designation of general purpose functionality as a risk factor reaches a high point with hyperlinks. Since terrorists and other potential perpetrators can use hyperlinks to point people to illegal material, hyperlinks can be designated as a risk factor despite not being inherently harmful.

It is worth recalling what the ECtHR said in Magyar Jeti Zrt (ECtHR) about the central role of hyperlinks in internet communication:

“Furthermore, bearing in mind the role of the Internet in enhancing the public’s access to news and information, the Court points out that the very purpose of hyperlinks is, by directing to other pages and web resources, to allow Internet users to navigate to and from material in a network characterised by the availability of an immense amount of information. Hyperlinks contribute to the smooth operation of the Internet by making information accessible through linking it to each other.”

General purpose functionality as a risk factor was foreshadowed in the June 2021 DCMS paper. Arguably it went further, asserting in effect that providing a platform for users to communicate with each other is itself a risk-creating activity:

          “Your users may be at increased risk of online harms if your platform allows them to:

• interact with each other, such as through chat, comments, liking or tagging
• create and share text, images, audio or video (user-generated content)”

In the context of the internet in the 21st century, this list of features describes commonplace aspects of the ability to communicate electronically. In a former age we might equally have said that pen, paper, typewriter and the printing press are risk factors, since perpetrators of wrongdoing may use written communications for their nefarious purposes.

Whilst Ofcom recognises the potential freedom of expression implications of treating general purpose functionalities as illegality risk factors, it always has to be borne in mind that from a fundamental rights perspective the starting point is that speech is a right, not a risk. Indeed the Indian Supreme Court has held that the right of freedom of expression includes the reach of online individual speech:

"There is no dispute that freedom of speech and expression includes the right to disseminate information to as wide a section of the population as is possible."

That is not to suggest that freedom of expression is an absolute right. But any interference has to constitute a sufficiently clear and precise rule (especially from the perspective of the user whose expression is liable to be interfered with), then satisfy necessity and proportionality tests.

Preventative technological measures

A preventative approach to safety by design can easily lean towards technological measures: since this is a technology product, technological preventative measures should be designed in to the service and considered at the outset.

Professor Woods [3], argues that:

“Designing for safety (or some other societal value) does not equate to techno-solutionism (or techno-optimism); the reliance on a “magic box” to solve society’s woes or provide a quick fix.”

However, in the hands of government and regulators it has a strong tendency to do so.[4].  Indeed the draft SSP devotes one of its five key priorities to Technology and Innovation, opening with:

“Technology is vital to protecting users online and for platforms fulfilling their duties under the Act.”

Later:

“It is not enough that new, innovative solutions to known problems exist – online service providers must also adopt and deploy these solutions to improve user safety. … The government … encourages Ofcom to be ambitious in its [code of practice] recommendations and ensure they maintain pace with technology as it develops.”

We have already seen that in the draft SSP, safety by design is said to include deploying technology in content moderation processes.

On the basis of prevention, an inbuilt technological design measure that reduces the amount of (or exposure to) illegal user speech or activity should be preferable to hiring legions of content moderators when the platform starts operating.

However, translating duties of care or safety by design into automated or technology-assisted content moderation can come into conflict with an approach in which non-content-specific safety features are seen as preferable.

Professor Woods said in the same paper:

“At the moment, content moderation seems to be in tension with the design features that are influencing the creation of content in the first place, making moderation a harder job. So, a “by design” approach is a necessary precondition for ensuring that other ex post responses have a chance of success.

While a “by design” approach is important, it is not sufficient on its own; there will be a need to keep reviewing design choices and updating them, as well as perhaps considering ex post measures to deal with residual issues that cannot be designed out, even if the incidence of such issues has been reduced.”

As to what ex post measures might consist of, in a letter to The Times in August, Professor Woods said:

“Through a duty of care, service operators are required to ensure that their products are as safe as reasonably possible and to take steps to mitigate unintended consequences. Essentially this is product safety, or health and safety at work. This approach allows a range of interventions that do not rely on content take-down and, indeed, could be content-neutral. One example might be creator reward programmes that incentivise the spreading of clickbait material. (emphasis added)].

Maeve Walsh, writing for the Online Safety Network shortly before publication of the draft SSP [5], contrasted safety by design with thinking about the OSA “primarily as a takedown-focused regime, centering on individual pieces of content.”

Content-neutrality suggests that a safety measure in relation to a functional feature should, rather than relating specifically to some kind of illegal or harmful content, either have no effect on content as such or, if it does affect user content, do so agnostically.

Some measures have no direct effect on user content: a help button would be an example. Others may affect content, but are not targeted at particular kinds of content: for instance, a friction-reducing measure like capping the permissible number of reposts, or other measures inhibiting virality.

A measure such as a quantititive cap on the use of some feature has the advantage from a rule of law perspective that it can be clearly and precisely articulated. However, by virtue of the fact that it constrains legitimate as well as illegitimate user speech across the board, it is potentially vulnerable to proportionality objections.

Thanks to the difficulty of making accurate illegality judgements, automated content filtering and blocking technologies are potentially at risk on both scores.

[1] Trust & Safety Professional Association. Safety by Design Curriculum chapter.

[2] Australian eSafety Commissioner. Safety by Design.

[3] Professor Lorna Woods, for the Online Safety Network (October 2024). Safety by Design. 

[4] Maria P. Angel, danah boyd (12 March 2024). Proceedings of 3rd ACM Computer Science and Law Symposium (CSLAW’24) Techno-legal Solutionism: Regulating Children’s Online Safety in the United States.

[5] Maeve Walsh, for the Online Safety Network (11 October 2024). Safety by design: has its time finally come?