The Online Harms edifice takes shape

The government has now published the Final Response to its Consultation on the April 2019 Online Harms White Paper.

Background

To recap, in the White Paper the government proposed to impose a “duty of care” on companies whose services host user-generated content or facilitate public or private online interaction between users. The duty of care would also apply to search engines.

An intermediary in scope would have to take reasonable steps to prevent, reduce or mitigate harm occurring on its service, including lawful content and activity deemed to be harmful. By its nature the duty placed on the intermediary would be to prevent the risk of one third party user causing harm to someone else.

This proposal differed from offline duties of care in two main respects: First, the White Paper did not limit or define the notion of harm. Comparable safety-related duties of care in the offline world are about objectively ascertainable physical injury and damage to property. An  undefined concept of harm arising from online speech was inevitably subjective and malleable. It raised objections of impermissible vagueness, consequent arbitrariness, and the prospect of online speech being judged by the standard of the most easily offended reader, viewer or listener.

Second, in the offline world a safety-related duty of care that imposes liability for failure to prevent third parties injuring each other is the exception rather than the norm - and in any event has not been applied to speech.

The White Paper proposed that the intermediaries’ duty of care would be overseen and enforced by a discretionary regulator - subsequently indicated as likely to be Ofcom - reminiscent of the world of television and radio. This represented a radical departure from the offline world, in which individual speech is governed only by settled and certain general law, not broadcast-style regulation by regulator.

All this was presented under the banner of offline-online equivalence.

The effect of the proposed Online Harms regime, although presented as regulating the tech companies, is that the regulator would indirectly govern our own individual speech via the proxy of online intermediaries acting under the legal compulsion of the duty of care. If harm were left undefined and unlimited, then the regulator would in effect have the ability to write its own parallel rulebook for online speech – both as to what amounted to harm, and what steps an intermediary should take to mitigate the risk of speech that the regulator deemed to be harmful.

In February 2020 the government published an Initial Response to the White Paper signalling some revisions to the regime, in particular a ‘differentiated’ duty of care that would apply more lightly to content that was harmful but not illegal. There was still no attempt to define or limit the concept of harm.

The government has now confirmed that Ofcom will be the scheme’s discretionary regulator. The Final Response proposes a number of significant changes to the regime described in the White Paper.

Harms in scope

The most significant development is that the government has now:

• Proposed a general definition of “harmful” content and activity: it must give rise to a “reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals”. [2.2] 
• Significantly limited what counts as illegal user content and activity for the purposes of the duty of care: excluding civil liability altogether and also limiting the kinds of criminal offences in scope to those that meet the general definition of “harmful” [2.24].

It has also confirmed previous indications that harms to organisations will not be in scope. [2.2, 4.1] Nor would intellectual property breaches, data protection breaches, fraud, breaches of consumer protection law, cyber security breaches or hacking. Harm arising from dark web activity would also be excluded. [2.3]

The combined effect of these steps is that the subject matter of the duty of care has moved in the direction of comparable offline duties of care. It is now more focused towards personal safety properly so-called, rather than resting on unbounded notions of harm. That is also reflected in the new name for the legislation: the Online Safety Bill.

By way of example, the government now explains that disinformation should not be regarded as per se dangerous, and that to do that would trespass unacceptably on freedom of speech:

“the duty of care will apply to content or activity which could cause significant physical or psychological harm to an individual, including disinformation and misinformation. Where disinformation is unlikely to cause this type of harm it will not fall in scope of regulation. Ofcom should not be involved in decisions relating to political opinions or campaigning, shared by domestic actors within the law.” [2.81]

This paragraph recalls the difference of opinion between Home Office and DCMS Ministers over 5G conspiracy theories when giving evidence to the Home Affairs Committee in May 2020.

Nevertheless, the definition of harmful remains problematic: not least because inclusion of ‘psychological impact’ may suggest that the notion of harm is still tied to variable, subjective reactions of different readers. Subjectivity opens the door to application of a standard of the most readily upset user. And while the subject matter of the duty of care may be more closely aligned with traditional duties of care, its nature – a duty to prevent third parties from harming each other – remains the exception, not the norm, in the offline world.

The Final Response proposes the creation, by secondary legislation, of specific ‘priority categories’ of harmful content and criminal offences, posing the greatest risk to individuals. [24], [2.3], [2.20]. The significance of these categories would be in underpinning a reformulated version of the ‘differentiated’ duty of care that was floated in the government’s Initial Response (see further below).

Providers and services in scope

Under the revised proposals, in-scope providers would be split into two categories of provider, subject to versions of the duty of care differing both as to what steps would be required to discharge the duty of care, and in respect of what kinds of harmful content. Only services designated as Category 1 would be duty-bound to address legal but harmful content.

Ofcom would determine which services meet the criteria for Category 1, according to thresholds previously set by the government. The relevant factors would be set out in the legislation: size of audience and functionalities offered.

According to the Response, functionalities such as the ability to share content widely or contact users anonymously are more likely to give rise to harm. [2.16]. When world-wide availability is an inherent feature of the internet, to treat the ability to share content widely as inherently risky is challenging for a government that proclaims that freedom of expression is at the heart of the proposed regulatory framework [1.10]. Contrary to the popular slogan, freedom of reach is indeed an aspect of freedom of speech - as the Supreme Court of India has held:

"There is no dispute that freedom of speech and expression includes the right to disseminate information to as wide a section of the population as is possible. The wider range of circulation of information or its greater impact cannot restrict the content of the right nor can it justify its denial." 

In the offline world, providing a venue specifically for activities that create a risk of danger is one situation in which a duty to prevent visitors injuring each other can arise. But to suggest that merely enabling individuals to speak to a large audience is a dangerously risky activity verges on an existential challenge to freedom of speech.

The Response excludes from scope:

• certain ‘low-risk’ activities: user comments on digital content in relation to content directly published by a service. This would exclude online product and service reviews and ‘below the line’ reader comments on news website articles. [1.7]
• three kinds of service: (a) B2B services as previously signalled in the Initial Response, (b) online services managed by educational institutions already subject to sufficient safeguarding duties or expectations, and (c) e-mail, voice telephone and SMS/MMS services. [1.6]

As to (c), the Response observes that “It is not clear what intermediary steps providers could be expected to take to tackle harm on these services before needing to resort to monitoring communications, so imposing a duty of care would be disproportionate.”

The result of the exclusions appears to be that the John Lewis customer review section would now be out of scope, but a site such as Mumsnet would still be in scope.

OTT private messaging services remain in scope [1.5]. The Response takes an approach to those that differs markedly from SMS/MMS services. Messaging providers may be required to monitor communications on private communications services, potentially by two routes.

First, it appears that Ofcom may have discretion to include monitoring in a Code of Practice. (Strictly speaking, however, this would not be mandatory, since it is always open to a provider to demonstrate to Ofcom that it can fulfil its duty of care as effectively in some other way [2.48].) The non-statutory interim code of practice on online child sexual exploitation and abuse (CSEA) published by the Home Office alongside the Response provides that automated technology should be considered on a voluntary basis.

Second, Ofcom would have express power to require companies to use “automated technology that is highly accurate” to identify illegal CSEA content and activity. This power would be usable where alternative measures cannot effectively address CSEA. Whilst the Response comments that this power is more likely to be considered proportionate on public platforms than private services, private services are not excluded. Ofcom would be required to seek approval from Ministers before exercising the power, on the basis that sufficiently accurate tools exist. The Response notes that the government assesses that, currently, sufficiently accurate tools exist to identify CSEA material that has previously been assessed as illegal. [2.59. 2.60]

Encryption is not mentioned in the Response.

News media and journalism The potential application of the legislation to news media and journalism has been fraught from the outset. The White Paper did not mention the issue, following which the then Secretary of State wrote to the Society of Editors assuring them that “where these services are already well regulated, as IPSO and IMPRESS do regarding their members' moderated comment sections, we will not duplicate those efforts. Journalistic or editorial content will not be affected by the regulatory framework.”

This left questions unanswered, for instance the position of mainstream news media not regulated by IPSO or IMPRESS. Nor did it address the position of newspapers’ own social media pages and feeds, which would count as user generated content and thus be indirectly regulated by Ofcom via the intermediaries’ duty of care.

The Final Response is, if anything, less clear than previously. It confirms that comment sections on news publishers’ websites would be out of scope, by virtue of the ‘low risk’ user comments exclusion mentioned above.  For social media feeds, it says that legislation will include ‘robust protections’ for journalistic content shared on in-scope services. As to what those protections might be, and what might count as journalistic content, the Response is silent. [1.10, 1.12]

Differentiated duty of care

The Initial Response proposed a differentiated duty of care, whereby for legal but harmful material and activities in-scope providers would be required only to enforce transparently, consistently and (perhaps) effectively, the standards that they chose to incorporate in their terms and conditions.

It always did seem unlikely that, for ‘legal but harmful’ content, the government intended to leave intermediaries completely to their own devices as to what standards (if any) to incorporate in their user terms and conditions. In 2018, after all, the government had said in its consultation response to the Internet Safety Strategy Green Paper that:

“The government has made clear that we require all social media platforms to have [inter alia]: Terms and conditions that provide a minimum level of safety and protection for users”.]

So it has proved.  The proposal in the Final Response is complex and nuanced. Its main features are:

• Providers that exceed specified audience and functionality thresholds will be designated as Category 1 providers (see above). 
• All in-scope providers will be expected to assess whether children are likely to access their services and, if so, to take additional protections for children using them [2.15] 
• Only Category 1 providers will be required to take action with regard to legal but harmful content and activity accessed by adults [2.15].
• The duty of care of non-Category 1 providers for adults would therefore apply only in relation to criminal content and activities (of a kind not otherwise excluded) that present a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals.

It should follow, although the Response does not spell this out completely clearly, that for non-Category 1 providers the general obligations listed below (such as risk assessment) would apply only in relation to the risk of such criminal content activities – and that ‘safety’ should also be understood in that sense. 

For Category 1 providers the general obligations would apply additionally to legal content and activity presenting a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals. 

General obligations

• All in-scope providers have a primary responsibility to take action to prevent user-generated content or activity on their services causing significant physical or psychological harm to individuals. To do this they will complete an assessment of the risks associated with their services and take reasonable steps to reduce the risks of the harms they have identified occurring. [2.7]
• Providers will fulfil the duty of care by putting in place systems and processes that improve user safety on their services – including, for example, user tools, content moderation and recommendation procedures. [2.9]
• Providers will be required to consider users’ rights, including freedom of expression online, both as part of the risk assessment and when making decisions on what safety systems and processes to put in place. [2.10]
• Regulation will ensure transparent and consistent application of terms and conditions relating to harmful content. This will include preventing companies from arbitrarily removing content. [2.10]
• Users must be able to report harm when it does occur and seek redress, challenge wrongful takedown and raise concerns about companies’ compliance with their duties. [2.11]
• All providers will have a specific legal duty to have effective and accessible reporting and redress mechanisms. This will cover harmful content and activity, infringement of rights (such as over-takedown), or broader concerns about a company’s compliance with its regulatory duties [2.12]

Illegal content and activities

• For in-scope criminal activity, all providers will need to ensure that illegal content is removed expeditiously and that the risk of it appearing and spreading across their services is minimised by effective systems [2.19]
• Priority categories of offences, against which providers will be required to take particularly robust action, will be set out in secondary legislation. [2.20] For CSEA and terrorism this may include proactively identifying and blocking or removing this type of material if other steps have not been effective and safeguards are in place. [2.21]

The Response is silent as to how such an obligation may be consistent with the prohibition on general monitoring obligations under Article 15 of the eCommerce Directive. The government has said, in the context of Brexit, that it has no current plans to change the UK’s approach to prohibition on general monitoring requirements.

Legal but harmful content and activity accessed by adults (Category 1 providers only)

• The legislation will not require removal of specific pieces of legal content [2.28], unless specified as not permitted by the provider’s terms and conditions [2.33] Terms and conditions could be about, for example, labelling and de-prioritising [2.32].
• Priority categories of legal but harmful material will be set out in secondary legislation. These will be categories of legal but harmful material that Category 1 providers should, at a minimum, address through their terms and conditions. The Response gives the examples of content promoting self-harm, hate content, online abuse that does not meet the threshold of a criminal offence, and content encouraging or promoting eating disorders. [2.29]
• Category 1 providers will be obliged to state how they will handle other categories of legal but harmful material identified in their risk assessment and make clear what is acceptable on their services for that content. [2.31]

Controversial viewpoints

• Category 1 companies will not be able to arbitrarily remove controversial viewpoints and users will be able to seek redress if they feel that content has been removed unfairly. [2.34]
• User redress mechanisms will enable users to challenge content that unduly restricts their freedom of expression. This appears to apply to all in-scope providers (Annex A).

These provisions appear to be the ‘impartiality’ requirements that were trailed in the press before the release of the Final Response, reportedly at the instigation of 10 Downing Street. It is unclear whether these provisions are intended to override substantive policies set out in providers’ terms and conditions. They appear to be unrelated to, or at least to go wider than, issues about illegal or harmful content.

Children

• All companies in scope will required to assess the likelihood of children accessing their service. [2.36] Only services likely to be accessed by children will be required to provide additional protections for children accessing them, starting with conducting a specific child safety risk assessment. [2.36], [2.37]
• The government will set out in secondary legislation priority categories of legal but harmful content and activity impacting children, meeting the general definition of harmful content and activity already described. These will be categories impacting children that companies in scope should, at a minimum, take action on. [2.38]
• Age assurance and age verification technologies are expected to play a key role in fulfilling the duty of care. [2.41]

Codes of Practice

The Final Response has increased the amount of influence that the government will have over Ofcom’s Codes of Practice. Ofcom will be required to send the final draft of a Code of Practice to the Culture Secretary and the Home Secretary, who will have the power to reject a draft code and require the regulator to make modifications for reasons relating to government policy.

Parliament will have the opportunity to debate and vote on the high level objectives set out by the government for the Codes of Practice by the affirmative resolution procedure. Completed codes will be laid in Parliament, subject to negative resolution. [4.10]

Search engines

Little is said in the Final Response about how the proposed duty of care would apply to search engines, beyond a brief summary of actions that they can take to mitigate the risk of harm and proportionate systems and processes that they would be expected to put in place to keep their users safe.

Search engines would need to assess the risk of harm occurring across their entire service. Ofcom would provide guidance specific to search engines regarding regulatory expectation

The government proposes that given the distinct nature of search engines, legislation and codes of practice would include specific material for them. It says that all regulatory requirements would be proportionate, and respect the key role of search engines in enabling access to information online. [1.3]

Territoriality

For the first time, the Final Response has set out the proposed territorial reach of the proposed legislation. Somewhat surprisingly, it appears to propose that services should be subject to UK law on a ‘mere availability of content’ basis. Given the default cross-border nature of the internet, this is tantamount to legislating extraterritorially for the whole world. It would follow that any provider anywhere in the rest of the world would have to geo-fence its service to exclude the UK in order to avoid engaging UK law. Legislating on a mere availability basis has been the subject of criticism over many years since the advent of the internet. [1.1]

Overall commentary

The fundamental issues with the government’s White Paper proposals have been exhaustively discussed on previous occasions. Reminiscent of a sheriff in the Wild West, to which the internet is so often likened, Ofcom would enlist deputies - social media platforms and other intermediaries acting under a legal duty of care - to police the unruly online population. Unlike its Wild West equivalent, however, Ofcom would get to define its territory and write the rules, as well as enforce them.

The introduction of a general definition of harm would tie Ofcom’s hands to some degree in deciding what does and does not constitute harmful speech. Limiting the scope of ‘harm’ to a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals goes some way to align the proposed duty of care more closely with analogous offline duties of care, which are specifically safety-related.

Nevertheless, when applied in the context of speech there remain significant problems.

1. What is an adverse psychological impact? Does it have to be a medically recognised condition? If not, how wide is it meant to be? Is distress sufficient? The broader the meaning, the closer we come to a limitation that could mean little or nothing more than being upset or unhappy. The less clear the meaning, the more discretion would be vested in Ofcom to decide what counts as harm, and the more likely that providers would err on the side of caution in determining what kinds of content or activity are in scope of their duty of care.

2. The difficulty, not to say virtual impossibility, of the task faced by the regulator and providers should not be underestimated. Thus, for the lawful but harmful category, the government has said that it will include online abuse as a priority category in secondary legislation. However, on the basis of these proposals that must be limited to abuse that falls within the general definition of harm – i.e. abuse that presents a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals. The provider’s actions under the duty of care should relate only to such harmful abuse. Where, concretely, is the dividing line between abuse that does and does not carry a foreseeable risk of adverse psychological impact? What content falls on either side of the line?

The provider would also have to take into account the proposed obligation not to remove controversial viewpoints and the possibility of user redress for unduly restricting their freedom of expression. Coincidentally, the Divisional Court in Scottow v CPS has in the last few days issued a judgment in which it referred to “the well-established proposition that free speech encompasses the right to offend, and indeed to abuse another”.

These issues illustrate the care that has to be taken with using terms such as ‘online abuse’ to cover everything from strong language, through insults, to criminal threats of violence.

3. What is the threshold to trigger the duty of care? Is it the risk that someone, somewhere, might read something and claim to suffer an adverse psychological impact as a result? Is it a risk gauged according to the notional attributes of a reasonably tolerant hypothetical user, or does the standard of the most easily upset apply? How likely does it have to be that someone might suffer an adverse psychological impact if they read it? Is a reasonably foreseeable, but low, possibility sufficient? 

The Media Minister John Whittingdale, writing in the Daily Mail on the morning of the publication of the Final Response, said:

“This is not about an Orwellian state removal of content or building a ‘woke-net’ where causing offence leads to instant punishment.  Free speech includes the right to offend, and adults will still be free to access content that others may disapprove of.”

If risk and harm thresholds are sufficiently low and subjective, that is what would result.

4. Whatever the risk threshold might be, would it be set out in tightly drawn legislation or left to the discretion of Ofcom? It will not be forgotten that Ofcom, in a 2018 survey, suggested to respondents that ‘bad language’ is a harmful thing. A year later it described “offensive language” as a “potential harm”.

5. Lastly, in the absence of deliberate intent an author owes no duty avoid causing harm to a reader of their work, even though psychological injury may result from reading it. That was confirmed by the Supreme Court in Rhodes. The government’s proposals would therefore mean that an intermediary would have a duty to consider taking steps in relation to material for which the author itself has no duty of care.

These are difficult issues that go to the heart of any proposal to impose a duty of care. They ought to have been the subject of debate over the last couple of years. Unfortunately they have been buried in the rush to include every conceivable kind of harm - however unsuited it might be to the legal instrument of a duty of care - and in discussions of ‘systemic’ duties of care abstracted from consideration of what should and should not amount to harm.

It should be no surprise if the government’s proposals became bogged down in a quagmire resulting from the attempt to institute a universal law of everything, amounting to little more than a vague precept not to behave badly online. The White Paper proposals were a castle built on quicksand, if not thin air.

The proposed general definition of harm, while not perfect, gives some shape to the edifice. It at least sets the stage for a proper debate on the limits of a duty of care, the legally protectable nature of personal safety online, and its relationship to freedom of speech – even if that should have taken place two years ago. Whether regulation by regulator is the appropriate way to supervise and police an appropriately drawn duty of care in relation to individual speech is another matter.