Internet jurisdiction revisited

As legal and policy topics go, cross-border internet jurisdiction is evocative of a remote but restless volcano: smouldering away mostly unnoticed by public and lawyers alike, only to burst spectacularly into life at odd intervals.

The latest eruption has occurred in Australia, where last month the Australian eSafety Commissioner launched legal proceedings for an injunction against X Corp (Twitter) requiring it to remove or hide from all users worldwide a video of a stabbing attack on an Australian bishop. X Corp argues that geo-blocking the content from Australian users is sufficient. The eSafety Commissioner disagrees, since Australian users equipped with VPNs can evade the block. In a judgment published on 14 May 2024 Kennett J, a judge of the Federal Court of Australia, sided with X Corp on an interim basis and declined to continue a previously granted emergency injunction.

The underlying policy issue is that the internet is readily perceived as undermining local laws, since material posted on the internet outside the local jurisdiction is, by default, available worldwide. The Canadian Supreme Court in Equustek put it thus:

“Where it is necessary to ensure the injunction’s effectiveness, a court can grant an injunction enjoining conduct anywhere in the world. The problem in this case is occurring online and globally. The Internet has no borders — its natural habitat is global. The only way to ensure that the interlocutory injunction attained its objective was to have it apply where Google operates — globally.”

The countervailing concern is that when a court acts in that way in order to secure the effectiveness of its local law, it is asserting the right to impose that law on the rest of the world, where the material in question may be legal. Assertion of extraterritorial jurisdiction has always had the potential to create friction between nation states. When the internet arrived, its inherent cross-border nature created additional policy tensions that, 30 or more years on, have yet to be fully resolved.

The background to the current dispute is Australia’s Online Safety Act 2021. A social media service is in scope of the Act unless “none of the material on the service is accessible to, or delivered to, one or more end-users in Australia” (S.13(4)).  Thus any social media service in the world is within the reach of the Australian legislation, unless it can and does take steps that prevent all Australian users from accessing its content.  

However, the general territorial scope of the Act is not the end of the story. Under the Act the eSafety Commissioner can issue a removal notice in respect of ‘Class 1’ material if (among other things) the Commissioner is satisfied that the material can be accessed by end-users in Australia. A removal notice requires the service provider to “take all reasonable steps to ensure the removal of the material from the service”.  

Echoing S.13(4), S.12 provides that material is ‘removed’ if “the material is neither accessible to, nor delivered to, any of the end-users in Australia using the service.” (The court interpreted this as meaning all users physically located in Australia.)

The Commissioner sought continuation of the previously granted emergency injunction pending trial. The court therefore had to decide whether there was a real issue to be tried that the final injunction sought by the Commissioner would go further than the “reasonable steps” that were all that a removal notice could require.

X Corp had agreed to geoblock the 65 URLs specified in the removal notice, so that they are not accessible to users with IP addresses in Australia. The eSafety Commissioner sought an injunction that would require X Corp to remove the 65 URLs from its platform altogether, or make them inaccessible to all users. The Commissioner argued that such action was within the “all reasonable steps” that the removal notice required to be taken. X Corp argued that a requirement for worldwide removal or blocking of the material goes beyond what is “reasonable”.

The court held that although a voluntary decision by X Corp to remove the 65 URLs altogether would be reasonable (in the sense of easily justified), that was not the test where the Act imposes its requirements regardless of the wishes of providers and of individual users. “Reasonable” should therefore be understood as limiting what must be done to the steps that it is reasonable to expect or require the provider to undertake. Such steps include not only considerations of expense, technical difficulty and time for compliance, but (the issue that divided the parties) the other interests that are affected.

Significantly, when considering the other interests affected, the court brought into consideration the ‘comity of nations’. At an earlier point in the judgment Kennett J had said:

“The policy questions underlying the parties’ dispute are large. They have generated widespread and sometimes heated controversy. Apart from questions concerning freedom of expression in Australia, there is widespread alarm at the prospect of a decision by an official of a national government restricting access to controversial material on the internet by people all over the world. It has been said that if such capacity existed it might be used by a variety of regimes for a variety of purposes, not all of which would be benign. The task of the Court, at least at this stage of the analysis, is only to determine the legal meaning and effect of the removal notice. That is done by construing its language and the language of the Act under which it was issued. It is ultimately the words used by Parliament that determine how far the notice reaches.”

Nevertheless, when it came to consider reasonableness as a matter of construction of the language of the Act, something very like those considerations reappeared:

“49    If s 109 of the OS Act provided for a notice imposing such a requirement, it would clash with what is sometimes described as the “comity of nations” in a fundamental manner. …

50    If given the reach contended for by the Commissioner, the removal notice would govern (and subject to punitive consequences under Australian law) the activities of a foreign corporation in the United States (where X Corp’s corporate decision-making occurs) and every country where its servers are located; and it would likewise govern the relationships between that corporation and its users everywhere in the world.

The Commissioner, exercising her power under s 109, would be deciding what users of social media services throughout the world were allowed to see on those services. The content to which access may be denied by a removal notice is not limited to Australian content.

In so far as the notice prevented content being available to users in other parts of the world, at least in the circumstances of the present case, it would be a clear case of a national law purporting to apply to “persons or matters over which, according to the comity of nations, the jurisdiction properly belongs to some other sovereign or State”. Those “persons or matters” can be described as the relationships of a foreign corporation with users of its services who are outside (and have no connection with) Australia. What X Corp is to be permitted to show to users in a particular country is something that the “comity of nations” would ordinarily regard as the province of that country’s government.

51    The potential consequences for orderly and amicable relations between nations, if a notice with the breadth contended for were enforced, are obvious. Most likely, the notice would be ignored or disparaged in other countries. (The parties on this application tendered reports by experts on US law, who were agreed that a US court would not enforce any injunction granted in this case to require X Corp to take down the 65 URLs.)”

In similar vein the judge went on to consider the balance of convenience, in case he was wrong on the construction of the statute:

“56    If the considerations relating to the comity of nations (discussed at [48]–[51] above) had not led me to the view that the Commissioner has not made out a prima facie case, the same considerations would have led me to conclude that the balance of convenience does not favour extending the interlocutory injunction in its current (or any similar) form.

57    On the one hand the injunction, if complied with or enforced, has a literally global effect on the operations of X Corp, including operations that have no real connection with Australia or Australia’s interests. The interests of millions of people unconnected with the litigation would be affected. 

Justifying an interlocutory order with such a broad effect would in my view require strong prospects of success, strong evidence of a real likelihood of harm if the order is not made, and good reason to think it would be effective. At least the first and the third of these circumstances seem to be largely absent. The first is discussed above. 

As to the third, it is not in dispute that the stabbing video can currently be viewed on internet platforms other than X. I was informed that the video is harder to find on these platforms. The interim injunction is therefore not wholly pointless. However, removal of the stabbing video from X would not prevent people who want to see the video and have access to the internet from watching it.

58    On the other hand, there is uncontroversial expert evidence that a court in the US (where X Corp is based) would be highly unlikely to enforce a final injunction of the kind sought by the Commissioner; and it would seem to follow that the same is true of any interim injunction to similar effect. This is not in itself a reason why X Corp should not be held to account, but it suggests that an injunction is not a sensible way of doing that. Courts rightly hesitate to make orders that cannot be enforced, as it has the potential to bring the administration of justice into disrepute.”

A notable aspect of these passages is the approach to comity of nations, especially in the balance of convenience section which refers to the effect on millions of people unconnected with the litigation. It stands in significant contrast with the approach of the Canadian Supreme Court in Equustek (a trade mark and confidential information case).

The court in that case took an approach to comity that was both more abstract and more state-centric than that of Kennett J. It was abstract in that it was apparently sufficient that other countries would recognise the notion of intellectual property rights – without needing to consider the concrete question of whether the plaintiff in fact owned equivalent intellectual property rights throughout the world. It was more state-centric in that it focused entirely on the sensibilities of other states, without consideration of the individual interests and rights of users throughout the world.

Both differences are apparent from a passage in the British Columbia Court of Appeal judgment under appeal in Equustek, endorsed by the Canadian Supreme Court:

"In the case before us, there is no realistic assertion that the judge’s order will offend the sensibilities of any other nation. It has not been suggested that the order prohibiting the defendants from advertising wares that violate the intellectual property rights of the plaintiffs offends the core values of any nation." [BCCA 93]

The notion that international law is about more than mere state interests gains some support from the academic Jeremy Waldron. He has referred to:

‘the peaceful and ordered world that is sought in [international law] – a world in which violence is restrained or mitigated, a world in which travel, trade and cooperation are possible. . . . [This, he says, is] something sought not for the sake of national sovereigns themselves, but for the sake of the millions of men, women, communities, and businesses who are committed to their care’ [J. Waldron, ‘Are Sovereigns Entitled to the Benefit of the International Rule of Law?’ (2011) 22 European Journal of International Law 325.]

The Australian case is due to go forward to a full trial in July 2024. It has the potential to become a test of the circumstances in which courts will exercise jurisdictional self-restraint over the internet.

Internet legal developments to look out for in 2022

Another instalment of my annual round-up of what is on the horizon for UK internet law [Updated 29 April and 2 November 2022]. It does stray a little beyond our shores, noting some significant EU developments (pre-Brexit habits die hard). As always, it does not include data protection (too big, not really my field).

Draft Online Safety Bill The UK government published its draft Online Safety Bill in May 2021. The Parliamentary Joint Pre-Legislative Scrutiny Committee published its report on the draft Bill on 14 December 2021. A sub-committee of the Commons DCMS Select Committee also published a report on 24 January 2022, as did the Lords Communications and Digital Committee Inquiry on Freedom of Expression Online on 22 July 2021.

The government is expected to introduced a Bill into Parliament by on 17 March 2022. The Bill had its Second Reading on 19 April 2022. Its Report Stage is paused, likely to be recommenced this month.  Among many things for which the draft legislation is notable, its abandonment of the ECD Article 15 prohibition on general monitoring obligations stands out.

EU Digital Services Act The European Commission published its proposals for a Digital Services Act and a Digital Markets Act on 15 December 2020. The proposed Digital Services Act includes replacements for Articles 12 to 15 of the ECommerce Directive.  Following a vote in the European Parliament on 20 January 2022, the proposed legislation will now entered the trilogue stage. Political agreement was reached on 23 April 2022. The final text was published in the Official Journal on 27 October 2022.

Terrorist content The EU Regulation on addressing the dissemination of terrorist content online will come into effect on 7 June 2022.

Erosion of intermediary liability shields by omission One by-product of Brexit is that the UK is no longer bound to implement the conduit, caching and hosting shields provided by the EU eCommerce Directive. The government says that it “is committed to upholding the liability protections now that the transition period has ended”.

However, implementation of that policy requires every new piece of legislation that could impose liability on an intermediary explicitly to include the protections. If that is not done, then, owing to the fact that the original Electronic Commerce Directive Regulations 2002 do not have prospective effect, the protections will not apply to that new source of liability.

Two examples are already progressing though Parliament: the statutory codification of the public nuisance offence in the Policing Bill (which, following Royal Assent, came into force on 26 June 2022), and the electronic election imprints offences in the Elections Bill (Royal Assent 28 April 2022, not yet in force), neither of which includes the conduit, caching and hosting shields.

Such omissions have been known in the past, and were cured by statutory instrument under the European Communities Act 1972. That option is no longer available. As time goes on, accretion of such omissions in new legislation will gradually erode the intermediary protections to which the government is committed.

Law Commission Reports The Law Commission has issued two Reports making recommendations that are relevant to online speech. The first is its Report on Reform of the Communications Offences (notably, recommending replacing S.127 Communications Act 2003 and  the Malicious Communications Act 1988 with a new harm-based offence). The second report is on Hate Crime Laws. The recommendations on communications offences, at least, are being considered for incorporation have been included in the Online Safety Bill.

Copyright The Polish government’s challenge to Article 17 (Poland v Parliament and Council, Case C-401/19) is pending was decided on 26 April 2022. Poland argued that Article 17 makes it necessary for OSSPs, in order to avoid liability, to carry out prior automatic filtering of content uploaded online by users, and therefore to introduce preventive control mechanisms. It contended that such mechanisms undermine the essence of the right to freedom of expression and information and do not comply with the requirement that limitations imposed on that right be proportionate and necessary.

The Advocate-General’s Opinion was delivered on 15 July 2021. It was something of an Opinion of Solomon: recommending that the challenge be rejected, but only on the basis that the Directive is implemented in a way that minimises false positives. The Advocate General also, in a postscript, challenged aspects of the Article 17 guidance issued by the Commission subsequent to the drafting of the Opinion. The judgment largely followed the Opinion, dismissing the challenge but on the basis of an interpretation of Article 17 that included strict safeguards against removal of lawful content.

Policing Bill The Police, Crime, Sentencing and Courts Bill has ignited significant controversy over its impact on street protests, including through its statutory codification of the common law offence of public nuisance. The potential application of the new statutory offence to online speech, however, has gone virtually unnoticed.  

Product Security and Telecommunications Infrastructure Bill An honourable mention for this Bill: a framework for imposing all kinds of security requirements on (among other things) internet-connectable products.

Back from the dead? The Digital Economy Act 2017 The non-commencement of the age verification provisions of the Digital Economy Act 2017 has long been a source of controversy. In November 2021 the High Court gave permission to two members of the public to commence judicial review proceedings. This may now in practice have been overtaken by the inclusion of pornography sites in the Online Safety Bill.

Cross-border data access The US and the UK signed a Data Access Agreement on 3 October 2019, providing domestic law comfort zones for service providers to respond to data access demands from authorities located in the other country. No announcement has yet been made that Agreement has entered into operation. It came into force on 3 October 2022.

The Second Additional Protocol to the Convention on Cybercrime on enhanced co-operation and disclosure of electronic evidence is was open for signature from 12 May 2022 and presented to the UK Parliament in July 2022.

State communications surveillance The kaleidoscopic mosaic of cases capable of affecting the UK’s Investigatory Powers Act 2016 (IP Act) continues to reshape itself. In this field CJEU judgments will continue to be relevant in principle, since they form the backdrop to future reviews of the European Commission’s June 2021 UK data protection adequacy decision.

Domestically, Liberty has a pending judicial review of the IP Act bulk powers and data retention powers. Some EU law aspects (including bulk powers) were stayed pending the Privacy International reference to the CJEU. Those aspects are now proceeding and, according to Liberty, are likely to be in court in early 2022. The Divisional Court rejected the claim that the IP Act data retention powers provide for the general and indiscriminate retention of traffic and location data, contrary to EU law. That point may in due course come before the Court of Appeal. The Divisional Court gave judgment on the stayed aspects on 24 June 2022. Liberty's claims were rejected except for one aspect concerning the need for prior independent authorisation for access to some retained data. 

Investigatory Powers Act review The second half of 2022 will see the Secretary of State preparing the report on the operation of the IP Act required under Section 260 of the Act.

Electronic transactions The pandemic focused attention on legal obstacles to transacting electronically and remotely. Whilst uncommon in commercial transactions, some impediments do exist and, in a few cases, were temporarily relaxed. That may pave the way for permanent changes in due course.

Although the question typically asked is whether electronic signatures can be used, the most significant obstacles tend to be presented by surrounding formalities rather than signature requirements themselves. A case in point is the physical presence requirement for witnessing deeds, which stands in the way of remote witnessing by video or screen-sharing. The Law Commission Report on Electronic Execution of Documents recommended that the government should set up an Industry Working Group to look at that and other issues. The Working Group has now been formed. It issued an Interim Report on 1 February 2022.

[Updated 29 April 2022 and 2 November 2022.]

The Online Harms edifice takes shape

The government has now published the Final Response to its Consultation on the April 2019 Online Harms White Paper.

Background

To recap, in the White Paper the government proposed to impose a “duty of care” on companies whose services host user-generated content or facilitate public or private online interaction between users. The duty of care would also apply to search engines.

An intermediary in scope would have to take reasonable steps to prevent, reduce or mitigate harm occurring on its service, including lawful content and activity deemed to be harmful. By its nature the duty placed on the intermediary would be to prevent the risk of one third party user causing harm to someone else.

This proposal differed from offline duties of care in two main respects: First, the White Paper did not limit or define the notion of harm. Comparable safety-related duties of care in the offline world are about objectively ascertainable physical injury and damage to property. An  undefined concept of harm arising from online speech was inevitably subjective and malleable. It raised objections of impermissible vagueness, consequent arbitrariness, and the prospect of online speech being judged by the standard of the most easily offended reader, viewer or listener.

Second, in the offline world a safety-related duty of care that imposes liability for failure to prevent third parties injuring each other is the exception rather than the norm - and in any event has not been applied to speech.

The White Paper proposed that the intermediaries’ duty of care would be overseen and enforced by a discretionary regulator - subsequently indicated as likely to be Ofcom - reminiscent of the world of television and radio. This represented a radical departure from the offline world, in which individual speech is governed only by settled and certain general law, not broadcast-style regulation by regulator.

All this was presented under the banner of offline-online equivalence.

The effect of the proposed Online Harms regime, although presented as regulating the tech companies, is that the regulator would indirectly govern our own individual speech via the proxy of online intermediaries acting under the legal compulsion of the duty of care. If harm were left undefined and unlimited, then the regulator would in effect have the ability to write its own parallel rulebook for online speech – both as to what amounted to harm, and what steps an intermediary should take to mitigate the risk of speech that the regulator deemed to be harmful.

In February 2020 the government published an Initial Response to the White Paper signalling some revisions to the regime, in particular a ‘differentiated’ duty of care that would apply more lightly to content that was harmful but not illegal. There was still no attempt to define or limit the concept of harm.

The government has now confirmed that Ofcom will be the scheme’s discretionary regulator. The Final Response proposes a number of significant changes to the regime described in the White Paper.

Harms in scope

The most significant development is that the government has now:

• Proposed a general definition of “harmful” content and activity: it must give rise to a “reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals”. [2.2] 
• Significantly limited what counts as illegal user content and activity for the purposes of the duty of care: excluding civil liability altogether and also limiting the kinds of criminal offences in scope to those that meet the general definition of “harmful” [2.24].

It has also confirmed previous indications that harms to organisations will not be in scope. [2.2, 4.1] Nor would intellectual property breaches, data protection breaches, fraud, breaches of consumer protection law, cyber security breaches or hacking. Harm arising from dark web activity would also be excluded. [2.3]

The combined effect of these steps is that the subject matter of the duty of care has moved in the direction of comparable offline duties of care. It is now more focused towards personal safety properly so-called, rather than resting on unbounded notions of harm. That is also reflected in the new name for the legislation: the Online Safety Bill.

By way of example, the government now explains that disinformation should not be regarded as per se dangerous, and that to do that would trespass unacceptably on freedom of speech:

“the duty of care will apply to content or activity which could cause significant physical or psychological harm to an individual, including disinformation and misinformation. Where disinformation is unlikely to cause this type of harm it will not fall in scope of regulation. Ofcom should not be involved in decisions relating to political opinions or campaigning, shared by domestic actors within the law.” [2.81]

This paragraph recalls the difference of opinion between Home Office and DCMS Ministers over 5G conspiracy theories when giving evidence to the Home Affairs Committee in May 2020.

Nevertheless, the definition of harmful remains problematic: not least because inclusion of ‘psychological impact’ may suggest that the notion of harm is still tied to variable, subjective reactions of different readers. Subjectivity opens the door to application of a standard of the most readily upset user. And while the subject matter of the duty of care may be more closely aligned with traditional duties of care, its nature – a duty to prevent third parties from harming each other – remains the exception, not the norm, in the offline world.

The Final Response proposes the creation, by secondary legislation, of specific ‘priority categories’ of harmful content and criminal offences, posing the greatest risk to individuals. [24], [2.3], [2.20]. The significance of these categories would be in underpinning a reformulated version of the ‘differentiated’ duty of care that was floated in the government’s Initial Response (see further below).

Providers and services in scope

Under the revised proposals, in-scope providers would be split into two categories of provider, subject to versions of the duty of care differing both as to what steps would be required to discharge the duty of care, and in respect of what kinds of harmful content. Only services designated as Category 1 would be duty-bound to address legal but harmful content.

Ofcom would determine which services meet the criteria for Category 1, according to thresholds previously set by the government. The relevant factors would be set out in the legislation: size of audience and functionalities offered.

According to the Response, functionalities such as the ability to share content widely or contact users anonymously are more likely to give rise to harm. [2.16]. When world-wide availability is an inherent feature of the internet, to treat the ability to share content widely as inherently risky is challenging for a government that proclaims that freedom of expression is at the heart of the proposed regulatory framework [1.10]. Contrary to the popular slogan, freedom of reach is indeed an aspect of freedom of speech - as the Supreme Court of India has held:

"There is no dispute that freedom of speech and expression includes the right to disseminate information to as wide a section of the population as is possible. The wider range of circulation of information or its greater impact cannot restrict the content of the right nor can it justify its denial." 

In the offline world, providing a venue specifically for activities that create a risk of danger is one situation in which a duty to prevent visitors injuring each other can arise. But to suggest that merely enabling individuals to speak to a large audience is a dangerously risky activity verges on an existential challenge to freedom of speech.

The Response excludes from scope:

• certain ‘low-risk’ activities: user comments on digital content in relation to content directly published by a service. This would exclude online product and service reviews and ‘below the line’ reader comments on news website articles. [1.7]
• three kinds of service: (a) B2B services as previously signalled in the Initial Response, (b) online services managed by educational institutions already subject to sufficient safeguarding duties or expectations, and (c) e-mail, voice telephone and SMS/MMS services. [1.6]

As to (c), the Response observes that “It is not clear what intermediary steps providers could be expected to take to tackle harm on these services before needing to resort to monitoring communications, so imposing a duty of care would be disproportionate.”

The result of the exclusions appears to be that the John Lewis customer review section would now be out of scope, but a site such as Mumsnet would still be in scope.

OTT private messaging services remain in scope [1.5]. The Response takes an approach to those that differs markedly from SMS/MMS services. Messaging providers may be required to monitor communications on private communications services, potentially by two routes.

First, it appears that Ofcom may have discretion to include monitoring in a Code of Practice. (Strictly speaking, however, this would not be mandatory, since it is always open to a provider to demonstrate to Ofcom that it can fulfil its duty of care as effectively in some other way [2.48].) The non-statutory interim code of practice on online child sexual exploitation and abuse (CSEA) published by the Home Office alongside the Response provides that automated technology should be considered on a voluntary basis.

Second, Ofcom would have express power to require companies to use “automated technology that is highly accurate” to identify illegal CSEA content and activity. This power would be usable where alternative measures cannot effectively address CSEA. Whilst the Response comments that this power is more likely to be considered proportionate on public platforms than private services, private services are not excluded. Ofcom would be required to seek approval from Ministers before exercising the power, on the basis that sufficiently accurate tools exist. The Response notes that the government assesses that, currently, sufficiently accurate tools exist to identify CSEA material that has previously been assessed as illegal. [2.59. 2.60]

Encryption is not mentioned in the Response.

News media and journalism The potential application of the legislation to news media and journalism has been fraught from the outset. The White Paper did not mention the issue, following which the then Secretary of State wrote to the Society of Editors assuring them that “where these services are already well regulated, as IPSO and IMPRESS do regarding their members' moderated comment sections, we will not duplicate those efforts. Journalistic or editorial content will not be affected by the regulatory framework.”

This left questions unanswered, for instance the position of mainstream news media not regulated by IPSO or IMPRESS. Nor did it address the position of newspapers’ own social media pages and feeds, which would count as user generated content and thus be indirectly regulated by Ofcom via the intermediaries’ duty of care.

The Final Response is, if anything, less clear than previously. It confirms that comment sections on news publishers’ websites would be out of scope, by virtue of the ‘low risk’ user comments exclusion mentioned above.  For social media feeds, it says that legislation will include ‘robust protections’ for journalistic content shared on in-scope services. As to what those protections might be, and what might count as journalistic content, the Response is silent. [1.10, 1.12]

Differentiated duty of care

The Initial Response proposed a differentiated duty of care, whereby for legal but harmful material and activities in-scope providers would be required only to enforce transparently, consistently and (perhaps) effectively, the standards that they chose to incorporate in their terms and conditions.

It always did seem unlikely that, for ‘legal but harmful’ content, the government intended to leave intermediaries completely to their own devices as to what standards (if any) to incorporate in their user terms and conditions. In 2018, after all, the government had said in its consultation response to the Internet Safety Strategy Green Paper that:

“The government has made clear that we require all social media platforms to have [inter alia]: Terms and conditions that provide a minimum level of safety and protection for users”.]

So it has proved.  The proposal in the Final Response is complex and nuanced. Its main features are:

• Providers that exceed specified audience and functionality thresholds will be designated as Category 1 providers (see above). 
• All in-scope providers will be expected to assess whether children are likely to access their services and, if so, to take additional protections for children using them [2.15] 
• Only Category 1 providers will be required to take action with regard to legal but harmful content and activity accessed by adults [2.15].
• The duty of care of non-Category 1 providers for adults would therefore apply only in relation to criminal content and activities (of a kind not otherwise excluded) that present a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals.

It should follow, although the Response does not spell this out completely clearly, that for non-Category 1 providers the general obligations listed below (such as risk assessment) would apply only in relation to the risk of such criminal content activities – and that ‘safety’ should also be understood in that sense. 

For Category 1 providers the general obligations would apply additionally to legal content and activity presenting a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals. 

General obligations

• All in-scope providers have a primary responsibility to take action to prevent user-generated content or activity on their services causing significant physical or psychological harm to individuals. To do this they will complete an assessment of the risks associated with their services and take reasonable steps to reduce the risks of the harms they have identified occurring. [2.7]
• Providers will fulfil the duty of care by putting in place systems and processes that improve user safety on their services – including, for example, user tools, content moderation and recommendation procedures. [2.9]
• Providers will be required to consider users’ rights, including freedom of expression online, both as part of the risk assessment and when making decisions on what safety systems and processes to put in place. [2.10]
• Regulation will ensure transparent and consistent application of terms and conditions relating to harmful content. This will include preventing companies from arbitrarily removing content. [2.10]
• Users must be able to report harm when it does occur and seek redress, challenge wrongful takedown and raise concerns about companies’ compliance with their duties. [2.11]
• All providers will have a specific legal duty to have effective and accessible reporting and redress mechanisms. This will cover harmful content and activity, infringement of rights (such as over-takedown), or broader concerns about a company’s compliance with its regulatory duties [2.12]

Illegal content and activities

• For in-scope criminal activity, all providers will need to ensure that illegal content is removed expeditiously and that the risk of it appearing and spreading across their services is minimised by effective systems [2.19]
• Priority categories of offences, against which providers will be required to take particularly robust action, will be set out in secondary legislation. [2.20] For CSEA and terrorism this may include proactively identifying and blocking or removing this type of material if other steps have not been effective and safeguards are in place. [2.21]

The Response is silent as to how such an obligation may be consistent with the prohibition on general monitoring obligations under Article 15 of the eCommerce Directive. The government has said, in the context of Brexit, that it has no current plans to change the UK’s approach to prohibition on general monitoring requirements.

Legal but harmful content and activity accessed by adults (Category 1 providers only)

• The legislation will not require removal of specific pieces of legal content [2.28], unless specified as not permitted by the provider’s terms and conditions [2.33] Terms and conditions could be about, for example, labelling and de-prioritising [2.32].
• Priority categories of legal but harmful material will be set out in secondary legislation. These will be categories of legal but harmful material that Category 1 providers should, at a minimum, address through their terms and conditions. The Response gives the examples of content promoting self-harm, hate content, online abuse that does not meet the threshold of a criminal offence, and content encouraging or promoting eating disorders. [2.29]
• Category 1 providers will be obliged to state how they will handle other categories of legal but harmful material identified in their risk assessment and make clear what is acceptable on their services for that content. [2.31]

Controversial viewpoints

• Category 1 companies will not be able to arbitrarily remove controversial viewpoints and users will be able to seek redress if they feel that content has been removed unfairly. [2.34]
• User redress mechanisms will enable users to challenge content that unduly restricts their freedom of expression. This appears to apply to all in-scope providers (Annex A).

These provisions appear to be the ‘impartiality’ requirements that were trailed in the press before the release of the Final Response, reportedly at the instigation of 10 Downing Street. It is unclear whether these provisions are intended to override substantive policies set out in providers’ terms and conditions. They appear to be unrelated to, or at least to go wider than, issues about illegal or harmful content.

Children

• All companies in scope will required to assess the likelihood of children accessing their service. [2.36] Only services likely to be accessed by children will be required to provide additional protections for children accessing them, starting with conducting a specific child safety risk assessment. [2.36], [2.37]
• The government will set out in secondary legislation priority categories of legal but harmful content and activity impacting children, meeting the general definition of harmful content and activity already described. These will be categories impacting children that companies in scope should, at a minimum, take action on. [2.38]
• Age assurance and age verification technologies are expected to play a key role in fulfilling the duty of care. [2.41]

Codes of Practice

The Final Response has increased the amount of influence that the government will have over Ofcom’s Codes of Practice. Ofcom will be required to send the final draft of a Code of Practice to the Culture Secretary and the Home Secretary, who will have the power to reject a draft code and require the regulator to make modifications for reasons relating to government policy.

Parliament will have the opportunity to debate and vote on the high level objectives set out by the government for the Codes of Practice by the affirmative resolution procedure. Completed codes will be laid in Parliament, subject to negative resolution. [4.10]

Search engines

Little is said in the Final Response about how the proposed duty of care would apply to search engines, beyond a brief summary of actions that they can take to mitigate the risk of harm and proportionate systems and processes that they would be expected to put in place to keep their users safe.

Search engines would need to assess the risk of harm occurring across their entire service. Ofcom would provide guidance specific to search engines regarding regulatory expectation

The government proposes that given the distinct nature of search engines, legislation and codes of practice would include specific material for them. It says that all regulatory requirements would be proportionate, and respect the key role of search engines in enabling access to information online. [1.3]

Territoriality

For the first time, the Final Response has set out the proposed territorial reach of the proposed legislation. Somewhat surprisingly, it appears to propose that services should be subject to UK law on a ‘mere availability of content’ basis. Given the default cross-border nature of the internet, this is tantamount to legislating extraterritorially for the whole world. It would follow that any provider anywhere in the rest of the world would have to geo-fence its service to exclude the UK in order to avoid engaging UK law. Legislating on a mere availability basis has been the subject of criticism over many years since the advent of the internet. [1.1]

Overall commentary

The fundamental issues with the government’s White Paper proposals have been exhaustively discussed on previous occasions. Reminiscent of a sheriff in the Wild West, to which the internet is so often likened, Ofcom would enlist deputies - social media platforms and other intermediaries acting under a legal duty of care - to police the unruly online population. Unlike its Wild West equivalent, however, Ofcom would get to define its territory and write the rules, as well as enforce them.

The introduction of a general definition of harm would tie Ofcom’s hands to some degree in deciding what does and does not constitute harmful speech. Limiting the scope of ‘harm’ to a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals goes some way to align the proposed duty of care more closely with analogous offline duties of care, which are specifically safety-related.

Nevertheless, when applied in the context of speech there remain significant problems.

1. What is an adverse psychological impact? Does it have to be a medically recognised condition? If not, how wide is it meant to be? Is distress sufficient? The broader the meaning, the closer we come to a limitation that could mean little or nothing more than being upset or unhappy. The less clear the meaning, the more discretion would be vested in Ofcom to decide what counts as harm, and the more likely that providers would err on the side of caution in determining what kinds of content or activity are in scope of their duty of care.

2. The difficulty, not to say virtual impossibility, of the task faced by the regulator and providers should not be underestimated. Thus, for the lawful but harmful category, the government has said that it will include online abuse as a priority category in secondary legislation. However, on the basis of these proposals that must be limited to abuse that falls within the general definition of harm – i.e. abuse that presents a reasonably foreseeable risk of a significant adverse physical or psychological impact on individuals. The provider’s actions under the duty of care should relate only to such harmful abuse. Where, concretely, is the dividing line between abuse that does and does not carry a foreseeable risk of adverse psychological impact? What content falls on either side of the line?

The provider would also have to take into account the proposed obligation not to remove controversial viewpoints and the possibility of user redress for unduly restricting their freedom of expression. Coincidentally, the Divisional Court in Scottow v CPS has in the last few days issued a judgment in which it referred to “the well-established proposition that free speech encompasses the right to offend, and indeed to abuse another”.

These issues illustrate the care that has to be taken with using terms such as ‘online abuse’ to cover everything from strong language, through insults, to criminal threats of violence.

3. What is the threshold to trigger the duty of care? Is it the risk that someone, somewhere, might read something and claim to suffer an adverse psychological impact as a result? Is it a risk gauged according to the notional attributes of a reasonably tolerant hypothetical user, or does the standard of the most easily upset apply? How likely does it have to be that someone might suffer an adverse psychological impact if they read it? Is a reasonably foreseeable, but low, possibility sufficient? 

The Media Minister John Whittingdale, writing in the Daily Mail on the morning of the publication of the Final Response, said:

“This is not about an Orwellian state removal of content or building a ‘woke-net’ where causing offence leads to instant punishment.  Free speech includes the right to offend, and adults will still be free to access content that others may disapprove of.”

If risk and harm thresholds are sufficiently low and subjective, that is what would result.

4. Whatever the risk threshold might be, would it be set out in tightly drawn legislation or left to the discretion of Ofcom? It will not be forgotten that Ofcom, in a 2018 survey, suggested to respondents that ‘bad language’ is a harmful thing. A year later it described “offensive language” as a “potential harm”.

5. Lastly, in the absence of deliberate intent an author owes no duty avoid causing harm to a reader of their work, even though psychological injury may result from reading it. That was confirmed by the Supreme Court in Rhodes. The government’s proposals would therefore mean that an intermediary would have a duty to consider taking steps in relation to material for which the author itself has no duty of care.

These are difficult issues that go to the heart of any proposal to impose a duty of care. They ought to have been the subject of debate over the last couple of years. Unfortunately they have been buried in the rush to include every conceivable kind of harm - however unsuited it might be to the legal instrument of a duty of care - and in discussions of ‘systemic’ duties of care abstracted from consideration of what should and should not amount to harm.

It should be no surprise if the government’s proposals became bogged down in a quagmire resulting from the attempt to institute a universal law of everything, amounting to little more than a vague precept not to behave badly online. The White Paper proposals were a castle built on quicksand, if not thin air.

The proposed general definition of harm, while not perfect, gives some shape to the edifice. It at least sets the stage for a proper debate on the limits of a duty of care, the legally protectable nature of personal safety online, and its relationship to freedom of speech – even if that should have taken place two years ago. Whether regulation by regulator is the appropriate way to supervise and police an appropriately drawn duty of care in relation to individual speech is another matter.

A Tale of Two Committees

Two Commons Committees –the Home Affairs Committee and the Digital, Culture, Media and Sport Committee – have recently held evidence sessions with government Ministers discussing, among other things, the government’s proposed Online Harms legislation. These sessions proved to be as revealing, if not more so, about the government’s intentions as its February 2020 Initial Response to the White Paper.

As a result on some topics we know more than we did, but the picture is still incomplete. Some new issues have surfaced. Other areas have become less clear than they were previously.

Above all, nothing is set in stone. The Initial Response was said to be indicative of a direction of travel and to form an iterative part of a process of policy development. The destination has yet to be reached – if, that is, the government ever gets there at all. It may yet hit a road block somewhere along the way, veer off into a ditch, or perhaps undergo a Damascene conversion should it finally realise the unwisdom of creating a latter-day Lord Chamberlain for the internet. Or the road may eventually peter out into nothingness. At present, however, the government is pressing ahead with its legislative intentions.

I’m going to be selective about my choice of topics, in the main returning to some of the key existing questions and concerns about the Online Harms proposals, with a sprinkling of new issues added for good measure. Much more ground than this was covered in the two sessions.

Borrowing from the old parlour game, each topic starts with what the White Paper said; followed by what the Initial Response said; then what the Ministers said; and lastly, the Consequence. The Ministers are Oliver Dowden MP (Secretary of State for Digital, Culture, Media and Sport); Caroline Dinenage MP (Minister for Digital and Culture) and Baroness Williams (Lords Minister, Home Office).  

Sometimes the government’s Initial Response to Consultation recorded consultation submissions, but came to no conclusion on the topic. In those instances the Initial Response is categorised as saying ‘Nothing’. Some repetitive statements have been pruned.

Since this is a long read, here is a list of the selected topics:

• Will Parliament or the regulator decide what “harm” means?

• The regulator’s remit: substance, process or both?
• For “lawful but harmful” content seen by adults, will the regulator be interested only in whether intermediaries are enforcing whatever content standards they choose to put in their TandCs?

• Codes of Practice for specific kinds of user content or activity?

• Search engines in scope?

• Everything from social media platforms to retail customer review sections?
• Will journalism and the press be excluded from scope?

• End to end encryption

• Identity verification

• Extraterritoriality

1. Will Parliament or the regulator decide what “harm” means?

The White Paper said:

“… government action to tackle online content or activity that harms individual users, particularly children, or threatens our way of life in the UK, either by undermining national security, or by reducing trust and undermining our shared rights, responsibilities and opportunities to foster integration.”

“This list [Table 1, Online harms in scope] is, by design, neither exhaustive nor fixed. A static list could prevent swift regulatory action to address new forms of online harm, new technologies, content and new online activities.”

The Initial Response said:

Nothing.

The Ministers said:

Oliver Dowden: “The only point that I have tried to make is that I am just keen on this proportionality point because it is often the case that regulation that starts out with the best of intentions can, in its interpretation if you do not get it right, have a life of its own. It starts to get interpreted in a way that Parliament did not intend it to be in the first place. I am just keen to make sure we put those kinds of hard walls around it so that the regime is flexible but that in its interpretation it cannot go beyond the intent that we set out in the first place in the broad principles.” (emphasis added)

Caroline Dinenage: “For what you might call the “legal but harmful” harms, we are not setting out to name them in the legislation. That is for the simple reason that technology moves on at such a rapid pace that it is very likely that we would end up excluding something….  We want to make sure that this piece of legislation will be agile and able to respond to harms as they emerge. The legislation will make that clearer, but it will be for the regulator to outline what the harms are and to do that in partnership with the platforms.” (Q.554) (emphasis added)

The Consequence: It is difficult to reconcile the desire of the Secretary of State to erect “hard walls”, in order to avoid unintended consequences, with the government’s apparent determination to leave the notion of harm undefined, delegating to the regulator the task of deciding what counts as harmful. This kind of approach has serious implications for the rule of law.

Left undelineated, the concept of harm is infinitely malleable. The Home Office Minister Baroness Williams suggested in the Committee session that 5G disinformation could be divided into “harmless conspiracy theories” and “that which actually leads to attacks on engineers”, as well as a far-right element. One Committee member (Ruth Edwards M.P.) responded that she did not think that any element of the conspiracy theory could be categorised as ‘harmless’, because “it is threatening public confidence in the 5G roll-out” — a proposition with which the DCMS Minister Caroline Dinenage agreed.

Harm is thus equated with people changing their opinion about a telecommunications project. This unbounded sense of harm is on a level with the notorious “confusing our understanding of what is happening in the wider world” phraseology of the White Paper.  

Statements such as the concluding peroration by Baroness Williams: “I, too, want to make the internet a safer place for my children, and exclude those who seek to do society harm” have to be viewed against the backdrop of an essentially unconstrained meaning of harm.

When harm can be interpreted so broadly, the government is playing with fire. But it is we – not the government, the regulator or the tech companies – who stand to get our fingers burnt.

2. The regulator’s remit: substance, process or both?

The White Paper said:

“In particular, companies will be required to ensure that they have effective and proportionate processes and governance in place to reduce the risk of illegal and harmful activity on their platforms, as well as to take appropriate and proportionate action when issues arise. The new regulatory regime will also ensure effective oversight of the take-down of illegal content, and will introduce specific monitoring requirements for tightly defined categories of illegal content.” (6.16)

The Initial Response said:

“The approach will be proportionate and risk-based with the duty of care designed to ensure companies have appropriate systems and processes in place to improve the safety of their users.”

“The focus on robust processes and systems rather than individual pieces of content means it will remain effective even as new harms emerge. It will also ensure that service providers develop, clearly communicate and enforce their own thresholds for harmful but legal content.

“The kind of processes the codes of practice will focus on are systems, procedures, technologies and investment, including in staffing, training and support of human moderators.”

“As such, the codes of practice will contain guidance on, for example, what steps companies should take to ensure products and services are safe by design or deliver prompt action on harmful content or activity.”

“Rather than requiring the removal of specific pieces of legal content, regulation will focus on the wider systems and processes that platforms have in place to deal with online harms, while maintaining a proportionate and risk-based approach.”

“In fact, the new regulatory framework will not require the removal of specific pieces of legal content. Instead, it will focus on the wider systems and processes that platforms have in place to deal with online harms, while maintaining a proportionate and risk-based approach.”

“Of course, companies will be required to take particularly robust action to tackle terrorist content and online Child Sexual Exploitation and Abuse. The new regulatory framework will not remove companies’ existing duty to remove illegal content.”

The Ministers said:

Caroline Dinenage: “the codes of practice are really about systems and processes, rather than naming individual harms in the legislation. There are two exceptions to that: there will be codes of practice around child sexual exploitation and terrorist content, because those are both illegal.” (Q554)

“It is for the regulator to set out codes of practice, but they won’t be around individual harms; they will be around systems and processes—what we expect the companies to do. Rather than focusing on individual harms, because we know that the technology moves on so quickly that there could be more, it is a case of setting out the systems and processes that we would expect companies to abide by, and then giving the regulator the opportunity to impose sanctions on those that are not doing so.” (Q.556)

Q562 Stuart C. McDonald: “…if the regulator feels that algorithms are working inappropriately and directing people who have made innocent searches to, say, far-right content, will they be able to order, essentially, the company to make changes to how its algorithms are operating?

Caroline Dinenage: Yes, I think that they will. That is clearly something that we will set out in the full response. The key here is that companies must have clear transparency, they must set out clear standards, and they must have a clear duty of care. If they are designing algorithms that in any way put people at risk, that is, as I say, a clear design choice, and that choice carries with it a great deal of responsibility. It will be for the regulator to oversee that responsibility. If they have any concerns about the way that that is being upheld, there are sanctions that they can impose.”

The Consequence: As with the specific issue around the status of terms and conditions for “lawful but harmful” content (see below), it is difficult to see how a bright line can be drawn between substance and process.  Processes cannot be designed, risk-assessed or their effectiveness evaluated in the abstract — only by reference to goals such as improving user safety and reducing risk of harm. A duty of care evaluated without reference to the kind of harm intended to be guarded against makes no more sense than the smile without the Cheshire Cat. 

In Caparo v Dickman Lord Bridge cautioned against discussing duties of care in the abstract:

"It is never sufficient to ask simply whether A owes B a duty of care. It always necessary to determine the scope of the duty by reference to the kind of damage from which A must take care to save B harmless."

Risk assessment is familiar in the realm of safety properly so-called: danger of physical injury, where there is a clear understanding of what constitutes objectively ascertainable harm. It breaks down when applied to undefined, inherently subjective harms arising from users' speech. If "threatening public confidence in the 5G roll-out” (see above) can be labelled an online harm within scope of the legislation, that goes far beyond any tenable concept of safety.

The government’s approach appears to be to adopt different approaches to illegal and “legal but harmful”, the latter avowedly restricted to process (although see next topic as to how far that can really be the case). 

In passing, the Initial Response is technically incorrect in referring to “companies’ existing duty to remove illegal content”. No such general duty exists. Hosting providers lose the protection of the ECommerce Directive liability shield if they do not remove unlawful content expeditiously upon gaining actual or (for damages) constructive knowledge of the illegality. Even then, the eCommerce Directive does not oblige them to remove it. The consequence is that they become exposed to the risk of possible liability (which may or may not exist) under the relevant underlying law (see here for a fuller explanation). In practice that regime strongly incentivises hosting providers to remove illegal content upon gaining relevant knowledge. But they have no general legal obligation to do so.

3. For “lawful but harmful” content seen by adults, will the regulator be interested only in whether intermediaries are enforcing whatever content standards they choose to put in their TandCs?

The White Paper said:

“As indication of their compliance with their overarching duty of care to keep users safe, we envisage that, where relevant, companies in scope will:

• Ensure their relevant terms and conditions meet standards set by the regulator and reflect the codes of practice as appropriate.
• Enforce their own relevant terms and conditions effectively and consistently. …”

“To help achieve these outcomes, we expect the regulator to develop codes of practice that set out: 

• Steps to ensure products and services are safe by design.
• Guidance about how to ensure terms of use are adequate and are understood by users when they sign up to use the service. …
• Steps to ensure harmful content or activity is dealt with rapidly. …
• Steps to monitor, evaluate and improve the effectiveness of their processes.”

The Initial Response said:

“We will not prevent adults from accessing or posting legal content, nor require companies to remove specific pieces of legal content. The new regulatory framework will instead require companies, where relevant, to explicitly state what content and behaviour is acceptable on their sites and then for platforms to enforce this consistently.”

“To ensure protections for freedom of expression, regulation will establish differentiated expectations on companies for illegal content and activity, versus conduct that is not illegal but has the potential to cause harm. Regulation will therefore not force companies to remove specific pieces of legal content. The new regulatory framework will instead require companies, where relevant, to explicitly state what content and behaviour they deem to be acceptable on their sites and enforce this consistently and transparently. All companies in scope will need to ensure a higher level of protection for children, and take reasonable steps to protect them from inappropriate or harmful content.”

“Recognising concerns about freedom of expression, the regulator will not investigate or adjudicate on individual complaints. Companies will be able to decide what type of legal content or behaviour is acceptable on their services, but must take reasonable steps to protect children from harm. They will need to set this out in clear and accessible terms and conditions and enforce these effectively, consistently and transparently.”

The Ministers said:

Oliver Dowden: “The essence of online harms legislation is holding social media companies to what they have promised to do and to their own terms and conditions. My focus in respect of those is principally on two things: underage harms and illegal harms. Clearly, the trickiest category is legal adult harms. In respect of that, we are looking at how we tighten the measures to ensure that those companies actually do what they promised they would do in the first place, which often is not the case.” (Q20) (emphasis added)

“Clearly, in respect of legal adult harms, that is the underlying principle anyway in the sense that what we are really trying to do is say to those social media companies and tech firms, “Be true to what you say you are doing. Just stick by your terms and conditions”. We would ask the regulator to make sure that it is enforcing them, and then have tools at our disposal to require it to do so.” (Q89) (emphasis added)

Caroline Dinenage: “A lot of this is about companies having the right regulations and standards and duty of care, and that will also be in the online harms Bill and online harms work. If we can have more transparency as to what platforms regard as acceptable—there will be a regulator that will help guide them in that process—I think we will have a much better opportunity to tackle those things head-on.” (Q513) (emphasis added)

“With regard to our role in DCMS, it is more as a co-ordinator bringing together the work of all the different Government Departments and then liaising directly with the platforms to make sure that their standards, their regulations, are reflective of some of the concerns that we have—make sure, in some cases, that harmful content can be anticipated and therefore prevented, and, where that is not possible, where it can be stopped and removed as quickly as possible.” (emphasis added) (Q525)

Baroness Williams: “There is obviously that which is illegal and that which breaches the CSPs’ terms of use. It is that latter element, particularly in the area of extremism, on which we have really tried to engage with CSPs to get them to be more proactive.” (emphasis added) (emphasis added) (Q.527)

The Consequence: This is now one of the most puzzling areas of the government’s developing policy. The White Paper expected that codes of practice would ensure that terms and conditions meet “standards set by the regulator” and that terms of use are “adequate”. These statements were not on the face of them limited to procedural standards and adequacy. They could readily be interpreted as encompassing standards and adequacy judged by reference to harm reduction goals determined by the regulator (which, as we have seen, would be able to decide for itself what constitutes harm) – in other words, extending to the substantive content of intermediaries' terms and conditions.

When the Initial Response was published, great play was made of the shift to a differentiated duty of care: that it would be up to the intermediary to decide – for lawful content for adults - what standards to put in its terms and conditions. 

The remit of the regulator would be limited to ensuring those standards are clearly stated and enforced “consistently and transparently” (or “effectively, consistently and transparently”, depending on which part of the Initial Response you turn to; or “effectively and consistently”, according to the White Paper). Indeed the Secretary of State said in evidence that "The essence of online harms legislation is holding social media companies to what they have promised to do and to their own terms and conditions" 

But it seems from the other Ministers’ responses that the government has not disclaimed all interest in the substantive content of intermediaries’ terms and conditions. On the contrary, the government evidently sees it as part of its role to influence (to put it at its lowest) what goes into them. If the regulator’s task is to ensure enforcement of terms and conditions whose substantive content reflects the wishes of a government department, that is a far cry from the proclaimed freedom of intermediaries to set their own standards of acceptable lawful content.

Ultimately, what can be the point of emphasising how, in the name of upholding freedom of speech, the role of an independent regulator will be limited to enforcing the intermediaries’ own terms and conditions, if the government considers that part of its own role is to influence those intermediaries as to what substantive provisions those TandCs should contain?

This is one aspect of an emerging issue about division of responsibility between government and the regulator. It is tempting to think that once an independent regulator is established the government itself will withdraw from the fray. But if that is not so, then reducing the remit of the independent regulator concomitantly increases the scope for the government itself to step in.

That is especially pertinent in the light of the government’s desire to cast itself as a ‘trusted flagger’, whose notifications of unlawful content the intermediaries should act upon without question. Thus Caroline Dinenage appears to regard the platforms as obliged to remove anything that the government has told them it considers to be illegal (with no apparent requirement of prior due process such as independent verification), and would like them to take seriously anything else that the government notifies to them:

“We have found that we have become—I forget the proper term, but we have become like a trusted flagger with a number of the online hosting companies, with the platforms. So when we flag information, they do not have to double-check the concerns we have. Clearly, unless something is illegal, we cannot tell organisations to take it down; they have to make their own decision based on their own consciences, standards and requirements. But clearly we are building up a very strong, trusted relationship with them to ensure that when we flag things, they take it seriously.” (Emphasis added)

4. Codes of Practice for specific kinds of user content or activity?

The White Paper said:

“[T]he White Paper sets out high-level expectations of companies, including some specific expectations in relation to certain harms. We expect the regulator to reflect these in future codes of practice.”

It then set out a list of 11 harms, accompanied in each case by a list of areas in relation to that harm that it expected the regulator to include in a code of practice. For instance, in relation to disinformation a list of 11 specific areas included:

“Steps that companies should take in relation to users who deliberately misrepresent their identity to spread and strengthen disinformation.”; and

“Promoting diverse news content, countering the ‘echo chamber’ in which people are only exposed to information which reinforces their existing views.”

The Initial Response said:

“The White Paper talked about the different codes of practice that the regulator will issue to outline the processes that companies need to adopt to help demonstrate that they have fulfilled their duty of care to their users. … We do not expect there to be a code of practice for each category of harmful content, however, we intend to publish interim codes of practice on how to tackle online terrorist and Child Sexual Exploitation and Abuse (CSEA) content and activity in the coming months.”

The Ministers said:

“Caroline Dinenage: I think I need to clear up a bit of a misunderstanding about the White Paper. The 11 harms that were listed were really intended to be an illustrative list of what we saw as the harms. The response did not expect a code of practice for each one, because the codes of practice are really about systems and processes, rather than naming individual harms in the legislation. There are two exceptions to that: there will be codes of practice around child sexual exploitation and terrorist content, because those are both illegal.” (Q.554) (emphasis added)

The Consequence: The different approach to CSEA and terrorism probably owes more to the different areas of responsibility of the Home Office and the DCMS than to any dividing line between illegality and non-illegality. The White Paper covers many more areas of illegality than those two alone.

5. Search engines in scope?

The White Paper said:

“… will apply to companies that allow users to share or discover user-generated content, or interact with each other online.” (emphasis added)

“These services are offered by…  search engines” (Executive Summary)

The Initial Response said:

“The legislation will only apply to companies that provide services or use functionality on their websites which facilitate the sharing of user generated content or user interactions, for example though comments, forums or video sharing” (emphasis added)

The Ministers said:

“Caroline Dinenage: Again, we are probably victims of the fact that we published an interim response, which was not as comprehensive as our full response will be later on in the year. The White Paper made it very clear that search engines would be included in the scope of the framework and the nature of the requirements will reflect the type of service that they offer. We did not explicitly mention it in the interim response, but that does not mean that anything has changed. It did not cover the full policy. Search engines will be included and there is no change to our thoughts and our policy on that.”   (Q.560)

The Consequence: Notwithstanding the Minister’s explanation, the alterations in wording between the White Paper and the Initial Response (omitting “discover”, adding “only”) had the appearance of a considered change. The lesson for the future is perhaps that it would be unwise to parse too closely the text of anything else said or written by the government.

6. Everything from social media platforms to retail customer review sections?

The White Paper said:

“… companies of all sizes will be in scope of the regulatory framework. The scope will include… social media companies, public discussion forums, retailers that allow users to review products online, along with non-profit organisations, file sharing sites and cloud hosting providers.” (emphasis added)

The Initial Response said:

“To be in scope, a business would have to operate its own website with the functionality to enable sharing of user-generated content, or user interactions.”

The Ministers said:

Oliver Dowden: “We are a Europe leader in this. I have seen, as I am sure you have seen, the unintended consequences of good-intended legislation then having bureaucratic implications and costs on businesses that we want to avoid.

For example, in respect of legal online harms for adults, if you are an SME retailer and you have a review site on your website for your product and people can put comments underneath that, that is a form of social media. Notionally, that would be covered by the online harms regime as it stands. The response to that is they will go through this quick test and then they will find it does not apply to them. My whole experience of that for SMEs and others is that it is all very well saying that when you are sat have no idea what this online harms thing is, this potentially puts a big administrative burden on you. (emphasis added)

Are there ways in which we can carve out those sorts of areas so we focus on where we need to do it? Those kinds of arguments pertain less to illegal harms and harms to children. I hope that gives you a flavour of it.” (Q.88)

Q89 Damian Hinds: “Yes, quite so. I think in the previous announcement there was quite a high estimate of the number of firms or proportion of total firms that would somehow be counted in the definition of an online platform, which was rather a disturbing thought. It would be very welcome, what you can do to limit the scope of who counts as a social media platform.”

The Consequence: This exchange does shine a light on the expansive scope of the proposed legislation. The Secretary of State said that SME retailers with review sections were “notionally” covered. However, there was nothing notional about it.  Retailer review sections were expressly included in the White Paper, as were companies of all sizes.

As the Secretary of State suggests, it is little comfort for an SME to be told “don’t worry, you’ll be low risk so it won’t really apply to you” if: (a) you are in scope on the face of it, and (b) it is left to the regulator to decide whether the duty of care should bear less heavily on some intermediaries than others. 

There are, of course, many other kinds of non-social media platform intermediary who are in scope as well as SME retailers with review sections: apps, online games, community discussion forums, non-profits and many other online services.  The Initial Response said “Analysis so far suggests that fewer than 5% of UK businesses will be in scope of this regulatory framework.” Whether 5% is considered to be small or large in absolute terms (not to mention the apparent indifference to non-UK businesses), there has been no indication of the assumptions underlying that estimate.

7. Will journalism and the press be excluded from scope?

The White Paper said:

Nothing. In a subsequent letter to the Society of Editors the then DCMS Secretary of State Jeremy Wright said:

“… as I made clear at the White Paper launch and in the House of Commons, where these services are already well regulated, as IPSO and IMPRESS do regarding their members' moderated comment sections, we will not duplicate those efforts. Journalistic or editorial content will not be affected by the regulatory framework.”

The Initial Response said:

Nothing. It limited itself to general expressions of support for freedom of expression, such as:

“…freedom of expression, and the role of a free press, is vital to a healthy democracy. We will ensure that there are safeguards in the legislation, so companies and the new regulator have a clear responsibility to protect users’ rights online, including freedom of expression and the need to maintain a vibrant and diverse public square.”

The Ministers said:

“Caroline Dinenage: Obviously, we know that a free press is one of the pillars of our society, and the White Paper, I must say from the outset, is not seeking to prohibit press freedom at all, so journalistic and editorial content is not in the scope of the White Paper. Our stance on press regulation has not changed.” (Emphasis added)

“As for what has been in the papers recently, the Secretary of State wrote a letter to the Society of Editors, and this was about what you might call the below-the-line or comments section. They were concerned that that might be regulated. I think what the Secretary of State is saying is that, where there is already clear and effective moderation of that sort of content, we do not intend to duplicate it. For example, there is IPSO and IMPRESS activity on moderated content sections. Those are the technical words for it. This is still an ongoing conversation, so we are working at the moment with stakeholders to develop proposals on how we are going to reflect that in legislation, working around those parameters. (Q.558)

“Stuart C. McDonald: But there is no suggestion that below-the-line remains unregulated. It is where that regulation should lie that is the issue.

Caroline Dinenage: Exactly.” (Q.559)

The Consequence: There are three distinct issues around inclusion or exclusion of the press from the regulatory scope of the Bill:

1. User comments on newspaper websites.  On the face of it, news organisations would be subject to the duty of care as regards user comments on their websites. The position of the government appears to be that whether the duty of care would apply would depend on whether the comments are already subject to another kind of regulation (or at least the existence of “clear and effective moderation”). Potentially, therefore, newspapers that are not regulated by IPSO or IMPRESS would be in scope for this purpose. Whether this demarcation would be achieved by a hard scope exclusion written into the Bill is not clear.

2. Journalistic or editorial material. Whilst the Minister may say that the government’s stance on press regulation has not changed, her statement that journalistic and editorial content is not “in the scope” of the White Paper is new — at least if we are to understand that as meaning that the Bill would contain a hard scope exclusion for journalistic or editorial content. Previously the government had said only that such content would not be affected by the regulatory framework. A general exclusion of journalistic or editorial material would on the face of it go much wider than newspapers and similar publications. It would be no surprise to find this statement being “clarified” at some point in the future.

3. Newspaper social media feeds and pages. Newspapers and other publications maintain their own pages, feeds and blogs on social media and other platforms. Newspapers would not themselves be subject to a duty of care in relation to their own content. But as far as the platforms are concerned the newspapers are users, so that their pages and feeds would fall under the platforms’ duty of care. As such, they would be liable to have action taken against their content by a platform in the course of fulfilling its own duty of care.

The government has said nothing about whether, and if so how, such press content would be excluded from scope. If the government is serious about excluding “journalistic or editorial” material generally from scope, that would achieve this. However that would create immense difficulties around whether a particular feed or page is or is not journalistic or editorial material (what about this Cyberleagle blog, or the Guido Fawkes blog, for instance?), and how a platform is supposed to decide whether any particular content is or is not in scope.  

8. End to end encryption

The White Paper said:

Nothing. (Although the potential for the duty of care to be applied to prevent the use of end to end encryption was evident.)

The Initial Response said:

Nothing.

The Ministers said:

Baroness Williams: “[Facebook] then announced that they were going to end-to-end encrypt Messenger. That, for us, is gravely worrying, because nobody will be able to see into Messenger. I know there is going to be a Five Eyes engagement next week, and I do not know if the Committee knows, but the Five Eyes wrote to Mark Zuckerberg last year, so worried were we about this development.” (Q538)

Q566 Chair: “On that basis, does end-to-end encryption count as a breach of duty of care?

Baroness Williams:It is criminal activity that would breach the duty of care. Allowing criminal activity to happen on your platform would be the breach of duty of care. End-to-end encryption, in and of itself, is not a breach of duty of care.

Chair: Presumably, for this regulation to have any bite at all, they will have to be able to take some enforcement against the policies that fail to prevent criminal activity. On that logic, introducing the end-to-end encryption, if it knowingly stops the company from preventing illegal activity—for example, the kind of online child abuse you have talked about—that would surely count as a breach of duty of care.

Baroness Williams: I fully expect that that is what some of the Five Eyes discussions, which will be happening very shortly, will look at.”

The Consequence: This is the first indication that the government is alive to the possibility that a regulator might be able to interpret a duty of care so as to affect the ability of an intermediary to use end to end encryption. The “in and of itself” phraseology used by the Minister appears not to rule that out. This issue is related to the question of how the legislation might apply to private messaging providers, a topic on which the government has consulted but has not yet published a conclusion.

9. Identity verification

The White Paper said:

“The internet can be used to harass, bully or intimidate. In many cases of harassment and other forms of abusive communications online, the offender will be unknown to the victim. In some instances, they will have taken technical steps to conceal their identity. Government and law enforcement are taking action to tackle this threat.”

“The police have a range of legal powers to identify individuals who attempt to use anonymity to escape sanctions for online abuse, where the activity is illegal. The government will work with law enforcement to review whether the current powers are sufficient to tackle anonymous abuse online.”

“Some of the areas we expect the regulator to include in a code of practice are:

• Steps to limit anonymised users abusing their services, including harassing others. …
• Steps companies should take to limit anonymised users using their services to abuse others.”

The Initial Response said:

Nothing.

The Ministers said:

Q25 John Nicolson: Would you like to see online harms legislation compel social media companies to verify the identity of users, not of course to publish them but simply to verify them before the accounts are up and running?

Oliver Dowden: There is certainly a challenge around, as you mentioned, bots, which are sometimes used by hostile state activity, and finding better ways of verifying to see whether these are genuine actors or whether it is co-ordinated bot-type activity. That is through online harms but there is obviously a national security angle to that as well.”

Q530 Ms Abbott: “Finally, would you consider changing the regulation, so you could post anonymously on a website or Twitter or Facebook, but the online platform would have your name and address? In my experience, when you try to pursue online abuse, you hit a brick wall because the abuser is not just anonymous when they post, the online platform doesn’t have a name and address either.

Caroline Dinenage: That is a really interesting idea. It is definitely something that we have been discussing. With regard to the online harms legislation that we are putting together at the moment, we have said very clearly that companies need to be much more transparent. They need to set out standards and they need to clarify what their duty of care is and to have a robust complaints procedure that people can use and can trust in. That is why we are also appointing a regulator that will set out what good looks like and will have expectations but also powers to be able to demand data and information and to be able to impose sanctions on those that they do not feel are abiding by them.

Q531 Chair: What does that actually mean? Does that mean that you think that the regulator should have the power to say that social media companies should not allow people to be … [a]nonymous to the platform?

Caroline Dinenage: This is something that we are considering at the moment. There are a number of things here. In the online harms legislation, the regulator will set out their expectations.

Chair: We can’t devolve everything to the regulator. Something like this is really important—should social media companies be allowed to not know who it is that is using their platforms? That feels like a big question that Parliament should take a view on, not something we just hand over to a regulator and say, “Okay, whatever you think,” later on.

Caroline Dinenage: Yes, exactly. That is why we are considering it at the moment, as part of the online harms legislation, and that, of course, will come before Parliament.”

Q545 Tim Loughton: “… If I want to set up a bank account and all sorts of other accounts, I must prove to the bank or organisation who I am by use of a utility bill and other things like that. It is quite straightforward. What is the downside of a similar requirement being enforced by social media platforms before you are allowed to sign up for an account? This is an issue that we have looked at before on the Committee. Many of us have suggested that we should go down that route. I gather that it already happens in South Korea. You say that you are looking at it, Minister Dinenage. What, in your view, is the downside of having such scrutiny?

Caroline Dinenage: You make a very compelling argument, Mr Loughton. A lot of what you said is extremely correct. The only thing we are mulling over and trying to cope with is whether there is any reason for anonymity for people who are victims, who want to be able to whistleblow, and who may be overseas and might not want to identify themselves because they fear for their lives or other harm. There are those issues of anonymity and protecting someone’s safety and ability to speak up. That is what we are wrestling with.

Q546 Tim Loughton: By the same token, you could have somebody with a fake identity who is falsely whistleblowing or pushing around propaganda, so it cuts both ways. I fail to see the downside of having a requirement that you have to prove who you are—not least because we know what happens when people are caught and have their sites taken down. Five minutes later, they set up another new anonymous site peddling the same sort of false information.

Caroline Dinenage: You make a very compelling argument. This is such an important piece of legislation, and we have to get it right. As I say, it is world-leading. Everybody is looking at us to see how we do it. We need to make sure that we have taken into consideration every angle, and that is what we are doing at the moment.”

The Consequence: Identity verification is evidently an issue that is bubbling to the surface. The most fundamental objection is that the right of freedom of expression secured by Article 19 of the Universal Declaration of Human Rights is not conditioned upon identity verification. It does not say:

"Everyone has the right to freedom of opinion and expression upon production of any two of the following: driving licence, passport, recent utility or council tax bill...".

In South Korea, legislation imposing online identity verification obligations was declared unconstitutional in 2012.

The Home Affairs Committee raised, to the best of my knowledge for the first time in any Parliamentary deliberation on the Online Harms project, the question of what should be decided by Parliament and what delegated to a regulator. That is not limited to the question of identity verification. It is an inherent vice of regulatory powers painted with such a broad brush that many concrete issues will lie hidden behind abstractions, to surface only when the regulator turns its light upon them – by which time it is far too late to object that the matter should have been one for Parliament to decide. That vice is compounded when the powers affect the individual speech of millions of people.

10. Extraterritoriality

The White Paper said:

“The new regulatory regime will need to handle the global nature of both the digital economy and many of the companies in scope. The law will apply to companies that provide services to UK users.” (6.9) (emphasis added)

“We are also considering options for the regulator, in certain circumstances, to require companies which are based outside the UK to appoint a UK or EEA-based nominated representative.” (6.10)

The Initial Response said:

Nothing of relevance.

The Ministers said:

“Q569: Andrew Gwynne: Presumably the regulations will apply to all content visibly available in the UK—is that correct?

Baroness Williams: Yes.”

The Consequence: Charitably, perhaps we should assume that the Minister misspoke. There is a vast difference between providing services to UK users and mere visibility in the UK. Given the inherent cross-border nature of the internet, asserting a country’s local law against content on a mere visibility basis is tantamount to asserting world-wide extra-territoriality. 

It would be more consistent with the direction in which internet jurisdictional norms have moved over the last 25 years to apply a test of whether the provider is targeting the UK.