Wednesday, 7 September 2016

A trim for bulk powers?

David Anderson Q.C.’s Bulk Powers Review made only one formal recommendation (a Technical Advisory Panel to assist the proposed Investigatory Powers Commission).

However the report drops a tantalising hint of the debate that might have taken place if the Review had been commissioned before the Bill started its passage through Parliament instead of almost at the end.

At [9.17] Anderson says:

“I have reflected on whether there might be scope for recommending the “trimming” of some of the bulk powers, for example by describing types of conduct that should never be authorised, or by seeking to limit the downstream use that may be made of collected material. 
But particularly at this late stage of the parliamentary process, I have not thought it appropriate to start down that path.  Technology and terminology will inevitably change faster than the ability of legislators to keep up.  The scheme of the Bill, which it is not my business to disrupt, is of broad future-proofed powers, detailed codes of practice and strong and vigorous safeguards.  If the new law is to have any hope of accommodating the evolution of technology over the next 10 or 15 years, it needs to avoid the trap of an excessively prescriptive and technically-defined approach.”
Let us put aside whether it is sensible or appropriate to try to future-proof powers – my view is that to do so repeats the error of RIPA – and then put aside the debate about whether bulk powers should exist at all. How might one go about a task of trimming bulk powers? What types of conduct might be candidates for never being authorised? What sort of limits on downstream use might be desirable and feasible?

The Report illustrates, perhaps more clearly than before, the very wide range of techniques that are brought to bear on bulk data (whether sourced from interception, equipment interference, bulk communications data acquisition or Bulk Personal Datasets). They range from real-time application of 'strong selectors' at the point of interception (akin to multiple simultaneous targeted interception), through to generalised pattern analysis and anomaly detection (utilised by MI6 on Bulk Personal Datasets in Case Study A11/2) designed to detect suspicious behaviour, perhaps in the future using machine learning and predictive analytics.

Pattern analysis is similar to data mining techniques described in A Question of Trust (AQOT):
"14.43. It is sometimes assumed that GCHQ employs automated data mining algorithms to detect target behaviour, as is often proposed in academic literature. That, it would say, is realistic for tasks such as financial fraud detection, but not for intelligence analysis."
AQOT included possible future developments of such techniques as one of several examples of capabilities that, at least cumulatively, would go beyond Bentham's Panopticon:

"13.19(d) A constant feed of data from vehicles, domestic appliances and healthmonitoring personal devices would enable the Government to identify suspicious (or life-threatening) patterns of behaviour, and take pre-emptive action to warn of risks and protect against them."
AQOT commented on those examples:

"13.20 Much of this is technically possible, or plausible. The impact of such powers on the innocent could be mitigated by the usual apparatus of safeguards, regulators and Codes of Practice. But a country constructed on such a basis would surely be intolerable to many of its inhabitants. A state that enjoyed all those powers would be truly totalitarian, even if the authorities had the best interests of its people at heart.
13.21. There would be practical risks: not least, maintaining the security of such vast quantities of data. But the crucial objection is that of principle. Such a society would have gone beyond Bentham’s Panopticon…"
Between the two ends of the spectrum are seeded analysis techniques, applied to current and historic bulk data. AQOT again:

"Much of [GCHQ's] work involves analysis based on a fragment of information which forms the crucial lead, or seed, for further work. GCHQ’s tradecraft lies in the application of lead-specific analysis to bring together potentially relevant data from diverse data stores in order to prove or disprove a theory or hypothesis. As illustrated by the case study on GCHQ’s website, significant analysis of data may be required before any actual name can be identified. This tradecraft requires very high volumes of queries to be run against communications data as results are dynamically tested, refined and further refined. GCHQ runs several thousand such communications data queries every day. One of the benefits of this targeted approach to data mining is that individuals who are innocent or peripheral to an investigation are never looked at, minimising the need for intrusion into their communications."
A similar explanation of seeded analysis of bulk data was given by Lord Evans in evidence to the Commons Public Bill Committee 24 March 2016. 

A "strong selectors" technique whereby the full catch from a transmission is stored for only a few seconds for processing before being discarded may rate relatively low on the Orwell scale.  Seeded analysis rates fairly high, since it relies on bulk data (albeit filtered to some degree) being stored for later querying. Unseeded pattern analysis and anomaly detection is off the scale.  It is closest to the characterisation by M. Delmas-Marty, a French lawyer quoted in the Review report: "Instead of starting from the target to find the data, one starts with the data to find the target."  
As it stands the Bill's bulk powers regime would empower all these techniques with no distinction between them, leaving it to the judgement of the Secretary of State, the Judicial Commissioners and after the event oversight to regulate and possibly limit their use under principles of necessity and proportionality.

An informed debate about trimming bulk powers could entail discussion of whether unseeded pattern analysis and anomaly detection should be permitted, and if so whether only for very specific and limited purposes.  It could also look at whether specific rules should govern seeded analysis.  It might also consider whether individual sets of "strong selectors" should require separate warrants, by analogy with non-thematic targeted interception warrants. Regrettably, in part due to the late stage at which the Bulk Powers Review has taken place, very little such nuanced debate has taken place.
Trim in the Bill, not Codes of Practice
Limitations on the scope of powers belong in the Bill and should not be left to Codes of Practice.

Although the government often states that the Codes of Practice 'have statutory force' (see e.g. Letter from Lord Keen to Lord Rooker, 8 July 2016, they do not have the same force as a statute. Their status and effect are limited to that set out in Schedule 7 para 6 (which possibly confers on Codes of Practice a weaker general interpretative role than does RIPA s.72).
Trimming approaches
Different kinds of analytical techniques apart, possible approaches to trimming bulk powers can be considered by reference to different facets of the powers.  I give some illustrative examples below, not necessarily to advocate them but more as an aid to understanding.
A.    Purposes
The Bill as currently drafted applies three cumulative sets of purposes to the interception and equipment interference bulk powers:
1.       The statutory purposes (national security etc).  Some have called for national security to be defined.
2.      Operational purposes. A new government amendment in response to a suggestion from the Intelligence and Security Committee provides that a list of purposes approved by the Secretary of State must be maintained by the heads of the intelligence services. The Secretary of State must be satisfied that an operational purpose to be included in the list is specified in a greater level of detail than the statutory purposes.
3.      Overseas-related purpose. The Bulk Powers Operational Case places considerable weight on the fact that the bulk interception and equipment interference powers are overseas-related.  Thus BI is described at 7.1 as a 'capability designed to obtain foreign-focused intelligence'. Similarly BEI is described at 8.2 as 'foreign-focused'. However:
a.      Obtaining 'overseas-related' data need only be the main, not the sole, purpose of the warrant.
b.      Overseas-related communications include those in which the individual overseas is communicating with someone (or something) in the UK.
c.       The 'overseas-related' limitation on purpose is exhausted once the information has been acquired by means of the bulk interception or interference (see the comments on RIPA S.16 in the Liberty IPT case, para 101 et seq. The Bill is structured in a similar way.)
d.     As the Operational Case acknowledges, non-overseas-related communications and information (and associated secondary data and equipment data) may be incidentally acquired. While the Operational Case attempts to downplay the significance of this, it provides no evidence on which to conclude that collateral acquisition may not be on a substantial scale.
e.      There is no obligation to discard, or attempt to discard, or discard upon gaining awareness of its presence, non-overseas-related material acquired in this way.
f.        The need to obtain a targeted examination warrant in relation to persons within the British Islands applies only to content, not to secondary data or equipment data.
g.      Secondary data and equipment data will under the Bill include some material extracted from content that under RIPA would be regarded as content. The expanded categories appear to go wider than what might intuitively be thought of as communications data (see Section F below).
h.      The purposes for which the Operational Case contemplates that secondary data and equipment data may be analysed go far beyond the limited purpose of ascertaining the location of a person ventilated in the Liberty IPT case (see Section G below).
Some possible approaches to trimming:
(1)   Limit the downstream use that can be made of collected material (whether content or secondary data/equipment data) to match the overseas-related main purpose for which it can be collected.
(2)  An obligation to seek out and remove, or remove upon gaining awareness of its presence, non-overseas-related material.
(3)  Raise the location threshold, so that a British Islands resident does not automatically lose content protection merely by venturing half-way across the English Channel (cf Keir Starmer, Commons Committee, 12 April 2016 at col. 116)].

B.    Types of data and communication
With one exception the bulk powers in the Bill make no distinction between types of communication. They range from human to human messaging of various types through to automated communications and single-user activities such as browsing websites.
The one exception arises from the definition of overseas-related communications, applicable to interception and equipment interference bulk powers: communications sent by or received by individuals who are outside the British Islands. 
This would include an e-mail sent by an individual within the British Islands to an individual outside the British Islands and vice versa. It would exclude a search request sent by an individual within the British Islands to an overseas server (since there is a server, not an individual, at the other end). But it would include a search request sent by an individual outside the British Islands to a UK server.
The significance of this exclusion is, however, reduced by the ‘by-catch’ provisions.  Unless the agencies are able to filter out excluded material at the point of collection then, as with RIPA, it is collectable as a necessary incident and falls into the general pool of selectable data.
The Bill contains no indication of when a communication is to be regarded as sent by or received by an individual. An e-mail or text message addressed to an individual clearly is so. What about an e-mail addressed to, or sent by, a corporate account? What about machine-generated e-mails? When is a communication generated by or sent to an individual’s device without the knowledge of the individual to be regarded as sent or received by the individual? Background smartphone communications are an obvious example. What if a car, without the owner/driver/passenger’s knowledge, automatically generates and sends an e-mail requesting a service or an emergency message, including associated location data?
Some possible approaches to trimming:
(1)   Limit the extent to which background and machine generated communications may be regarded as sent or received by an individual.
(2)  An obligation as in B(2) above to remove non-overseas-related material would imply an obligation to remove kinds of overseas communication not sent or received by an individual.
(3)  Should powers apply to all types of communication, or only human to human messaging?
C.    Types of conduct authorised
Some possible approaches to trimming:
(1)   Limit scope by reference to concrete types of conduct that can (or specifically cannot) be authorised. The Centre for Democracy and Technology submission to draft Bill Joint Committee at [42], repeated in CDT evidence to the Public Bill Committee at [20] to [25], suggested this kind of approach for equipment interference warrants in relation to the possibility of mandating encryption back doors.
D.   Use of incidentally collected data
As discussed in my evidence to the Joint Committee ([117]to [137]) and above in relation to overseas-related communications there is a fundamental issue concerning the extent to which domestic content and secondary data collected as a by-product of the overseas-related bulk powers can be used in non-overseas-related ways.
Some possible approaches to trimming:
(1)   As above (B(1)).

E.    Extent of secondary data and equipment data
The Bill embodies a significant shift (compared with RIPA) towards classifying various types of content as secondary data or equipment data (see my blog post).  The Bill appears to go further than extracting communications traffic data (e-mail addresses and the like) from the body of a communication such as an e-mail. It appears to include the ‘who where and when’ not just of communications, but of people’s real world activities per se. 
Some possible approaches to trimming:
(1)   Limit extracted metadata to true communications data (i.e. data about communications).
F.     Types of use of bulk secondary, equipment and communications data
Various uses of bulk metadata have been ventilated. The Bulk Powers Review contains numerous examples. They types of use can differ significantly from each other. For instance:
-         To determine whether the sender or recipient of a communication is within or outside the British Islands (the very limited purpose advanced by the government in the Liberty IPT case – see my evidence to the Joint Committee at [128] to [130])
-         To have visibility of a full historic record so that authorities can go back and find out after the event about a malefactor’s communications and online activities
-         Seeded analysis to find a target’s associates or more about a target’s identity (as discussed above)
-         Target discovery based on patterns of behaviour, as discussed above (see also Operational Case [3.3] and [3.6]).
These various uses have different implications for the rationale for collecting data in bulk. At one end of the spectrum bulk collection is seen as a necessary evil, required only because for technical reasons (e.g. fragmentation of packets or presence of the target in other countries) target communications cannot be separated at point of collection from the rest. That may hold out the prospect that as technology improves it becomes possible to carry out more targeted bulk collection, particularly as real time capabilities increase. 
At the other end of the spectrum (pattern detection and predictive analysis) bulk collection is can become more of an end in itself: amassing data so as to provide the most accurate ‘normal’ baseline against which ‘suspicious’ behaviour patterns can be detected. This appears to carry no prospect of reducing the quantity of metadata collected – probably the opposite.
The Bill is almost completely devoid of concrete limitations on, or distinctions between, the types of use that can be made of bulk metadata. The limits are the statutory purposes, operational purposes and necessity and proportionality. The Bulk Powers Review proposes a Technical Advisory Panel to assist the Investigatory Powers Commission in keeping technological developments under review.
Some possible approaches to trimming:
Limitations on use could be based on e.g.
(1)   the justification provided to the IPT in Liberty;
(2)  specific seeded analysis versus more generalised pattern detection
(3)  limitations on numbers of hops when following possible associations (Twitter followers, Facebook friends etc)
(4)  applying the non-British Islands examination restriction to metadata searches (note Operational Case paras 5.14 to 5.19).
G.    Types and location of conduct authorised by warrants
The bulk warrantry system seems to allow for three possibilities:
(1)   Unilateral conduct by the intercepting or equipment interfering agency without the knowledge or assistance of the CSP
(2)  Assisted conduct under a warrant supported by a technical capability notice
(3)  Assisted conduct under a warrant without the support of a technical capability notice
The Bill does not specify any specific circumstances in which these different approaches are or are not appropriate (other than technical capability notices for equipment interference limited under Clause 228(10)/(11) to UK CSPs). Nor are the different approaches addressed in the Operational Case. Similarly AQOT:
"Implementing a s8(1) warrant generally relies on the cooperation of service providers, acting typically in response to a direction from the Government under RIPA s12. A copy of the intercepted communication is passed by the companies to the intercepting agencies who examine it using their own staff and facilities. External communications may be obtained under a s8(4) warrant either directly by GCHQ, using its own capabilities, or through a service provider." (emphasis added)
Some possible approaches to trimming:
(1)   Limitations (perhaps territorial) on unilateral conduct under bulk warrants.
(2)  Special thresholds for the use of (say) bulk equipment interference warrants.
(3)  Limits on what a technical capability notice can require.

H.   Intermediate stages
Bulk interception and use of its product may take place in several stages, such as: collection, culling (discard of unwanted types of data), filtering (use of positive selectors), storage for subsequent querying by analysts.  Whether these techniques are typically applied to secondary data to the same extent as to content is unclear.
The Bill says nothing detailed about the culling and filtering stages, other than restrictions by reference to someone's location within the British Islands on selection of content for examination.
Some possible approaches to trimming:
(1)   Specific obligation to apply data minimisation techniques at intermediate stages, applicable to both content and metadata
(2)  Specific provisions controlling culling and selector types (for instance requiring individual warrants for "strong selectors")
I.      Real time versus periodic
Is the bulk communications data acquisition power meant to be one that should be exercised occasionally when specific circumstances justify it, or can it be exercised routinely? If the latter, could it be used as a near-real time or quasi-real time feed?

A one-off data dump in exceptional circumstances is a rather different animal from a near real-time tool. In this context the recent IOCCO report speaks of ‘regular feeds’ acquired under S.94 Communications Act 1984.   The Bill appears to cover both possibilities.
Some possible approaches to trimming:
(1)   Specially justified occasions versus frequent routine feeds.

J.     Interaction with communications data retention
The bulk communications data acquisition power is closely linked to the communications data retention power.  The more broadly the data retention power is exercised, the greater the range of datatypes that will be available to be acquired in bulk.
It is significant in this context to recall that the data retention power (a) goes far wider than the internet connection records that the government has so far discussed and budgeted for in its Impact Assessment; and (b) unlike DRIPA, can be used to require relevant communications data to be generated or obtained, not merely retained. 
Some possible approaches to trimming:
(1)   Limit bulk acquisition power to concretely specified types of communications data; and/or
(2)  Require specified public consultation and procedures if any extension of compelled retention or acquisition is contemplated.
K.    Types of operator
The Bill significantly extends the classes of operator to which the various powers can be applied.  The table below compares the powers in current legislation (mainly RIPA, but bearing in mind the extension effected by DRIPA) with those in the Bill.

Compliance and assistance obligations expressly applicable to private operators are highlighted in green.  "Telecommunications operator" under the Bill definition at Clause 233(10) includes private networks (and 'service' is not restricted to a commercial service).  

The draft Codes of Practice suggest that most powers would be exercised more sparingly.




Data retention notice

Public telecommunications operator (DRIPA)

Telecommunications operator (89(1))

Communications data acquisition notice

Provider of a telecommunications service  (RIPA)

Telecommunications operator (62)

Interception warrant

(1) Public telecommunications service (2) telecommunication system wholly or partly within UK (RIPA)

Telecommunications operator (41, 139(5))

Interception capability notice

Public telecommunications services (RIPA) > 10,000 persons in UK) (regulations)

Relevant operator (includes telecommunications operator) (226(1)/228(9))

Other technical capability notices


Relevant operator (includes telecommunications operator) (226(1)/228(9)); (some UK enforceable only)(228(10)/(11))

Equipment interference warrant

? (ISA 1994)

Telecommunications operator (120, 167); UK enforceable only (120(7), 175(5))

Bulk communications data acquisition warrant

Public electronic communications network providers (TA 1984)

Telecommunications operator (157); (UK enforceable only) (157(5))

National security notice

Public electronic communications network providers (TA 1984)

UK telecommunications operator (225(1), 228(9) and (10)).

Some possible approaches to trimming:

(1)   Stricter definitions of the kind of operators that can be subjected to duties to assist or comply, or in what circumstances. 

L.    Technical capability notices
The power to give technical capability notices is open-ended, not limited to the list of examples given in Clause 226(5). 

Some possible approaches to trimming:

(1)   Convert powers to make regulations and give technical capability notices from illustrations into a clearly specified list that limits the exercise of the powers.


  1. Great analysis. I have one qualification to the suggestions, which I'm sure you're already aware of but I'm not sure how it fits.

    With respect to trimmings in A and B, specifically on filtering out any external-internal communications and any nonhuman communications, one objection is that unseeded machine learning processing is already apparently quite effective at detecting patterns indicative of cyberattacks from foreign actors against UK networks. It seems from leaked material and the hints provided in the bulk powers review that What we could call unseeded analysis of data streams, based on pure analytics of data rather than an initial reference point, is much more suitable for detecting hostile computer generated signals than detecting suspicious human behaviour. Trimming out such data on the basis of it being machine generated or being addressed to UK based IP addresses, enhancing human privacy, would seem to eliminate that form of cyber defence.

    1. That example of a specific or limited purpose was what I had in mind when I said: "An informed debate about trimming bulk powers could entail discussion of whether unseeded pattern analysis and anomaly detection should be permitted, and if so whether only for very specific and limited purposes". However I agree that wasn't reflected in the trimming examples.