Sunday 21 December 2014

A Cheltenham Carol

On the Twelfth Day of Christmas my true love sent to me:

Twelve Zettabytes

Eleven Encryption Layers

Ten Coders Coding

Nine Hackers Hacking

Eight Routers Routing

Seven Inspected Packets

Six Spies-a-Spying

Five Back Doors

Four Fishing Warrants

Three Haystacks

Two Secret Laws

And a Paean to Proportionality

Wednesday 3 December 2014

Another round of data retention

[Updated 4 December 2014]
[Further updated 20 January 2015 to add tweet.]
[Also updated 5 January 2015 with this brief commentary on the Home Office Factsheet:

Page 1: Top Lines

"IP resolution is the ability to identify who in the real world was using an Internet IP address at a given point in time." Data retention at best identifies the device or connection being used and any associated subscriber details. The subscriber is not necessarily the user. Page 2 of the Factsheet is accurate: "This data can help identify who has made a communication, when, where and how." (emphasis added) 

Page 1: Background

"However, some IP addresses are shared and allocated dynamically." True, but dynamic allocation is not what Clause 17 is about. Dynamic IP address allocation is sequential temporary allocation of a public IP address to one customer after another. Dynamic IP addresses are already explicitly mentioned in the DRIPA datatypes (Data Retention Regulations 2014, Schedule, Paras 13(1)(b) and 11(3)). It is evident from the diagram on page 3 of the Factsheet that the problem being addressed by Clause 17 is simultaneous sharing of a single public IP address by multiple ISP customers. 

Page 3 : Diagram

"At 4pm 2,500 people are using a single IP address on the internet." Exactly. The issue is simultaneous sharing of a single IP address, not dynamic (sequential) allocation of an IP address. 

"The e-mail service provider now provides police with IP address and port number used to send the e-mail and accurate time."  In order to do this the e-mail service provider in the diagram example will have had to retain IP address, port number and timing data.  Will such providers, as well as internet access providers, be subject to mandatory retention?

"Police seek details from internet access provider. Internet access provider now identifies the individual using the unique combination of IP address and port number provided at 4pm." The internet access provider identifies the customer, who may be but is not necessarily the individual who used the device in question.] 

Four months after DRIPA and 18 months after putting down a marker in the May 2013 Queen’s Speech, the UK government has embarked on a new round of legislation for mandatory retention of communications data. This time it is under the banner of IP address matching.

The Counter-Terrorism and Security Bill had its Second Reading yesterday and is expected to go into Committee on 9 December. Clause 17 will extend DRIPA to new categories of communications data.

DRIPA’s existing data retention obligations, rushed through Parliament in four days in July, are of course controversial. They are the subject of a threatened legal challenge by David Davis MP and Tom Watson MP.  The proposal to add IP address matching dates back to a recommendation of the Joint Committee on the draft CommunicationsData Bill in December 2012.

What new categories of communications data would have to be retained?

Clause 17, like so much UK legislation in this field, is difficult to understand. The Explanatory Notes and the Impact Assessments are more detailed, but still confusing. (The Home Office has subsequently issued a Factsheet.) MPs suggested in the Second Reading that the drafting of Clause 17 needs to be examined critically.  They are right.

The overall aim seems to be to mandate retention of data that can link a given communication made via a simultaneously shared public IP address to one of many devices or connections that may have been using that IP address at a given time.  Clause 17 labels this “relevant internet data”. We might call it linking data.

This appears to break down something along the following lines (the first two of these are illustrated in the useful diagram in the Home Office Factsheet).
  • Some ISP and mobile operator systems don’t allocate one public IP address to one customer device or connection, but have many customers sharing an IP address simultaneously. They could be required to retain linking data such as port numbers.
  • Even if an ISP retains IP address and (say) port number records, it cannot be sure of identifying a single device or connection unless law enforcement can provide it with a both a port number and an IP address to look up. So a cloud storage or web e-mail provider accessed by the user could also be required to retain logs of linking data visible to it, such as port numbers.
  • Operators such as public Wi-Fi hotspots could be required to log MAC addresses.
Weblog data (records of websites accessed by customers) would be excluded from mandatory retention by internet access providers such as ISPs and mobile operators.

The Overarching Impact Assessment provides this summary:

“IP Resolution: Allow for a power to require communications service providers to retain the data necessary to attribute an IP address to an individual.”

Taken literally, that is a power to require the impossible. We don’t have IP addresses tattooed on our foreheads. Even if we did that would not identify us, as opposed to someone else, as the user of the device at any given time. An IP address at best identifies a device or a connection. The ISP may then be able to link that with the identity of its subscriber customer, but no more. The subscriber may or may or not be the user. The Factsheet diagram, unfortunately, perpetuates the myth that an IP address identifies a user.

DRIPA in fact already covers retention of subscriber data for IP addresses (both where the IP address is static and where it is dynamically allocated in sequence to different customer devices and connections). What it doesn’t cover is the single public IP address simultaneously shared among many of an ISP’s customers.

The Bill is meant to be only about IP address matching. So it is not immediately obvious why the Impact Assessments say that the Bill will expand DRIPA to cover a wider range of internet services. On the other hand Clause 17 does not seem to do this, since it only amends the categories of data to be retained. DRIPA has already adopted an extremely broad underlying definition of telecommunication services.

The new obligations would be subject to the same 31 December 2016 sunset clause as DRIPA. As with DRIPA itself, mandatory retention will apply only to data generated or processed in the UK by public providers in the process of providing the telecommunications services concerned; and then only to those on whom the government serves a notice. The Impact Assessment says that the service providers most likely to be affected by the Bill have been consulted.

That is my current stab at what Clause 17 is trying to do.  However it is a puzzling piece of drafting. Here are some questions worth considering.

What is ‘relevant internet data’?
Clause 17(3)(b) defines this as communications data relating to an internet access service or an internet communications service which:

“may be used to identify, or assist in identifying, which internet protocol address, or other identifier, belongs to the sender or recipient of a communication (whether or not a person)”.

This is the most curious part of Clause 17. The problem is surely not identifying which IP address ‘belongs’ to a given sender or recipient of the communication, but identifying which device or connection (of many) was used to make a given communication via a given shared public IP address. Is it drafted the wrong way round?

What is an ‘identifier’?
The Clause says that “identifier” means “an identifier used to facilitate the transmission of a communication”.  More helpfully, Clause 17(3)(b) tells us that an IP address is an identifier. The Explanatory Notes seem to conflate linking data and the shared identifier that we are trying to tie to a device or connection:

“…  An IP address can often be shared by hundreds of people at once – in order to resolve an IP address to an individual other data ("other identifier" in this clause) would be required.”

Whatever the ‘other data’ may be, surely it is not the ‘other identifier’ in Clause 17(3)(b)?

What else might be covered by ‘identifier’? A MAC address, although it operates at a lower (physical) layer than an IP address, would seem to qualify. But Clause 17 is not avowedly about retention of new categories of identifiers, only retention of data capable of linking shared identifiers (such as IP addresses) to an individual device or connection. If a MAC address is itself an identifier, does that prevent it being linking data? The Explanatory Notes suggest that a MAC address could also be linking data:

“Data necessary for the resolution of IP addresses could include port numbers or MAC (media access control) addresses.

Are there circumstances in which a MAC address could be used to identify the particular device that sent a communication via a shared IP address? Public Wi-Fi hotspots seem a likely candidate. However a MAC address would presumably be less useful than a port number, assuming that the MAC address is not visible from outside the hotspot and so could not be logged at the other end of the communication.

What are an internet access service and an internet communications service? 
These are the foundation stones of Clause 17. Communications data cannot be required to be retained unless it relates to an internet access service or an internet communications service. These terms are also critical to the scope of the weblog data exclusion. Many will be surprised, therefore, to find that neither term is defined.

What do the terms mean? The glib answer is ‘whatever they meant in the EU Data Retention Directive’. That is their origin. They were used (but not defined) in the Directive.

The 2009 Data Retention Regulations, which implemented the Directive, followed its terminology. When the Directive was invalidated DRIPA re-enacted the datatypes that were in the Schedule to the 2009 Regulations. So the 2014Data Retention Regulations that were made under DRIPA again used the two terms, notably in the definition of ‘User ID’: “a unique identifier allocated to persons when they subscribe to, or register with, an internet access service or internet communications service.” Perhaps unsurprisingly given the government’s commitment to re-enact the 2009 datatypes identically, the 2014 Regulations again left the terms undefined. 

That is a plausible historical reason why the terms have been left undefined in Clause 17. But even though there is a breadcrumb trail back to the Directive, the lack of definitions in the Directive means that uncertainty remains particularly over ‘internet communications service’. Does it relate to any type of communication, or is it more limited, for instance to e-mail, messaging or telephony providers? The diagram in the Factsheet uses the example of an e-mail provider. However the Impact Assessment suggests that the government believes it has a broad meaning, covering for instance cloud storage services:

“For example w[h]ere a user uploads an illicit file to a cloud server that server provider, if subject to a data retention notice, would be required to retain sufficient information to enable the internet access provider to identify the user.”

We look forward to illumination of these and no doubt other points as the Bill proceeds. Meanwhile, the bigger question of whether any of this is compatible with the European Convention on Human Rights and the EU Charter of Fundamental Rights remains to be fought out. 

[My 8 point tweet of points on Clause 17:

[Updated 4 December 2014 with references to the Home Office Factsheet and minor clarifications and edits. Further update 5 January 2015 with comments on the Home Office Factsheet. Further updated 20 January 2015 to add tweet.]

Saturday 15 November 2014

Of straws and haystacks

Much post-Snowden attention has been directed to GCHQ’s TEMPORA programme, authorised (so it is thought) by a rolling series of external interception warrants under section 8(4) of RIPA. (See foot of this post for an explanation of Section 8(4) warrants and the restrictions, particularly for communications of persons within the British Isles, on their use.)

TEMPORA captures communications in bulk from transatlantic fibre optic cables, then filters them by computer leaving a residue of sifted material that GCHQ and NSA analysts can examine. It is said to process 40 billion items a day.

The often repeated justification for bulk collection and sifting is that we have no method of identifying and separating individual communications at the point of collection, so we must gather the straws and sift the resulting haystack. The usual metaphor is looking for needles, implying objective distinctions. It may be better to think of looking for straws.

What kind of straws can be looked for? The haystack can, within the restrictions laid down by RIPA, be sifted to detect the straws of pre-existing persons of interest. However Section 8(4) warrants go beyond that.  The captured material can also be searched and analysed to form new suspicions.   Home Office official Charles Farr said of RIPA in his witness statement in the current Investigatory Powers Tribunal proceedings:
“Other information that is obtained via interception is used to identify other previously unknown communications of existing targets, and to identify new targets for investigation. Indeed, a significant proportion of initial intelligence leads derive from interception operations.” (emphasis added)
We do not know what proportion of initial leads are false positives, casting suspicion on blameless people. We do not know how many true positives the system misses. Moreover suspicion is a highly subjective matter.

History suggests that general collection and subject matter analysis was an established approach to external communications long before today’s separation problems arose.

The ancestor of RIPA Section 8(4) was Section 4 of the Official Secrets Act 1920, legislated in the immediate aftermath of the First World War following the lapsing of wartime powers.  It empowered the Secretary of State to issue a warrant requiring a telegraph operator to hand over telegrams entering or leaving the country:
“Where it appears to a Secretary of State that such a course is expedient in the public interest, he may, by warrant under his hand, require any person who owns or controls any telegraphic cable or wire, or any apparatus for wireless telegraphy, used for the sending or receipt of telegrams to or from any place out of the United Kingdom, to produce to him, or to any person named in the warrant, the originals and transcripts, either of all telegrams, or of telegrams of any specified class or description, or of telegrams sent from or addressed to any specified person or place, sent or received to or from any place out of the United Kingdom by means of any such cable, wire, or apparatus, and all other papers relating to any such telegram as aforesaid.”
The Attorney General Sir Gordon Hewart introduced the provision in Parliament as a measure for detecting foreign spies:
“The postal and cable censorship which we had during the War, and which was of the greatest possible value and importance, was removed shortly after the Armistice. That being so, it is necessary that there should be power at least to compel the production of the originals and the transcripts of certain telegrams. It is not a power to stop telegrams. It is merely a power to compel the production of the originals and transcripts sent to, or received from, any place out of the United Kingdom; and the main purpose of that provision is to enable the authorities to detect and deal with attempts at spying by foreign agents.”
Earl Winterton invoked a familiar mix of foreign threats and ‘nothing to hide, nothing to fear’:   
“Everyone knows we do not live in ordinary times. Everyone knows there are plots and conspiracies against this Realm which are being carried out in foreign countries and some parts of the British Empire, and that, however one may dislike the idea of imposing additional restrictions on the subject, it is necessary for the Government to have that power. I suggest there is nothing to interfere with a person going about his legitimate business. The right hon. Gentleman, for example, made great play with Clause 4 of the Bill. … Surely he does not suggest that in the critical time in which we are living to-day a Secretary of State should not have power, if it seem desirable in the opinion of the Government that he should exercise that power, to find out what is being cabled to and from this country. Of course, it is a most necessary power, which every government ought to have.”
John Thorpe MP put the State firmly ahead of the individual:
“… In my view the State is in great danger, and no power which would tend to protect it should be withheld from the Government. We heard something from the same right hon. Gentleman of the liberty of the subject. In my view, the subject has no liberty when it is in conflict with the good-being of the State. When the liberty of the individual conflicts in any way with the well-being of the State, then it becomes license.  
… The law-abiding citizen, the man who says that his country is his first consideration, need have nothing to fear whatever from the Clauses of this Bill. … The only man who has anything to fear is the man who puts self before country, the man who says, "I want liberty, and the State can look after itself." He is a danger, and I congratulate the Government on the efficient manner in which they propose to deal with him.”
The legislation duly passed. For nearly 50 years Section 4 did its work in obscurity. The 1957 Birkett Inquiry into interception of communications did not mention it. (The Birkett Committee’s terms of reference were limited to the executive power to intercept, which was different and separate from the statutory power to issue warrants under Section 4.)

Things changed in February 1967.  Section 4 came to public notice when journalist Chapman Pincher revealed in the Daily Express that cablegrams sent out of Britain were being collected from the Post Office and private cable companies for scrutiny. This incident is most famous for sparking the ensuing D-Notice row. But the substance of Pincher’s ‘Cable Vetting Sensation’ story is of interest here. He revealed that:
“There is no hold-up or censorship of the cables. But on the morning after they have been sent or received they are collected and sifted by a Post Office department concerned with security. Then any cables believed to be of special interest are passed to the Security Services. 
They are studied there, copied if necessary, and returned to the Post Office and cable offices after being held for 48 hours. 
Most of the original cables and telegrams go out through the Post Office, which owns the former Cable and Wireless Company. Cables passed through private companies—mainly branches of foreign concerns operating in Britain—are collected in vans or cars each morning and taken to the Post Office security department. 
The probe is conducted under a special warrant, signed by a Secretary of State under Section 4 of the Official Secrets Act and regularly renewed to keep it valid.”
A week later Alan Watkins in the Spectator wrote:
“Indeed, sources confirm that a Ministry of Works van regularly takes cables—it is not clear whether they form a random sample, or come from a particular sender or class of senders—along to the Ministry of Defence for examination. The authority for such action is section 4(1) of the Official Secrets Act, 1920.”
The Radcliffe Report on the D-Notice affair confirmed the substantial accuracy of Pincher’s story:
“It does involve a regular collection of copies of messages transmitted by the Post Office and other cable offices with a view to the total collected being sorted and certain defined categories of them being set aside for inspection by the intelligence agents of Her Majesty’s Government. … 
The practice is authorised in law by section 4 of the Official Secrets Act, 1920 … . According to the information given to us, this power has been regularly exercised against transmitting companies since the coming into operation of the Act. … 
In fact only a small percentage of the total telegrams handled is put aside [by the sorters for inspection]. … The Daily Express article was … not inaccurate in any sense that could expose it to hostile criticism on that score.”
The Government White Paper published simultaneously with the Radcliffe Report said it would be contrary to the public interest to say in what detailed respects the article was misleading. It also said:
“It was precisely because national security was threatened that, from the outset, the Government regarded the publication of certain information in the Daily Express of 21 February as a matter of the utmost gravity. … It is the duty of the government, in the light of all the advice they have received and the information they possess, to record that the effect on national security of that publication has been to cause damage, potentially grave, the consequences of which cannot even now be fully assessed.”
The White Paper complained that the article created:
“the sensational impression that the Government were responsible for introducing new and sinister procedures.  There were, and are, no such new and sinister procedures. The activities involve no element of prying into the private affairs of the citizen. Such activities are, in fact, carefully controlled and confined and the article was misleading when it inferred the Government might use them improperly.”
Although the government denied (supported by the Report) that any new practice had recently been introduced, the possibility that routine vetting was a long established practice was left open.

Several themes from this episode resonate today:
  • Bulk collection, sifting and examination
  • Periodically renewed warrants
  • Revelations about the extent of use of powers, answered by denials that the powers are abused
  • Assertions, to be taken on trust, that publicity has caused damage to national security
  • Intrusion into privacy rebutted on the basis of close control over the intrusion
  • Bulk collection defended on the basis that only a small percentage of the items collected is inspected

Like the 1957 Birkett Report, the Home Office and Diplock Interception Reports of 1980 and 1981 made no mention of the Section 4 powers.  The reports were limited to statistical information about non-statutory warrants.  

The reports recognised the invasion of privacy involved in interception warrants. Lord Diplock said:
“The exercise by the State of any power to read or listen to communications taking place between private citizens involves an invasion of their privacy which has always been looked upon by the public with suspicion and distaste.”
The 1920 powers lasted until 1985, when they were replaced by the Interception of Communications Act (IOCA). The preceding White Paper had promised that the legislation would include provisions “along the lines currently covered by the Official Secrets Act 1920.” Whilst IOCA folded interception of external communications into the new statutory system for issuing warrants, the warrantry power for external communications continued to be broader than for internal communications.

So what is now the Section 8(4) warrant trod its own quiet path from 1920 to 1985, exposed to public scrutiny only once as a result of the Chapman Pincher cable vetting story – to which the Government of the day reacted almost identically as did the government of today to Edward Snowden’s TEMPORA disclosures.

A vanload of cables is on a smaller scale than 40 billion items of data per day, but the principle and method is the same: general capture, selection, examination. Long before any technical argument that targeted interception is impossible, the 1920 legislation enabled the government to engage in suspicionless bulk capture followed by subject-matter analysis of external communications.

In Chapman Pincher’s day collected telegrams and cables were evidently sorted manually. Human beings looked at them all and decided which were worthy of further examination. Now the initial capture, sift and discard is computerised.  The government argues that capture involves only a technical interference with privacy compared with a human being examining intercept material:
"The Respondents accept that the interception of a communication under a s. 8(4) warrant may be regarded as giving rise to a technical interference with the Art. 8 rights of the parties to the communication even if that communication is not and/or cannot be read, looked at or listened to by any person." (Open Response, IPT proceedings)
Going back further than 1920, in 1765 Lord Camden, the judge in Entick v Carrington, held that general search warrants had no legal basis. It is perhaps idle to speculate how he might have reacted had Lord Halifax (the then Secretary of State) said:
“Fear not, Mr Entick.  True we have ransacked your home, broken the locks on your desks and cupboards and seized your papers and correspondence.  But, since we have not yet examined any of them, that is a merely technical breach of privacy.  We have strict safeguards in place to ensure that we will only look for material about that renegade Wilkes who is outside the British Isles, skulking in Paris.”

Footnote: How does a Section 8(4) warrant work?

The Foreign Secretary can issue a RIPA warrant for purposes of national security; for preventing or detecting serious crime; for safeguarding the economic well-being of the United Kingdom (if related to national security); or, in relation to serious crime, mutual legal assistance treaties with other countries. He must believe the warranted interception and disclosure to be proportionate to what it seeks to achieve; and must take into account whether the information he thinks it necessary to obtain could reasonably be obtained by other means.

A Section 8(4) warrant, unlike an ordinary RIPA Section 8(1) warrant, does not have to be targeted at the communications of a particular person or premises. It can authorise general bulk collection at the level of the cable. But while a targeted Section 8(1) warrant can be used to intercept internal communications (those sent and received within the British Isles), the overall purpose of a Section 8(4) warrant must be the collection of external communications (sent or received outside the British Isles). So external communications are those where both ends of the communication, or only one end, are outside the British Isles.

Internal and external communications tend to be inseparably mingled within a single fibre-optic cable. So RIPA allows a S.8(4) warrant to authorise the capture not only of external communications, but any internal communications unavoidably swept up with them.

After capture of the communications come selection and examination. RIPA constrains these in different ways.

Captured communications (whether internal or external) can be examined if they are within a description certified by the Secretary of State in the warrant. That description could be very broad. However they can be selected for examination only in a way permitted by RIPA’s selection rules. These govern the automated filtering down of the captured communications to a database of material and also the queries made by analysts against the database.

The rules restrict the use of selection factors targeting the communications of people known for the time being to be in the British Isles. But despite this there are several gateways via which a communication sent or received by someone in the British Isles and captured under a S8(4) warrant could end up being examined by a GCHQ analyst.

Foreign Secretary Philip Hammond touched on two of the gateways in a clarification of his recent evidence to the Intelligence and Security Committee of Parliament. He posited a communication (say an e-mail) between someone in the British Isles and someone abroad. In general terms it could not be selected where the subject of interest is the person in the British Isles. That would require a further step such as the Secretary of State’s modification of the S.8(4) warrant under the exception in RIPA S.16(3). However the e-mail could be selected for examination if the person outside the British Isles is the subject of interest.

Sunday 9 November 2014

A Catechism of Privacy

Q. What is the State’s duty?
A. To protect us.

Q. How does the State protect us?
A. Through watchfulness.

Q. Whom does the State watch?
A. All who present a threat.

Q. Who watches the State?
A. We do.

Q. What may we see?
A. That which the State, which is wise, permits.

Q. May the State watch us?
A. We have nothing to hide.

Q. Must we obey the State?
A. The law must be observed.

Q. Does the State obey the law?
A. The State acts as necessary and proportionate in accordance with the law.

Q. Does the law protect privacy?
A. Privacy is not absolute.
Q. Should we fear the State?
A. The servants of the State are conscientious and dedicated.

Q. What does the State require of us? 
A. That we obey the law and act responsibly.

Q. What is our responsibility?
A. To enable the State to perform its duty.

Q. What is the State’s duty?
A. To protect us.

Friday 10 October 2014

Submissions to the Investigatory Powers Review

[Update 11 June 2015.  David Anderson's report 'A Question of Trust' has been published today and is available on his website, together with two volumes of submissions made to his review. My own submission is also available here (PDF).]

David Anderson QC (@terrorwatchdog) is the UK's Independent Reviewer of Terrorism Legislation. He is tasked under the Data Retention and Investigatory Powers Act 2014 (DRIPA) with conducting a review of investigatory powers. This includes interception of communications (e.g. by GCHQ and law enforcement) and powers to compel retention and production of communications data. His Call for Evidence closed on 3 October 2014.  Here are some of the submissions to the Review now being made public.


Bingham Centre for the Rule of Law

Centre for Democracy and Technology

Dr Andrew Defty and Professor Hugh Bochel (University of Lincoln)

Equality and Human Rights Commission

Global Network Initiative

Human Rights Watch

Interception of Communications Commissioner


The Law Society of England and Wales


The Newspaper Society

UCL LLM Students


And although not strictly speaking a submission to the Review, GCHQ Director Sir Iain Lobban's valedictory speech.

More to follow.

Saturday 6 September 2014

Whose domain space is it anyway?

Governments shouldn’t get in the way of the people who run the internet. Fine sentiments reported by the Guardian from UK Culture Minister Ed Vaizey at the Internet Governance Forum in Istanbul this week.  They echo his speech to the ICANN meeting in London in June: "What governments shouldn’t be doing is attempting to manage how the internet is run."

Fine sentiments, but does the UK government live up to them?
Regrettably the UK government has not been immune from the temptation to take powers over internet governance institutions.  Sections 19 to 21 of the Digital Economy Act 2010 gave it power to take direct control of the .uk domain by putting a manager into Nominet.  The sections have not been brought into force, let alone the powers exercised.  But the government hardly needs to once the potential exists.
In the current interstate tug-of-war over global internet governance every State accuses every other State of donning fig leaves to conceal self-interest.  Here is an opportunity for the UK government to plant a flag in the high ground, to say ‘We mean what we say.  We have backed off, how about you?’
So make the bold move, repeal Sections 19 to 21 and issue the challenge. 
Or would the government backpedal? We can hear it now. “Reserve powers, only to be deployed in the last resort in the interests of UK plc, the Secretary of State cannot use them unless there is a serious failure in limited circumstances…” (See here the reasons put forward at the time the powers were legislated). 
That won't wash.  If failings are for a national government, not the internet governance community to sort out then fine sentiments are just so much vapour.  Letting go of powers is more than desirable, it is a litmus test.

Sunday 20 July 2014

The other side of communications data

[Updated 24 May 2021 with errors for 2019 based on IPCO report for that year.]
[Updated 6 March 2020 with errors for 2018 based on IPCO report for that year; and on 4 May 2020 with corrections to the 2017 and 2018 error figures.]
[Updated 31 Jan 2019 with errors for 2017 based on IPCO report for that year.]
[Updated 16 July 2015, 10 September 2016 and 20 December 2017 with errors for the years 2014, 2015 and 2016 based on the IOCCO reports for those periods.]

Now that the dust has settled for the moment on DRIPA (the Data Retention and Investigatory Powers Act 2014) we should perhaps not forget that, even though many will regard it as worth paying, a tangible price attaches to the authorities’ use of communications data for the investigation and prosecution of crime.

This is a human, not a money price.  Mistakes are made with communications data and can have (in the words of the Interception Commissioner’s Report for 2008) catastrophic consequences for members of the public.

Calculated as a percentage of requests for communications data, the proportion of errors is arithmetically small – in the region of .2%, or 1 in 500.  But when the police arrive at an innocent front door to execute a warrant, that is not an arithmetical event. Since 2008 that, or something equally serious such as the arrest of an innocent person or a wrong accusation, has happened eleven times. [Now 73 times including 2014, 2015, 2016, 2017 and 2018 figures.]

The errors are set out in the Interception Commissioner’s Annual Reports.  These are the statistics since formal oversight of communications data requests began in 2005, covering requests by all public authorities.

Total communications data requests
Arrests, accusations, warrants executed [or other coercive power]1
2005-6 (15 months)
2006 (9 months)
- (from Oct 2007 only privacy-intrusive errors are included in statistics)
(The Report separates 640 overall errors and a further 1061 arising from two technical faults in an intelligence agency's systems, treated in the Report as one error.)
761,702 (items of data, not comparable with previous years' count of number of requests) 
754,559 (items of data) 
757,977 (items of data) 
926 934
808,214 (items of data) 
903 949
768,080 (items of data) 
[Note 1: Figures for 2017, 2018 and 2019 include instances where no search warrant was executed or arrest made, but equipment was seized. 2019 includes an instance of forced entry where IPCO determined that no serious harm resulted.]

[Note 2. The figure of 903 for 2018 is subject to clarification as to whether it includes 46 intelligence agency CD errors.]

[Note 3. Following correspondence with the IPCO press office, corrections have been made to both the 2017 and 2018 error figures. It appears that the 2017 total did not include intelligence agency figures.]

Or graphically:

[These numbers count the numbers of persons arrested, rather than the number of incidents. Thus if two persons were arrested on the same occasion as a result of the same error I have counted that as two arrests.]

The first reported catastrophic incident was in 2008.  That was the result of confusion over interpretation of international time zone information relating to an IP address. The then Interception Commissioner Sir Paul Kennedy reported it thus:

“In this particular example the police took swift action when information from a reliable source suggested that a number of very young children were at immediate risk of falling into the hands of a paedophile ring. Subscriber information relating to an Internet Protocol (IP) Address was obtained in order to locate an address for the children but unfortunately it would appear this was not correct. The police entered the address and arrested a person who was completely innocent and further enquiries are continuing. This was a very unfortunate error and the whole process of obtaining data relating to IP addresses has been re-examined.”
No incidents of this nature were reported for 2009 and 2010, but in 2011 two occurred.  Sir Paul Kennedy again:

“Unfortunately in two separate cases where a CSP disclosed the incorrect data, the mistakes were not realised and action was taken by the police forces on the data received. Regrettably, these errors had very significant consequences for two members of the public who were wrongly detained / accused of crimes as a result of the errors. I cannot say more about these two instances at this time as investigations are ongoing. … I am pleased to say that this CSP has since put in place some very sensible measures which will hopefully prevent recurrence of similar errors in future. Fortunately errors with such severe consequences are rare.”
The next year, 2012, saw a rise in the number of errors that had severe consequences.

“Regretfully in six separate cases this year, the mistake was not realised and action was taken by the police forces / law enforcement agencies on the data received. In four of the cases the mistake was made by the public authority (either the applicant or SPoC acquiring data on either the incorrect communications address or time period) and in the remaining two the mistake was made by the CSP (disclosing data on the incorrect communications address). All of these cases were requests for internet data (Internet Protocol or node name resolutions). Regrettably, five of these errors had very significant consequences for six members of the public who were wrongly detained/accused of crimes as a result of the errors. The remaining one error also caused an intrusion into the privacy of an individual, as an address was mistakenly visited by police looking for a child who had threatened to commit self harm.”
2013 saw two such incidents, described in the first Report of the current Interception Commissioner Sir Anthony May:

“I have to report that 7 errors with very serious consequences have occurred this year. Regrettably these errors resulted in police action relating to wrongly identified individuals. In 5 of these cases the mistakes caused a delay in the police checking on young persons who were intimating suicide or on an address where it was believed that someone had been the victim of a serious crime. Fortunately the police were able to identify quickly in these instances that the persons visited were not connected with their investigation. In the remaining instances warrants were executed at the homes of innocent account holders and this is extremely regrettable. [The report does not state how many such homes or people were involved. We have assumed two.]
4.52 All but one of these errors occurred in relation to requests for Internet Protocol (IP) data to identify the account that was accessing the internet at a particular date and time. There were 3 specific causes for the errors: data applied for over the wrong date or time, the incorrect time zone conversion or a transposition error in the IP address.”
[For 2014 the Interception Commissioner's Report said:
"These 21 errors (12 technical and 9 human) resulted in action being taken against the wrong individual (for example, an innocent individual’s address being visited by officers, or a warrant being executed at the wrong address) in 12 instances; and on 4 occasions caused a delay in the police conducting welfare checks on a person in crisis. Some of these errors occurred in relation to the resolution of Internet Protocol addresses and the consequences of these are particularly acute. An IP address is often the only line of enquiry in a child protection case, and it may be difficult for the police to corroborate the information before taking some form of action against the individual identified. Any police action taken erroneously in such cases, such as the search of an innocent individuals house, can have a devastating impact on the individual concerned. These errors are extremely regrettable and it is fortunate that errors with such severe consequences are rare."]
[The Interception Commissioner's Half-Yearly Report of July 2015 goes into considerable detail about the serious errors investigated during 2014.  The six mistaken search warrants or arrests listed for 2014 in the table above were caused as follows:

- Omission of an underscore when transcribing an e-mail address led to the wrong subscriber information being provided and a search warrant being executed at the premises of an individual unconnected with the investigation.

- A CSP's data warehouse system change affected how GMT and British Summer Time were treated. This was not communicated to staff using the data retention disclosure system. This led to a one hour error in subscriber information disclosed in relation to IP address usage. Of 98 potential disclosure errors identified, 94 were in fact incorrect and four returned the same results when re-run. Of the 94 incorrect disclosures, in three cases a search warrant was executed at premises relating to individuals unconnected with the investigation (and one individual was arrested).

- Due to a technical fault causing a time zone conversion to be out by seven hours, a CSP voluntarily disclosed an incorrect IP address to a public authority.  That led to a search warrant being executed at premises relating to individuals unconnected with the investigation.]

The Annual Report for 2015 also gives more detail in Section on the causes  of the errors (both human and technical).]

[The 2016 Annual Report contains a chapter on IP Address Resolution Errors, included in order to raise the profile of the issue with public authorities and among victims and their legal representation. The Commissioner is concerned by the increasing number of errors that occur when public authorities try to resolve IP addresses. He says that there needs to be a change of mindset away from the assumption that technical intelligence, such as an IP address resolution, is always correct.

Annex D to the Report contains details of 29 serious error investigations.]   

[Annex B to the 2017 Annual Report contains details of 24 serious error cases resulting from 33 serious error investigations.]

[Annex C to the 2018 Annual Report contains details of 22 serious error cases resulting from 24 serious error investigations.]

In all, since 2008 accountholders have mistakenly been the subject of arrests, accusations [seized equipment] or search warrants on 11 occasions [now 74 occasions]. This does not include the five 2013 cases [, four 2014, six 2015, nine 2016, ten 2017 and three 2018 cases] in which people were visited [, contacted or spoken to] by the police, since the Reports does not state that anyone was wrongly accused.

A point of subsidiary interest is where the responsibility for errors may lie as between the CSPs producing communications data information and the requesting public authorities. The Commissioner’s statistics split overall errors into those attributable to the CSP and those to the requesting authority. 

This graph is based on the figures in the Annual Reports [to 2018]. 

The split for 2010 is as reported by the Interception Commissioner, based on an overall figure of 640 errors and excluding a further 1061 errors treated as one error. If those had been treated as individual errors the split for 2010 would have been 7% CSPs and 93% public authorities.

The 2013 Interception Commissioner's Report states that the overall figures for communications data requests in 2011, 2012 and 2013 exclude urgent oral applications, which in 2013 totalled 42,293. It does not comment on whether the same is true for previous years.