1. Background

The NSA, founded in 1952, is the USA’s signals intelligence agency, and the biggest of the country’s myriad intelligence organisations. It has a strict focus on overseas, rather than domestic, surveillance. It is the phone and internet interception specialist of the USA, and is also responsible for codebreaking.

It is run by General Keith Alexander, who answers to Obama’s Director of National Intelligence James Clapper. The NSA is overseen by congressional intelligence committees, who have security clearance, and the Foreign Intelligence Surveillance Court, which sits in secret.

GCHQ – an acronym of Government Communications Headquarters – is the UK’s answer to NSA, and its predecessor organisations were founded in 1919. The very existence of the agency was not officially admitted until 1983. It is permitted to spy in the interests of national security, preventing serious crime, or defending the UK’s economic interests. The agency answers to foreign secretary William Hague, and has parliamentary oversight from the Intelligence and Security Committee, chaired by Sir Malcolm Rifkind. In fact, the Guardian revelations show that it vastly exceeded this remit.

2. The story in a nutshell

The Snowden files reveal a number of mass-surveillance programs undertaken by the NSA and GCHQ. The agencies are able to access information stored by major US technology companies, often without individual warrants, as well as mass-intercepting data from the fibre-optic cables which make up the backbone of global phone and internet networks. The agencies have also worked to undermine the security standards upon which the internet, commerce and banking rely.

The revelations have raised concerns about growing domestic surveillance, the scale of global monitoring, trustworthiness of the technology sector, whether the agencies can keep their information secure, and the quality of the laws and oversight keeping the agencies in check. The agency is also required to abide by the European Convention on Human Rights.

3. The programs

3.1. PRISM

Prism is a top-secret $20m-a-year NSA surveillance program, offering the agency access to information on its targets from the servers of some of the USA’s biggest technology companies: Google, Apple, Microsoft, Facebook, AOL, PalTalk and Yahoo. The UK’s spy agency GCHQ has access to Prism data.

NSA documents suggest the agency can use Prism to access information “directly from the servers” of US companies – a claim they strongly deny. Other documents showed the NSA had paid out millions of dollars to “Prism providers”, and showed Microsoft had helped the NSA circumvent its users’ encryption.

3.2. Tempora

The UK’s GCHQ spy agency is operating a mass-interception network based on tapping fibre-optic cables, and using it to create a vast “internet buffer”, named Tempora – a kind of Sky+ for huge amounts of data flowing in and out of the UK. The content of communications picked up by the system are stored for three days, while metadata – sender, recipient, time, and more – is stored for up to thirty days. Metadata is effectively the "envelope" of a communcation: who it's from, when it was sent and from where, and who it's to, and where - but not the actual contents of the communication.

The system, part of GCHQ’s stated goal to "Master the Internet"">, is enabled using a little-known clause of a law passed in 2000 for individual warranted surveillance, known as RIPA. The telecoms companies involved in the surveillance program were later named as BT, Verizon Business, Vodafone Cable, Global Crossing, Level 3 Viatel and Interoute.

3.3. Phone collection

The very first story from the NSA files showed the agency was continuing a controversial program to collect the phone records (“metadata”) of millions of Americans – a scheme begun under President Bush. The scheme was widely believed to have been scrapped years before.

The program, which was re-authorised in July, allows the agency to store who Americans contact, when, and for how long. The agency is not, however, allowed to store the contents of calls. The Obama administration later released hundreds of pages of confidential documents about the program, showing aspects of the surveillance had at one stage been judged unconstitutional by secret oversight courts.

3.4. Upstream

“Upstream” refers to a number of bulk-intercept programs carried out by the NSA, codenamed FAIRVIEW, STORMBREW, OAKSTAR and BLARNEY. Like similar GCHQ programs, upstream collection involves intercepting huge fibre-optic communications cables, both crossing the USA and at landing stations of undersea cables.

The collection, which relies on compensated relationships with US telecoms companies, allows the NSA access to huge troves of phone and internet data, where at least one end of the communication is outside of the country. Later disclosures revealed the NSA keeps all the metadata it obtains through Upstream and Prism in a database system called MARINA for 12 months.

3.5. Cracking cryptography

The NSA and GCHQ have been undertaking systematic effort to undermine encryption, the technology which underpins the safety and security of the internet, including email accounts, commerce, banking and official records.

The NSA has a $250m-a-year program working overtly and covertly with industry to weaken security software, hardware equipment, and the global standards on security, leading experts to warn such actions leave all internet users more vulnerable.

Both agencies’ codenames for their ultra-secret programs are named after their countries' respective civil war battles: BULLRUN for the NSA, and EDGEHILL for GCHQ.

4. The issues

4.1. Corporate cooperation

The extent to which private companies are cooperating with intelligence agencies has been a major source of concern for internet users across the world. The technology companies in the PRISM slides were keen to stress they do not go beyond what they are forced to do under law in handing over user data, but other documents suggest some internet and telecoms companies on occasion go beyond what is mandatory.

Such relationships create issues of customer trust for US and UK technology giants, as Facebook founder Mark Zuckerberg publicly acknowledged, as well as raising questions as to whether what the law allows represents the limits of surveillance, or merely a starting point.

Documents suggest some payments to Prism, cryptography and cable-intercept providers, but the scope of such transactions, and the recipients, are to date unknown.

In October, the Electronic Frontier Foundation withdrew from the Global Network Initiative, the biggest multi-stakeholder group on human rights online, over concerns that corporate members were unable or unwilling to speak out on surveillance.

4.2. The law

Revelations on GCHQ and NSA activities to date have led to lawmakers, particularly in the USA, raising concerns that the interpretations of the law used by agencies were not the intent of lawmakers when the rules were set.

NSA mass-surveillance is authorized under a law known as the Fisa Amendments Act of 2008, which was renewed in 2012. It allows for the collection of communications, without a warrant, where at least one end of the communication is a non-US person. Collection of Americans' phone data comes under a different law, section 215 of the Patriot Act. A congressional motion to defund such collection was defeated by just 12 votes in the wake of the program's revelation.

Previously secret court-imposed rules published by the Guardian showed a wide range of circumstances where the data of US people collected without a warrant could be stored, used, and viewed. Later documents showed the agency is even allowed tosearch for US people within such data.

GCHQ mass-surveillance is authorized under Section 8(4) of the Regulation of Investigatory Powers Act (RIPA), which allows for bulk surveillance provided the Secretary of State signs certificates authorising it for particular purposes every six months.

The agency is also required, however, to be compliant with the right to privacy within the European Convention of Human Rights. Three UK privacy groups are currently mounting a legal challenge to GCHQ surveillance in the European courts.

4.3. Oversight

Oversight for the NSA comes from the Foreign Intelligence Surveillance Court, which operates in secret. In the wake of the Snowden revelations, there has been widespread public and congressional pushback against the court’s efficacy, leading Obama to consider reforms to its operations and to declassify hundreds of pages of rulings from the court.

Ron Wyden and Mark Udall, both members of the Senate intelligence committee, which also oversees NSA operations, have repeatedly stated concerns about the scope of NSA surveillance, even accusing Director of National Intelligence James Clapper and NSA head General Alexander of misleading the committee in the wake of the first NSA revelations.

In the UK, GCHQ oversight comes from parliament’s Intelligence and Security Committee, which is chaired by Sir Malcom Rifkind, who said it was part of his role to “defend” the UK’s intelligence agencies.

In public statements, GCHQ says it works within "the strongest systems of checks and balances for secret intelligence anywhere in the world". Internal legal briefings, however, acknowledge the agency has “a light oversight regime compared with the US", adding that the parliamentary committee responsible for GCHQ has “always been exceptionally good at understanding the need to keep our work secret”.

GCHQ documents further note the UK’s investigatory powers tribunal has "so far always found in our favour".

4.4. Trust in technology

NSA and GCHQ efforts to undermine global encryption garnered a strong reaction from the world’s internet security community. Experts warned systems were more open to hacking by foreign governments or criminal gangs, and accused the agency of “subverting” the internet.

Several organisations have begun redesigning their products so as not to use standards approved by the US government, for fear they are insecure, while others have suggested that surveillance overreach could damage US technology companies’ standing and sales in the world, as well as undermining the USA’s moral authority as custodian of the internet.

Phillip Zimmerman, the architect of the PGP email security software, has said in the wake of the NSA revelations that secure email is largely impossible, and a new product would need building from scratch.

4.5. Privacy and mass surveillance

Revelations from the Snowden cache show that even the NSA’s own internal auditors found its agents broke privacy rules thousands of times each year, but some governments and advocates alike have warned mass-surveillance itself, even if not abused, can be a major problem.

Freedom of expression advocates have warned routine surveillance of communications can stifle free speech, while Germany’s justice minister described GCHQ’s Tempora programme as like something from “a Hollywood nightmare”.

In the USA, a coalition of academics has formally submitted a 15-page document to Obama’s intelligence review panel warning of the serious threat mass surveillance poses to journalism in the USA and across the world.

5. The story of the leak

5.1. Edward Snowden

On 9 June 2013, 29-year-old Edward Snowden revealed himself as the source of the NSA revelations published that week in the Guardian and the Washington Post, in a video interview with Glenn Greenwald and Laura Poitras.

Snowden, an IT specialist working for US contractor Booz Allen Hamilton, had left his home in Hawaii for Hong Kong days before, and met with Poitras, Greenwald, and another Guardian journalist, Ewen Macaskill.

Saying he wanted to launch a global debate on the limits of NSA surveillance, Snowden said "I understand that I will be made to suffer for my actions," but "I will be satisfied if the federation of secret law, unequal pardon and irresistible executive powers that rule the world that I love are revealed even for an instant."

5.2. The hunt for Snowden

Snowden had already left the USA for Hong Kong for fear of legal retribution as a result of his leaks. For several days, he remained in an undisclosed location in Hong Kong. However, just days after the USA issued an international warrant for his arrest on espionage charges, the whistleblowing website WikiLeaks announced Snowden had boarded a flight to Russia, as a stop-off to an undisclosed country in South America.

Snowden was expected to board a plane to Cuba the following morning, but did not. It emerged his temporary travel authorisation, issued by an Ecuadorian diplomat, had been revoked. After spending several weeks trapped airside in Moscow’s Sheremetyevo airport, he was granted one year’s asylum in Russia. He is currently staying at an undisclosed location in the country. In October, he was visited by a group of US whistleblowers who presented him with the Sam Adams award.

5.3. GCHQ and the Guardian

The Guardian has had a running series of conversations with GCHQ about its access to material disclosed by Edward Snowden, and publication of stories based on it. Such conversations intensified in July – weeks after the first publication – when a senior Cabinet Office official told Guardian editor Alan Rusbridger to hand back or destroy the material, saying “You’ve had your fun”.

Wary of the risk of injunction or state censorship, the Guardian had already shared material with ProPublica and the New York Times. As such, when pressure from the Cabinet Office and GCHQ intensified, the newspaper offered to destroy all hard disks and computers on UK soil which contained information with the Snowden files. This offer was accepted, and the computers were smashed in a Guardian basement as GCHQ officials watched.

Rusbridger said the decision was taken in order to prevent a situation in which the paper would be legally required to hand over the material, or injuncted. The White House deputy press secretary later said it was “difficult to imagine” the US government ever requiring an American newspaper to do the same.

5.4. David Miranda

On Sunday 18 August 2013, David Miranda, the partner of Guardian journalist Glenn Greenwald, was detained for nine hours at Heathrow airport under anti-terror laws.

Miranda was transiting through the UK on his return to his home in Rio de Janeiro after visiting filmmaker Laura Poitras when he was detained by the Metropolitan Police at the airport under section 7 of the Terrorism Act.

He was required to hand over all electronic devices and storage in his possession, and give over passwords. Among the material was heavily encrypted journalistic source material being used in Greenwald and Poitras’ reporting. Later police statements said Miranda was also carrying a password which opened a small number of files in his possession.

Miranda is challenging the legality of his detention, while the UK’s Independent Police Complaints Commission has separately applied for judicial review of the Met Police’s use of Section 7 of the Terrorism Act.

Charlie Falconer, the author of the Act, wrote in the Guardian that enabling the detention of Miranda was not the intention of lawmakers when passing the legislation, and his detention was not “within the spirit nor the letter of the law”.