The Baltimore Sun’s Siobhan Gorman has a major story about NSA data mining programs. This really deserves more attention than it has received. One aspect to this story stands out, the very thing that has gone virtually unnoticed so far: The lawyers for the NSA strongly opposed, in the late 1990s, implementing a program that would have analyzed domestic telephone data on the grounds that it was against the law. After 9/11, however, the NSA lawyers reversed themselves and implemented another program to analyze domestic calls that had significantly less stringent protections for civil liberties.
The basis for such a stunning reversal? Merely the tired argument that the President obtains expansive powers during wartime. Even apart from the absurdity of that argument as applied to constitutional civil liberties, there is this difficult conundrum. What are we to do about this inconvenient fact, that the U.S. was already at war in the Balkans in the late 1990s? Let’s not mention the never-concluded war in Korea.
The data-mining program in question, named `ThinThread’, was developed by the NSA in the 1990s partly because of fears about terrorist attacks at millennium celebrations. Controversially, it would have analyzed both foreign and domestic electronic communication. The NSA was aware that some foreign-based terrorists might be communicating with others inside the U.S., and the Agency was looking for a way around the rule that domestic surveillance had to be conducted by the FBI.
To make a domestic surveillance program less repulsive, ThinThread incorporated several measures to safeguard the civil liberties of Americans, at least to some degree. All the data was encrypted before analysis to ensure privacy; an automated system watched how analysts were using the data, to prevent misuse; and after sifting out most of the encrypted data, if the remaining data showed evidence of a terrorist threat, only then would the Agency request a court for permission to decrypt the relevant data.
The program turned out to be highly adept at identifying threats; allegedly it was the best one available to the NSA. Yet the NSA lawyers blocked it because they refused absolutely to allow the Agency to spy on domestic communication. Here are the relevant sections of the Sun report:
Officials say that after the successful tests of ThinThread in 1998, [project director Richard] Taylor argued that the NSA should implement the full program. He later told the 9/11 Commission that ThinThread could have identified the hijackers had it been in place before the attacks…
But at the time, NSA lawyers viewed the program as too aggressive. At that point, the NSA’s authority was limited strictly to overseas communications, with the FBI responsible for analyzing domestic calls. The lawyers feared that expanding NSA data collection to include communications in the United States could violate civil liberties, even with the encryption function.…
After the 2001 attacks, the NSA lawyers who had blocked the program reversed their position and approved the use of the program without the enhanced technology to sift out terrorist communications and without the encryption protections.
The NSA’s new legal analysis was based on the commander in chief’s powers during war, said former officials familiar with the program. The Bush administration’s defense has rested largely on that argument since the warrantless surveillance program became public in December.
There are more angles to this story, which I won’t explore in detail partly because others have focused on them. Consult this diary by clammyc at Booman, and these three diaries from Daily Kos, here, here, and here. Also, the story has been discussed by Kevin Drum, Josh Marshall, and Digby.
For Gorman, the main issue is that ThinThread was shelved largely because of bureaucratic infighting. Gen. Hayden was heavily promoting a data-mining program he’d initiated, `Trailblazer’, which was plagued by problems. It is the program that he was closely questioned about this morning by Sen. Wyden, in as much as Wyden thought the General had grossly exaggerated Trailblazer’s success in testimony to the Senate. As the Sun noted yesterday,
The decision [to shelve ThinThread], which one official attributed to “turf protection and empire building,” has undermined the agency’s ability to zero in on potential threats, sources say. In the wake of revelations about the agency’s wide gathering of U.S. phone records, they add, ThinThread could have provided a simple solution to privacy concerns….
Despite its success in tests, ThinThread’s information-sorting system was viewed by some in the agency as a competitor to Trailblazer, a $1.2 billion program that was being developed with similar goals. The NSA was committed to Trailblazer, which later ran into trouble and has been essentially abandoned.
For the on-going analysis of domestic communication, done without warrant, the NSA has been relying upon a less effective rump version of this data-mining program, one without encryption, without automated oversight, without the need to seek a court order to decrypt the most important information, and therefore without the rigor of ThinThread that forced analysts to focus narrowly on the most salient threats that the program could single out.
NSA managers did not want to adopt the data-sifting component of ThinThread out of fear that the Trailblazer program would be outperformed and “humiliated,” an intelligence official said.
In short, before 9/11 the NSA rejected a more effective program in favor of a pet boondoggle, on the grounds that the civil liberties safeguards in the better program still fell afoul of the law. But later the NSA did not apply even those basic safeguards either to the ineffectual program (Trailblazer), or to the rump of a data-mining program that it began to apply to domestic communications after 9/11.
It is the incompetence of Hayden’s leadership that most other commentators on the Baltimore Sun report have dwelled upon. Yet the most important aspect by far is the ease with which the NSA reversed course on the absolute necessity of avoiding domestic surveillance and of protecting civil liberties.
Once President Bush gave the go-ahead for the NSA to secretly gather and analyze domestic phone records — an authorization that carried no stipulations about identity protection — agency officials regarded the encryption as an unnecessary step and rejected it, according to two intelligence officials knowledgeable about ThinThread and the warrantless surveillance programs.
“They basically just disabled the [privacy] safeguards,” said one intelligence official.
From a surface reading of this article, it would appear that the NSA reversed course on safeguarding civil liberties with alacrity late in 2001.
Officials familiar with Thin Thread say some within NSA were stunned by the legal flip-flop.
And Bush has said in the past that it was Hayden who volunteered to create the domestic surveillance program, without prompting from Bush (he implied). But did the NSA give in without a fight? A recent report in the NY Times argues that Dick Cheney tried to force Gen. Hayden to direct the NSA to engage in widespread surveillance of domestic communication, which Hayden resisted in favor of a more limited violation of the law. My own analysis of the NYT story, for what it’s worth, argued that the details don’t add up unless we infer that it was Cheney, rather than Hayden, who conceived of the NSA’s domestic surveillance program. I speculated that Hayden, or more likely the NSA lawyers, fought a rear-guard action against Cheney until they finally agreed to permit a less expansive version of domestic surveillance.
In any event, the thing to stress about this week’s reports in the NYT and Baltimore Sun is that until quite recently the NSA aggressively resisted attempts to drag it into the business of domestic surveillance on the grounds that it was illegal for the Agency to engage in it. The only support for the reversal under the Bush administration is a very thin reed, the notion that the Congressional Authorization of the Use of Force gave the President the power to spy domestically without warrant.
If the post at NRO gets nothing else right, at least it recognizes that this ought to be the central issue that comes out of the Sun story.
Either you think this type of intelligence-gathering falls within the powers of the executive to command the military, or you don’t.
I don’t, because it did not fall within Bill Clinton’s powers in the 1990s either.