Department of Defense’s ‘Hack the Pentagon’ Bug Bounty Program Helps Fix Thousands of Bugs
Credit to Author: Lily Hay Newman| Date: Fri, 10 Nov 2017 12:00:00 +0000
The United States government doesn't get along with hackers. That's just how it is. Hacking protected systems, even to reveal their weaknesses, is illegal under the Computer Fraud and Abuse Act, and the Department of Justice has repeatedly made it clear that it will enforce the law. In the last 18 months, though, a new Department of Defense project called "Hack the Pentagon" has offered real glimmers of hope that these prejudices could change.
The government's longstanding defensive posture makes some sense in theory—it has important secrets to keep—but in practice security experts have long criticized the stance as a fundamental misunderstanding of how cybersecurity works. The inability of researchers and concerned citizens to disclose vulnerabilities they find inevitably makes the government (or any institution) less secure. So in the wake of numerous government agency breaches, including the devastating Office of Personnel Management hack, DoD's Defense Digital Services group, the Office of the Secretary of Defense Cyber Policy group, and then-Defense Secretary Ash Carter saw a possible opportunity to spur change by introducing the DoD to bug bounties—programs that offer cash rewards to independent hackers who find and disclose software bugs.
"DoD has a framework of doing penetration testing and doing their own vulnerability assessment, but this is in the constraints of federal government," says Michael Chung, the Product and Technology Lead at Defense Digital Services. "So our gut feeling was that bringing in private sector practices would show that there were more vulnerabilities that hadn't been found."'
With the help of bug bounty facilitator firm HackerOne and after coordinating with the Department of Justice, DDS kicked off the pilot Hack the Pentagon bug bounty on April 16, 2016. Over a 24-day period, dozens of pre-selected security researchers hunted down vulnerabilities in certain public-facing DoD websites, in what was the first federal bug bounty ever run at a federal agency. The department ended up resolving more than 138 unique vulnerabilities, and paid tens of thousands of dollars to 58 hackers. One made a total of $15,000 by reporting multiple bugs.
"What HackerOne and the Pentagon have done seems like a feat of wizardry," says Dan Tentler, a founder of the attack simulation and remediation firm Phobos Group, which contributed to the first Hack the Pentagon bug bounty (but chose not to be eligible for rewards). "Up until very recently, the government’s way of keeping people in the US from hacking them was to basically threaten that black helicopters would show up over your house if you tried. Then one day I’m stuck at the airport and I’m brute-forcing various Pentagon hosts with no fear of repercussions. It’s pretty cool."
'Up until very recently, the government’s way of keeping people in the US from hacking them was to basically threaten that black helicopters would show up over your house if you tried.'
Dan Tentler, Phobos Group
To follow up on the success of Hack the Pentagon, DoD launched another bounty, Hack the Army, last November, to assess public-facing websites related to Army enrollment. That program included hundreds of hackers who found more than 100 unique bugs, and received about $100,000 in total payouts.
After Hack the Pentagon, DoD had noticed that with limited-time bounties, bugs still trickled in days and weeks after the open call concluded. So the feds announced an open-ended Vulnerabilities Disclosure Policy that didn't offer rewards, but would legally allow people to submit bugs any time related to public-facing websites and web applications owned by DoD.
In the year since, about 650 people have submitted almost 3,000 unique, valid vulnerabilities. A year ago, they would have been breaking the law.
"The VDP has just really taken off and started providing value in a way that I don’t think anyone was anticipating when we first launched it," says Alex Rice, CEO of HackerOne. "It was some learning. DoD realized that…if someone was still working on something there was no legal channel for them to get it to the government."
Hack the Air Force came next, at the end of May, awarding more than $130,000 for 207 unique vulnerabilities. Through the bounties and VDP, DoD has found out about and fixed thousands of vulnerabilities in its systems so far, along with more than a hundred highly critical flaws. These have included vulnerabilities that allow remote code execution, SQL code injection bugs on various websites, and methods for bypassing authentication protections.
"For the past 12 months we’ve learned a lot and we’ve really reached a tipping point where now we’re getting a lot of requests, a lot of interest to do these bug bounties across all DoD," Chung says. "We’re trying to do away with the guy in sunglasses and a hoodie in his basement image, and trying to put an actual person behind the whole white-hat hacker persona. It really is a shift in thinking."
That newfound acceptance has spread. Over the last year, DoD has also run a few private bug bounties on more sensitive systems through the penetration testing firm Synack, which was awarded a contract to focus on assessing internal platforms. And outside the Department, the General Services Administration and Department of Homeland Security are both working on bug bounties as well. Chung eventually wants to ramp up to as many as two bug bounties per month within DoD alone. Similarly, Lieutenant General Edward Cardon, who worked on the Army's first bounty last year, says the they're working toward running one bug bounty per quarter to assess a diverse array of public-facing systems.
The momentum Hack the Pentagon now has within DoD belies the challenges and struggles of the last 18 months, though. And the initial pilot alone required a hard-won ideological evolution. "When we first launched Hack the Pentagon it was pretty much a non-starter," Chung says. "The idea of hacking into the Pentagon scared a lot of people."
One of the original proponents of the project at Defense Digital Services, Lisa Wiswell, is actually known as DDS's "bureaucracy hacker."
The DoD's existing digital defense practitioners and contractors also expressed skepticism. "There was a little pushback in the beginning by some of the incumbents there, some of the pen testers, some of the contractors," Chung says. "But they know that there’s a mission involved with this. I can’t stress enough how much of this work is valuable to national security."
Even after the successful pilot, real doubts still existed within DoD about doing additional bug bounties. The Army runs combat simulations and war games, of course, to train, improve its tactics, and identify weaknesses. But Lieutenant General Cardon says it was a process to explain that the same concepts apply in cyberspace.
'The idea of hacking into the Pentagon scared a lot of people.'
Michael Chung, Defense Digital Services
"I’m a big believer in this sort of approach. I think it’s good for the government. Some of these vulnerabilities, if attackers took it to the end, would be a serious problem for us," he says. "With the bug bounties, there was obviously a lot of concern about the risks. The rules for how to do this were mature enough, though, that we could provide an understanding of the risks. That then made the senior leadership of the Army much more amenable to this type of a program."
There were also hurdles in hammering out the processes for executing the bug bounties themselves. Tentler, the researcher who worked on Hack the Pentagon, says that at first there were issues establishing the scope of the bug bounty, to keep participants from submitting vulnerabilities for systems DoD didn't intend them to look at.
"I can't speak for everyone, but the people that I was working with said well, this doesn’t make any sense. We’re eyeballs-deep in their systems and now they’re saying that what we’re doing is out of scope," Tentler says. "Apparently there were four or six actual web hosts that were permitted, and I was like it would have helped to just have those from the start. What I’ve seen, though, over time, is a gradual lessening of tension. In the last year they’ve come quite a long way."
Bug bounties and vulnerability disclosure processes alone can also only go so far. You have to actually fix the flood of bugs after hackers find them. Establishing an effective remediation process takes time and resources, challenges that Chung and Cardon both attest to within DoD. And Tentler notes that one vulnerability he found during the pilot Hack the Pentagon took months for the DoD to resolve. That came in part because the vulnerability was outside the scope of the bounty and it was difficult to determine how best to submit it for actual consideration.
But HackerOne's Rice says he has been impressed with the infrastructure DoD has established over time. "Their remediation time has been well below average for these programs that we’ve run," Rice says, "and they’ve resolved everything within a pretty condensed period of time afterward. We have private companies that have vulnerabilities that still aren’t resolved after a year."
Given all the breaches of government agencies over the last few years, from OPM to an embarrassing hack of the Pentagon's own non-classified email system, Hack the Pentagon could have amounted to a one-off publicity stunt to make the DoD seem tuned in during a rocky phase. Instead its newfound openness to security feedback seems like it may genuinely be propagating throughout the government rather than being quickly shut down. In the face of such entrenched resistance there are still no guarantees, but given that none of this seemed possible even recently, the accomplishments of Hack the Pentagon's first year are noteworthy.
"It’s one thing for a company to come forward and work with their general counsel to do a bug bounty," Rice says. "It’s a completely different thing entirely for the organization that really initiated the Computer Fraud and Abuse Act and that early hostility toward security researchers to openly start engaging and working with them. The weight that the DoD brings when they pair with the DoJ to say 'hackers can do good,' that just doesn’t exist anywhere else."