It began as routinely as any other passenger flight. At gate 15 of New York City’s JFK Airport, more than 200 men, women, and children stood in line as they waited to board a Boeing 747. They were on their way to Seoul, South Korea’s capital city. But none would ever make it to their destination. About 14 hours after its departure, the plane was cruising at around 35,000 feet not far from the north of Japan when it was shot out of the sky.

The downing of Korean Airlines Flight 007 occurred on September 1, 1983, in what was one of the Cold War’s most shocking incidents. The plane had veered off course and for a short time entered Soviet airspace. At Dolinsk-Sokol military base, Soviet commanders dispatched two fighter jets and issued an order to “destroy the intruder.” The plane was hit once by an air-to-air missile and plummeted into the sea, killing all passengers and crew. President Ronald Reagan declared it a “crime against humanity,” marking the dawn of a volatile new chapter in relations between the United States and the Soviet Union. Soon, tensions would escalate to a level not seen since the Cuban missile crisis, which 20 years earlier had brought the world to the brink of nuclear war.

Sisters of a passenger on Korean Airlines Flight 007 weep as a South Korean government spokesman announced that it was “almost certain” the jetliner had been shot down en route to Seoul on Sept. 2, 1983.

As the international confrontation between the two adversaries played out publicly, behind closed doors another problem — which has never before been revealed — was developing. The U.S. and one of its closest allies, Japan, were embroiled in a dispute involving secret surveillance. Soviet officials were flat-out denying they had any role in shooting down the jet. At a spy base on Japanese territory, however, communications had been intercepted proving the Soviet military was the perpetrator. The U.S. wanted to obtain copies of the tapes but had to first receive approval from the head of a shadowy Japanese surveillance organization known as the “G2 Annex.”

After some bureaucratic wrangling, the Japanese eventually signed off on the release and the highly sensitive recordings were sent to Washington. From there, the tapes were forwarded to New York City, where U.S. Ambassador Jeane Kirkpatrick brought them to the United Nations headquarters in Manhattan. On September 6 — just five days after the Korean Airlines jet was shot down — Kirkpatrick attended a meeting at the U.N. Security Council where she blasted the Soviet Union for telling “lies, half lies and excuses” about its involvement in the downing of the plane. She then proceeded to play the copy of the intercepted conversations, stating that the evidence was being presented in “cooperation with the government of Japan.”

The case Kirkpatrick put forward against the Soviets was irrefutable and damning. But Japan’s spying capabilities had now been exposed — and the country’s officials were not pleased about it. The G2 Annex received new orders limiting its cooperation with the U.S., which affected the NSA’s relationship with its Japanese counterparts for the better part of a decade, at least until the Cold War ended in the early 1990s.

The details about the Korean Airlines case are revealed in classified National Security Agency documents, obtained by The Intercept from the whistleblower Edward Snowden. The documents, published Monday in collaboration with Japanese news broadcaster NHK, reveal the complicated relationship the NSA has maintained with Japan over a period of more than six decades. Japan has allowed NSA to maintain at least three bases on its territory and contributed more than half a billion dollars to help finance the NSA’s facilities and operations. In return, NSA has kitted out Japanese spies with powerful surveillance tools and shared intelligence with them. However, there is a duplicitous dimension to the partnership. While the NSA has maintained friendly ties with its Japanese counterparts and benefited from their financial generosity, at the same time it has secretly spied on Japanese officials and institutions.

The NSA declined to comment for this story.

View of the radioactive plume from the bomb dropped on Nagasaki, as seen from 9.6 kilometers away, in Koyagi-jima, Japan, on Aug. 9, 1945.

On August 14, 1945, Japan announced its unconditional surrender just days after U.S. Air Force planes dropped two atomic bombs on the cities of Nagasaki and Hiroshima, killing more than 100,000 people. The war was over, but as part of the peace agreement, Japan agreed to U.S. military occupation. American forces — led by Gen. Douglas MacArthur — drafted a new Japanese constitution and reformed the country’s parliamentary system. In April 1952, Japan’s sovereignty was restored, but the U.S. continued to maintain a major presence in the country — and that is where the NSA’s story begins.

According to the agency’s documents, its relationship with Japan dates back to the 1950s. NSA’s presence in the country was for many years managed out of a “cover office” in the Minato area of downtown Tokyo, within a U.S. military compound called the Hardy Barracks. From there, NSA maintained close relations with a Japanese surveillance agency that it refers to as Japan’s Directorate for Signals Intelligence, or SIGINT.

At first, the NSA appears to have kept a low profile in Japan, concealing details about its presence and operating undercover. But as its relationship with the country developed, that changed. By 2007, the agency had determined that “cover operations are no longer required” and it relocated its main office in Japan to a space within the U.S. Embassy in Tokyo. “NSA’s partnership with Japan continues to grow in importance,” the agency noted in a classified October 2007 report, adding that it planned to take the country “to the next level as an intelligence partner with the U.S.”

Beyond Tokyo, NSA has a presence today at several other facilities in Japan. The most important of these is located at a large U.S. airbase in Misawa, about 400 miles north of Tokyo. At what it calls its “Misawa Security Operations Center,” the agency carries out a mission under the code name LADYLOVE. Using about a dozen powerful antennas contained within large golf ball-like white domes, it vacuums up communications — including phone calls, faxes, and internet data — that are transmitted across satellites in the Asia-Pacific region.

Army Lt. Gen. Keith Alexander, former director of the National Security Agency, reviews his notes while testifying on Capitol Hill in Washington on May 1, 2007.

As of March 2009, Misawa was being used to monitor “over 8,000 signals on 16 targeted satellites,” one NSA document noted. At the same time, the agency was working on beefing up the spy hub’s systems, so that it could meet a challenge set by then-Director Keith Alexander to “collect it all” — meaning, to sweep up as many communications as possible. Misawa’s NSA employees responded to Alexander’s call by developing technology to automatically scan and process more satellite signals. “There are multitude of possibilities,” one Misawa-based NSA engineer reported, predicting that the base would soon be “one step closer to ‘collecting it all.’”

Strategically, Japan is one of the NSA’s most valuable partners. Because of its close proximity to major U.S. rivals like China and Russia, it has been used as a launching pad to spy on those countries. But NSA’s operations in Japan are not limited to monitoring the communications of nearby adversaries. At Misawa, the NSA deployed programs called APPARITION and GHOSTHUNTER, which pinpoint the locations of people accessing the internet across the Middle East and North Africa. NSA documents detailing GHOSTHUNTER’s deployment at the NSA’s British base Menwith Hill state the program was used to facilitate lethal strikes, enabling “a significant number of capture-kill operations” against alleged terrorists. One November 2008 document noted that Misawa had proved particularly useful in tracking down terror suspects in Afghanistan and Pakistan, and was also being used in an effort to identify targets in Indonesia.

Over the past decade, the NSA’s tactics have evolved dramatically — and it has rolled out new and more controversial methods. By 2010, with the internet surging in popularity, the agency was continuing to focus on long-established spying tactics like eavesdropping on phone calls, but it was increasingly adopting more aggressive methods, such as hacking into its targets’ computers.

At Misawa, the NSA began integrating hacking operations into its repertoire of capabilities. One such method it deployed at the base is called a “Quantum Insert” attack, which involves monitoring the internet browsing habits of people targeted for surveillance, before covertly redirecting them to a malicious website or server that infects their computers with an “implant.” The implant then collects data from the infected computer and returns it to the NSA for analysis. “If we can get the target to visit us in some sort of web browser, we can probably own them,” an NSA employee claims in one document describing the hacking techniques. “The only limitation is the ‘how.’”