According to AFP news agency, Palij confessed when investigators arrived at his door in 1993, saying: "I would never have received my visa if I told the truth".
Palij's US citizenship was revoked in August 2003 based on his alleged involvement in war crimes and his immigration fraud.
As Palij admitted to Justice Department officials in 2001, he was trained at the SS Training Camp in Trawniki, in Nazi-occupied Poland, in the spring of 1943.
But none of the European countries to which he could have been deported would accept him, sparking concerns he would be allowed to live out his final years peacefully at his Jackson Heights Home, which has witnessed protests from anti-Nazi protesters. He said the new German government, which took office in March, brought "new energy" to the matter.
The deportation came after weeks of diplomatic negotiations.More news: Australian PM Malcolm Turnbull on leadership contest
Interior to reverse pesticide ban at wildlife refuges MORE last August, urging him to negotiate a deportation agreement for Palij so that their constituents were not forced to live beside "a painful reminder for Americans who fought against the Nazis or lost loved ones in the Holocaust".
The Bild newspaper reported that German prosecutors had opened a criminal investigation against Palij in 2015 but closed the case for lack of evidence.
Palij, with a fluffy white beard and a brown, newsboy-style cap atop his head, was wrapped in a sheet as the agents carried him down a brick stairway in front of his home and into a waiting ambulance.
He was granted United States citizenship in 1959 after lying about his Nazi service to immigration officers.
Representatives for Palij didn't respond to ABC News request for comment.
The Frankfurter Allgemeine Zeitung said Palij had arrived by military plane at Duesseldorf airport in western Germany and taken to a home for the elderly in the area.More news: Are Contact Lenses a New Hazard to Our Sealife?
"He (Trump) told me directly to make it a priority, to get the Nazi out", Mr Grenell told reporters. He also said he joined the Nazis only to save his family. Documents subsequently filed in court by the Justice Department showed that men who trained at Trawniki participated in implementing the Third Reich's plan to murder Jews in Poland, code-named "Operation Reinhard".
The United States had asked Germany and other nations to take Palij in the past, but those requests were repeatedly denied.
Jakiw Palij, former Nazi concentration camp guard, lives in the Queens borough of NY.
After falsely telling authorities he spent the war at his hometown in Germany, Palij gained entry into the U.S.in 1949. Palij said he never served in the military.
Germany has jailed former Nazi camp guards, despite their old age, in recent years.
Palij was finally deported thanks to the work of Justice Department "Nazi hunter" Eli Rosenbaum, whose efforts have been praised by Attorney General Jeff Sessions. ICE deports Nazi who was supposed to be deported in 2004, but neither George W. Bush nor Barack Hussein Obama had been able to convince Germany to accept him back.More news: Multiple rockets hit Afghan capital Kabul, clashes underway
Since 2017, Poland has been seeking the extradition of Ukrainian-born Michael Karkoc, an ex-commander in an SS-led Nazi unit that burned Polish villages and killed civilians during the war. The 99-year-old who now lives in Minneapolis was the subject of a series of 2013 reports by the AP that led Polish prosecutors to issue an arrest warrant for him.