Police sources reportedly said the shooter fled the scene in a blue pickup truck.
The New York fire department (FDNY) was called to a home in Staten Island at 9.18pm local time after a caller reported hearing six or seven gunshots.
'The man was on the ground face-up.
They said distraught family members ran outside after the shots were fired. Papa!' as a woman shouted into her phone: 'Why doesn't the ambulance come?More news: Abbas aide blasts U.S. report that drops references to Israeli 'occupation'
Cali, who had roots in Sicily and ties to the Sicilian Mafia, was the highest-ranking organized crime figure to be murdered in New York City since 1985, when Paul Castellano was assassinated outside Sparks Steakhouse in Manhattan, according to WPIX-TV.
The organization reportedly focused its efforts on heroin and Oxycontin trafficking under his leadership.
Gambino family mobster Francesco "Frank" Cali has been murdered in what local media say is the first major NY gang killing in more than 30 years.
He was described by mafia watchers as the opposite of the exuberant former boss John Gotti because "no one ever sees him". He served a 16-month sentence.More news: Google Hardware makes cuts to laptop and tablet development, cancels products
The Gambino criminal organisation is one of the "Five Families" of organised crime in NY, alongside the Bonanno, Colombo, Genovese and Lucchese families.
"He was a real quiet old-school boss", a police source told The Post.
The Gambino crime operation is one of the five historic Italian-American Mafia families in New York, Reuters has previously reported, and it makes money through violence and extortion. That hit was ordered by Gotti, who later took over the family until being convicted of racketeering and Castellano's murder in 1992. John Gotti and his turncoat underboss, Sammy "Bull" Gravano watched the action from Third Avenue.
The family was headed by Domenico Cefalu until 2015, when Frank Cali took over as capo.More news: Beto O'Rourke launches 2020 bid for president, Texas TV station confirms