The man was pulled from the water and given CPR but later died in hospital from his injuries, police in the town of Wellfleet said.
The Cape Cod beach was closed to swimming Saturday after the bad incident.
Beach-goers carry a shark attack victim down Newcomb Hollow Beach.More news: Man drives auto into crowd in southern France leaving two injured
Local fisherman and surfer Joe Booth said he was on the shore and witnessed the attack. The man's friend had to drag him to shore.
Great white shark sightings in Cape Cod have soared this year, with experts blaming increased seal levels close to the region's beaches. "Fifteen feet wide. I saw a tail and a lot of thrashing", Booth said. "This has turned into Amity Island real quick out here", he admitted to WBUR.
The last shark attack in the state to result in a death before this happened in 1936.
State police spokesman Dave Procopio said the victim's family has been notified, but police are withholding his identity for now. The Cape Cod District Attorney's office and the Massachusetts State Police are both investigating, Hurley said. Great white sharks, apex predators, inhabit the waters off the Cape.More news: Florence, a wet and unwanted visitor, besieges Carolinas
On Aug. 15, a great white shark bit a NY man swimming off Long Nook Beach in Truro, Mass.
Newcomb Hollow Beach in Wellfleet, MA. appears in this undated stock photo.
Hayley Williamson, a Cape Cod resident and former lifeguard who was on the beach at the time, was in disbelief after the man was rushed into an ambulance. He's now recovering in a Boston hospital.
Mr Booth said the attack victim and his friend were boogie-boarding when the attack happened. "Right spot, wrong time, I guess". A shark's powerful jaws clamped down William Lytton's leg a few yards from the shoreline. Great White Sharks are predators and should be considered unsafe.More news: Buckeyes edge TCU in neutral site thriller
Saturday's incident was the second shark attack off of Cape Cod this summer, though multiple sharks have been spotted in the area in recent months, according to WBUR.