The 66-year-old billionaire bought Little St. James Island off this U.S. Caribbean territory more than two decades ago and began to transform it — clearing the native vegetation, ringing the property with towering palm trees and planting two massive U.S. flags on either end. When guides took scuba divers to spots near the island, security guards would walk to the water’s edge.

It was off-putting to residents of St. Thomas — a lush tropical island east of Puerto Rico with winding roads through mountains dotted with dainty Danish colonial-era homes. Then, when Epstein pleaded guilty in a 2008 to soliciting and procuring a minor for prostitution, his need for privacy began to appear more sinister.

“Everybody called it ‘Pedophile Island,’” said Kevin Goodrich, who is from St. Thomas and operates boat charters. “It’s our dark corner.”

Many people who worked for Epstein told The Associated Press this week that they had signed long non-disclosure agreements, and refused to talk.