Many people consider Ground Zero “hallowed ground.” “One has to,” he insists. “How could it not be?”

But the encounters with widows and relatives were “brutal, absolutely brutal.” Their interests weren’t aligned. He wanted to build office towers and they wanted to commemorate their loss. Though nearly 3,000 people were killed at the old World Trade Center, “at the end of the day, we have to recognize that there are millions down here,” he says. “People live down here, people work down here. They desperately want to have this rebuilt. So we have to do what’s good for New York.”

The spiritual status of this site and its surroundings is once again a contentious issue in the wake of plans to open an Islamic cultural center two blocks north of here. “The last thing I need to do is get involved in that one,” says Mr. Silverstein, ducking a question about his own views. I suggest he might empathize at least with the political pickle the developer there finds himself in. Mr. Silverstein isn’t, for a change, happy to talk. “He’s got his issues, I’ve got mine,” he says. End of discussion.