“I personally have always felt that somebody must have had some sense of what was happening at this compound,” said Panetta about the fugitive’s hideout in an interview to air Sunday on CBS’s 60 Minutes.

The terrorist leader was killed by U.S. Navy SEALs in a raid on his compound in Abbottabad, Pakistan in May. The operation was carried out without the cooperation of Pakistani authorities.

“Don’t forget, this compound had 18-foot walls. It was the largest compound in the area. So you would have thought that somebody would have asked the question, ‘What the hell’s going on there?’” said Panetta.