But many also point out that Libya’s Muslim Brotherhood, as well as Islamist militia leaders like Abdel Hakim Belhaj — once a terror suspect tortured and extradited by the CIA, and now the head of one of the better organized political parties — never had the popularity that their counterparts had in neighboring Egypt. After all, Egypt’s ousted authoritarian, Hosni Mubarak, had allowed the Brotherhood to cultivate charity networks and even run for parliament. It may have all been part of a decades-long scheme to convince Egyptians and Egypt’s allies that the country’s options for governance were limited to two extremes, but the end result was that the Egyptian Muslim Brotherhood was more prepared. Gaddafi on the other hand never tolerated the Islamists — or even weak political parties. Men with beards or political sympathies were so regularly monitored and rounded up, that many Libyans said it was a crime to be religious or have opinions. Few bothered to try.

That rise from exile and repression may have given Libya’s Islamists an early boost when it came to political organizing during the uprising, but it also meant that they were starting at square one — just like everybody else. When TIME met with one of the Muslim Brotherhood’s senior leaders in Tripoli late last year, he admitted that he had no idea how many members the group even had inside the country. “Yesterday was the first time we met in Tripoli not underground,” Alamin Belhaj said shortly after the rebels took control of the capital. “The Brotherhood has been around for a long time, since 1951. But after Gaddafi came, it vanished.”