“This was not a vote of no confidence in the Speaker”