Another argument is that the Electoral College bears the moral stain of slavery. But the debate over how to select the president that took place at the Constitutional Convention—whether to do it by popular vote, or via Congress, or another method—was between the large and small states. Slavery wasn’t mentioned, except in an ambiguous remark by James Madison.
The Electoral College was indirectly touched by the notorious slavery compromise only because states were allocated electors based on their senators and congressional districts, and slaves were counted as 3/5ths of a person for purposes of congressional representation. The Electoral College wasn’t in any way dependent on the 3/5th clause or defined by it. The clause was abolished 150 years ago—and yet the Electoral College persisted.