One possibility is that this machine is super-capable, exceeding our human capacity for cognitive or analytical tasks. Such an AI might be exceedingly hard to understand, either in terms of its underlying motivation or because of practical barriers of communication bandwidth. For this device, talking to us might be like talking to an infant. Or trying to discuss the collected works of Shakespeare using pictographs. An alien system optimised for processing vast data streams might not even be able to downgrade its pace enough to notice that we’re trying to talk, whether we use technology or not.
An extraterrestrial (ET) AI could also be seriously intimidating and scary simply because of its machine nature: a thing animated from non-living pieces, just like the classic tale of the golem moulded from clay or mud. By comparison, while a biological alien might be shocking, it would surely have some traits in common with us. We could convince ourselves that evolution leads to recognisable, even sympathetic behaviours and intentions. An artificial entity need not follow all of those evolutionary rules, taking alienness to a whole other level.