China is a communist police state that places the good of the state above the good of any particular individual. That results in a nation where anyone who speaks critically about the government or its leader Xi Jinping is liable to get a visit from the police and maybe disappear until they adjust their attitude.
There was always a limit to how many people the country could effectively surveil at once and to how effectively it could respond to challenges. But artificial intelligence might put an end to those limits. In theory, big brother could be watching and keeping track of challenges to the ruling party all the time. Those same tools could go beyond watching to creating propaganda designed to influence and counteract any messages the CCP doesn't like. Leaked documents, which apparently came from an unhappy employee, were examined by researchers at Vanderbilt University.
One company’s internal documents show how it has undertaken influence campaigns in Hong Kong and Taiwan, and collected data on members of Congress and other influential Americans.
While the firm has not mounted a campaign in the United States, American spy agencies have monitored its activity for signs that it might try to influence American elections or political debates, former U.S. officials said...
A new technology can track public debates of interest to the Chinese government, offering the ability to monitor individuals and their arguments as well as broader public sentiment. The technology also has the promise of mass-producing propaganda that can counter shifts in public opinion at home and overseas.
The company in question is called GoLaxy. There's a company of the same name that produces an app for playing the game Go, but this appears to be a separate AI company with the same name. Naturally, they deny they are up to anything threatening.
In a statement, GoLaxy denied that it was creating any sort of “bot network or psychological profiling tour” or that it had done any work related to Hong Kong or other elections. It called the information presented by The New York Times about the company “misinformation.”...
After being contacted by The Times, GoLaxy began altering its website, removing references to its national security work on behalf of the Chinese government.
Think of GoLaxy as personal, targeted propaganda which is not limited by manpower issues or cultural barriers.
In traditional influence operations — think Russia’s social media efforts in 2016 to spread chaos in the United States — disinformation had to be made piece by piece. Automated bots could amplify certain messages, but countries like Russia needed to build troll farms, operations using dozens or hundreds of people, to write fake posts, sometimes in halting English. The disinformation the troll farms could spread was limited.
GoLaxy’s technology has the potential to upend the influence business.
The addition of AI means people with wrong opinions can be identified and targeted almost instantly.
What sets GoLaxy apart is its integration of generative A.I. with enormous troves of personal data. Its systems continually mine social media platforms to build dynamic psychological profiles. Its content is customized to a person’s values, beliefs, emotional tendencies and vulnerabilities. According to the documents, A.I. personas can then engage users in what appears to be a conversation — content that feels authentic, adapts in real-time and avoids detection. The result is a highly efficient propaganda engine that’s designed to be nearly indistinguishable from legitimate online interaction, delivered instantaneously at a scale never before achieved.
While the documents offered no specific examples of these conversations, they describe how the technology develops personalized content. By extracting user data and studying broader patterns, A.I. can build synthetic messaging designed to appeal to a wide spectrum of the public. It can adapt to a user’s tone, values, habits and interests, according to the documents. Then it can mimic real users by liking posts, leaving comments and pushing targeted content.
They call this the GoPro system and the leaked documents suggest it has already been deployed to work for the Chinese Ministry of State Security. The documents also make clear that the ultimate target of this new tool isn't the domestic audience or even Taiwan, it's the United States.
Evoking a phrase used by Mao Zedong, the company suggests that its technology will be essential to help China prevail over the West.
The GoPro system, the company says, will become a “technological platform that can truly tell China’s story, amplify China’s voice and expand China’s influence, as well as providing comprehensive technological support to quickly make ‘the easterly wind overpowering the westerly wind.’”
One document claims the company has collected data on 117 members of congress, all of them Republicans. Plus another 2,000 political public figures and 4,000 "influencers" on the right. What do they plan to do with all of this information? The documents don't say. But as AI's ability to monitor and control conversations improves, it could be a real concern. The only thing that's certain is that this is now part of China's influence arsenal and that they see the US as their chief rival. At some point it seems inevitable they will turn this on us.
Join the conversation as a VIP Member