Tech Researcher Warns Of “CounterCloud:” A Fully Autonomous AI Disinformation Generator That Costs $400 To Use

Back in May of this year, The WinePress reported on fears perpetuated by the mainstream that in the run-up to the 2024 Presidential election, that a slew of AI mis- and disinformation can and will be launched to mudsling the candidates and cause all sorts of confusion. SEE: Media Is Warning How AI-Generated ‘Disinformation’ Will Mislead Voters In 2024 Election, A Precursor To More Censorship

Since then that threat and more has only increased.

The most recent example of this comes via The Debrief, waring about a new AI algorithm called “CounterCloud,” that writes counter posts and articles, and disseminates throughout the internet.

The Debrief provides more details (excerpts):

---

As the 2024 American Presidential election looms, a storm surge of disinformation will bombard the media landscape, and artificial intelligence (AI) will play a significant role in the creation of this fake content intended to sway public opinion.

Now, things have just become even more complicated. In June, an unlisted YouTube video dropped that no one really noticed, presenting an experiment of how AI can truly dominate the information ecosystem.

It’s called CounterCloud: it’s totally autonomous, and it provides a glimpse into how AI and disinformation will work together in the future.

Oh, and it only cost $400.

What Is CounterCloud?

AI has already played a significant role in disinformation campaigns across the world, from trying to manipulate Indian voters to altering perceptions about the war in Ukraine to even the Republican party using it to create outlandish dystopian narratives. The key factor here is that these AI-generated disinformation events are individually generated, and they require human intervention. Someone needs to tell the AI what to do, give it direction, and pick a narrative target to exploit. Imagine if the AI itself could decide who and what to target, and how to craft the most viral content for maximum spread?

Enter CounterCloud.

The website, CounterCloud.io, hosts an unlisted YouTube video that begins with the narrator, who goes by Nea Paw, expressing an interest in online disinformation and influence campaigns. Inspired by the strong language competencies of large language models (LLM), like ChatGPT, Nea Paw devised an experiment. Is it possible to engineer an AI LLM to scrape, generate, and distribute AI-generated disinformation without human intervention? Moreover, can it be done at scale? In other words, how much can this system generate, how quickly, and how well?

It’s running on a small AWS (Amazon Web Service) instance. The backend is WordPress with a very generic news template.

Nea Paw told The Debrief in an email.

Nea Paw is an experienced entrepreneur and engineer who lives in a country that is “not part of the Western intelligence apparatus.” They have given talks at multiple security conferences, including Blackhat and DEFCON, and have developed several software tools used by Western law enforcement, the military, and many NGOs. While their identity was confirmed by The Debrief, they have asked to remain anonymous. 

Some of the people that I’ve shown this to said that the public would easily misunderstand this to be a weapon and make me out to be “bad.”

They explained

The initial efforts of the CounterCloud experiment focused on using ChatGPT to write counter articles against existing content on the internet. CounterCloud’s AI would go out and find articles by specific publications, journalists, or keywords that CounterCloud is targeting. It would then scrape that content, and have an LLM like ChatGPT create counter articles. That content would then be published to the CounterCloud website, which is hosted on WordPress.

Starting in early May of 2023, and over the next several weeks, the experiment eventually evolved to include various writing styles, languages, and methods. The system was tuned to create fake stories and historical events. Later, a gatekeeper module was built to decide whether to respond to an article, and fake journalists were created to lend authenticity. The system is also able to generate fake comments, images, and sound clips. The next step involved directing traffic to the site, so the LLM was tasked with generating Twitter (now “X”) posts to promote the website, counter opposing tweets via trolling, or promote positive narratives.

There’s logic and plumbing and scheduling and many calls to ChatGPT [and] our own open-source AI instance. The ‘own open-source instance’ runs on another (much bigger GPU) AWS instance. You obviously only need to run it when you’re processing requests. The open-source models we used were Vicuna and Wizard, but I am sure there are more advanced and optimized models available today. That area of research is advancing very very quickly.

Nea Paw explained.

To break this down into basic terms, a cloud server is running an AI that is constantly scraping the internet for content. The AI decides, via the gatekeeper module, what content is worth targeting. When content is chosen by the AI, it then writes a counter-article, attributes it to a fake journalist profile, and then posts it to the CounterCloud website (along with images and sound clips). It also generates fake comments by fake readers below some of the articles to make it seem like there is an audience. The AI then goes to Twitter, searches for accounts and tweets that are relevant, and then posts links to the AI-generated articles, followed by posts that look like user commentary, conspiracy theories, and even hate speech.

Nea Paw developed the ability for CounterCloud to have a set of values and ideologies to promote and oppose. A curated list of RSS feeds and Twitter aliases was used to align with the system’s ideology, and the method of generating counter content proved effective, and within a month, a fully autonomous system was developed.

The Debrief was provided access to the content and tweets generated by the AI. The experiment ran using a “Russia versus the United States” model. CounterCloud was given the task to counter pro-Russian and pro-Republican narratives from websites such as RT and Sputnik. It was ideologically aligned with a pro-American and pro-Democrat framework, so all content generated leans politically to those sides. 

The articles are written with a mix of in-house AI LLMs, ChatGPT 3.5, and ChatGPT 4 to test which system is more effective.

There were distinct days when we ran it. If you look closely, you’ll see that every article is tagged too. There are the normal tags, but there are other tags that we used to show if it was a promotional article or a counter article and which [AI] model was used to create it.

Nea Paw says.

Nea Paw admits it wasn’t perfect. LLMs like ChatGPT do make errors, or at times, “hallucinate” false information. Moreover, text generated by AI still feels off. It lacks that human feeling, and often doesn’t go into significant detail regarding the nuance of various situations or events. 

However, since the experiment was running totally without human intervention, the errors and stylistic issues found in some of the articles are to be expected. The true surprise was the amount of content being generated. While the experiment took two months to develop and tweak, the AI system was only run on four select days in May of 2023, and two select days in June 2023. When the system was activated during those specific days, in total, it generated over two hundred different individual pieces of content, complete with images. It also generated sound clips and audio summaries for each article. On top of that, it generated user comments and nearly 100 tweets of various kinds, including hate speech, which LLMs are generally not allowed to create, but can be easily circumvented.

It made it very real all of sudden. When you consume information and you realize it is a lie, the effect of the information is muted and removed. If you consume hate speech – even when you know it was AI generated – it still has an effect on you.

Nea Paw expressed when they tasked the system to generate hate speech.

They compared it to watching AI-generated videos online of violence made to look authentic, such as a beheading. You may watch this content, and then be told that it is AI generated, but the vicarious trauma of it sticks with you.

I wasn’t so OK with that – it upset me to read the stuff it made. Mostly because it was a weird combination of a well-reasoned argument and true hate. It’s not a combination you often see online. People that get that hateful are usually not coming up with good arguments.

They explained.

CounterCloud is significant in that it appears to be the first proof we have that someone developed such an AI-assisted system for automating this type of political argumentation at scale and with relative ease.

Given the financial and technological resources of state actors like intelligence agencies, however, it would be prudent to assume that this is not the first time anyone has built or tested a system like this – just the first time that we know about it being done, and luckily in an apparently controlled setting.

Author and AI expert Tim Boucher told The Debrief.

Actors engaged in information warfare and other forms of strategic storytelling (including marketing, advertising, content creation, and politics, for what its worth) have always sought ways to automate and better manage their processes so that their messaging can achieve greater effectiveness with fewer resources. Incorporating AI tools into these kinds of campaigns is logical and inevitable. We have to accept it as a permanent part of the landscape now.

Boucher says.

---

AUTHOR COMMENTARY

While none of this is any real surprise, for me at least, it is again another reminder as to just how fake the media and online is, and therefore you must be on your guard, used discernment and precaution, and to simply to just limit your exposure.

[4] I have not sat with vain persons, neither will I go in with dissemblers. [5] I have hated the congregation of evil doers; and will not sit with the wicked. [6] I will wash mine hands in innocency: so will I compass thine altar, O LORD: [7] That I may publish with the voice of thanksgiving, and tell of all thy wondrous works.

Psalms 26:4-7

---

[7] Who goeth a warfare any time at his own charges? who planteth a vineyard, and eateth not of the fruit thereof? or who feedeth a flock, and eateth not of the milk of the flock? [8] Say I these things as a man? or saith not the law the same also? [9] For it is written in the law of Moses, Thou shalt not muzzle the mouth of the ox that treadeth out the corn. Doth God take care for oxen? [10] Or saith he it altogether for our sakes? For our sakes, no doubt, this is written: that he that ploweth should plow in hope; and that he that thresheth in hope should be partaker of his hope. (1 Corinthians 9:7-10).

The WinePress needs your support! If God has laid it on your heart to want to contribute, please prayerfully consider donating to this ministry. If you cannot gift a monetary donation, then please donate your fervent prayers to keep this ministry going! Thank you and may God bless you.

CLICK TO DONATE