> ## Content Index
> Fetch the complete content index at: https://nordisk-ai.ghost.io/llms.txt
> Use this file to discover other available public pages before exploring further.

# The Black Market for AI Chatbots Is Growing Fast
- URL: https://nordisk-ai.ghost.io/the-black-market-for-ai-chatbots-is-growing-fast/
- Published: 2024-10-11T20:09:13.000Z
- Updated: 2026-08-27T17:21:17.000Z
- Description: Illegal AI chatbots on the black market are thriving and can generate large sums of money. A study shows these malicious LLMs can be used to create phishing emails and malicious code, posing significant risks to both individuals and companies.
- Author: Anders Hagen
- Tags: Cybersecurity, AI Security, Generative AI, LLM, phishing, #Migrated-1787841634512, #wp, #wp-post, #Import 2026-08-27 14:41, #Import 2026-09-07 13:41

ChatGPT, with its 200 million active users every week, has helped OpenAI reach a $100 billion valuation. But there's also a parallel market where large sums are being made - particularly in the underworld. According to a study published on arXiv, a preprint server owned by Cornell University, illegal large language models (LLMs) can generate up to $28,000 in just two months through sales on underground marketplaces.

#### Two Types of Malicious AI Models

The study, which examined over 200 examples of malicious LLMs (or "malas") listed on underground marketplaces between April and October 2023, identified two main types of models. The first type is completely uncensored LLMs, often based on open source. The second type involves commercial LLMs that have been jailbroken from their safety guardrails through special prompts.

Xiaofeng Wang, professor at Indiana University Bloomington and one of the study's co-authors, emphasizes the importance of studying these phenomena at an early stage: "We want to stay ahead and prevent major harm from already having occurred."

[See the Study Here](https://arxiv.org/abs/2401.03315?ref=nordisk-ai.ghost.io)

#### Financial Motives Behind Malicious LLMs

While hackers can sometimes bypass the restrictions of mainstream LLMs, these cases are relatively rare. Instead, illegal LLMs have emerged to meet demand, and unsurprisingly, those behind them are interested in making money. "We found that most malas on underground forums exist primarily to make money," says Zilong Lin, another researcher at Indiana University Bloomington.

#### Different Uses for Black-Market LLMs

The malicious LLMs are used in a variety of ways, from writing phishing emails to developing malicious code that attacks websites. A separate study estimates that LLMs can reduce the cost of producing phishing emails by 96%. The capability of these black-market LLMs to carry out their tasks can vary widely, although some are particularly powerful.

Two uncensored LLMs, DarkGPT and EscapeGPT, stood out in the study by producing working code about two-thirds of the time, without the code being detected by antivirus software - giving them a higher chance of successfully attacking a computer. Another model, WolfGPT, has proven highly effective at crafting phishing emails that manage to evade most spam filters.

#### Risks and Countermeasures

Andrew Hundt, a researcher at Carnegie Mellon University who wasn't involved in the study, points out that the authors show that malicious users are able to resell companies' products for harmful purposes. Hundt believes policymakers should require AI companies to develop and implement customer identification policies to verify users' identities. "We also need legal frameworks to ensure that companies building these models and providing these services do so responsibly, in order to reduce the risks from bad actors," he says.

Wang points out that research like his team's is only the beginning in the fight against cybercriminals. "We can develop technologies and provide insights to help," he says, "but we can't stop these phenomena entirely because we lack the resources."