Do you really want AI to cancel customers?

Computerworld.com reported that “When Anthropic cancelled the AI account of a Swiss company that depended on the service, the move was entirely automated. A lawyer got involved and the account was restored within a day — minus 80% of the data. Oops.”  The December 18, 2025 article entitled “Using AI to automatically cancel customers? Not a smart move” (https://www.computerworld.com/article/4108169/using-ai-to-automatically-cancel-customers-not-a-smart-move.html) included this story:

The tale began when Tom Hoffman, the CEO of a Swiss cybersecurity company called Wicked Design, on Monday received an alert from Anthropic’s system that the company’s entire account had been cancelled due to an automated review that supposedly found unspecified policy violations.

Hoffman knew the best way to get that addressed was with social media support, so he wrote about it on LinkedIn. He also alerted some people who worked at Anthropic, including the head of product legal. The latter responded, saying he’d flagged the issue internally for review. “We are working on the automated ban stuff,” said the Anthropic attorney.  (Hoffman shared multiple Anthropic screen captures with Computerworld.) 

Within a day, the account was restored — sort of. The new alert told him: “Earlier this week, your account was disabled by an automated system for being in violation of our Terms of Service or Acceptable Use Policy. Upon further investigation, we believe this was an error and your account has been reinstated. We apologize for the inconvenience and for your patience.” 

Hoffman briefly celebrated, logged in and found that most of the account — 80% of company projects and data, he said — was missing.

When he asked Anthropic’s automated system how to restore those files, the system replied, “I understand how frustrating this must be. When a user is removed and later re-added to an organization, previous projects and their associated chats are not restored — even if you use the same email address. Unfortunately, there’s no way to restore or transfer these previous projects back to your account once you’ve been re-added.”

But the message also said the files could be restored if the company paid. Apparently, that “there’s no way to restore” reference doesn’t apply if money changes hands. “Reactivating your subscription will restore access to all your previous projects,” Anthropic said. 

(After Hoffman paid the money, the files were restored, he said.)

As amusing — and simultaneously terrifying — as that back-and-forth is, it highlights key issues for enterprise IT.

Any surprised?