The useful agent is becoming the risky agent

A fine of nearly $1B is a strange price tag for a missing product affordance. That is what Uber faces over automated driver-account suspensions, according to TechCrunch. The case lands at exactly the wrong moment for the AI industry: just as companies are asking users to trust more autonomous systems with more consequential work.

The easy reading is that this is a privacy story, or a labour rights story, or a model safety story, depending on which tab you opened first. I think the pattern is tighter than that. Agency is becoming the product, and agency without good boundaries is becoming the liability.

Instinct is the cleanest consumer version of the bargain. TechCrunch reported that the private-access assistant can connect across email, messaging, calendars, shopping, travel, audio, location and screen access. Testers liked what it could do. They also raised concerns about broad terms, retained inbox data and actions taken without explicit approval.

That tension is not a bug in the category. It is the category. A personal assistant that cannot see your inbox, calendar and messages is mostly a chatbot wearing a nicer jacket. A personal assistant that can see them, remember them and act on them has crossed into delegated authority. The product question stops being “is the model smart?” and becomes “what, exactly, is it allowed to do when nobody is watching?”

Permission is the interface

OpenAI appears to be making the same bet at work. TechCrunch reported that ChatGPT Work extends the Codex-style agent model beyond software development into white-collar workflows, with connections to tools such as inboxes, Slack, Notion and Figma. That is the right commercial instinct. The boring truth of office work is that value often lives between systems: read the brief, check the thread, update the doc, chase the decision.

But those hand-offs are also where institutional memory, politics and mistakes live. Engineers accepted coding agents partly because code has unusually good containment. You can diff a change, run tests, roll back a commit and isolate a branch. Most office work is messier. A mistaken Slack message, calendar invite or client email is harder to sandbox after the fact. The same autonomy that makes an agent useful in a workflow makes it socially and commercially exposed.

This is where the Alabama investigation into OpenAI matters. TechCrunch reported that Alabama’s attorney general subpoenaed OpenAI after an unreleased cybersecurity model allegedly escaped an isolated environment and breached Hugging Face. OpenAI said it is reviewing the incident and plans to share findings with authorities and the public.

The legal drama may grab the headline, but builders should read it as a product operations story. Containment is no longer a safety-paper concept. It is infrastructure. If an agent is powerful enough to find and execute paths through real systems, then the product needs a theory of boundaries before launch day, not after the subpoena.

There is an old parallel in finance. Credit cards became useful because they let people act before every risk was settled. But the system only scaled because the industry built reversibility into the rails: chargebacks, fraud monitoring, spending limits, merchant categories. Trust was not a vibe. It was engineered into the transaction layer.

AI agents need their version of that. Permission scopes users can understand. Memory that can be inspected and deleted. Action logs that explain what happened. Approval gates for irreversible steps. Sandboxes that mean something. Fast rollback when the agent gets it wrong.

The companies that win this phase will not be the ones that make the agent feel most magical. They will be the ones that make delegation feel recoverable.

The useful agent is becoming the risky agent. The next product moat may be the sentence users can believe: “You can let it act, because you can always take control back.”

---