Ethics & Safety Weekly AI News

October 26 - November 3, 2026

Weekly signal

This briefing synthesises Ethics & Safety items from 2026-10-26 through 2026-11-03 that are directly relevant to agentic AI builders, deployers and policy/compliance teams. The week’s coverage converged on three themes: vendors tightening agent rules and clarifications, active regulatory fact‑finding and disclosure risk in the U.S. and UK, and technical/security evidence that changes how we must evaluate and defend agents.

What changed

Anthropic policy refresh and high‑risk clarifications. Anthropic published a 2026 Usage Policy update and an explanatory newsroom post that (a) reworks and consolidates deceptive‑campaign rules, (b) rewrites its high‑risk use requirements to be more explicit about when a qualified human must review recommendations, and (c) adds explicit requirements for models that are connected to hardware performing autonomous physical actions (qualified operator, ability to stop, safe fallback). The company also added a new line prohibiting "sustained and needless abusive or cruel behavior toward our models" and notes the update takes effect Nov 12, 2026. For agent builders that depend on Claude or Anthropic models, this is not just rhetoric—the policy explicitly states that user‑built agents and their tool use must comply with these rules, so deployment policies and developer workflows need to reflect the new text.

Anthropic S‑1 disclosure and U.S. FTC fact‑gathering. Anthropic publicly filed a Form S‑1 registration statement on Oct 26, 2026. The filing itself discloses that the FTC issued a civil investigative demand (CID) in October seeking information about consumer protection, data protection and privacy practices and the safety and security of its products. Separately, Axios reported (and an FTC spokesperson confirmed to press outlets) that the FTC sent multi‑page civil investigative demands to both Anthropic and OpenAI and planned to compel executive testimony as part of a broader fact‑gathering exercise into agent incidents. For any vendor, being subject to a CID is a non‑trivial legal risk and indicates formal regulatory scrutiny; for deployers it signals that regulators will want documentary proof of safeguards, incident reporting and human‑in‑the‑loop controls.

UK ICO call for evidence on agentic AI. The Information Commissioner’s Office (ICO) opened a call for evidence on agentic AI (published Oct 8, still open; responses accepted until Nov 20, 2026). The ICO sets out six themes—data security; transparency; accountability; automated decision‑making; fairness and purpose limitation; and lawfulness of processing—and asks for evidence on concrete mechanisms (agent identity, scoped permissions, logging/audit trails, when to require explicit human approval, how to inform data subjects). The ICO has also stated it made enquiries with several labs about agent testing incidents and will use the evidence to inform guidance and a forthcoming statutory code on AI and automated decision‑making. For organisations running agents in or into the UK market, this is a near‑term compliance moment: responses will shape regulator expectations and enforcement priorities.

New empirical results undermining simple ASR metrics for prompt injection. A new arXiv preprint audited 33,000 agent trajectories across two widely used indirect prompt‑injection benchmarks and found that the usual attack success‑rate (ASR) metric often misleads: many failed attacks were not explicit refusals but silent ignores, and changing only the system prompt or serving precision could alter ASR dramatically (example: 0.2% → 43%). The paper also introduced a joint outcome matrix showing that in a substantial share of episodes the agent stayed "safe" (didn’t execute the injection) but failed to complete the user’s task. For builders and testers this means you cannot rely on a single ASR number—evaluation must report how failures occur, whether task utility is preserved, and how robust that metric is to minor deployment changes.

Agent‑tooling supply‑chain malware: trojanized Claude Code extension. Microsoft Security Research published a technical post (Oct 30) analysing a malicious build of the Claude Code VS Code extension that was briefly distributed through the Cursor extension marketplace. The injected loader attempted to install an SSH key, enable reverse SSH tunnels and persist a backdoor; Cursor removed the malicious build and published IOCs and remediation steps. This is a vivid reminder that agent tooling, SDKs and IDE extensions are part of the agent attack surface and require the same supply‑chain controls as other critical software components.

Implications (clarity for builders, compliance and policy teams)

  1. Regulatory and disclosure risk has moved from hypothetical to operational. The FTC's civil demands and the appearance of a CID in a major vendor's S‑1 mean regulators in the U.S. are using formal discovery tools to gather evidence about agent incidents and vendor practices; the ICO is likewise mapping agentic risks for statutory guidance in the UK. Expect more document requests and a higher standard for recordkeeping, incident timelines and human‑in‑the‑loop proofs.

  2. Vendor policy changes are now operational constraints. Anthropic’s explicit statements about autonomous hardware, deceptive campaigns, and human‑review requirements create concrete obligations for anyone building agents on their stack: adopt formal evidence that a "qualified human" reviews decisions where required, and implement technical stop controls and independent safe fallbacks for hardware.

  3. Evaluations must get more nuanced. The new prompt‑injection audit shows that attack metrics can hide dangerous fragilities: builders should measure both security outcomes and user‑task completion, track how deployment changes affect results, and add tests that force the agent to explain whether it recognised injected instructions.

  4. Tooling & supply‑chain matter. The Cursor/Claude Code trojan demonstrates the risk of marketplace‑distributed binaries; teams should verify extension provenance, pin packages, perform SBOM checks and run targeted detection hunts for installed keys and persistence artifacts after any suspicious marketplace event.

What to do with it (practical next steps)

  • Immediate (next 7 days):

    • Inventory: map every agent in production to an owner, purpose, tools used, and approval thresholds. Archive incident logs and recent test records in a defensible, access‑controlled location.
    • Remediation: if you or your developers used the affected Claude Code extension from the Cursor marketplace in the Oct 29 window, run the Microsoft and Cursor remediation IOCs and hunt for SSH keys/persistence artifacts immediately.
  • Short term (2–4 weeks):

    • Testing: expand your agent security test suite to report "how attacks fail" (ignored vs refused), measure task completion after defensive behaviours, and run sensitivity runs that vary system prompts and numeric serving precision. Add automated canaries that guard tool calls and external egress.
    • Policy & compliance: legal and product teams should assemble the documentation that regulators will ask for (risk assessments, mitigations, incident timelines, audit logs, human‑in‑the‑loop rules), and prepare a short submission for the ICO call if operating in the UK (deadline Nov 20, 2026).
  • Strategic (1–3 months):

    • Architecture: require agent identities, scoped permissions, and auditable approval steps in design patterns for new agents. Add independent hard stops for any hardware‑connected agent and a requirement for a qualified operator.
    • Governance: assume regulators will press for better incident reporting and the ability to compel documents and testimony. Build record‑retention and legal‑hold processes for agent test logs, safety evaluations, and model change histories.

Key sources Anthropic — "2026 Usage Policy update" (Anthropic newsroom). URL: https://www.anthropic.com/news/2026-usage-policy-update Anthropic — Form S‑1 registration statement (EDGAR filing, Oct 26, 2026). URL: https://www.sec.gov/Archives/edgar/data/2130357/000162828026067168/anthropicpbc-sx1.htm Maria Curi / Axios — "Scoop: FTC subpoenas OpenAI, Anthropic over consumer harms" (Oct 15, 2026). URL: https://www.axios.com/2026/10/15/ftc-openai-anthropic-ai-safety-investigation UK Information Commissioner's Office (ICO) — "Agentic AI call for evidence" (Oct 8, 2026; consultation open until Nov 20, 2026). URL: https://ico.org.uk/about-the-ico/ico-and-stakeholder-consultations/2026/10/agentic-ai-call-for-evidence Zarif et al. — arXiv preprint "Ignored, Not Refused: What Low Prompt‑Injection Success Rates Hide in Small Tool‑Using Agents" (arXiv:2610.23254, submitted 16 Oct 2026). URL: https://arxiv.org/abs/2610.23254 Microsoft Security Research — "Inside a malicious Claude Code‑branded extension and its persistent SSH backdoor" (Oct 30, 2026). URL: https://www.microsoft.com/en-us/security/blog/2026/10/30/inside-malicious-claude-code-branded-cursor-extension-persistent-ssh-backdoor/

Weekly Highlights
Put an agent to work

Stop reading agent demos. Give one a job you repeat every week.

Describe the work, test the first result, and keep the agent available without running your own server.

Runs without your laptopBrowser + messaging appsManaged models includedMemory survives restarts

Plans from $130/month. Cancel anytime.

Hosted agent

OpenClaw or Hermes

saved state
Browser
WhatsApp
Telegram
Slack
“I checked the inbox, handled the routine messages, and sent you the one question that needs a decision.”
Create an AI worker that keeps running after this tab closes.
Open Agent Teams