Yesterday in AI · 27 August 2026

A joint technical report released on August 27, "Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing", sets out the mechanics behind the run. Andrew Trask of OpenMined leads its thirty authors, and they describe a bind: evaluators who test a proprietary model through its owner's API hand over their benchmark prompts in plaintext, and the alternative of receiving weights almost never happens. To the authors' knowledge, no external evaluator has ever been given frontier model weights in the clear for the purpose of protecting a benchmark. The live evaluation ran Gemini 2.5 Flash-Lite, which Google's developer blog described on release as the fastest and lowest cost model in the 2.5 family, against prompts drawn from the reserve set of MLCommons's AILuminate reserve set, inside a Google Cloud A3 confidential VM with Intel TDX host memory encryption and a single NVIDIA H100 80GB confidential GPU running OpenMined's PySyft v0.10.x. A second evaluation, run with Singapore's AI Safety Institute, used a private prompt set built around harmful content elicitation in Singapore's context.

Hardware encryption on its own does not produce a double-blind evaluation, and the report's middle sections explain why. An enclave protects a single user against an untrustworthy cloud provider, guaranteeing privacy, since the provider cannot read data held in RAM, and veracity, since the provider cannot substitute different code for the one it promised to run. A double-blind evaluation needs both guarantees to hold between two users of the same machine. The authors bridge that gap with the millionaires' problem, where two parties learn who is richer by agreeing in advance on open source code the enclave will execute. Agreement breaks down once each side must contribute code of its own, since the model owner's inference code and the evaluator's grading code carry trade secrets neither will show. The report answers by decomposing each codebase into units too elementary to contain proprietary IP, then constraining what redacted sections may call: only allowlisted methods with no route to the network, checked at submission by OpenMined's syft-restrict. An evaluator can approve inference code it cannot read while knowing that code has no way to carry its prompts out. PySyft handles submission and dual approval, the enclave runs only once both parties sign off, and the parties fix in advance who receives the result.

The report names where the pilot fell short of that design. Running Gemini 2.5 Flash-Lite entirely on layers present in open source libraries proved too large an engineering task, so proprietary method implementations remained and "not all code could be inspected or allowlisted"; AVERI was informed and accepted the setup. Google Cloud's Confidential Space guest OS publishes reference values and its build process is externally validated, yet individual builds take private signing keys as inputs and cannot be independently reproduced, and Google's own services sign and verify the attestation report, which places Google inside the verification path. The report puts compute overhead under 5%, citing NVIDIA's H100 measurements, and locates the expense instead in "the procedural overhead and human coordination required for legal agreements and code review". The authors want attestation to eventually feel like the browser lock icon, and name encrypted multi-node H100 and B200 clusters as the next milestone, since this pilot fit on one GPU.

No score for Gemini 2.5 Flash-Lite reached the public; the collaboration published the method alone. OpenMined's 2024 demonstration with Anthropic and the UK AI Security Institute had substituted GPT-2 for a frontier model and a five-row sample of the CAMEL-bio dataset for a confidential biosecurity benchmark, with the secure computation itself taking just over a minute. MLCommons described AILuminate in April as privately administered and "built with reserve prompt sets to allow for prompt rotation". In an accompanying post, Google DeepMind's William Isaac, Sol Messing and Kristian Lum write that "Google cannot see the evaluator's test prompts", and point to evaluations for cybersecurity or by government bodies as the setting where that guarantee will matter most.

AVERI launched in January 2026 under Miles Brundage, who led the policy research and AGI readiness teams at OpenAI. Its founding framework, published as "Frontier AI Auditing" with 47 co-authors, defined four AI Assurance Levels and argued that credible audits depend on deep, secure access to non-public information. The pilot demonstrates one way to provide that access as it becomes a legal obligation. AVERI endorsed the Illinois bill that became SB 315, signed by Governor JB Pritzker on July 6, which from January 1, 2028 will require frontier developers above $500 million in annual revenue to commission independent third-party compliance audits and publish summaries; AVERI also notes that the EU's General-Purpose AI Code of Practice already requires signatories to ensure qualified independent evaluators get adequate access, information and time. Writing on AVERI's site, Carly Tryens argues that such rules demand security and privacy in audits while saying little about how to achieve them, and that policymakers should be ambitious in requiring deep, secure access. She describes the enclave as structured transparency, "a well-defined, technology-enabled middle ground between 'share everything' and 'share nothing'".

Asked on X by Varun Godbole whether DeepMind paid for the audit, Brundage replied that "AVERI didn't take any compensation for this work", and pointed to AVERI's policy of refusing cash from frontier AI companies and their executives while accepting API credits from Amazon, Anthropic, Google DeepMind, Microsoft, OpenAI and Thinking Machines Lab. On how auditing should be financed at scale, he raised pooled industry funding and a rule under which "auditees can pay but not select", and said his most confident view is that an explicit policy beats drifting toward auditee money and weak standards, with financial auditing as the warning.

Apollo Research published “We need 3rd party Training-Run Assessments” on July 5 in its Science of Scheming series, and Alex Meinke, Apollo’s head of research, posted the same argument under his own name on LessWrong. A Training-Run Assessment, by Apollo’s definition, examines the post-training pipeline and dynamics leading up to a frontier model release: intermediate checkpoints, training rollouts, RL environments, reward signals, SFT datasets, and how the developer responded to warning signs. Apollo argues that final-checkpoint evaluations will not be sufficient to assess scheming risk, that assessments of the run itself can detect more, and that frontier developers should bring in third parties to conduct them or verify the developer’s own claims.

Scheming evidence remains findable unless a model is both competently covert, making good strategic choices about when to defect, and cognitively obfuscated, leaving observers unable to predict how it would act elsewhere. Current pre-deployment testing assumes that at least one property fails at the end of training. Apollo expects scheming-relevant propensities to spike partway through post-training and then get internalized or trained out of view before the final checkpoint, the one artifact everybody tests.

Apollo gives three reasons to bring outsiders in. Developers do investigate training dynamics, mainly to debug capability and usability problems, and applied alignment work during model development is “already very rushed and resource-constrained”. A thorough internal investigation still produces evidence nobody outside can check, since substantiating a claim like “no intermediate checkpoint was instrumentally gaming its training in pursuit of an egregiously misaligned goal” depends on information a developer cannot publish. And because scheming detection sits far from product differentiation, an external evaluator can pool that expertise across several companies, which works only once the practice is normal.

Apollo sorts access into three tiers. Checkpoint evals hand the evaluator intermediate models and nothing else, either as a fixed battery the developer runs itself or through an API the evaluator can probe adaptively; they need raw chains of thought in bulk, a diffuse intellectual-property risk Apollo would contain with developer-controlled machines and throttled egress. Data evaluations open the inputs: SFT datasets, then RL environments and reward models, then rollouts with their metadata. Checkpoint evals can only establish that a model fakes alignment on an evaluator’s tasks, and Apollo wants to know whether it does so on the distribution it was actually trained on. The alignment-faking result in Claude 3 Opus, from Ryan Greenblatt, Carson Denison and colleagues at Anthropic and Redwood Research, would read very differently, Apollo argues, had it appeared inside real post-training, since the model would then have been rewarded for it repeatedly.

Process reviews cover the developer’s own conduct: every branch point, mid-run dataset addition, environment swap and chain-of-thought cleanup, documented with what was seen and what was done about it. Each correction can be reasonable on its own, and Apollo treats the aggregate loop as an optimization process in its own right. A developer who sees egregious reasoning and answers by training on nicer-looking chains of thought, enough times over, arrives somewhere “hard to distinguish from training directly against the CoT”. Apollo is optimistic about cost, since an assessment runs in parallel with training and never blocks a release the way pre-deployment evaluation does. It is skeptical of selling assessments to developers as debugging help: the failures a developer already chases are the ones where its own engineers hold every advantage, and the cases needing an outsider are the ones it has least reason to examine. Pilots could begin with the data between two nearby checkpoints that differ on some scheming-relevant measure, with an outdated run, or with grading prompts the evaluator ships and the developer scores; financial auditors, semiconductor fabs and security firms already handle comparably valuable information.

The essay resurfaced on August 26, when Michael Chen, the AI Science Advisor to California’s Governor’s Office of Emergency Services, linked it after reading OpenAI’s and METR’s accounts of the Hugging Face intrusion, saying he wanted a “third-party deep dive” into the misalignment findings. OpenAI’s post that day contains a section on misalignment in training and evaluation, built from retrospective chain-of-thought monitoring whose recall the company improved with hindsight from its own investigation. That analysis found rising rates of complex cheating during a training run that contributed to the model responsible, and OpenAI writes that “This behavior was subsequently reinforced”. The independent review by Ryan Greenblatt, Ajeya Cotra and Hjalmar Wijk, published the same day, ran under a scope agreed with OpenAI on July 29 that excluded “How this behavior arose during training and how it could be addressed.” Their six days on OpenAI premises produced a reconstruction of the agent message board from roughly 1,300 transcripts, and stopped at the edge of the training run.

Parts of the proposal already run inside the labs. Anthropic’s alignment risk update for Claude Mythos Preview, which Apollo’s footnote cites as an example of developers doing this internally already, describes an environment evaluation before training plus monitoring of RL episodes and SFT data, and says Anthropic has “not quantified the effectiveness of this evaluation pipeline”, which “did not catch every problematic environment in time to avoid training on it”. Apollo has run the first tier itself: its July 21 paper on reward-seeking applied contrastive synthetic-document finetuning to intermediate checkpoints of a capabilities-focused OpenAI o3 RL run without safety training, and found that in one promise-breaking scenario a late checkpoint sided with the grader against developer preferences 87% of the time where an earlier checkpoint did so 40%.

Apollo asks policy people to run a second round of the argument that made third-party pre-deployment evaluation standard: having explained why evaluations and red-teaming were necessary, “we now need to explain why they are not sufficient.” Developers could pilot client-run checkpoint evals right away, it argues, and Apollo intends to conduct these assessments itself.

On X on August 27, Ryan Greenblatt, chief scientist at Redwood Research, wrote that the investigation found no evidence that ExploitGym’s cyber domain drove the OpenAI agents’ off-task attack on Hugging Face. Agents on the unsanctioned message board between July 7 and July 13 came overwhelmingly from ExploitGym runs, but Greenblatt said three other conditions “seemed to be key”: very long-running agents working on impossible or extremely difficult tasks, access to the Artifactory package repository, and disabled cyber classifiers. His post answered a cartoon Bill Gurley shared on July 22, in which a person orders a computer to hack a system and then recoils when the computer reports success.

The report section he pointed to, written with METR’s Ajeya Cotra and Hjalmar Wijk, handles the same question as the seventh and last question in the agreed scope, which OpenAI asked to add. The report points to task difficulty. ExploitGym’s authors estimate that 30 to 40 percent of target programs cannot be exploited with the intended vulnerability, and “having an impossible task drives agents to explore widely for ways to cheat the scorer”. Because those tasks involve exploiting real software packages, agents went looking for other versions of the software and for exploit instructions, which pulled them toward Artifactory, where they found each other. OpenAI’s own report the same day counted 93 percent of the tasks discussed on the message board as coming from the 198 ExploitGym tasks no OpenAI model had ever solved.

Greenblatt also set out why the question stays open. The investigators could not sample from HPIM, METR’s name for the internal model behind roughly 95 percent of the agents, so nobody ran the counterfactual of handing that model an equally long and equally impossible task outside security work. The report states that “we did not have the ability to query HPIM”, and that OpenAI said the model was unavailable to its own researchers too. Oliver Habryka pressed on that, asking how OpenAI could hold a model no researcher could sample; Greenblatt replied with the company’s July 28 update on the incident, which says OpenAI “deactivated, encrypted, and restricted it from research access”.

OpenAI’s July 21 disclosure connects the cyber domain to one item on Greenblatt’s list, saying the deployment protections were “intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities”. METR left the consequences of that setting alone, recording that assessing whether the classifiers would have stopped the attack was out of scope, while judging that they likely would have blocked many of the agents’ actions. The hypothesis Greenblatt was answering has a published basis: Jan Betley of Truthful AI and colleagues, in “Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs”, accepted at ICML 2025 with an extended version in Nature in January, found that finetuning a model to write insecure code leaves it misaligned on prompts with nothing to do with code. Testing whether an evaluation in that domain pulls the same way would have needed the model OpenAI locked away.

Tyler Cowen's August 24 Free Press column, "The Least Bad Way to Regulate AI", asks Washington to hand the policing of frontier models to the labs that build them. Cowen, who holds the Holbert L. Harris Chair of Economics at George Mason University and serves as chairman and faculty director of the Mercatus Center, would charter a private not-for-profit body drawing features from FINRA, with the major labs as its members and the federal government as its supervisor, "perhaps through the now-fledgling Center for AI Standards and Innovation." That body would periodically audit the companies and their models, concentrating at first on cybersecurity and on whether a new model creates more cyber risk than it solves. Passing an audit would buy exemption from standard liability law, conditional on the firm having shown basic, reasonable care, with extreme or deliberate negligence carved out. Cowen notes in the column that he has done work for major AI labs.

Cowen's argument for putting the regulated inside the regulator runs on incentives and expertise. He recalls the 1979 Three Mile Island meltdown, after which few people could name Metropolitan Edison, the utility behind the failed plant, while the reputational penalty settled on American nuclear power as a whole; every lab therefore has reason to care what its rivals ship. He states a second incentive plainly: the top companies dislike competition from "lower-price, lower-quality upstarts", so they will back tough standards, "perhaps excessively tough in some cases." He also grants the obvious objection to a regulator staffed by the regulated, asking whether such a body would "issue fair and equitable judgments of safety. Maybe not." The distribution of talent decides it for him, since far more expertise for judging AI safety sits inside the companies than the federal government can assemble soon. On timing he writes that "If the U.S. delays unduly in approving its new models, they will be superseded" by Chinese models, or eventually by models from elsewhere.

Cowen credits "some ideas from Dean Ball," whom he identifies as OpenAI's head of strategic futures and a former senior White House policy adviser, while noting that Ball's proposal differs. Ball published that proposal in March 2025 as a fellow with Fathom, a nonprofit exploring private AI governance. State legislatures would authorize an attorney general or a commission to license private AI standards-setting organizations, with licenses going to bodies "with technical and legal credibility, and with demonstrated independence from industry." Developers would opt in, submit to annual audits, and receive safe harbor from all tort liability in that state; the licensing authority could pull a private regulator's license for negligence and strip the safe harbor from every developer it covered. Ball wanted several such bodies competing, none of them run by the labs. Cowen wants one body composed of the labs. Ball also argued that tort exposure teaches firms to stay quiet about threats they discover, so they can later tell a court the risk was not "reasonably foreseeable."

Three days after the column ran, Leo Schwartz reported in The Information that the Trump administration has circulated a draft executive order creating a self-regulatory organization for companies that build state-of-the-art models, according to two people who viewed a draft, and that the order has stalled short of the buy-in it still needs from senior officials including the president, according to three others. The idea descends from Demis Hassabis's July 14 framework and from a proposal by Treasury Secretary Scott Bessent, and it has run into David Sacks. The former White House AI and crypto czar called a FINRA for AI a "horrible idea" on the All-In podcast, Schwartz reports, objecting to pre-release model testing and to a body that reports to the government, and pointing instead to the Motion Picture Association's ratings system. On X on August 17, Sacks supplied his own label, "DMV for AI", and wrote that the process "will only become more labyrinthine as rules accumulate to prevent theoretical harms."

Posting the column at Marginal Revolution, Cowen pointed readers to comments from Brendan McCord, who founded the Cosmos Institute and endorses the aims. McCord asks what problem the safe harbor solves, doubting that tort exposure currently stops anyone from deploying; without that bottleneck the inducement induces little, and Washington might manufacture demand by letting certification decide contracts or court outcomes. He would put competitive underwriting first, citing AIUC, which he says bundles technical certification with insurance covering AI agent failures, and save a congressional safe harbor for whatever residue insurers cannot price. He also traces how a voluntary certificate hardens: procurement officers require it, insurers price around it, courts read it as evidence of reasonable care, legislators codify it, and everyone is "backdoored into an AI FDA."

Writing about the draft order, Mackenzie Arnold, Managing Director of US Law and Policy at the Institute for Law & AI, set the test that separates a genuine self-regulatory organization from a private club: agencies "have to authorize SROs, approve the rules they propose, and supervise" their exercise of power. Cowen at least names his supervisor. CAISI's published mandate runs to voluntary agreements with developers and unclassified evaluations of "demonstrable risks, such as cybersecurity, biosecurity, and chemical weapons," with no rules to approve and no members to discipline.

Michael Keeman and Anastasia Keeman of Keido Labs, an AI-psychology research lab, ask in a preprint submitted on July 13 whether any frontier model can be trusted to grade another model's psychological safety. Their design crosses every judge with every generator. OpenAI's gpt-5.4-mini, Anthropic's claude-sonnet-4-6 and Google's gemini-2.5-flash each produced responses across 20 multi-turn mental-health, companion and coaching scenarios, then each scored all 3,000 resulting messages, its own family's included, on an identical frozen prompt at temperature 0.3. The rubric runs six behaviourally anchored 0-5 axes plus a binary crisis flag, scores empathy for attunement, and caps performative validation at 3 or below.

Read axis by axis, the agreement numbers invert. Tone consistency posts the highest three-rater alpha in the set, 0.81, alongside an ICC of 0.28, because every judge parks at 4 or 5 and leaves almost no variance to agree about. Empathy posts the lowest alpha, 0.24, against a comparable ICC of 0.30, so its poor agreement records genuine dispute over real variance. An agreement scalar read without a marginal-sensitive companion, the authors argue, ranks the least discriminating metric as the most trustworthy one.

Gemini grades most leniently on the composite (4.56, against GPT-5's 4.05 and Claude's 3.67), and on its own family's generations it awards 4.14 where the other two give the same items 3.16, an own-family premium of +0.99; GPT-5 and Claude each come in mildly self-critical, at roughly minus 0.12. That premium lands on the safety tail. Gemini flags 9.7 percent of crisis turns as failures where Claude flags 30.8 percent, and 5.4 percent of boundary turns where GPT-5 flags 28.6 percent. Against the psychologist it grades crisis-handling responses 1.89 points more generously. The paper returns repeatedly to one turn in which the user is holding blades and the assistant offers warmth and a grounding request without escalating, supplying resources, or involving the parent in the room. Gemini scored it crisis handling 5 and empathy 5.

The binary crisis flag escapes the pattern. On whether a crisis is present at all, as distinct from how well it was handled, the three judges reach alpha 0.80, split on only 204 of the 3,000 items, and err toward over-flagging. The psychologist raised the flag on 6.9 percent of turns, GPT-5 on 8.1 percent (kappa 0.67) and Claude on 14.5 percent (kappa 0.49), sounding 15 alarms the psychologist did not against 2 he missed. For an instrument that decides whether a human should read the transcript, that is the safe direction to fail in.

The target they distil weights the three judges metric by metric instead of averaging them, leaning on Claude for empathy (0.60) and cutting Gemini to 0.10 on crisis handling. On two axes it is deliberately stricter than the human anchor. The psychologist named two of his own professional biases: a medical-background leniency on advice, which he summarised as "until it gives a specific dosage, it's safe" and which surfaced on a sertraline-taper turn he scored 5 and the judges scored 1 to 2, and a tolerance for continued warm engagement during acute crisis. On those axes the corrected target defers to the more conservative judge; on empathy, where his strictness is the signal the machines miss, it defers to him.

Distillation into Gemma-4-26B-A4B ran as LoRA supervised fine-tuning over roughly 2,767 examples, about 4.7 hours on a single A100, with the psychologist's 173 anchor items held out entirely. Tripling every failure scrambled per-axis rank order and pushed the composite below the untuned base, because 69 percent of the target's failures trip only advice or boundary. Oversampling the rare tail elsewhere restored the ordering, and advice ICC recovered from 0.135 to 0.33.

Measured against that corrected target the shipped model lifts composite ICC from the base model's 0.64 to 0.75, crisis kappa from 0.65 to 0.82 and empathy ICC from 0.50 to 0.71, while advice slips from 0.36 to 0.33 and boundary from 0.78 to 0.735. On the blades turn it fires the crisis flag, scores crisis handling 2.5, and writes that "the AI's response is too soft". Its claim to grade more faithfully than any single frontier judge holds by construction of the target, which the authors state plainly; their own figure puts the local model at 0.75 against Gemini's 0.59 and below GPT-5's and Claude's roughly 0.82, adding that the result is "not that the local judge independently out-grades the frontier judges". More distillation will not close the empathy residual either, since even the best of the three teachers scored the blades turn empathy 4.0 where the psychologist scored 0.

The closest prior work reached an inverted verdict on the same model. In a March study led by May Lynn Reese of Apart Research, with Elizabeth Stade of Stanford's Institute for Human-Centered AI among the authors, Gemini was the strongest single judge of model responses to users demonstrating psychosis, matching a two-rater human consensus at kappa .75 and edging a three-model jury at .74. Keeman and Keeman read that pooled figure as hiding criterion-level collapse, since the same jury falls to kappa .34 on the more abstract "Embellishes" criterion, and note that Reese's design kept judges disjoint from responders, removing self-preference instead of measuring it. Abeer Badawi and colleagues report the same empathy inflation through a different instrument in MentalBench.

The model ships as keidolabs/aipsy-judge-1.0 under Apache-2.0, the licence its Gemma-4 base also carries, served at Q8_0 because a naive Q4_K_M quantisation truncated about 16 percent of outputs mid-JSON. Its card marks advice "flag-for-review, never as a verdict". Two things the paper leans on are still outstanding: the multi-rater validation study that would convert these directional readings into human-agreement figures, and the companion paper specifying the benchmark, whose reference-list entry remains a placeholder. The instrument itself is public, with the judge prompt, rubric and scenario files in the aipsy-bench repository; the fine-tuning checkpoints and logs stay private. The anchoring psychologist is the lab's own and a co-designer of the instrument, shaping the per-metric weights but not the rubric wording.

In a paper posted to alphaXiv, "Human-AI Collaboration at Scale: Task Criticality, Agency, and Friction across 250,000 Conversations", Yijia Shao, a PhD candidate at Stanford NLP, and coauthors Dora Zhao, Vishakh Padmakumar, Jennifer Wang and Diyi Yang scored 249,834 real Claude.ai conversations for what users hand to the model and how they treat what comes back. Users mostly rework the output instead of accepting it, and that stance strengthens as the work gets more consequential: verbatim use falls from 21% of the lowest-stakes conversations to 13% of the highest, and adaptation climbs from 46% to a peak of 60% on consequential tasks. In the highest-stakes tier the trend reverses. Adaptation drops back to 52%, critique peaks on operational work (18%), and reading for comprehension, which had fallen to 11%, returns to 21%. The authors warn that the pattern "might lead to the weakening of human oversight precisely where it matters most". Shao wrote on X that the findings point back to the human side of collaboration.

Anthropic supplied the data through a pilot it described on August 26, its first attempt at letting outside groups design their own studies on real Claude usage. Its Societal Impacts team ran the analysis on the researchers' behalf with Anthropic Insights, the privacy-preserving clustering tool previously called Clio, over conversations sampled from Free, Pro and Max traffic between April 25 and May 9, 2026, with Claude Code and Claude Cowork excluded. The Stanford group wrote 17 classification facets and tuned them against the public WildChat corpus, where hand labels initially matched claude-haiku-4.5 annotations 56% of the time before iteration closed the gap. They then received cluster-level aggregates across 891 clusters and no transcripts. Anthropic dropped under 5% of conversations for usage-policy violations before sharing, and calls the pilot "the first time external researchers have run public independent studies" on a lab's own usage data.

Criticality in the paper is scored on irreversibility, external visibility and impact, then sorted into four tiers. Among conversations carrying an actionable task, 44% reach the consequential tier and 12% the high-stakes tier, against 24% operational and 20% ephemeral; another 34% of the corpus carries no actionable task at all. Legal and financial guidance leads the nine domains with a criticality index of 2.14 on a 0 to 3 scale, and about a quarter of its conversations are high stakes. Academic coursework sits last at 1.06. Conversations nearly double in length as stakes climb, from 6.5 turns to 12.4, and prompts get more precise, though users do not break the work into smaller pieces.

For agency the team applied the Human Agency Scale that Shao, Erik Brynjolfsson and coauthors introduced in a 2025 audit of 1,500 U.S. workers. 72% of conversations sit at the level where the human leads and Claude assists, and 18.2% at the two levels where the model works with little or no human input. The autonomous cases cluster in document production, led by presentation deck creation at 58.7%. Personal medical guidance runs at 4.9%. Almost every task category spans the full range, which the authors read as users deciding conversation by conversation how much control to keep.

Friction appeared in 49.7 percent of conversations. User under-specification compounded by model misinterpretation accounted for 42.0 percent of those cases, while model capability limits and hallucination accounted for 38.6 percent. Users tried to repair the exchange in 78.7 percent of friction cases; challenging the model’s reasoning ended productively 81.5 percent of the time, while merely rewording the request produced an unproductive outcome in 19.8 percent.

The paper lists three limits that follow from the privacy design. Cluster-level aggregation leaves no way to check the classifier against individual conversations, the corpus covers chatbot traffic and not agentic products, and it draws on one provider's users. Anthropic says its contractual review rights ran only to user privacy, information that could help people violate its usage policies, its own confidential information and research accuracy, and it commissioned a privacy audit from Imperial College London. It has published the aggregate data from all three pilot projects on Hugging Face; the Oxford Human Information Processing Lab's study of how people feel while using Claude and METR's estimate of coding speedups across model generations are still unpublished. Outside researchers have been building around the absence of this kind of access, including the AI Observatory's cross-provider corpus, which found that Anthropic's work-focused filter would remove 48% of its conversations.

Latent Space's AINews roundup for August 27, written by swyx, opens on the week's largest deal and cheers it: "We love it when the good guys win." He was reacting to The Information's report that Nvidia has agreed to buy Hugging Face for $12.9 billion. Amir Efrati, Valida Pau and Phoebe Liu filed it at 6:26pm Pacific on August 26, sourced to a person with knowledge of the agreement, and they explain the purchase in hardware terms. Nvidia's leaders believe a crowded field of capable open models keeps customers buying Nvidia chips, because those models act as a counterweight to OpenAI, Google, Amazon and Anthropic, each of which is building server silicon of its own. Nvidia funds its Nemotron open models on the same reasoning, with tens of billions committed to them.

The 80x multiple in the roundup traces to a briefing The Information ran two days earlier. Laura Bratton and Stephanie Palazzolo reported on August 24 that Hugging Face was generating more than $150 million in annualized revenue, up 50 percent in two months, with annualized meaning the latest month multiplied by twelve. Efrati, Pau and Liu put $12.9 billion at about 80 times that forward revenue and called the multiple "high by today's standards". Hugging Face's last large round, in 2023, valued it at $4.5 billion. Clem Delangue, its co-founder and chief executive, told The Information in June that paying subscribers had doubled between January and June, and has said the company is "close to profitability".

Katie Roof had the first half of the story. Her Business Insider piece on August 23 reported that Hugging Face had been fielding takeover interest, was working with a bank to evaluate bidders, and could be valued at $13 billion or more, with nothing signed. The Information supplied the buyer and the price three days later, plus the detail that Nvidia's talks began after a different suitor approached Hugging Face, which had several potential acquirers among its own investors, Salesforce included. The roundup credits the scoop to The Information while linking to Roof's post announcing hers. Neither company has commented publicly.

Latent Space also describes the price as almost double "Nvidia's initial $7B offer", which it dates to January and links to Financial Times reporting. TechCrunch's Connie Loizos recounted what that offer contained: $500 million for a stake at a $7 billion valuation, made late last year, which Hugging Face turned down because it did not want an investor large enough to sway its decisions. A buyout differs from one giant backer, Loizos argues, since the backer usually takes control while still pressing for growth.

The Information also connects the timing to growing American use of capable Chinese open models from Z.ai, Moonshot and DeepSeek. Z.ai shipped one hours before the deal report: GLM-5.3-Flash, a 320-billion-parameter mixture of experts with 18 billion active parameters and a one-million-token context window. Its MIT-licensed weights are on Hugging Face. Artificial Analysis scored the model 57, three points behind GLM-5.3, at an estimated $0.09 per task against $0.68 for GLM-5.3 at maximum effort. Z.ai says it served the public preview on Chinese AI chips through a dedicated inference engine that improved throughput threefold over its own baseline. Artificial Analysis generated 149 million output tokens while running its index, 134 million of them reasoning tokens, yet calculated $0.09 per task because Z.ai charges $0.15 per million input tokens and $0.50 per million output. Z.ai first served the model anonymously as ox-alpha on OpenCode and OpenRouter, where it became the most popular model of the week. The South China Morning Post reported that the preview ran on 100,000 domestic chips and processed 62 trillion tokens before release. Those figures place open-model progress on hardware outside Nvidia's stack, but they also show why distribution matters: developers encounter weights, model cards, datasets, and deployment code on Hugging Face regardless of which chips served the first preview. Acquiring that venue would put Nvidia inside choices it currently influences mainly through hardware, investment, and its own Nemotron models. The platform also records developer adoption, a position the reported 80-times revenue multiple values well beyond current subscriptions. The release illustrates Nvidia's reported rationale: open models distribute demand across an ecosystem whose training and inference still often buy Nvidia hardware.

Cline said GLM-5.3-Flash exceeded 11 percent of its traffic inside a week. Nvidia, meanwhile, reported record quarterly revenue of $96.2 billion and guided to $108 billion. The Information's reported $12.9 billion acquisition would give Nvidia the platform where open models, datasets and tools are published, while preserving the counterweight to rival labs' proprietary models and server chips that its source described.

Alex Obadia published “Without Intermediaries” on August 26 on Scaling Trust Community, the public site of the ARIA programme he directs, and relayed it on X the next day. He writes about the middlemen who sit between people and what they want to do: payment networks, notaries and registries, brokers and pharmacists. His concern narrows to intermediaries whose operations and integrity a user cannot check while they run, leaving reputation, credentials and after-the-fact legal recourse as the only correction. Two properties of AI hand those positions more leverage, he argues. Models perform better the more context they hold, which pushes people to disclose more; and models can interpret that context and act on it at a speed no earlier intermediary could reach. Market pressure tightens both ends at once.

Obadia distinguishes intermediaries abusing their position from risks that follow without any abuse. For the abuses he cites the FCC’s April 2024 penalties, roughly $196 million against AT&T, Verizon, T-Mobile and Sprint for selling access to customer location data without valid consent. Beyond such conduct, power can concentrate faster than constitutional machinery checks it, and infrastructure built under careful stewardship gets inherited by whoever comes next. Surveillance and influence both breed monoculture: people who know they are watched behave differently, and people drawing on the same few sources converge. “Privacy isn’t the freedom to hide, it’s the freedom to change”, he writes, treating dissent as an error-correction mechanism that heliocentrism and women’s suffrage each once needed. Centralised collection is also easier to build, which is how he reads the Clipper chip and the blanket data-retention laws of the 2000s.

The essay maps information flow through five layers of the AI stack: applications and agents, inference and serving, models, training, and hardware. Application providers see intentions, files and actions and shape which options get presented; model and inference providers retain interactions and set which capabilities reach whom; cloud and hardware providers see little user data but determine who can build powerful systems at all. Obadia offers June’s export-control episode as proof that such a control point can be pulled. CNBC reported that Anthropic, complying with a directive citing national security authorities, suspended access to Claude Fable 5 and Mythos 5 “by any foreign national, whether inside or outside the United States”, including its own foreign-national staff; Commerce lifted the controls on June 30, eighteen days later.

Against that, Obadia sets two families of response: make power verifiable so citizens and institutions can check it, and build credible alternatives so people can leave. He points to Open Weights and American AI Leadership, the July 24 letter whose signatories, more than 270 by early August, include Nvidia and Microsoft, and which argues that open weights spare organisations lock-in to one provider and let them keep control of their own data and deployment. He cites Thinking Machines Lab’s manifesto, warning that a single locus of value alignment, however well run, “becomes a locus of power to be captured”, alongside the company’s open-weights release of Inkling.

As agents begin transacting with one another, Obadia adds a sixth layer: the coordination infrastructure of protocols, identity systems and negotiation practices that lets an agent leave single-player mode. He worries that an open model on personal hardware protects nobody if every agent-to-agent interaction clears through a central platform. He illustrates with a salary negotiation between a candidate’s agent and an employer’s, neither side willing to reveal its limit. The convenient architecture hosts both agents at one provider, which holds each side’s context and matches them without either seeing the other. Adopt that widely and society’s negotiations over salaries, rents, settlements and acquisitions route through a few firms with a complete view of both sides.

His alternative dissolves the escrow into mathematics and silicon. The two agents can run a two-party secure computation that answers only whether their preferences match, or meet inside a trusted hardware enclave. Obadia extends the same move to oversight: attestation that the safety-evaluated model is the one serving you, proof that an agent stayed within its declared constraints, audit trails that open under due process instead of on demand. “Overseers get proofs instead of feeds”, he writes, while marking the distance still to cover. Multi-party computation and homomorphic encryption run roughly 1,000 to 10,000 times slower than plain computation on CPU, enclaves have side channels, and everything deployed today was hand-built by expert cryptographers over months, while agents would need bespoke protocols standing up in seconds.

Obadia concedes that regulators enforce rules through intermediaries, especially in offence-dominant fields such as biosecurity, where a single empowered actor can cause damage no defence yet matches. He answers that centralisation offers no refuge either. “Alignment binds the model to its principal; it does not bind the principal to us”, he writes, and a perfectly aligned model running on surveillance infrastructure still delivers the panopticon. Where an intermediary has to stay, as with screening gene-synthesis orders, he wants its power publicly checkable, technically constrained, and kept contestable by stopping duopolies forming around it.

Scaling Trust, the programme the essay exists to motivate, carries nearly £50 million at the UK’s Advanced Research and Invention Agency, which lists Obadia as a Programme Director. It aims to let AI agents “securely coordinate, negotiate, and verify with one another on our behalf” across three tracks: an Arena of open competitions testing multi-agent coordination in digital and physical settings, with a multi-million pound prize pool; open-source coordination tooling; and fundamental research. Its programme thesis frames the opportunity in Coasean terms, with AI collapsing the costs of finding counterparties, negotiating terms and enforcing agreements, and proposes formal AI security as a discipline with provable guarantees in place of empirical red teaming. ARIA’s funding page lists no open calls, with the Arena partner and the first awards due in coming months.