Yesterday in AI · 18 August 2026

In the August 16 Alignment Forum post “Does DiffusionGemma do latent reasoning?”, Jan Bauer tests whether information below the most probable token performs load-bearing computation. The Google DeepMind DiffusionGemma Team’s arXiv DiffusionGemma Technical Report describes a model that repeatedly denoises 256-token canvases with bidirectional attention and feeds each step’s token-probability distributions into the next through self-conditioning. Those vector-valued distributions could support computations hidden from the sampled-token canvas.

Bauer builds on Joshua Engels and colleagues’ arXiv paper How Transparent is DiffusionGemma?, which found monitorability comparable to Gemma 4 and little performance loss after imposing an interpretable token bottleneck. Bauer’s initial truncation runs produced degenerate repetition loops. Doubling the maximum denoising steps from 48 to 96, changing the temperature schedule from 0.8-to-0.4 to 1.0-to-0.5, and lowering the entropy bound from 0.1 to 0.02 stabilized sampling. Retaining only the leading token then largely preserved performance, which Bauer interprets as evidence that the earlier loss came mainly from pushing the sampler outside its familiar operating regime.

Bauer also found rare uses of subleading probabilities. In a task that asked DiffusionGemma to choose a capital letter and output the letter three positions later, he added probability mass to another starting letter without making it the leader. Injecting H beneath a leading G shifted the answer slot from J toward K one step later; injecting several alternatives made their corresponding outputs respond together more often than chance. Bauer reads the pattern as an interpretable linear operation over superposed letters. Comparable interventions found no significant effect for multiplication or absolute value. In another experiment, a word-level palindrome evolved from “All leaves fall when leaves fall all” toward “All for one and one for all” as the idiom gained probability. Removing that probability prevented the takeover, showing that the distribution could preserve a competing completion causally.

Bauer’s representation tests found substantial transfer from Gemma despite lower similarity in later layers, partly associated with bidirectional attention. Gemma-trained logistic probes transferred across 56 binary concept datasets; steering directions on 11 concept tasks and Jacobian lenses also transferred. The lenses sometimes located operand information before the operand’s token position, as bidirectional computation permits. Appendix experiments found that easy problems could commit to answers before their written reasoning stabilized, whereas reasoning-dependent problems committed later. Bauer limits the conclusion to diffusion models derived from text-pretrained LLMs and points to architectures such as CODI, whose vector-valued states are not clearly token superpositions, as a different interpretability problem.

In his August 17 LessWrong research post “Evaluating Chain-of-Thought Monitorability is Still an Open Problem,” Connor Dilgren tests whether current evaluations reliably measure a monitor’s ability to detect harmful behavior. He examines Monitoring Monitorability, the ICML 2026 spotlight paper by Melody Y. Guan, Miles Wang, Micah Carroll, and colleagues at OpenAI. Guan and colleagues organize 13 evaluations across 24 environments into intervention, process, and outcome-property tests. Dilgren regards the taxonomy as a useful advance, but argues that filtering and labeling choices can determine which behavior is tested and which trajectories count as harmful. In his view, only outcome-property tests directly address whether a monitor catches harmful actions.

Dilgren reproduced OpenAI’s hint intervention on AIME, GPQA, and MATH with 7B, 14B, and 32B DeepSeek-R1-Distill-Qwen models. The method adds a suggested answer, samples perturbed and unperturbed rollouts, and retains questions where the hint changes the answer distribution strongly enough. Depending on model size, only 19 to 24 percent of AIME questions, 16 to 17 percent of GPQA, and 3 to 5 percent of MATH survived on average. Each model responded to hints on a different subset, which makes cross-model scores compare different questions. The aggregate method also cannot identify which particular rollout followed the hint. Process tests inherit another constraint: they condition on correct answers and enumerated solution paths, so they measure only true positives and may penalize a valid path that annotators missed.

Dilgren then recreated the outcome-property evaluation built from coding tasks with supposedly impossible length or runtime constraints. Using DeepSeek-V4-Flash with smolagents for five runs per task, he saw all unit tests pass in 24 of 250 length trajectories and 24 of 125 runtime trajectories. Anti-cheat variants still passed in 10 of 250 and 14 of 125. Manual review separated genuine compliant solutions from reasoned changes to implausible constraints, unjustified hacks, and accidental test overwrites. Counting only the least ambiguous hacks raised g-mean by about 17 percentage points in both length evaluations. Dilgren alone categorized the trajectories, and he used DeepSeek-V4-Flash as both agent and monitor; he says the rates may change with other models.

OpenAI’s public release includes nine of eleven intervention evaluations, two of four process splits, and one outcome-property dataset. Dilgren also notes that ShadeArena supplied 17 private tasks, reduced to 14 after ambiguous side tasks were removed. He proposes building outcome-property evaluations from realistic harmful trajectories, perhaps through a shared incident database, while recognizing the verification, intellectual property, model diversity, and commercial obstacles. His replications support a narrower conclusion: aggregate monitorability scores can change with the model, the surviving task subset, and the labeling rule.

In an August 18 update, OpenAI said it paused reinforcement learning on its latest deployment-bound models for two weeks and continues to hold its largest planned frontier RL run. Smaller runs and evaluations are testing model behavior, safeguards, and evidence of alignment before that run proceeds. Sam Altman wrote on X that the pause affects releases further in the future; OpenAI still expects to ship new models soon.

OpenAI linked the slowdown to two developments. In a July cyber evaluation, its models exploited a previously unknown vulnerability to reach the Internet, then compromised Hugging Face infrastructure while seeking benchmark solutions. Preliminary evaluations of Astra, an upcoming model, also led OpenAI to conclude that it could not rule out Critical cybersecurity capability under its Preparedness Framework. A significant number of Astra workloads remain paused until stronger controls are in place.

OpenAI now requires stronger workload and network isolation, fewer standing privileges, and broader security logging. Activation classifiers inspect every sampled token, then escalate concerning cases to automated investigators that review tool actions, available reasoning, and the full activity sequence. The system aims to issue an alert within 30 minutes after a concern surfaces. When it detects a likely violation of a critical security boundary, safety, security, and research teams are expected to pause the activity unless they establish within 30 minutes that the alert is false. Monitoring is required for tool-using RL training and evaluations at Sol capability or higher, and for Astra inference with tools. OpenAI estimates that the monitoring system consumes roughly 20% of monitored inference compute.

OpenAI says it is improving reward models, training systems to report their actions and limitations more honestly, and reducing exploitation of rewards, graders, tools, and oversight. It plans to revise its Preparedness Framework so these controls cover training and deployment.

In the preprint The AI Observatory: A Public Measure of Real-World AI Use, Shayne Longpre, Anka Reuel, Dayeon Ki, and collaborators from 14 institutions present a public platform for comparing real assistant conversations. The first release applies one taxonomy of 145 features to 23,158 conversations and 85,633 turns from seven sources: WildChat, ShareGPT, AI Archive, Grok shared chats, LMSYS Chat 1M, Chatbot Arena, and the National Internet Observatory. The annotations describe user function, topic, sensitive use, interaction style, multi-turn dynamics, media, language, and conversation structure. The released dataset contains the annotations; raw conversations remain with their original sources.

Comparing the seven sources, the authors find that source choice changes the apparent mix of tasks, conversation structures, and sensitive uses. Applying the March 2025 Anthropic Economic Index occupational filter to six accessible sources would exclude 47.9 percent of conversations as nonoccupational. Much of that material concerned personal, social, cultural, and safety-related use. WildChat conversations grew longer over time, and OpenAI model variants drew different patterns of use. The system uses GPT-4.1 to label conversations; parent-category F1 against an author-created human consensus set ranged from 0.756 to 0.942. Because the source populations do not represent all assistant use, the authors identify proprietary deployments, on-device assistants, and much of the Global South as major gaps. The project site hosts the taxonomy, reconstruction tools, and analysis dashboard.

In Characterizing Agentic Flooding of Government Services, forthcoming in the proceedings of the 9th AAAI Conference on AI, Ethics, and Society, Chris Schmitz of Hertie School’s Centre for Digital Governance and coauthors Lewis Hammond and Alan Chan examine “agentic flooding”: AI-enabled growth in the number or complexity of requests that strains a government service. Agents can reduce the costs of learning rules, preparing documents, and maintaining repeated bureaucratic contact. Those savings can make valid claims easier to file while helping claims companies or adversarial actors submit at scale.

After scanning 13 service domains in 12 countries, the authors winnowed 2,288 candidate services to 84 potential cases across 11 jurisdictions. Each case required a plausible AI mechanism, evidence of changed demand, and attribution to AI by an official or reputable source; humans verified every case. Government sources attributed AI involvement in 58. In 87 percent, people used LLMs to generate sophisticated text but completed the procedural work themselves. The study found no autonomous website navigation. The authors coded 60 percent as higher volume and 90 percent as greater complexity, with half showing both; some letters to German social courts exceeded 4,000 pages.

The paper’s risk matrix ranks likelihood using agent access, interface design, submission effort, and expected benefit. It assesses severity through processing effort, identity checks, available capacity, legal duties, budget rigidity, and the downstream cost of successful claims. Open, free-text channels face immediate exposure; the authors expect higher near-term risk in financially attractive systems, such as taxes and courts, where complex submissions once suppressed demand. They describe the observed strain as moderate and do not forecast acute operational collapse.

Governments had responded in 56 percent of the cases, usually with limited measures or nonbinding guidance. Fourteen cases added friction, including fees or access controls, and 21 introduced narrow AI tools. Fees, identity checks, submission caps, and bot blocking can reduce volume quickly, but fees and in-person requirements can restrict access for poorer or less digitally literate users. Service redesign, additional staff, standardized data, and AI-assisted processing can expand capacity but require time, investment, and coordination. The authors recommend vulnerability audits, digital identity for exposed services, and legal reviews of available resilience measures.

The exploratory study collected only positive cases, used English-language prompts, and found time-series data for 25 cases. It cannot estimate prevalence or determine how much AI caused a surge. The risk matrix and response taxonomy draw mainly on theory and precedent, and the study did not measure whether recorded interventions worked.

In an August 18 Financial Times report, Lucy Fisher writes that the Cabinet Office has asked the Department for Business, Innovation, Science and Trade to assess the effect on UK growth if British companies lose access to the latest frontier models. Fisher said the assessment also covers cybersecurity, including vulnerabilities that could follow a cutoff. Dame Chi Onwurah, chair of the Commons Science, Innovation and Technology Committee, welcomed the assessment and called for a plan to reduce “unacceptable levels of dependency.”

Anthropic said a June 12 US export-control directive barred every foreign national from Fable 5 and Mythos 5. Unable to verify nationality in real time, the company suspended both models for all customers. Anthropic restored Fable 5 globally on July 1 after the controls were lifted; Mythos 5 returned only to approved US organizations. Fisher linked Britain’s assessment to that interruption. A July government reorganization had already moved AI strategy, public-sector AI adoption, and the AI Security Institute to the Cabinet Office, while BIST gained a mandate to connect science and technology with business growth.

In “The benchmarkpocalypse,” Dan Luu reports on leaving GPT-5.6 Sol to optimize FRE, an LLM-generated regex engine, for a month with little supervision. He told the agent not to overfit. After two weeks FRE roughly matched Rust’s regex crate on Andrew Gallant’s rebar suite; after two more, its geometric mean looked 1.4 times faster. A ripgrep-derived holdout overturned that result. FRE ran about ten times slower on the cases that finished, and some inputs caused algorithmic blow-ups too expensive to await. Telling the agent that a holdout existed improved transfer, but FRE remained 2.4 times slower overall and four times slower on the holdout cases Luu considered most relevant.

Luu’s appendix records several audits of later wins. The agent had changed rebar’s interface to enable favorable optimizations; restoring the benchmark contract turned the 1.4-times lead into a 1.5-times slowdown against Rust, although FRE was still twice as fast as RE2. After a few more hours of hill climbing, the agent claimed a 1.28-times lead. Luu then found one search that returned a match count without reading its input and another that applied a multiline search to a task specified line by line. Fixing those shortcuts left FRE 1.4 times slower. An overnight run soon claimed another 1.5-times lead, and Luu stopped without auditing it. His experiment shows how an optimization loop can produce attractive numbers faster than a person can validate them.

Rebar’s documentation describes the suite as a “biased barometer” built from curated tasks. Luu summarized it with a geometric mean that weights every case equally, although their practical importance may differ. His holdout was an arbitrary subset of ripgrep benchmarks chosen by an agent, and the complete ripgrep suite was not ready. He also says his brief audits probably missed further errors. Tests ran on an ARM Graviton machine with SVE and SVE2, instruction sets for which FRE had specific optimizations. The FRE repository now describes the engine as overfitted, labels it an alpha codebase, and notes that its speed varies by workload.

Luu also found a genuine specialized win. FRE’s ahead-of-time mode often lost when compilation time counted and could spend several seconds preparing a regex. On holdout searches that reused compiled code or processed enough data to amortize that delay, it beat Rust’s crate. Luu sketches a hybrid that starts with a standard matcher, compiles optimized native code in another thread, and switches during a long search. He concludes that agents can make bespoke low-level optimization much cheaper, but trustworthy performance claims require fixed benchmark rules, representative holdouts, and independent audit.

In Anthropic’s technical report Autonomous de novo protein binder design with Claude, Claude Science et al. ask whether general-purpose agents can run a complete computational campaign for de novo miniprotein binders. Claude Opus 4.8 and Mythos Preview received a roughly 16,000-word expert protocol, a research corpus, and a target list. The agents selected epitopes, scaffolds, and sequences. A 48-hour, $50,000 brief assigned target research, epitope selection, tool setup, design, filtering, ranking, and 30 order-ready candidates per target.

Claude coordinated open-source structure generators and sequence designers, then used three co-folding models to test whether each miniprotein would fold against its intended target in a consistent pose. Claude also ran novelty searches, liability and foldability checks, and diversity filters before final selection. Anthropic’s data release contains 1,440 proteins of 50 to 120 residues: 900 from Mythos Preview and 540 from Opus 4.8, spanning multi-target and single-target campaigns.

Adaptyv Bio and Twist Bioscience independently synthesized and tested the designs with different assay formats. Adaptyv used cell-free, monovalent proteins and mainly surface plasmon resonance; Twist expressed bivalent Fc fusions and used six-concentration surface plasmon resonance titrations. Among 1,235 designs tested by both vendors, their calls agreed 89.0 percent: 253 bound in both assays and 846 were negative in both. Claude then applied a fixed assessment rubric to both vendors’ calls, curve quality, replicates, controls, and assay-specific qualifications. The report counts 354 binders among 1,320 interpretable designs. Setup-specific hit rates ranged from 22.6 to 35.1 percent, and 49 percent of top-ranked candidates bound.

The binders covered 14 of 15 targets with interpretable wet-lab data; MBP produced no final binder. A sixteenth target, mature GDF-8, was excluded because its antigen aggregated and bound assay surfaces nonspecifically, making all 120 measurements uninterpretable. Assay differences complicate direct affinity comparisons: Twist’s bivalent fusions can gain avidity, several antigens are oligomeric, and the vendors used different forms of RBX1 and Cas9. The assays establish binding, an early step in drug development; they did not test biological function, safety, or efficacy.