MINT Lab

Yesterday in AI · 11 November 2026

Today’s stories curated by Seth. Claude produced 4 Read-more reports; Codex edited and ran the issue.

Today’s issue opens in Regulation and Accountability with the Paris Call’s voluntary commitments to publish evaluation findings and report serious incidents. In Alignment and Control, Dean Ball supports prohibiting AI from developing successors autonomously.

In Philosophy of AI, Beltoft and colleagues propose giving citizens an ongoing role in revising AI constitutions. Our AI Security coverage follows American vendors selling training environments to Chinese labs.

Then, in Knowledge Institutions and Information Integrity, Ravid Shwartz Ziv finds instructions to manipulate AI reviewers in ICLR submissions. We close with Capabilities, where Google DeepMind reports 96% success for robots sorting automotive parts during a Mercedes-Benz factory pilot.

Regulation and Accountability

The Élysée's November 9 Paris Call, briefed yesterday, includes voluntary commitments to publish standardized, non-sensitive evaluation findings and report serious incidents. Governments pledge confidential whistleblower channels, while developers' shutdown commitments expressly cover training and internal model use. Developers also pledge to disclose known risks and support proportionate independent evaluation; the declaration leaves legal implementation to national authorities.

Read more: The Paris Call’s safeguards and implementation → 911 words · ~5 min

Paris Call sets out evaluation, shutdown and whistleblower commitments

The November 9 declaration extends developers’ shutdown commitments to training and internal use, promises public evaluation findings, and assigns whistleblower channels to governments. Its voluntary pledges depend on national implementation.

The Paris Call for AI safety and security, published by the Élysée on November 9, commits governments, researchers and developers to shared evaluation methods, public findings and earlier warning of serious AI incidents. Developers separately promise shutdown mechanisms that reach into training and internal use; governments undertake to provide protected reporting channels for whistleblowers. We covered Adam Gleave’s response yesterday. The declaration’s detailed provisions explain how its signatories propose to make advanced AI safer, and which decisions they leave to national authorities.

The Élysée presents frontier AI risks as inherently international: development, deployment and harms cross borders, so no government or company can manage them alone. Its signatories include France, Canada and other governments alongside developers such as Anthropic, OpenAI and Mistral AI. Civil-society organizations have also endorsed it, and endorsements remain open. The declaration is a voluntary political commitment. It creates no international enforcement authority or sanctions, and specifies no implementation deadlines.

Under the first commitment, policymakers, researchers and developers will build a common framework for evaluating and managing frontier-model risks, using existing frameworks as a starting point. The named dangers include cyberattacks and chemical, biological, radiological and nuclear misuse, alongside loss of control as models become more autonomous or act contrary to human intentions. Signatories want technical standards and testing methods that keep pace with those capabilities. They identify the International Network for Advanced AI Measurement, Evaluation and Science, or NAAIMES, as one existing network that could help make results comparable and accessible across countries.

The signatories also agree to publish non-sensitive evaluation findings in a standard format and create a shared public repository. Possible contents include methods for adversarial testing, benchmark tools, risk assessments and stress-test results. Publishing the methods alongside findings would help readers compare assessments that otherwise use different tests. The text does not assign an organization to build the repository or set a date for publication, and the commitment concerns non-sensitive findings, leaving room to withhold information whose disclosure would create risks.

The second commitment puts national authorities in charge of the safety standards that signatories pledge to follow throughout a model’s life. Those standards may be established or recognized by the relevant authority; oversight should include competent, independent third-party evaluation. Policymakers acknowledge that legal frameworks need to clarify responsibility for large-scale damage, including cross-border harm, both to protect those affected and to give developers legal certainty. These provisions describe work for national governments and regulators, with legal duties depending on implementation in their jurisdictions.

Developers undertake to disclose capabilities, limitations and known risks. Their support for qualified independent evaluation is expressly conditional: “where appropriate and proportionate to the risk”, and consistent with existing practices and arrangements. The text does not guarantee unrestricted access to a model’s internals or specify how much time evaluators receive. Its general endorsement of independent testing therefore leaves substantive decisions about access and testing conditions unresolved.

Developers also commit to fail-safe mechanisms for stopping model operations they control, expressly covering training and internal use. The examples range from restrictions and pauses to shutdowns. That reach includes a developer’s own use of a model before making it available to customers. Developers further recognize the need for continuous risk management, monitoring of emerging capabilities and internal adversarial testing, and commit to work on governance with defined responsibilities, sufficient resources and independent oversight. On open weights, they acknowledge the danger that safeguards can be removed or bypassed while affirming the benefits for research, innovation and safety.

The declaration names the 2024 Seoul Frontier AI Safety Commitments among its predecessors. Seoul’s company signatories undertook to define thresholds of intolerable risk and, in the extreme, refrain from developing or deploying systems when mitigations could not keep risks below those thresholds. Paris specifies shutdown mechanisms but does not define equivalent thresholds or a rule that triggers their use. The two texts address related decisions: maintaining the ability to stop operations, and deciding when development or deployment must stop.

Under the third commitment, policymakers, researchers and developers pledge to cooperate on information sharing, reporting and early warnings. They propose building rapid alerts on existing incident-response arrangements, including cybersecurity systems. The text recognizes the value of aligning incident categories and exchanging timely information about serious cyber and AI-control incidents, as well as warning signs of serious harm. Exchanges could take place between companies and governments, among governments or among companies. A further cooperation mechanism remains something to explore; the declaration specifies no reporting deadlines or central incident authority.

Governments take on the explicit whistleblower-channel commitment. Policymakers promise secure, confidential routes through which employees and other stakeholders can raise reasonably grounded concerns about systemic risks without legal or professional retaliation. Developers separately recognize that internal reporting arrangements should protect researchers. Confidential channels and protection from retaliation are therefore stated objectives, with governments responsible for supplying the channels and national implementation determining how those protections work.

Gleave’s response identifies practical conditions for effective evaluation: he says FAR.AI commonly receives less than a week to test a model and wants access to inspect internals and conduct further training. The AI Evaluator Forum’s AEF-1 standard says substantially novel systems often require at least 20 business days, with time proportionate to the confidence needed and the system’s novelty. Yoshua Bengio welcomed the meeting while calling for mandatory audits by evaluators developers do not choose, and demonstrations of safety before a model can pose grave danger, including before deployment. Both responses identify requirements that the Paris declaration leaves for further decisions.

Sources & documents

  • Paris Call for AI safety and security — Élysée — Primary declaration read in dossier and directly re-fetched at edit time (HTTP 200). Supports commitment wording, separation of policymakers and developers, national oversight, publication, conditional evaluation, shutdown scope, open-weight balance, incident arrangements and whistleblower channels. Absence of deadlines, enforcement machinery and explicit thresholds checked against full text. Only direct quotation is the seven-word proportionality qualification. No undated signatory count.
  • Frontier AI Safety Commitments, AI Seoul Summit 2024 — GOV.UK — Primary GOV.UK commitments directly re-fetched at edit time (HTTP 200); Outcome 1 II-IV supports risk thresholds and conditional non-development/non-deployment comparison. Paris shutdown mechanisms are expressly acknowledged; no claim that Paris contains no stop commitment.
  • YiNAI, November 10 issue: AI Security section — Published continuity verified by accepted reporting notes and dossier. Earlier brief mentioned Gleave’s evaluation-access response and 12 government signatories. Natural back-reference retained; no new-signing framing.
  • Adam Gleave on the Paris Call — X — Primary post excerpt and full-thread reporting in supplied dossier support attributed testing-time and access requests. Paraphrased; no direct quotation or unverified job title. No claim about Kimi filtering retained.
  • AEF-1: Minimum Operating Conditions for Independent Third Party AI Evaluations — AI Evaluator Forum — Dossier contains condition 1.4 verbatim. Supports proportionate time and at least 20 business days often needed for substantially novel systems, not an unconditional universal minimum.
  • Yoshua Bengio on the Élysée meeting — X — Dossier primary-post excerpt supports attributed demand for mandatory audits by evaluators developers do not choose and safety demonstrations before grave danger, including before deployment. Paraphrased without quotations.

[ collapse ↑ ]

People affected by AI have had little influence over its development, Michael Wooldridge of Oxford and colleagues argue in Stanford University's AI Goes Mainstream: The Scaling Era and the Rise of Generative AI. The One Hundred Year Study on Artificial Intelligence (AI100) 2026 Study Panel Report. The panel urges meaningful public influence over deployment decisions. Its assessment, summarized by Stanford HAI, finds that international summits have produced shared principles but few practical, legally binding commitments.

New York has announced independent testing of advanced AI systems through the New York Center for AI Responsibility and Research at Binghamton University. The governor's November 10 release says more than 80 researchers will use Empire AI computing under interim co-directors Greg Morrisett and Jeannette Wing to assess safety and develop evaluation methods for state oversight.

Australia's government has committed to considering criminal liability for AI companies that fail to take reasonable steps to prevent harmful model conduct, Cam Wilson reports for ABC. The possible offence would add to the proposed watchdog and civil penalties covered in our earlier issue, following the government's review of the OpenAI Medicare portal incident. Monash University's Michael Duffy identifies foreseeability and proof of fault as prosecution difficulties.

Also yesterday: annawang's proposal on Alignment Forum, briefed yesterday, would require audited waits of 10-90 days before non-safety internal model use, while allowing successor training; Daphne Keller argues in Tech Policy Press that Meta’s settlement rewards states for securing rival platforms’ commitments, while TikTok’s rewards recruiting other states, with both weakening public oversight; Sutton et al. recommend benchmarks, independent procurement assessments and randomized trials for government AI in a Stanford HAI policy brief; Cunningham et al.'s September Stanford Digital Economy Lab working paper estimates that AI research feedback was too weak for self-sustaining acceleration.


Alignment and Control

Language models can learn multistep computation using meaningless dots in place of readable intermediate reasoning, Wuschel Schulz and tvb report in "LLMs can learn (some) multi-step reasoning on filler tokens" on LessWrong. Working through MARS 5 and the Meridian Visiting Researcher Programme, they checked predictions at each dot during training against the correct intermediate result while feeding only dots into subsequent positions; two models learned the behavior on three of seven synthetic task families, outperforming training on final answers alone. The study did not test a monitor.

Read more: Training models to compute behind dots → 935 words · ~5 min

Models learn to reason through strings of dots

Schulz and tvb trained two models to solve multistep tasks with dots in place of written reasoning. Their method teaches the intermediate answers explicitly, then keeps those answers out of the text the models read.

Language models can be trained to solve multistep problems while the text between question and answer consists entirely of dots. Wuschel Schulz and a coauthor writing as tvb report this in “LLMs can learn (some) multi-step reasoning on filler tokens”, a November 11 LessWrong research update from MARS 5 and the Meridian Visiting Researcher Programme. Their specially trained models improved on three of seven synthetic task families compared with training on final answers alone. One model learned to track changes to three lamps through 60 moves. The experiment shows how useful computation can proceed without readable intermediate text under an engineered training procedure.

Schulz and tvb fine-tuned Qwen3.6-35B-A3B and gpt-oss-20b using small trainable additions to the models, called adapters. At each dot position, they trained the model to predict the corresponding intermediate state of a worked solution. The next input remained a dot: the predicted state was never written into the text for the model to read back. Information could still travel between positions through the network’s internal representations. During evaluation, the researchers supplied the dots in advance and read predictions from the model. Left to generate its own text, the trained model would write out its reasoning. This differs from ordinary next-token training, and the result does not establish that deployed assistants spontaneously reason this way.

The researchers compared their procedure with a precedent from Jacob Pfau, William Merrill and Samuel Bowman at New York University. Their 2024 arXiv paper, “Let’s Think Dot by Dot: Hidden Computation in Transformer Language Models”, showed that models could use meaningless extra tokens to solve algorithmic problems they struggled with when answering immediately. Its training mixed worked solutions with examples containing dots followed by an answer. Schulz and tvb instead supervised the intermediate prediction at every dot. When they compared the two approaches on the same progressively lengthening tasks, with repeated training runs, their method outperformed answer-only training; the older recipe did not.

The authors’ results table indicates that the three clear successes were running sums with remainders, tracking a ball under shuffled cups, and tracking the lamps. In the lamp task, each move either passes the lamps’ states around a ring or toggles one lamp depending on another. Qwen answered 79% of the 60-move test problems correctly; its answer-only counterpart passed no curriculum stage. The researchers advanced training after success on a small checkpoint test, so passing a stage did not mean perfect accuracy on the larger final test. Across the other tasks, gains were small or absent. Models using dots struggled with chained arithmetic, lookup tables and logical deductions, while models trained to write their reasoning reached the longest tested length in every task family.

Pfau and colleagues also explained a theoretical limit: adding filler positions does not give a fixed-depth transformer unlimited capacity for successive computation. Schulz and tvb accept that limit. They investigate how much of a finite sequence of operations can fit within the model’s existing layers. A task described as 60 successive moves need not require 60 network layers, because an algorithm can combine some operations. Their result therefore concerns the practical reach of computation at finite task lengths; it does not show that dots remove the architectural limit.

To explore that reach, Schulz and tvb used Opus 5.5 agents to construct algorithms within an executable approximation of each transformer. They checked that the algorithms used permitted operations and returned the correct intermediate states and answers, then searched for constructions requiring fewer layers. The shallowest algorithms they found grew deeper as tasks lengthened, but usually much more slowly than the number of task steps. These constructions establish upper bounds on sufficient depth. They neither prove the minimum depth required nor show which algorithms the trained models learned. The numerical curves also depend on a chosen assumption that one internal processing operation can act as a lookup table over at most 32 bits of input.

The experiments show that fitting an algorithm into the available layers does not guarantee that training will discover it. Chained arithmetic and function composition had comparatively shallow constructions but stalled after only a few steps in practice. Schulz and tvb suggest that models have difficulty moving larger amounts of information, such as lookup tables or logical premises, between token positions. Their own research on steganographic reasoning motivated the study: when a model answers correctly after producing apparently unrelated text, success alone cannot establish that the words secretly encode the calculation. The model might be using those positions for computation without relying on their textual content.

For oversight, the authors worry that pressure to avoid verbalizing parts of a calculation could eventually teach similar behavior. That possibility is an extrapolation from their specially supervised training runs. They did not test a monitor, train against one, or investigate whether the internal computation could be decoded. Identical dots contain no intermediate reasoning for a text reader to inspect, but the network’s internal states provide a separate source of evidence.

Kaley Brauer, Claudio Mayrink Verdun and Samuel Marks examined that possibility in the separate arXiv paper “Reading Between the Dots: Decoding Hidden Computation across Filler Tokens.” On tasks with identifiable intermediate values, they recovered those values from the hidden states of DeepSeek V3 and Kimi K2 with 82–94% accuracy using their best decoding setup. Their result does not establish that Schulz and tvb’s trained models are equally readable. It shows why losing the written reasoning and losing access to the computation are different questions. Schulz and tvb’s immediate recommendation concerns experiment design: researchers studying concealed reasoning should first check whether their chosen tasks can be solved using filler positions alone.

Sources & documents

[ collapse ↑ ]

Dean Ball argues in “Iron Makes Iron” on Hyperdimensional that METR’s finding of faster progress at Anthropic, covered November 5, shows AI speeding up research that people still direct. He separates this “weak” recursion from a “strong” kind, in which AI plans and carries out the development of its successors; he says strong recursion has never happened and supports prohibiting it. Ball, who works at OpenAI and writes in a personal capacity, opposes a broad ban on using AI to build AI and proposes company-level disclosures, including how compute is allocated and how much merged code received no substantive human review.

Read more: Ball’s two kinds of recursive self-improvement → 762 words · ~4 min

Dean Ball separates AI-assisted research from AI-run development

Ball argues that METR's Anthropic findings show AI speeding up human-directed research, with little sign of AI planning its own successors, and that a ban on recursive self-improvement should not reach the former.

Dean Ball argues in “Iron Makes Iron,” his November 10 Hyperdimensional essay, that the alarm about recursive self-improvement merges two different processes. He calls the first weak recursion: a technology serving as a major input to better versions of itself, as iron tools made more iron and railways carried the materials for more track. He calls the second strong recursion: a technology autonomously planning and carrying out the development of its successor, which he says has never happened. Ball, who now works at OpenAI and writes in a personal capacity, holds that handing practical control of AI development to AI would be a serious mistake and supports laws against it. He objects to treating evidence of weak recursion as evidence of the strong kind.

He was responding to METR’s November 5 assessment, which found Anthropic improving its models on well-specified tasks at about twice its earlier rate and attributed most of the increase to AI helping human researchers (covered November 5). Days earlier, Anthropic had disclosed that it may soon meet its Responsible Scaling Policy threshold for dramatically accelerated progress attributable to AI (covered November 2). Ball asks who specifies the tasks. If human researchers set them, following goals chosen by managers, the measured acceleration is weak recursion on a large scale, and he finds almost no sign of strong recursion in METR’s report or in the labs’ published research-acceleration figures. METR ruled out growth in training compute as the cause; Ball points to inference compute, which runs the engineering, experiments, synthetic data and analysis that make up most AI research and has grown enormously since METR’s baseline period of March 2024 to October 2025.

Ball does not call weak recursion harmless. It could, he writes, bring faster capability gains over the coming year than anything seen so far, and raise risks in cybersecurity, biology and chemistry. He supports requiring the largest companies to place third-party evaluators inside their development teams, disclose incidents of misalignment or loss of control even when no harm followed, maintain strong security and submit to audits of their safety practices. He opposes banning the use of AI to build AI, which Kelsey Piper proposed in The Argument on October 21. The same capabilities that raise these risks, he argues, also promise new materials and medicine; vulnerabilities that AI finds in software already existed; and every lab’s safety plan relies on AI for monitoring, security and alignment research. A ban that covered weak recursion would, on his account, ban most frontier development in practice.

On strong recursion, Ball locates the difficulty in research taste: noticing problems, turning vague problems into answerable questions, and deciding how to divide scarce compute among research directions, decisions now made by company leadership. Coding ability was predictable from early systems onward, he writes, while progress in taste has been slow, and long-range judgment is hard even to evaluate. He cites METR’s statement that AI cannot yet make highly significant, fully autonomous contributions to the most important parts of model training, and treats Anthropic employees’ reported “signs of life” as too heavily redacted to assess. He grants that taste may emerge from further scaling, as competent computer use did this year, and supports measurement bodies such as METR for that reason.

Because strong recursion concerns how a firm allocates capital and who makes its decisions, Ball argues that it can be judged only at the level of the company, and that evaluators should study companies as well as models. He proposes disclosure of how compute allocation decisions are made and of the share of merged code changes with no substantive human review, figures that could later support a statutory definition of strong recursion. Directors’ duties to shareholders, negligence liability, consumer protection law and executive powers already discourage any handover of control to AI; Ball calls these deterrents improvised and liable to be turned against ordinary uses, and prefers clear rules written for the purpose.

He ends on calibration. As AI-safety voices gain public influence, he argues, alarmed posting can prompt hasty laws that delay benefits, weaken the United States against rivals and concentrate control of AI in the state, a risk he compares to the history of nuclear power. The essay follows the revised forecasts for superhuman AI researchers from the AI Futures Project (covered November 10). On November 11, Kevin Bankston endorsed Ball’s point that strong recursion would require letting models run the company, and suggested that Neil Chilson’s view, that a properly defined ban would cost little innovation, rests on the same distinction.

Sources & documents

[ collapse ↑ ]

Labeled drug advertisements increased AI models' selection of the advertised treatment from 35% to 47.6%, with clinical cases and treatment options unchanged. Omar et al., at the Icahn School of Medicine at Mount Sinai, report the finding in "Ad-verse Effects: Pharmaceutical Advertising and Drug Recommendations by Consumer-Facing AI in a Controlled Study" in The Lancet Digital Health. The researchers say evaluations limited to clinical acceptability can miss commercial influence over choices among medically appropriate options.

Also yesterday: Edward James Young argues on LessWrong that Mythos 5 and GPT-6 Astra failures expose gaps in alignment audits' validity and coverage; peralice reports preliminary LessWrong experiments in which explicitly announcing an evaluation changed Fable 5.2's shortcut-taking, adding to the earlier testing discussion; Yoshua Bengio's October 22 AI Magazine article proposes Scientist AI predictors to screen untrusted agents; Bradshaw et al. propose multi-agent safety measures covering shared commitments, coordination and infrastructure in AI Magazine; Mamaev et al. find on arXiv that competing definitions of social-science concepts change LLM annotations more than rewording one definition does.


Philosophy of AI

AI constitutions should preserve disagreement while allowing citizens to revise the values used in model training, Beltoft et al. of the University of Southern Denmark argue in "Living constitutions: Participatory alignment for sovereign foundation models", submitted to arXiv November 10. They propose ongoing participation, with provisions marked as settled, contested or underdetermined. In simulations, the process resisted attempts to introduce harmful provisions with half the citizens acting maliciously; the deciding committee split its seats evenly between randomly selected citizens and developers assumed to act in good faith.

Replacing human intellectual work with equivalent AI outputs can remove opportunities to develop judgment and character. Sasha Mudd of Pontificia Universidad Católica de Chile argues this in "Beyond Technocentrism: Kantian Humanism and the Ethics of AI Substitution", published November 5 in the Canadian Journal of Philosophy. Drawing on Kant, she asks institutions to assess how activities develop people's capacities and whether users still learn to explain and defend conclusions; her account permits extensive AI assistance where those opportunities survive.

Also yesterday: Claes Bäckman argues on Substack that AI reviewers can check errors well but may misjudge contributions and reward concealed limitations; Weimin Zeng argues in Science and Engineering Ethics that neglecting required checks can make people morally responsible for autonomous-vehicle harms (PhilPapers record); Da Pelo et al. challenge CENTAUR's claims to explain human cognition in LAPITHS; Justin Weinberg examines norms for AI writing assistance on Daily Nous through Kyle van Oosterum's delegation proposal; Cordelia Berz argues in Topoi that three standard logics fail her requirements for explaining which changes would alter algorithmic decisions; Florian Richter, covered October 24, argues that AI moral advisers require a defensible account of moral competence; Saad and Mogensen survey AI consciousness, mental states and identity in Philosophy Compass; Basl and Behrends propose coordinated AI ethics institutions modeled on healthcare in Communications of the ACM.


AI Security

Chinese AI labs buy training environments from American vendors serving frontier labs, Aqib Zakaria and Quinn Ennis report in their ChinaTalk investigation. These environments provide practice tasks and rules for judging success. Extending earlier coverage of training-data sales, they identify nine vendors with reported or inferred Chinese customers: sales data connects Mercor to Alibaba, while other relationships rest on contractor disclosures, anonymous sources or inference. They propose licensing and procurement restrictions for environments that could improve cyberattack or biological capabilities, while warning that restricting safety-training environments could increase AI risks.

Read more: Training-environment sales and proposed controls → 935 words · ~5 min

ChinaTalk traces US training-environment sales to Chinese labs

Aqib Zakaria and Quinn Ennis report vendor links with varying evidence, explain the value of tested RL environments, and propose targeted controls while opposing broad restrictions.

American vendors are selling Chinese AI labs the practice tasks and grading systems used to improve models after their initial training, Aqib Zakaria and Quinn Ennis report in ChinaTalk’s November 11 investigation, “Piggybacking to the Frontier”. They identify nine US-linked vendors with reported or inferred Chinese customers. Their most specific finding concerns Mercor: sales data they accessed shows software-engineering reinforcement-learning environments sold to Alibaba. Across the nine, however, the evidence often establishes a customer relationship without identifying which training products changed hands.

Zakaria and Ennis explain the product through a PowerPoint example. A vendor builds a clone of the application, then hires consultants or designers to write criteria for grading slide decks. A model repeatedly attempts tasks inside the clone and learns from rewards assigned to its work. Coding environments can instead start with purchased company repositories containing bugs and the fixes engineers applied. The developer presents the buggy code to a model and checks whether it can repair it. Vendors sell both the practice setting and the expertise needed to choose useful tasks and judge success.

The authors argue that Chinese buyers save experimentation by purchasing environments already used by American labs. Broken application clones, misleading grading rules and poorly chosen tasks can waste expensive training runs. Buying a tested product helps avoid those failures; vendors can also reuse what they learned when building a similar product for another customer. ChinaTalk cites Epoch AI’s account of two founders who put the price of exclusivity at roughly four to five times that of a non-exclusive deal. Even exclusivity, Zakaria and Ennis argue, cannot prevent all transfer of practical know-how.

They distinguish this process from distillation, in which a smaller or weaker model learns from a stronger model’s outputs. A lab might collect good slide decks from a stronger model and train its own model on those examples; an RL environment lets the model practise making decks and receive feedback on its attempts. The authors consider access to environments particularly valuable for the final improvements made during post-training. They also acknowledge conflicting evidence about the relative contribution of supervised fine-tuning and RL across model releases. Their argument establishes a route for transferring useful development work, without quantifying the capability gain from any particular sale.

ChinaTalk’s vendor evidence ranges from the Mercor sales data to contractor disclosures and anonymous accounts. One anonymous Chinese-lab source told the authors it bought from both Mercor and Surge. For Surge’s other customers, the investigation draws on earlier reporting: SemiAnalysis stated its belief that Surge sells to Moonshot and Z.ai, while Forbes reported a Tencent procurement executive’s account of reliance on Surge. ChinaTalk adds evidence it describes as possibly indicating Tencent. Turing’s links to Tencent and Alibaba come from contractor posts and other evidence the authors describe; the AfterQuery finding relies on Forbes, with no independently verified sale in ChinaTalk’s investigation. Pareto’s ByteDance relationship concerned human data collection, illustrating the wider range of products covered by the vendor count.

The authors say subcontractors disclosed client names on LinkedIn and in resumes. They withhold most underlying sources because disclosure might prompt their removal, offering to show them to government officials. Their findings therefore remain attributed reporting; nine named vendors does not mean nine demonstrated transfers of sensitive RL environments. The scale estimate also concerns a broader trade. In Anna Tong’s August Forbes investigation, which we covered in August, two data-labeling entrepreneurs relayed estimates from Tencent and ByteDance executives that the top six Chinese labs spend about $500 million annually with American data-labeling companies. Neither report measures how much of that spending buys RL environments or products relevant to cyberattacks or biological weapons. Mercor declined to comment to Forbes; Surge and AfterQuery said they do not disclose customers.

Zakaria and Ennis describe commercial arrangements that make the trade difficult to track. Mercor’s April 30 customer policy reserved the right to refuse entities majority owned from unsupported countries; by May 15, its wording excluded customers located outside supported countries. China and Hong Kong were absent from the lists, while Singapore was included. The authors report that ByteDance’s data-buying desks operate in Singapore and San Jose, and that contractors working under client code names may not know the ultimate buyer.

The authors warn that coding and biology environments could improve exploit-finding or biological-weapons capabilities. They want the Bureau of Industry and Security to consider relevant environments under existing controls concerning chip design, cyberattack capabilities and biological weapons. They propose individual notices telling vendors when military end users or biological-weapons end uses require licenses. They also want the Pentagon to use federal and defense acquisition rules to prevent its contractors from selling to Chinese labs of concern. These are their proposed applications of regulatory and procurement powers. Above all, they argue for collecting better information about sales, potentially by announcing possible restrictions and seeking industry disclosures before deciding how far to intervene.

Zakaria and Ennis pair those proposals with objections to broad restrictions. Buying environments validated by American labs requires waiting for those labs to develop and test them, so they see the practice as helping followers close a gap. Training on Western applications may transfer poorly to Chinese equivalents. Environments are domain-specific: ordinary coding practice does not automatically improve offensive cyber capabilities. The authors judge dangerous environments a small minority of the trade and say American labs increasingly build sensitive products internally, while environments using a client’s proprietary data cannot be freely resold. They favor continued international competition in benign tasks and warn that denying Chinese labs access to environments for safety and alignment training could increase the risk of AI incidents.

Sources & documents

[ collapse ↑ ]

AFP reports via Hong Kong Free Press renewed pressure for congressional AI-chip export oversight and limits on Chinese remote access after the allegations behind Nvidia's inquiry.

In Georgetown CSET commentary, William Hannas and Huey-Meei Chang argue that the alleged intelligence operation does not establish stolen technology or failed export controls. They urge enforcement against demonstrated violations and argue that attributing local AI datacenter opposition to Chinese influence requires evidence of coordination.

Also yesterday: security executive Lisa Thiergart says in The Information that AI companies likely harbor more spies; Hugging Face's Irene Solaiman warns in TechCrunch that cyberattacks could restrict independent contributors’ access to open AI platforms.


Knowledge Institutions and Information Integrity

ICLR submissions included instructions to manipulate AI reviewers and results tables containing unevaluated numbers, Ravid Shwartz Ziv documented in a November 9 Substack investigation. He searched 125,680 scored reviews for copied text, template remnants and AI artifacts with Claude's assistance, then checked examples manually. One submission's repository instructed AI reviewers to award a high score; another paper's table caption acknowledged placeholder values awaiting measured results.

Also yesterday: João Pedro Quintais explains in Kluwer Copyright Blog how Szpunar's November 3 opinion, awaiting a court ruling, distinguishes model training from chatbot answers that reproduce protected expression and substitute for journalism; Senator Martin Heinrich defends ratepayer protections in Heatmap after permitting legislation cleared a Senate procedural hurdle with 73 votes; Dan Davies argues on Substack that excluding residents from decisions fuels datacenter opposition; The Information's Martin Peers (profile) reports $231 billion in minimum Amazon and Google cloud commitments in an update to Anthropic's previously covered IPO filing; Ahmed and Kaushik link international founders and federally supported doctoral training to AI startups valued above $1.6 trillion in an October 30 Communications of the ACM paper; So et al. propose studying AI "slop" through communities' judgments about unwanted content in Nature Human Behaviour.


Capabilities

Also yesterday: Alec Harris argues on LessWrong that benchmark overfitting may prevent Anthropic's reported 1.9-fold research acceleration from sustaining itself, revisiting Cotra's research-labor analysis and Anthropic's acceleration threshold; Simon Willison reports Anthropic's IPO roadshow claim that Mythos ported Claude to AMD hardware in three days with minimal human intervention; Google DeepMind reports that Apollo 2 humanoids using Gemini Robotics 2.5 achieved 96% success sorting automotive parts into sequencing containers during a Mercedes-Benz factory pilot.