🔍 Read the full analysis: Is Weight-Space Programming Legal At Home? Our Test Says Yes on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A home experiment edited the output weights of the local model Ornith 1.5 35B-A3B to reduce looping. On 10 published Lexassay tasks, the edited model completed all 10, compared with four in the same test setup, but its legal score remained zero and it produced 55 wrong or invented citations.
A home test of weight editing reduced looping in the local AI model Ornith 1.5 35B-A3B, helping it complete 10 of 10 published legal benchmark tasks in one setup, compared with four of 10 before the edit. The model’s legal score remained zero, however, and its 10 edited answers contained 55 wrong or invented citations, according to the experiment’s results on Lexassay.
The experiment followed a report from four MIT researchers about using an AI system to edit another model’s weights directly. The home test was a related but different method: a Claude Opus 5.5 session wrote code to run Ornith, measure an internal direction associated with looping, and remove that direction at half strength from the output weights of 27 of the model’s 40 layers. The test did not involve training the model, adding legal data, or asking an AI to interpret the weights as text.
The reported completion comparison uses the same test setup: the unedited model answered four of 10 published tasks, while the edited model answered all 10. On a separate run using sampling settings recommended in the model card, the unchanged model answered nine of 10. The source cautions that this was one run per setting and that the two unchanged-model setups are not directly comparable. On the edited answers, the model met 7 of 50 checked criteria, up from three, but its overall legal score remained zero.
The experiment’s author disclosed that Claude performed the edit and Claude agents carried out blind grading. OpenAI’s GPT-6 Astra, run in Codex, reviewed the code, figures and claims and identified faults, according to the account. The results and answers for the 10 published Lexassay tasks are public; five additional benchmark tasks are hidden. The article says the experiment is not legal advice.
Infographic 01 · What we changed
One loop direction, weakened at half strength
Inspired by the MIT post, but a simpler, classic technique (directional ablation): code written by a Claude Opus 5.5 session measured one loop direction by running the model and projected it out of the weights. No training, and no AI reading weights as text.
The six steps
- Write test material that is not the benchmark: 24 legal development questions plus 8 general controls, 0 overlap with benchmark prompts
- Find the true loop onset at token level: from there, at least 90% of the next 1,500 positions repeat earlier 32-token sequences
- Run the model and measure one loop direction per layer: average internal signal inside the loop minus the reasoning just before it
- Weaken it in the weights:
W ← W − α·r·(rᵀW)on the MLP output matrices of all 256 experts and the shared expert - Bake an edited copy; the original files stay untouched and the edited weights stay internal
- Benchmark original and edit on the same local setup, graded blind by three Claude agents; figures cover the 10 published tasks
Quick test: share of pure copying
Restart 20 tokens before a known loop onset, 8 cases, 3,000 tokens. Lower is better.
The cost of the edit
- Full strength: 23 of 24 legal questions finished, but control answers broke: tokens dropped out of short formulas, and month counting went wrong
- Half strength: 23 of 24 finished, 9 of 9 checked control facts right, and one German control question was answered in Chinese once
- Before the edit: 8 of 24 legal development questions did not finish; all 8 controls did
One run per setting. The loop detector was revised three times during the experiment, so the development numbers come from a moving yardstick. The direction is a statistical description, not a single “loop neuron”.
Sources: our experiment records, 4–5 Nov 2026 (executing Claude Opus 5.5 session; review by GPT-6 Astra in Codex); Ornith-1.5-35B-A3B model card.
Powered by Thorsten Meyer AI
Completion Is Not Legal Accuracy
The result illustrates a limited but meaningful distinction for people experimenting with local AI: changing a model may help it finish a response without making the response reliable. In this test, fewer unfinished answers did not translate into a passing legal score. The 55 incorrect or invented citations are a direct warning against treating a fluent, completed legal answer as a verified one.
For readers considering similar edits at home, the test shows that a technically successful intervention can still fail at the task that matters. Legal research depends on correct rules and traceable authorities, not just an answer that reaches the end. The published benchmark checks citations against primary sources, but the results cover only its 10 public tasks and one model under specific local conditions.
The experiment also does not establish that weight editing is generally safe, effective or legally consequential. It reports a model-performance test, not an assessment of the lawfulness of editing model weights. The title’s “legal at home” framing should not be read as a legal conclusion: the source provides no legal analysis of licensing, local regulations or other possible restrictions.
Popular right now
Vitamix Touch & Go Advance 32-Ounce Bar Blender, White
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
KitchenAid Artisan 5-Quart Tilt-Head Stand Mixer, Blue Velvet
As an affiliate, we earn on qualifying purchases.
From MIT’s Edit to a Home Test
The MIT work described in the source was presented by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab as “weight-space programming.” In their reported Qwen example, GPT-6 Astra reasoned about a model’s weights and removed a direction associated with looping. The source reports that the Qwen model’s math score rose from 7.9% to 48.1%; that is a separate experiment and should not be confused with the home test.
The home experiment used a more established approach called directional ablation: run the model, record internal signals, calculate a direction, then project it out of selected weights. The test chose Ornith because its benchmark runs showed looping: two answers did not finish before the 62,000-token limit, and the final portions of those transcripts contained few distinct lines. The model’s publisher describes it as a coding and agent model, not a legal model. The source says other candidates either lacked the same looping pattern or could not be tested under comparable conditions.
Lexassay asks models 15 questions involving German, US and EU law, without internet access or tools, and checks cited laws and court decisions against primary sources. At the time of the update, 10 tasks were published and five were held back. The article’s figures refer to the published tasks, not the full set of 15.
Limits of the Published Test
The results are preliminary: there was one run per setting, the development set was small, and the loop detector changed during the experiment. The source also says that three graders came from the same model family, despite the separate review by GPT-6 Astra. These conditions limit how confidently the result can be generalized to other prompts, models or runs.
The comparison has an additional constraint. The unchanged model answered eight of 10 tasks in a published run but only four of 10 in the setup used for the direct comparison; with the model-card sampling settings, it answered nine of 10 without an edit. The account says these setups are not directly comparable. It is not clear from the supplied material how the edit would perform across repeated trials or on the five hidden tasks, nor whether it would help other kinds of model errors.
The headline question about legality also remains unanswered by this experiment. The source reports an AI test and identifies the model’s MIT license, but does not establish whether every relevant use, modification or distribution is lawful in every jurisdiction. Readers should not treat the experiment’s title as a legal opinion.
The next useful test would be to compare edited and unedited models across repeated runs under identical settings, then evaluate them on Lexassay’s five hidden tasks. That could help show whether the completion change is reproducible and whether it extends beyond the questions used while developing the method. The source does not announce a date for such testing.
For now, the public Lexassay runs provide the available record: answers, grades and citation checks for the 10 published tasks, along with a method page. Until more comparable results are available, the supported conclusion is narrow: the edit helped this model complete more answers in one setup, but did not produce a passing legal score. Any claim that weight editing makes a local model legally reliable—or settles whether a particular home use is lawful—would go beyond the evidence reported.
Key Questions
What did the home experiment change?
It removed an internal direction associated with looping at half strength from the output weights of 27 of Ornith’s 40 layers. The method used code to run and measure the model; it did not train the model or add legal data.
Did the edited model become better at law?
Not according to its overall benchmark score. It completed all 10 published tasks in the comparison setup, but its legal score stayed at zero and the answers contained 55 wrong or invented citations.
Is weight editing legal to do at home?
This experiment does not answer that legal question. It reports a model test, not legal advice or an analysis of licensing and other rules. The source notes Ornith’s publisher released it under the MIT license, but that fact alone does not resolve every legal issue.
How reliable are the results?
The findings are limited: there was one run per setting, a small development set, and differences between test setups. The source says the results cannot establish how the edit would perform across repeated trials or on the five hidden benchmark tasks.
Source: ThorstenMeyerAI.com
