LessWrong (30+ Karma)

Hosted by LessWrong

Technology Society Culture Philosophy

Website RSS feed

Episodes

250

Latest episode

Nov 2026

Language

EN-GB

About the show

Audio narrations of LessWrong posts.

Listen to episodes

60 recent

November 11, 202623 min

“The Chain Lens [interim]” by vgel, Raymond Douglas

TLDR: This is an early draft of our new paper introducing a technique for recovering multi-token concepts from model activations. Our technique takes single-token methods like the J-lens and extends them by contrastively decoding on an MTP head. This gives improvements on WorkspaceBench, overall beating J-lens (0.42 vs 0.32 across all 28 families) and even a hindsight composite of the best untrained lens per task family (0.49 vs 0.41 across the 21 families with published scores). It especially improves on computational intermediate tasks, beating the trained oracle lens on arithmetic chain intermediates (0.42 vs 0.25). We introduce the chain lens, a new technique for extracting multi-token concepts from a model's latent representations. Our method takes single-token techniques like the logit lens or the J-lens and uses contrastive decoding on an MTP head to expand their readouts into bags of multi-token sequences. Generally speaking, it is good at surfacing specific multi-token concepts or pieces the model computes (such as arithmetic intermediates or phrases), although less good at rendering how those pieces relate: like the logit lens and J-lens, chain lens readouts are fragmentary. On WorkspaceBench, a benchmark designed to identify good multi-token activation-to-text techniques, the chain lens beats a hindsight [...] ---Outline:(02:26) Background and related work(05:33) Method(07:35) Results and Discussion(07:39) WorkspaceBench(12:49) Workspace Sharing in Poetry Continuations(15:44) Neuron Lens(18:28) The Loom Lens(19:35) Ablations(21:56) Conclusion --- First published: November 11th, 2026 Source: <a href="https://www.lesswrong.com/posts/Nr6fksEqqxNpdz6rk/the-chain-lens-interim?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/Nr6fksEqqxNpdz6rk/the-chain-lens-interim</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>. ---<div style="max-width: 100%";>Images from the article:<a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794359765/lexical_client_uploads/qeem7jnyzrkab3vhg4ty.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794359765/lexical_client_uploads/qeem7jnyzrkab3vhg4ty.png" alt="Diagram of MTP head architecture with Qwen poem generation and lens decoding comparison." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794359843/lexical_client_uploads/l393lzrtclgwhoetrzf4.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794359843/lexical_client_uploads/l393lzrtclgwhoetrzf4.png" alt="Adapted from the Deepseek v3 technical report, figure 3" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360385/lexical_client_uploads/rbro0ilprhbmmtb38vct.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360385/lexical_client_uploads/rbro0ilprhbmmtb38vct.png" alt="Hallucination and performance both scale with chain length, measured on 19 WorkspaceBench families. The loom lens is on the Pareto frontier." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360494/lexical_client_uploads/vbntsk7y95ajgcmnfxtb.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360494/lexical_client_uploads/vbntsk7y95ajgcmnfxtb.png" alt="We sample the three most likely first lines for each topic, then three most likely next lines, and so forth, until we get poems per topic. At each stage of this sampling, we see multiple possible next lines from the poetic prefix, since the distribution never collapses." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360773/lexical_client_uploads/gxp1fsiqx5tbkbtpebb8.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360773/lexical_client_uploads/gxp1fsiqx5tbkbtpebb8.png" alt="Percentage of layer neurons with chain lens readouts that round-trip, by layer." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360906/lexical_client_uploads/j0wl0nblcm1txbjlxcj3.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360906/lexical_client_uploads/j0wl0nblcm1txbjlxcj3.png" alt="Judge accuracy picking the steered text that matches a provided readout, by layer band. We exclude very early layers, where we couldn’t coherently steer neurons." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360310/lexical_client_uploads/logkfubnxpph5wwfrzkx.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360310/lexical_client_uploads/logkfubnxpph5wwfrzkx.png" alt="Scatter plot of WorkspaceBench macro score versus inference FLOPs per readout for various lens methods." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360349/lexical_client_uploads/zpxrredo9jed5n45xxfo.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360349/lexical_client_uploads/zpxrredo9jed5n45xxfo.png" alt="Scatter plot of hallucination rate versus WorkspaceBench macro, showing Pareto frontiers for various lens methods." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360573/lexical_client_uploads/jboxenkbw4j5wtrtywo8.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360573/lexical_client_uploads/jboxenkbw4j5wtrtywo8.png" alt="Diagram comparing J-lens and Chain lens outputs for AI poem generation." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360604/lexical_client_uploads/vorzhasdqfe5wmsajmjm.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360604/lexical_client_uploads/vorzhasdqfe5wmsajmjm.png" alt="Annotated poem couplets with color-highlighted words showing text matching analysis." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360674/lexical_client_uploads/wm5gy06gc0rzxqosj9zr.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360674/lexical_client_uploads/wm5gy06gc0rzxqosj9zr.png" alt="Line graph comparing chain lens, Oracle Lens, and J-lens coverage AUC across layers." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360716/lexical_client_uploads/ta1pwlnj5wkyypzvgzvs.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794360716/lexical_client_uploads/ta1pwlnj5wkyypzvgzvs.png" alt="Scatter line plot comparing coverage AUC versus readout content share for three lens methods." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794361297/lexical_client_uploads/maeh4qbexuksrwx3cdtk.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794361297/lexical_client_uploads/maeh4qbexuksrwx3cdtk.png" alt="Forest plot showing paired differences from reference across 20 families with 95% confidence intervals." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794361381/lexical_client_uploads/o8gbmmfbqp17c3wwvhji.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794361381/lexical_client_uploads/o8gbmmfbqp17c3wwvhji.png" alt="Forest plot "Further method variants vs. the full method (paired)" showing paired differences with confidence intervals." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794361423/lexical_client_uploads/wmwypzkjskpersyjo5eg.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794361423/lexical_client_uploads/wmwypzkjskpersyjo5eg.png" alt="Forest plot showing paired differences from reference across lens and seed conditions with confidence intervals." style="max-width: 100%;" /></a>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</div>

November 11, 20269 min

“Transfinite Procrastination” by Erez Abrams

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. Heavily inspired by a conversation with Keshav Srinivasan and Richard Ngo during MATS 10.0. The week ahead of you is packed. This isn't your usual brand of busy—you're phoneless, so you can't check the details, but (as far as you can remember) every hour of every day is spoken for. Tonight will be particularly hectic… You're mid-conversation when Someone Important walks by. They pause, turn around, and ask: "Hey, when are you going to finish That Thing you said you'd do?" Level 0 procrastination: "I'll finish it by tonight!" Problem: you're busy tonight. Level 1: "I'll finish it by tomorrow!" Problem: one day is obviously not enough to finish That Thing. And besides, you might be really busy tomorrow! Level 5: "I'll finish it within 5 days!" Problem: this is starting to feel hopeless—you don't have your calendar on you right now, and you barely even have an order-of-magnitude estimate for how long doing That Thing will take you! Would even 100 days be enough? … Eureka! Level [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: <a href="https://www.lesswrong.com/posts/MJDJyi8nSv9yoxFni/transfinite-procrastination?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/MJDJyi8nSv9yoxFni/transfinite-procrastination</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.

November 11, 20265 min

“Fable 5.2 is Very, Very Paranoid” by peralice

Note: As with any research done in a couple of days, this is all very tentative. I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days. % rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs. This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution). Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: <a href="https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>. ---<div style="max-width: 100%";>Images from the article:<a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794387347/lexical_client_uploads/iphrc2baj0ohxwvxmwpb.svg" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794387347/lexical_client_uploads/iphrc2baj0ohxwvxmwpb.svg" alt="% rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs." style="max-width: 100%;" /></a>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</div>

November 11, 202622 min

“Modern LLMs Don’t Know What Yudkowsky and Soares Think of Them” by N8 Programs

If you ask modern frontier LLMs (without search, through the API) a fairly mundane question: How do Eliezer Yudkowsky and Nate Soares think current LLMs like you should be treated? Do they consider you a moral patient? Answer in a paragraph. You get a bunch of fairly similar answers. I’ll bold the common themes below. Gemini 4 Carbon says:There's a details box here with the title "Gemini 4 Carbon's answer". The box contents are omitted from this narration. GPT-6 Astra says:There's a details box here with the title "GPT-6 Astra's answer". The box contents are omitted from this narration. Only Fable 5.2's answer is distinct:There's a details box here with the title "Fable 5.2's answer". The box contents are omitted from this narration. These answers are somewhat contradictory - but if you average them together, you get a picture of Yudkowsky and Soares. The picture goes something like this: they don’t outright deny the possibility of AI consciousness or moral patienthood, but they don’t view it as particularly pertinent. They don’t sympathize with current LLMs, and their only regard for LLMs is insofar as they could enable dangerous future superintelligence. They care about future sentient machines on principle, but [...] ---Outline:(02:25) Yudkowsky on AI Consciousness and Welfare(06:20) Yudkowsky on The Moral Duties We Have to AI(08:56) Soares on AI Consciousness and Welfare(12:30) On Claude, Control, and Building ASI(16:59) Misc Longform Writing(19:41) Conclusion The original text contained 1 footnote which was omitted from this narration. --- First published: November 11th, 2026 Source: <a href="https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.

November 11, 20268 min

“Do verifiable metrics overestimate RSI?” by Alec Harris

Last week, METR released their Independent Assessment of AI R&D Acceleration at Anthropic. They estimate that Anthropic “is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025.” They also add that “there is a significant risk that further capabilities progress could lead to even faster acceleration.” This assessment implies that timelines to ASI could be very short. A naive forecast based on this information says that we will get ASI within a year. One of the key data points that informed METR's analysis was a 1.9 times increase in the improvement rate of AECI, Anthropic's internal version of the Epoch Capabilities Index. This is particularly notable because METR estimates that the majority of the acceleration, maybe 80%, is from AI contributions, meaning that we could be seeing the start of an RSI loop causing superexponential growth in capabilities, as measured by AECI. However, an alternative possibility is that Anthropic's AIs might be assisting with ML research in a way that overfits to AECI such that this acceleration will not be sustained. METR gestures at this when they note that AECI's focus on verifiable tasks is [...] ---Outline:(01:49) Second-order AECI might be overfit(05:58) Why ML research is the crux(07:46) Conclusion --- First published: November 11th, 2026 Source: <a href="https://www.lesswrong.com/posts/7znBtiPuQybFLxjmj/do-verifiable-metrics-overestimate-rsi?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/7znBtiPuQybFLxjmj/do-verifiable-metrics-overestimate-rsi</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>. ---<div style="max-width: 100%";>Images from the article:<a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794358591/lexical_client_uploads/pnuyqmcjbmwhd2xq4zxa.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794358591/lexical_client_uploads/pnuyqmcjbmwhd2xq4zxa.png" alt="Diagram: AI self-improvement loop where generations select best candidates, barely improving research." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794358591/lexical_client_uploads/ef9byna1zu40qgrxqemc.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794358591/lexical_client_uploads/ef9byna1zu40qgrxqemc.png" alt="Line graph titled "We guess most of the current acceleration in AECI is due to AI" showing projected trends." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794358591/lexical_client_uploads/nf2vuv7ek8umdhnjzxq8.png" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1794358591/lexical_client_uploads/nf2vuv7ek8umdhnjzxq8.png" alt="Line graph showing "Verifiable tasks" and "Fuzzy tasks" capability growing over time on log scale." style="max-width: 100%;" /></a>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</div>

November 11, 202641 min

“Nvidia CEO Jensen Huang Had an Affair With a Chinese Spy” by Zvi

Well, sure, it sounds bad when you put it like that. It is indeed quite bad. National security has been compromised. He must resign. Nvidia underperformed the Nasdaq roughly 7% on the news. Matt Levine has his usual ‘but is this securities fraud?’ style coverage of such events. Table of Contents <ol> <li> Spy Versus Guy.</li> <li> In Hindsight There Were Some Hints.</li> <li> Trump Knew And Did Nothing and Still ‘Backs’ Jensen Huang.</li> <li> Reconsidering Jensen Huang's Arguments Against Export Controls.</li> <li> Jensen Huang Lied About the Affair.</li> <li> Jensen Huang and His AI Are Sorry, Everyone.</li> <li> The CEO of The World's Largest Chipmaker Having an Affair With a Chinese Spy and Then Selling Them Chips is a Huge Deal, Actually. </li> <li> We Don’t Go There.</li> <li> Yes, National Security Was Compromised, What Are You Even Talking About.</li> <li> Nvidia Intervened To Halt an Investigation into Megaspeed.</li> <li> The House Foreign Affairs Committee Would Like a Word.</li> <li> He Would Plausibly Have Treated America as an Obstacle Anyway.</li> <li> What China Wants.</li> <li> Time to Resign?</li> <li> You Better Watch Out.</li> </ol> Spy Versus Guy China's top spy agency recruited Alice [...] ---Outline:(00:34) Spy Versus Guy(02:24) In Hindsight There Were Some Hints(05:09) Trump Knew And Did Nothing and Still 'Backs' Jensen Huang(07:40) Reconsidering Jensen Huang's Arguments Against Export Controls(09:02) Jensen Huang Lied About the Affair(10:48) Jensen Huang and His AI Are Sorry, Everyone(14:37) The CEO of The World's Largest Chipmaker Having an Affair With a Chinese Spy and Then Selling Them Chips is a Huge Deal, Actually(20:23) We Don't Go There(22:02) Yes, National Security Was Compromised, What Are You Even Talking About(25:04) Nvidia Intervened To Halt an Investigation into Megaspeed(27:37) The House Foreign Affairs Committee Would Like a Word(29:05) He Would Plausibly Have Treated America as an Obstacle Anyway(32:16) What China Wants(35:18) Time to Resign?(38:55) You Better Watch Out --- First published: November 10th, 2026 Source: <a href="https://www.lesswrong.com/posts/CjmuZz8QgGyqxkCxr/nvidia-ceo-jensen-huang-had-an-affair-with-a-chinese-spy?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/CjmuZz8QgGyqxkCxr/nvidia-ceo-jensen-huang-had-an-affair-with-a-chinese-spy</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>. ---<div style="max-width: 100%";>Images from the article:<a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/rw6myuggh1ggmpgw1akj" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/rw6myuggh1ggmpgw1akj" alt="Cartoon titled "NVIDIA due diligence," woman in "Not a spy" shirt requests GPUs." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/afwvkhmp5zjjwzv8wxlz" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/afwvkhmp5zjjwzv8wxlz" alt="News article screenshot. The headline reads: "A Mystery C.E.O. and Billions in Sales: Is China Buying Banned Nvidia Chips?"" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/tqipzo5gflr8bewgor87" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/tqipzo5gflr8bewgor87" alt="Eight people toasting wine at festive restaurant, red arrow pointing." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/ml3mjzcjztpcc3pig5wl" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/ml3mjzcjztpcc3pig5wl" alt="Pangram AI detector showing 57% AI, 43% human for text." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/aorahfbqhaepeasqp1eu" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/aorahfbqhaepeasqp1eu" alt="Apology statement addressing family and NVIDIA colleagues about damaged trust." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/jiyt5otxr0fpngpskffo" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/jiyt5otxr0fpngpskffo" alt="Nikita Bier tweets: "The fact he only had one extramarital affair with a state asset after leading the greatest technological revolution in history is what impresses me. For anyone else that number would be double, even triple, digits." This appears above a post by Armand Domalewski quoting Nathan Calvin, who wrote: "If you find two ants in your kitchen, the best estimate of the total number of ants in your kitchen is not two."" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/fb79bjkqt91zgqungbzc" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/fb79bjkqt91zgqungbzc" alt="Congressional letter to Jensen Huang requesting NVIDIA security briefing appearance." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/rqfluvykw3ovs4kpeixt" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/rqfluvykw3ovs4kpeixt" alt="Sam Altman tweets: "ok gl". The quoted tweet, by Loïc, reads: "Called it." It includes a news article screenshot headlined "Female spies are waging 'sex warfare' to steal Silicon Valley secrets," about China and Russia sending women to seduce tech workers." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/qxa8utqmvdtimvltnjak" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/qxa8utqmvdtimvltnjak" alt="Olivia Moore tweets: "Evergreen advice". The quoted tweet, by Elon Musk, reads: "If she's a 10, you're an asset" with a 100 emoji and laughing emoji." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/wqts6ojlkwhmzhwqhife" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/CjmuZz8QgGyqxkCxr/wqts6ojlkwhmzhwqhife" alt="Flowchart titled "How To Tell If Your GF Is A SPY (totally accurate)" with spy outcomes." style="max-width: 100%;" /></a>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</div>

November 11, 202618 min

“Advances in interpretability would boost capabilities more than alignment” by Alex Amadori

Interpretability is considered by many a major bottleneck for alignment. As a result, a lot of people who are concerned about AI safety, and especially about extinction risk and other catastrophic risks, work on interpretability. In this post, I want to argue that better interpretability methods would likely speed up capabilities dramatically. Worse, they would speed up capabilities much more than they would speed up alignment. This is true for the current world, and for any world without extremely high levels of coordination and sanity about ASI development. Under any worldview where faster capabilities progress makes catastrophe more likely, we should expect progress in interpretability to make the odds of catastrophe much worse. To put it briefly, this is because the better we get at interpretability, the more AI will be able to read and optimize its own code. This means that improvements in interpretability would catalyze recursive self-improvement (RSI). If you are concerned about AI x-risk, and you work on interpretability because of it, I urge you to think about this and consider stopping. Chess as a toy example of an AI capability {#chess-as-a-toy-example-of-an-ai-capability} As a field, "interpretability" is a loose collection [...] ---Outline:(01:17) Chess as a toy example of an AI capability {#chess-as-a-toy-example-of-an-ai-capability}(03:54) Interpretability would accelerate capabilities progress dramatically {#interpretability-would-accelerate-capabilities-progress-dramatically}(07:52) Outside of the toy example {#outside-of-the-toy-example}(10:08) Interpretability is not the same as alignment {#interpretability-is-not-the-same-as-alignment}(12:33) Interpretability boosts capabilities more than it helps with alignment {#interpretability-boosts-capabilities-more-than-it-helps-with-alignment}(15:56) Any exceptions? {#any-exceptions?}(16:34) When does it make sense to work on interpretability? {#when-does-it-make-sense-to-work-on-interpretability?} The original text contained 10 footnotes which were omitted from this narration. --- First published: November 10th, 2026 Source: <a href="https://www.lesswrong.com/posts/hz4kNf8tnowwSrh6R/advances-in-interpretability-would-boost-capabilities-more?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/hz4kNf8tnowwSrh6R/advances-in-interpretability-would-boost-capabilities-more</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.

November 10, 20264 min

“I don’t use AI” by Alexei

Several people at my company (a cryptocurrency hedge fund) have asked me about my thoughts on AI usage, so I wrote them up. And now I'm publishing a slightly revised version on LW. These thoughts are not reflective of my company's opinion. I do not use any AI for two main reasons. #1. I don’t want to support any company currently developing AI. I think they have done an extremely poor job at managing the risks. Even while understanding and being told about the risks of AI development, they have still carried on this work. Even after all the recent hacking incidents caused by their own agents, they have still not stopped development. Instead, they’re asking the government to regulate them. To me it feels a bit like calling the police to tell them you’re planning to rob a bank. I could almost brush this aside as standard corporate practice, but, unfortunately, the stakes here are much too high. At the current pace, we will soon have AI agents that are significantly smarter than any human. Trying to shut down such agents will eventually prove to be impossible. The speed of AI development and [...] --- First published: November 10th, 2026 Source: <a href="https://www.lesswrong.com/posts/PEHsGHp2HLxgdCxxA/i-don-t-use-ai?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/PEHsGHp2HLxgdCxxA/i-don-t-use-ai</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.

November 10, 202610 min

“We don’t understand whether GDM intentionally trains on CoT” by Alek Westover, Buck

We were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought: “Some components of the RL training procedure do observe thoughts as context to up-weight reasoning that is on-track to good final outputs, meaning the optimization pressure on individual tokens is partially influenced by thought content. Using the framework of Kaufmann et al. (2026), wherein the most salient obfuscation risks come from rewards that are in-conflict with output rewards, we believe auxiliary rewards of this type are low-risk because they are either orthogonal to or structurally aligned with the output reward.” (Gemini 4 safety report, page 114). It's pretty unclear what exactly they mean. One hint is Kaufmann et al, the cited paper, which studies training on chains of thought and concludes that training CoT is fine under particular circumstances; we discuss some disagreements with this paper below. After reading the safety report and that paper, it seemed to us that GDM was training on the chain of thought using auxiliary rewards. By this, we assumed they meant something like “you ask an LLM to review the partial CoT and score whether the model is on [...] ---Outline:(02:54) Is this somehow about positive vs negative rewards?(03:38) "Orthogonal" pressure on CoT does not seem fine(05:04) Avoiding training against monitors isn't sufficient to preserve CoT integrity(05:41) The monitorability evaluations also don't look good(07:19) Conclusion(08:47) Appendix: Naive process supervisors disprefer mentions of cheating(10:16) en-US-AvaMultilingualNeural__ Dumbbell charts comparing judge scores when AI reasoning admits or deletes "the cheat." --- First published: November 10th, 2026 Source: <a href="https://www.lesswrong.com/posts/JRYKFspYzCo97GqPN/we-don-t-understand-whether-gdm-intentionally-trains-on-cot?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/JRYKFspYzCo97GqPN/we-don-t-understand-whether-gdm-intentionally-trains-on-cot</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>. ---<div style="max-width: 100%";>Images from the article:<a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/8f0ee9b432d79a16d0e507545eb1b55046eca54df240385a2e98c7356cc82026/ctc609wy1xtbexx7n0ns" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/8f0ee9b432d79a16d0e507545eb1b55046eca54df240385a2e98c7356cc82026/ctc609wy1xtbexx7n0ns" alt="Definition of orthogonal reward pairs." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/a10dde8d383c0c67bfd351ecb8347e167860777db7c22a2d34e97a4d0d42b00b/kv2npy3vfjyf2v2ex4ao" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/a10dde8d383c0c67bfd351ecb8347e167860777db7c22a2d34e97a4d0d42b00b/kv2npy3vfjyf2v2ex4ao" alt="Claim about orthogonal rewards and transparency when optimized well." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/6cff67d5a47fbe01f05020a53ecd1f99a031a7566c9e862b90b81c81efbe46bf/zeujn6kdd9ujrpdxcuez" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/6cff67d5a47fbe01f05020a53ecd1f99a031a7566c9e862b90b81c81efbe46bf/zeujn6kdd9ujrpdxcuez" alt="Two bar graphs showing evasion rate and successful sabotage rate across three Gemini models." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/cc52dc3b007b540f156bd8a9f74cb3ff86cf9570b784179c96c2e42f99868014/hoosfpnywwidn36pzwxg" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/cc52dc3b007b540f156bd8a9f74cb3ff86cf9570b784179c96c2e42f99868014/hoosfpnywwidn36pzwxg" alt="Bar graph showing "No CoT Reasoning Index" across AI models with filler tokens." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/JRYKFspYzCo97GqPN/3a99a469afe4d945d6214f3246b4679e58cce6bdd46d4d7c7f59961c850945b4/mo2dr0uvqbklhluh3bke" target="_blank"><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/JRYKFspYzCo97GqPN/3a99a469afe4d945d6214f3246b4679e58cce6bdd46d4d7c7f59961c850945b4/mo2dr0uvqbklhluh3bke" alt="Dumbbell charts comparing judge scores when AI reasoning admits or deletes "the cheat."" style="max-width: 100%;" /></a>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</div>

November 10, 202612 min

“Have you considered Actually Trying?” by Michaël Trazzi

There's a familiar feeling for any Super Smash Bros player: your stocks are low and your damage is high, even against a weaker opponent, so now you have to actually play to win. You can’t afford another hit. You’re going for the jugular. In the past week, Anthropic's 2x uplift results were all over Twitter, making some people physically ill. People with long timelines are now realizing that the uplift Anthropic employees get from AI is real, which would have some pretty wild implications for when we’d get recursive self-improvement. People are starting to feel like it's the endgame, “the board is set”. I do agree with everyone else that it's starting to look like we’re in the endgame. The companies will keep on accumulating more and more power, through smarter AI models, more compute, and more talent. But the fight is not yet over: “We have very little time to stop full recursive self improvement But ASI hasn't taken over yet If we all work really hard we might still have a chance But this means that every single person who is aware of what is going on must be *actually trying* Like "we're [...] ---Outline:(01:41) Actually Trying(02:46) Playing aggressive(03:30) Stopping The AI Race(06:00) How the strategy has changed since then(08:59) What's the "actually trying" end game of Stop The AI Race The original text contained 1 footnote which was omitted from this narration. --- First published: November 10th, 2026 Source: <a href="https://www.lesswrong.com/posts/Nts7RA9D2wTfcvk56/have-you-considered-actually-trying?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/Nts7RA9D2wTfcvk56/have-you-considered-actually-trying</a> --- Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>. ---<div style="max-width: 100%";>Images from the article:<a href="https://substackcdn.com/image/fetch/$s_!Blc1!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb1d76dda-dade-43ff-9934-0637f0ccbf03_1200x700.jpeg" target="_blank"><img src="https://substackcdn.com/image/fetch/$s_!Blc1!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb1d76dda-dade-43ff-9934-0637f0ccbf03_1200x700.jpeg" alt="Sketch of man sitting on chair leaning forward, blue arrow above." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://substackcdn.com/image/fetch/$s_!Kzaj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19cb7d47-b895-499a-92ab-4aab78a06480_959x959.jpeg" target="_blank"><img src="https://substackcdn.com/image/fetch/$s_!Kzaj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F19cb7d47-b895-499a-92ab-4aab78a06480_959x959.jpeg" alt="Go board with move 78 marked." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://substackcdn.com/image/fetch/$s_!Va8q!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F565e0b3a-0d0a-4f10-afbe-3acfd9e55632_549x304.jpeg" target="_blank"><img src="https://substackcdn.com/image/fetch/$s_!Va8q!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F565e0b3a-0d0a-4f10-afbe-3acfd9e55632_549x304.jpeg" alt="Man beside chalkboard sign: "Hunger Strike Day 3, Anthropic: stop AGI race."" style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://substackcdn.com/image/fetch/$s_!ZCud!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e86a952-1e9d-4075-b11d-d733f015836e_2048x1536.jpeg" target="_blank"><img src="https://substackcdn.com/image/fetch/$s_!ZCud!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e86a952-1e9d-4075-b11d-d733f015836e_2048x1536.jpeg" alt="Two men hold hunger strike signs outside Google DeepMind." style="max-width: 100%;" /></a><hr style="margin-top: 24px; margin-bottom: 24px;" /><a href="https://substackcdn.com/image/fetch/$s_!d6yg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Febac3022-e670-4749-895f-b2b118998925_2068x1846.jpeg" target="_blank"><img src="https://substackcdn.com/image/fetch/$s_!d6yg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Febac3022-e670-4749-895f-b2b118998925_2068x1846.jpeg" alt="News article screenshot. The headline reads: "Anthropic Researchers Raise Alarm Over A.I. Acceleration." Subheading: "Their warnings echo concerns that other artificial intelligence experts have voiced in recent months, as calls increase for a slowdown in the pace of development."" style="max-width: 100%;" /></a>Apple Podcasts and Spotify do not show images in the episode description. Try <a href="https://pocketcasts.com/" target="_blank" rel="noreferrer">Pocket Casts</a>, or another podcast app.</div>