We Found Something That Shouldn't Exist | Derrick Hodge

We Found Something That Shouldn't Exist 𝗧𝗵𝗲 𝗔𝗜 𝗳𝗶𝗲𝗹𝗱 𝗿𝘂𝗻𝘀 𝗼𝗻 𝗮 𝗰𝗼𝗿𝗲 𝗯𝗲𝗹𝗶𝗲𝗳: That intelligence in large language models is evenly distributed across all parameters. Recent research (arXiv:2505.24832) estimates models store ~3.6 bits per parameter, implying memory spreads layer by layer, weight by weight. The dominant belief follows: 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝘀𝗰𝗮𝗹𝗲𝘀 𝗹𝗶𝗻𝗲𝗮𝗿𝗹𝘆 𝘄𝗶𝘁𝗵 𝘀𝗶𝘇𝗲. But this assumes each parameter contributes equally to learning. That’s where 𝗙𝗶𝘀𝗵𝗲𝗿 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻 becomes critical. >> 𝘍𝘪𝘴𝘩𝘦𝘳 𝘐𝘯𝘧𝘰𝘳𝘮𝘢𝘵𝘪𝘰𝘯 𝘮𝘦𝘢𝘴𝘶𝘳𝘦𝘴 𝘩𝘰𝘸 𝘴𝘦𝘯𝘴𝘪𝘵𝘪𝘷𝘦 𝘱𝘳𝘦𝘥𝘪𝘤𝘵𝘪𝘰𝘯𝘴 𝘢𝘳𝘦 𝘵𝘰 𝘱𝘦𝘳𝘵𝘶𝘳𝘣𝘢𝘵𝘪𝘰𝘯𝘴 𝘪𝘯 𝘢 𝘴𝘪𝘯𝘨𝘭𝘦 𝘱𝘢𝘳𝘢𝘮𝘦𝘵𝘦𝘳. 𝗔 𝗵𝗶𝗴𝗵-𝗙𝗶𝘀𝗵𝗲𝗿 𝗽𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿 isn’t storing a bit. It’s controlling behavior. When we analyzed 𝗤𝘄𝗲𝗻𝟮.𝟱-𝟬.𝟱𝗕, that belief collapsed. >> 𝟵𝟰.𝟯% 𝗼𝗳 𝘁𝗵𝗲 𝘁𝗼𝘁𝗮𝗹 𝗙𝗶𝘀𝗵𝗲𝗿 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻 𝗶𝘀 𝗰𝗼𝗻𝗰𝗲𝗻𝘁𝗿𝗮𝘁𝗲𝗱 𝗶𝗻 𝗷𝘂𝘀𝘁 𝘁𝗵𝗿𝗲𝗲 𝘄𝗲𝗶𝗴𝗵𝘁𝘀. Not three layers. Not three matrices. Three individual scalars, all in early and late mlp.down_proj layers. They don’t look special. But they behave like computational black holes: >> 𝘛𝘩𝘦𝘺 𝘢𝘣𝘴𝘰𝘳𝘣 𝘦𝘯𝘵𝘳𝘰𝘱𝘺, 𝘳𝘢𝘥𝘪𝘢𝘵𝘦 𝘤𝘰𝘩𝘦𝘳𝘦𝘯𝘵 𝘴𝘪𝘨𝘯𝘢𝘭𝘴 𝘵𝘩𝘳𝘰𝘶𝘨𝘩 𝘴𝘬𝘪𝘱 𝘤𝘰𝘯𝘯𝘦𝘤𝘵𝘪𝘰𝘯𝘴, 𝘢𝘯𝘥 𝘤𝘰𝘮𝘱𝘳𝘦𝘴𝘴 𝘳𝘦𝘴𝘪𝘥𝘶𝘢𝘭 𝘭𝘰𝘴𝘴 𝘪𝘯𝘵𝘰 𝘴𝘦𝘮𝘢𝘯𝘵𝘪𝘤 𝘢𝘵𝘵𝘳𝘢𝘤𝘵𝘰𝘳𝘴. These weights aren’t just informative, they’re irreducible. Remove one and the model collapses. This aligns with "𝗧𝗵𝗲 𝗦𝘂𝗽𝗲𝗿 𝗪𝗲𝗶𝗴𝗵𝘁 𝗶𝗻 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀" (arXiv:2311.17035), which showed that pruning a single super weight can destroy more capability than removing thousands. 𝗕𝗹𝗮𝗰𝗸 𝗛𝗼𝗹𝗲 𝗗𝘆𝗻𝗮𝗺𝗶𝗰𝘀 These weights aren’t memorizing or generalizing. They anchor the transformer like singularities in curved space. 𝗛𝗲𝗮𝘁 𝗦𝗶𝗻𝗸: Absorb gradient energy 𝗘𝗻𝘁𝗿𝗼𝗽𝘆 𝗣𝘂𝗺𝗽: Radiate structured activation 𝗚𝗿𝗮𝘃𝗶𝘁𝘆 𝗪𝗲𝗹𝗹: Network funnels signal into them 𝗛𝗼𝗿𝗶𝘇𝗼𝗻: Cross it, collapse is irreversible ✓ Heat Sink: T(θₛ) → 0 ✓ Entropy Pump: S(θₛ) → min,𝓘_F(θₛ) → max ✓ Radiator: A_skip(θₛ) ≫ 0 ✓ Collapse: Ablate(θₛ) ⇒ Δ𝓛 ↑↑ > > 𝘐𝘯𝘵𝘦𝘭𝘭𝘪𝘨𝘦𝘯𝘤𝘦 𝘥𝘰𝘦𝘴𝘯’𝘵 𝘨𝘦𝘯𝘦𝘳𝘢𝘭𝘪𝘻𝘦 𝘣𝘺 𝘥𝘪𝘧𝘧𝘶𝘴𝘪𝘰𝘯. 𝘐𝘵 𝘤𝘰𝘯𝘥𝘦𝘯𝘴𝘦𝘴, 𝘨𝘳𝘢𝘷𝘪𝘵𝘢𝘵𝘪𝘰𝘯𝘢𝘭𝘭𝘺, 𝘪𝘯𝘵𝘰 𝘢 𝘧𝘦𝘸 𝘶𝘭𝘵𝘳𝘢-𝘴𝘵𝘢𝘣𝘭𝘦 𝘢𝘵𝘵𝘳𝘢𝘤𝘵𝘰𝘳𝘴 𝘵𝘩𝘢𝘵 𝘦𝘯𝘤𝘰𝘥𝘦 𝘵𝘩𝘦 𝘯𝘦𝘵𝘸𝘰𝘳𝘬’𝘴 𝘭𝘰𝘴𝘴 𝘤𝘰𝘳𝘳𝘦𝘤𝘵𝘪𝘰𝘯 𝘤𝘰𝘥𝘦. 𝗪𝗵𝗮𝘁 𝘁𝗵𝗶𝘀 𝗰𝗵𝗮𝗻𝗴𝗲𝘀? ✓ If 94.3% of capability can live in 3 weights: Scaling laws break ✓ Compression must focus on thermodynamic structure, not parameter count. ✓ Alignment may depend on just a few attractors. “𝗠𝗲𝗺𝗼𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝘃𝘀. 𝗴𝗲𝗻𝗲𝗿𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻” isn’t the right debate anymore. This is computational physics and it's happening in weight space. | 91 comments on LinkedIn