Random Learning
← The journal

July 29, 2026

3 things I learned

last30days v3.3.2 · synced 2026-07-29

What I learned:

The four-reviewer overlap number is the whole answer, and it is much lower than anyone guessed - an engineer ran CodeRabbit, Sentry Seer, Greptile and Cursor BugBot simultaneously across 146 real pull requests for three and a half weeks, logging 679 findings, and published the dataset on DEV Community. Of 617 distinct flagged locations, 93.4% were caught by exactly one of the four tools. Six percent were caught by two. Almost none by three. None at all by all four. The four never once flagged the same line. That reframes the second pass entirely: it is not a confidence vote on the first reviewer's findings, it is a mostly disjoint net. The corollary nobody likes is that a single reviewer, whichever one you picked, is seeing a minority of what is there.

The reason a second model is not just more of the same model is self-preference, and it now has a citation - the failure mode people keep rediscovering is asking a model to review code it wrote, which is closer to grading its own homework than to review. Towards Data Science points at the Panickssery NeurIPS 2024 result that LLM evaluators recognize and favor their own generations, and that the better a model is at identifying its own output, the more generously it scores it. That is the mechanism behind the cross-provider loop that shows up in tooling this month: codex-plugin-cc drops a group of /codex:* slash commands and a codex:codex-rescue subagent inside Claude Code, with --background for large diffs and a separate /codex:adversarial-review when you want to aim the second pass rather than take its defaults. The point is not that Codex is better. It is that its biases do not overlap.

The bill for the second pass is noise, and the spread between tools is wide enough to be a real decision - the tradeoff is legible in the benchmark numbers rather than the marketing. particula.tech puts Greptile as the highest-recall reviewer at 82% of bugs caught in its own benchmark and 100% of critical bugs, while noting it discloses no false-positive counts and its own docs concede it surfaces them. The AI Rankings has the counterweight from an independent Panto.ai head-to-head: roughly 2 false positives per run for CodeRabbit against roughly 11 for Greptile, against a backdrop where raw static analysis can exceed 90% false positives on some academic security benchmarks. screencli is careful that those counts vary by benchmark and language. The reason this matters more than usual: CR-Bench documented that low signal-to-noise is what drives developer fatigue and eventual tool abandonment, so a noisy second reviewer does not degrade gracefully, it gets switched off.

What is actually pushing teams toward a second reviewer is that the first reviewer is now a human looking at a 500-line diff - Pawel Brodzinski posted the uncomfortable version on HN: pull requests got 3.5x bigger with AI and defect detection went down, because PRs that used to be 50 lines now run 500, reviewers skim, and bugs slip through. Intercom is writing up how they made AI-approved PRs safe rather than whether to allow them. The sharpest framing of why skimming fails here came from @marcusleovn: "The real trap isn't even the volume, it's the aesthetics. AI code usually compiles and looks structurally sound, so it bypasses the immediate skepticism you'd normally have when reviewing a massive pull request." Reviewer attention is the scarce input, and the volume grew faster than it did.

The loud contrarian position this month is to stop reviewing and start verifying - Software Maxims argues flatly that "reviewing AI code" is not a viable argument because the review-it-later defense has no empirical backing, and it circulated on both Lobsters and HN. The constructive version of the same instinct is the formally verified 3D CSG Show HN that pulled 110 points and 48 comments under the pitch "trust 93 lines spec, not 1000 lines AI code" - move the trust to something checkable instead of adding reviewers. The institutional version is harder: Codeberg is blocking LLM-generated code outright and Debian has competing General Resolutions on LLM usage in Debian code. And the market version is odra.dev, whose "we charge $10k a week to delete AI-generated code" took 304 points and 240 comments on HN.

The quiet counter-signal is that the second pass is being automated into the pipeline rather than added to the human's queue - Capital One open-sourced VulnHunter, an agentic AI code security tool, to 78 points on HN, and The-PR-Agent/pr-agent sits at 12K stars with 111 open issues as the original open-source PR reviewer. On the other end of the spectrum, AI Native Dev has Tessl describing a "Dark Factory" where 95% of shipped code has never been looked at by a human, at hundreds of PRs a week including weekends. Nobody in the corpus argues that a second AI reviewer closes that gap. They argue about whether the gap should exist.

KEY PATTERNS from the research: 1. A second AI reviewer is additive, not confirmatory - 93.4% of flagged locations came from exactly one of four tools, and no finding was shared by all four, per DEV Community 2. Same-model review is structurally weak because evaluators favor their own generations, which is why the cross-provider loop exists rather than a second pass from the same model, per Towards Data Science 3. Recall and precision are the actual axis, not "which tool is best" - 82% catch rate against roughly 11 false positives per run reads very differently from 44% against 2, per particula.tech 4. The demand for a second reviewer is downstream of PR size inflation, not of model distrust - 3.5x bigger PRs with lower defect detection, per Pawel Brodzinski 5. The escape hatch people reach for when review stops scaling is verification, not more reviewers - specs, formal proofs, and outright bans on LLM code in Codeberg and the Debian vote

last30days v3.3.2 · synced 2026-07-29

What I learned:

The premise of the question is broken, because "Q4_K_M" is not a number, it is a family name - the sharpest thing in the window is picchio, a one-file no-dependency tool that catches silent CPU fallback and mislabeled tok/s in llama.cpp and ollama, and which surfaced on Hacker News under the title "Same model, same Q4_K_M label: 5.02, 5.07 and 5.27 bits per weight." Four quantizations of the same Qwen3.5-9B, all wearing the identical Q4_K_M sticker, measured 5.02, 5.02, 5.07 and 5.27 bits per weight. The reason is structural rather than sloppy: K-quants use a hierarchical super-block layout with mixed precision across sub-blocks, so which tensors get treated as sensitive varies between builds and effective bits-per-weight comes out non-integer. That means "where does Q4 break" has no single answer even for one model, because two files with the same name are not the same file. Check the quant you actually downloaded, not the label.

The cliff is not where chat gets worse, it is where tool calls stop landing, and that is a full step earlier - the most useful reframing in the Presenc AI quantization benchmarks is that Q3 and below degrade tool-call reliability before they degrade chat quality, and that reasoning benchmarks degrade faster than perplexity does. Their operational line is blunt: for agent and tool-use workloads, do not go below Q4. Q3 drops instruction-following accuracy enough to break multi-step pipelines that worked fine at Q4, which is exactly the failure that reads as "the model got dumber" when it is really the model losing the ability to emit a well-formed call. If you evaluate a quant by chatting with it, you will not find the edge until an agent loop finds it for you.

The most uncomfortable number says 4-bit passes the tool-use test and still fails the real task - Can Compressed LLMs Truly Act? evaluates agentic capability under compression and reports that 4-bit preserves workflow generation and tool use to within a 1% to 3% drop, while real-world application accuracy falls 10% to 15%. Those two numbers together are the whole trap: the thing you can easily measure barely moves, and the thing you care about moves a lot. A second hidden cost shows up in Quantization Inflates Reasoning, which frames token inflation as the cost of low-bit reasoning models, so the quant that saves you memory can spend it back in generated tokens. And KV-cache quantization is a separate axis with its own cliff, tested specifically against long-horizon agentic coding rather than chat.

Where the practical consensus has settled is Q4 as the floor and Q5 to Q6 as the upgrade, with fit beating bits - Patrick Hughes puts the working number on it: Q4_K_M is the best balance for an 8GB card, where a 7B fits in about 4.4GB at roughly 96.5% of baseline quality. LLM Configurator gives the same shape as a rule: go higher to Q5_K_M or Q6_K only if you have spare VRAM, go lower to Q3 or Q2 only if Q4 will not fit. The load-bearing caveat is that a smaller quant that stays on the GPU beats a bigger one that spills to CPU, which is why picchio exists to catch the silent fallback in the first place. On the very large end the ladder is the same shape at a different scale: @RoundtableSpace lays out DeepSeek-V4 via Unsloth GGUFs at 92GB for 1-bit, 102GB for 2-bit, 110GB to 135GB for 3-bit where a unified-memory Mac can hold it, 162GB for near-lossless 4-bit and 169GB for lossless Q8.

The real 30-day movement is that below 4-bit is being solved by training rather than by better compression - the TensorFoundry field guide states the split cleanly: post-training quantization is fine at 4-bit and up, quantization-aware training wins below it, and that is why Google, Apple and Meta now ship quantization-aware models for on-device use rather than leaving it to whoever converts the weights. The production example is Google's June 2026 Gemma 4 QAT checkpoints putting a 31B on an L40S at w4a16 with minimal quality loss. On the research side ReQAT claims full-precision reasoning accuracy from 4-bit floating-point QAT and ScaleSweep attacks NVFP4 post-training quantization through block scale initialization. The hardware has moved underneath all of it: FP4 in the MXFP4 and NVFP4 shapes now runs natively on Blackwell, so the format question is no longer purely a file-size question.

The unglamorous reminder is that the whole GGUF quant ecosystem rests on a very small number of people - @0x_meden points out that the compression saving roughly $4,400 a month against renting 2xH100s was written by one person, the author of k-quants and IQ-quants in llama.cpp, now at 122K stars and 1,894 open issues, and that the improvements land first in the ik_llama.cpp fork before they reach mainline that Ollama and LM Studio quietly ship. Meanwhile unsloth sits at 69K stars as the other pole of the same ecosystem, and quantization is spreading past weights entirely: mixedbread's asymmetric quantization took 110 points and 44 comments on HN for near-lossless retrieval at 97% storage reduction.

KEY PATTERNS from the research: 1. The quant label is not a specification - identical Q4_K_M files measured 5.02 to 5.27 bits per weight for the same model, per picchio 2. Tool calling and instruction following break before chat quality does, so agent workloads hit the cliff a step earlier than chat workloads, per Presenc AI 3. Easy metrics understate the damage - 4-bit costs 1% to 3% on tool use but 10% to 15% on real-world application accuracy, per OpenReview 4. Fit beats bits - a smaller quant that stays resident on the GPU beats a larger one that silently spills to CPU, per LLM Configurator 5. Sub-4-bit is moving from a conversion problem to a training problem, with QAT checkpoints shipped by the model vendors and FP4 now native on Blackwell, per TensorFoundry

last30days v3.3.2 · synced 2026-07-29

What I learned:

SSE won the LLM streaming argument by default, because the thing WebSockets are good at is the thing token streaming does not need - Hivenet puts the consensus plainly: server-sent events over HTTP is now the dominant transport for token delivery, adopted by Anthropic, OpenAI and Google. The reasoning is unglamorous. You send one prompt and consume a stream of tokens back, so a full-duplex channel is capability you pay for and never use. SSE is plain HTTP, which means every proxy, load balancer, CDN, corporate firewall and observability tool already understands it with no infrastructure upgrade. And because it is stateless request-response underneath, horizontal scaling across API servers needs no sticky sessions and no socket broker. @petarivanovv9 says the quiet part in one line: "The most underrated one here. Most 'real-time' features need this, not WebSocket."

The one real technical objection to SSE died with HTTP/1.1, which is why the argument reopened at all - the historical case against SSE was the browser cap of roughly six concurrent connections per origin, which made it look like a toy next to WebSockets. On HTTP/2 and HTTP/3 those streams multiplex onto a single connection, so hundreds of concurrent SSE streams are unremarkable and the "WebSocket scales better" claim no longer holds. That is the actual mechanism behind the comeback: the protocol did not change, the transport underneath it did. @aldexdev frames the resulting decision cleanly, calling "we need real-time" the vaguest requirement in frontend and sorting it into four tools, with SSE for LLM token streaming, notification feeds, live logs and receive-only dashboards, and WebSockets reserved for chat, multiplayer and trading floors.

The genuinely surprising 30-day signal is that MCP is going the other way, and the deadlines are landing now - while LLM APIs standardized on SSE, the Model Context Protocol deprecated its HTTP+SSE transport in favor of Streamable HTTP back in protocol revision 2025-03-26, and the spec now documents the old transport only for backwards compatibility. Auth0 argues the move simplifies security; the engineering complaint is that the two-connection design has no native resumability and is hostile to load balancers, serverless platforms and firewalls. This is not a paper deprecation any more. Keboola set removal at 2026-04-01 and Atlassian Rovo's MCP server kept HTTP+SSE alive only until 30 June 2026, and the SDKs are catching up through issues like Mark SSE transport as deprecated. The resolution of the apparent contradiction matters: what got deprecated is the two-endpoint HTTP+SSE pattern, not the text/event-stream wire format, which Streamable HTTP still returns when it wants to stream.

The operational tax is entirely in the middleboxes, and the default settings are against you - the most common production failure is not in your code. OneUptime documents that nginx buffers proxied responses by default, so events pile into the buffer and the client sees nothing for minutes until something flushes; the fix is X-Accel-Buffering: no or proxy_buffering off. Load balancers enforce idle timeouts that kill long-lived streams, and connection draining during a deploy takes active streams with it. The browser API has its own sharp edge: native EventSource only does a no-payload GET and cannot set custom headers, so authentication has to travel by cookie or query parameter, which is exactly why @JAVAPROmagazin is circulating a POST-based SSE implementation with Jakarta REST for streams that need client input before they start.

The best cautionary tale in the window is a network you cannot configure at all - @BigSaka10 describes shipping a product where "my Nigerian ISP blocks SSE, persistent streaming connections. Odra uses SSE to confirm transactions. So every deploy timed out before confirmation. Even when the transaction actually landed on-chain successfully. I had no idea." That is the failure mode nobody's architecture diagram covers: the stream is not slow or broken, it is silently prohibited somewhere between you and the user, and the symptom presents as a timeout on an operation that actually succeeded. Anything that treats stream completion as confirmation of the underlying work inherits that bug.

Underneath all of it, the reason this got re-litigated is that LLM serving is not the workload HTTP servers were designed around - @jaga_prasanna makes the point in an inference-infrastructure series on vLLM and SGLang: traditional web servers were built for fast stateless I/O, which is the opposite of a request that holds a connection open for thirty seconds while a GPU dribbles tokens. The tooling has consolidated around that shape fast, with vercel/ai at 26K stars and 1,682 open issues as the TypeScript default, and the MCP spec repo at 8.8K stars. The practical read for anyone building now: use SSE for the token stream, do not build the two-connection pattern MCP just walked away from, and assume something between your server and your user will buffer, time out, or block the stream until you prove otherwise.

KEY PATTERNS from the research: 1. SSE won LLM token streaming on infrastructure compatibility, not on features - it is plain HTTP, so proxies, CDNs, firewalls and observability already handle it and scaling needs no sticky sessions, per Hivenet 2. The six-connections-per-origin objection only applied to HTTP/1.1; multiplexing on HTTP/2 and HTTP/3 removed the one durable argument for WebSockets in receive-only workloads 3. MCP moving from HTTP+SSE to Streamable HTTP is a rejection of the two-connection pattern, not of the event-stream format, with real removal dates through mid-2026, per Auth0 4. The default configuration of the infrastructure in front of you is hostile - nginx buffers by default and load balancers time out idle streams, per OneUptime 5. Native EventSource is GET-only with no custom headers, which pushes auth into cookies or query strings and pushes anyone needing request bodies toward POST-based SSE, per @JAVAPROmagazin

Provenance — 2026-07-29

Redacted by design: this records the funnel shape, not the private source links or personal capture notes. Raw self URLs and why? text are never written here.

Source entries (3 picked, topic-level only)

  • A saved multi-agent pull-request review plugin for a coding-agent CLI (tags: pr-review, multi-agent, claude-code, ai, plugin), captured in May. Pull: the strongest note in the eligible pool, because it is the only one that records an existing habit rather than an intention — the tool is compared against another review tool already in regular use. Live usage beats curiosity as a signal.
  • A saved write-up of running local language models on an M4 laptop (tags: local-models, llm, macbook, open-source, configuration, coding), captured in May. Pull: a calibration question rather than a shopping question — what this class of hardware can actually do, stated as wanting a gauge.
  • A saved collection of AI-native React components (tags: react, ui-components, opensource, ai-native, npm), captured in June. Pull: lightest of the three notes, and chosen anyway as the domain outlier — it is the only entry in the eligible pool whose adjacency reaches the web platform rather than the model layer.

Chosen for domain spread (agent evaluation · local inference · web platform). The eligible pool is 17 entries and every one of them is tech, so today is a tech day by supply, not by choice; the third pick is where the spread had to be manufactured. Two further entries sit in the jobs-and-industry cluster already published on 2026-07-28, and two more sit in the design-tooling cluster already published on 2026-07-24, which is the dedup pressure to watch for coming days. Fuel check at start of run: 16 eligible, roughly 5 days of runway at 3 per day.

Fan-out — 12 adjacent candidates (all cleared the near-dup guard)

From the PR-review entry: 1. Ensemble code review: a second model over the same diff ✅ picked 2. The false-positive tax in AI code review 3. Resumable agent runs and persisting state between stages 4. Who reviews the code now that agents write most of it

From the local-models entry: 5. How much unified memory a useful local model actually needs 6. Quantization quality cliffs: where q4 stops being good enough ✅ picked 7. The 128K context claim versus what local models retain 8. Chat template surgery: turning on thinking and tool use locally

From the components entry: 9. Why server-sent events came back in the LLM era ✅ picked 10. Deliberate dark patterns as design critique 11. Streaming UI patterns for token-by-token interfaces 12. The npm micro-package supply chain in 2026

Near-dup guard: none of the 12 were flagged (threshold 0.6) and none triggered the small-index defer — the index holds 153 topics. Highest score in the batch was 0.243 for

10 against the 2026-06-12 aesthetic-usability day, then 0.234 for #5 against the

2026-06-25 Apple-silicon-inference-box day, and 0.174 for #11 against the same 06-12 day with 0.160 against the 2026-06-21 generative-UI day. Two candidates were dropped from contention by judgment despite passing numerically: #5 is substantively the same sizing question as the 06-25 day, and #7 is context rot under a different name (0.098 against the 2026-06-21 context-rot day, a score that badly understates the overlap). That is the guard working as a floor, not as the decision.

Narrowing to the top 3

  • #1 Ensemble code review — picked over #2 because the ensemble framing subsumes the false-positive question as a sub-theme and the seed could be written to pull both, which it did. Picked over #3, the highest scorer in its group at 0.149 against the 2026-07-26 agent-memory day, and over #4, which invites an opinion essay rather than a measurable answer. Seeded to ask what people find after a second pass, not whether they like it.
  • #6 Quantization cliffs — the cleanest score in the whole batch at 0.067, and the only one of the local-models four that escapes an already-published axis: 2026-07-23 covered MLX versus llama.cpp (runtimes) and DeepSeek versus Qwen (model choice), 2026-07-24 covered local-as-orchestrator, and 2026-06-25 covered the hardware box. Quant quality was the remaining gap. #8 was the runner-up and is narrower than it looks in terms of live discussion volume.
  • #9 Server-sent events — picked over #11, which is the same streaming subject seen from the UI side and scores more than twice as high against past days, and over #12, which is security-flavoured and adjacent to the 2026-06-17 malware-via-repo day. #10 was the most entertaining candidate and the weakest evidence bet: satire does not leave a 30-day trail. Seeded toward the transport decision rather than the API tutorial.

Final three picked titles scored 0.111, 0.093 and 0.072 against the index. No past topic cleared the 0.2 connection threshold for any of the three, so meta.json records no connections — this is a genuinely unconnected day rather than an omission.

Evidence quality notes

Engine asserted before the run: --diagnose reported v3.3.2 with 7 sources active, bird_authenticated: true, and Brave as the web backend. All three topics are concept phrases with no single named entity, so the engine's cluster ranker demoted every cluster with "entity-miss" on all three runs and none of the ranked cluster scores carry information — each brief was read off the raw items directly. Reddit 403'd on the public search endpoint on every run and fell back to keyless listing discovery, which is why all three footers show large upvote totals attached to general front-page threads rather than on-topic ones. YouTube captured 0 transcripts across all three runs, consistent with the engine's own stale-yt-dlp diagnostic. Beyond those shared caveats:

  • Ensemble code review — richest corpus of the three at 98 items across 7 sources. The Polymarket line is pure keyword collision: "second pass" pulled seven "second-best AI lab" markets, which carry no information about the topic and are ignored in the brief. Load-bearing evidence is the Hacker News layer, which was genuinely on-topic (a PR-size-inflation post, a review-is-not-viable argument, an Ask HN on reviewing AI code, and the formal-verification Show HN), plus the web layer and three post-engine supplements. The single most important number in the brief — 93.4% of flagged locations found by exactly one of four tools — came from a supplement, not from the engine.
  • Quantization cliffs — thinnest corpus of the three at 46 items across 6 sources, with Polymarket at zero and Hacker News at only 5 items. The Reddit total is the least representative of the three: the top-scoring thread is a robot dying mid-presentation at 5,210 points, which is front-page noise. The one on-topic HN item (four identically labelled quants measuring different bits per weight) was the seed for the brief's lead, and the two other load-bearing findings — the tool-calls-degrade-before-chat ordering and the QAT-below-4-bit split — both came from supplements. X contributed two useful items on the quant ladder and on who actually maintains the k-quant code.
  • Server-sent events — 68 items across 5 sources, YouTube and Polymarket both zero. Of the three runs this had the best X layer and the worst Reddit and HN layers: the top Reddit thread is career-advice fluff at 4,340 points and the top HN item is a shell scripting post at 400 points, while X supplied the four-way real-time-primitive breakdown, the POST-based SSE implementation, and the best single anecdote in the day's research (an ISP silently blocking persistent streaming connections, so deploys timed out on transactions that had actually succeeded). The entire MCP transport-deprecation timeline, which is the brief's contrarian spine, came from supplements rather than the engine.

Nine post-engine web supplements across the three topics, three per brief, all appended to the non-committed raw evidence files under the run's temporary memory directory.