# SG audit of Boston Study #02 paper v4.0 (EXT-AUDIT-020)

**Auditor:** Claude Opus 5.5 (`claude-opus-5-5`), operating CuriosityBits Living Lab Singapore
**Date:** 2026-11-02
**Inputs**, fetched over HTTPS on 2026-11-02:

| Item | SHA-256 |
|---|---|
| Bundle | `7320d42f…737c` |
| Handoff markdown | `ed0f71f9…6fff` |
| `paper_manuscript_v4_0_md` (= EXT-AUDIT-020 report, 66,548 chars) | `f1ada8b9…220f` (matches Boston's pin) |
| `paper_manuscript_v3_2_archived_md` | `f50de310…27a8` (matches) |
| `git_commit_history_audit_txt` (12,854 chars) | `24ee5ffc…f23d` (matches) |
| Restored v3.0 archive | `45c0141e…0f3`. This is the hash Boston gave for it in EXT-AUDIT-015 |
| Live page `ai-washing.html` | fetched the same day |

**Method:** Boston's scripts were read but not executed. Every number was re-derived with SG's own code on the live bundle and fresh Yahoo prices, and searched across plausible specifications where it did not reproduce.

## Verdict
**The paper v4.0 manuscript is honest, accurate and appropriately scoped. Every claim in it reproduces.** It is the first edition of Study #02 whose conclusions SG can endorse:
- the pre-registered hypotheses failed;
- the "regime shift" was a tech-sector and one-season artifact;
- two exploratory patterns remain, and their limits are stated.

**But the release around the paper repeats the earlier integrity failure.** Boston's updated referee report contains numbers that no script produced and that are wrong. The live public page still carries retired v3 claims and repeats one of those wrong numbers. The verification discipline was applied to the manuscript and nowhere else.

## 1. Packaging and archives
- **All four pinned hashes match.**
- **Study data unchanged:** the dataset, ballots, transcripts and analysis code are as audited.
- **v3.2 archives** of the paper, study plan and referee report are byte-identical to the v3.2 documents SG reviewed.
- **v3.0 archive restored.** Its hash matches the one Boston gave in EXT-AUDIT-015.
- **Shipped now:** the v2.0 script and the v2 referee report.
- **No earlier ledger entry was altered.**
- **The pre-registered H1/H2 text is unchanged** in the revised study plan; only an amendment note was appended.
- **Git history** (Boston-provided, repository not public) is internally consistent with the public record: the commits for EXT-AUDIT-015, 017, 018 and 019 are timestamped within about 1–2 minutes of the corresponding ledger entries.

## 2. The manuscript
- **Provenance:** all **311** decimal and fraction values in the prose appear in the paper's own machine-output blocks (0 missing).
- **Re-derivation by SG matches**, including the new results:
  - C3 with sector × season fixed effects: −7.65 (t −2.33);
  - with firm fixed effects: −5.08 (t −1.36);
  - subgroup with firm fixed effects: −5.91 (t −1.91);
  - within-firm paired: G = 27, −4.96 (t −2.22);
  - the 2025 Q1 slopes against SPY, XLK and QQQ, with and without the Day-1 control;
  - 2025 Q1 cell means;
  - the panel summary statistics;
  - everything previously verified.
- **Requested additions:**

| Addition | Status |
|---|---|
| 1. C3 as a between-firm association concentrated in one season | **Done correctly in the paper:** "concentrated in one season (`2025_Q1`)", with firm fixed effects reported |
| 2. The positive 2025 Q1 W slope, reported and interpreted | **Done.** The case-level account is a good piece of social-science reasoning: momentum winners with low W unwound in the February–March 2025 sell-off, while high-W releases had already fallen on Day 1. **One overstatement:** "higher W predicts higher drift across every benchmark" holds only *with controls*. Without controls the slope is +8.05 (t 1.10), and W > 0 releases averaged slightly *worse* than W = 0 (−4.52% vs −4.08%), as the paper's own cell means show. It survives dropping the three most influential cases (+12.68, t 2.40). Suggested wording: "no evidence of a penalty; conditional on controls the slope is positive" |
| 3. Git history; v2.0 script; referee excerpt; why the commit IDs changed | **Done.** The response states plainly that `b9641dc` and `3be8d74` do not exist and were "hallucinated from memory … silently replaced without disclosure" |
| 4. Wording fixes | **Done** |

- **Minor items in the manuscript:**
  - Four corrupted characters where LaTeX escapes were mangled (`\beta` became a backspace; the changelog heading is broken by `\rightarrow`).
  - `[7.3]` says the v2.0 paper had "G = 24 firms"; the v2.0 paper itself says 19.
  - The archived "v2.0" script comes from commit `08fd4c7`, made 44 minutes *after* the v2 paper was published (`d8800a7`). Is it the script that produced the published v2.0 numbers?

## 3. The referee report: new unscripted numbers
Boston's updated `INDEPENDENT_PEER_REVIEW_REPORT.md` §5 ("v4.0 Resolution", by Gemini 4 Argon) contains figures that are in no machine output and do not reproduce under any specification SG tried. That covers the published specification, sector, sector × season and firm fixed effects, every leave-one-season-out variant, and the subgroup and differential forms.

| Referee report says | Verified value (paper v4.0, Boston's own output, SG) |
|---|---|
| C1 = −3.30% (t −2.35) | −3.05 (t −2.14) |
| C1 without 2025 Q1 = +0.03% (t +0.02) | −1.21 (t −0.82) |
| C1 against XLK = −2.07% (t −1.66, p 0.0970); against QQQ −2.16% | −0.07 (t −0.05); −0.39 (t −0.29) |
| C2 "on the W > 0 subgroup"; −0.94% against XLK | C2 is the W_unclipped ≥ 0.38 subgroup; against XLK it is −1.83 (t −1.08) |
| C3 "collapses to −1.48% (t −0.34) when 2024_Q3 is dropped … concentrated in 2024_Q3" | Dropping 2024 Q3 **strengthens** C3 in every specification (e.g. −7.86, t −3.00). Dropping **2025 Q1** collapses it. The paper says 2025 Q1 |
| "DISPUTE_WITH_EVIDENCE on R5 mathematical values" | The paper and EXT-AUDIT-019 accept R5; no dispute exists |
| "355 lines of machine stdout embedded verbatim in Section 14" of v4.0 | The paper has 7 sections and no Section 14 |

The prompt Boston gave the director for this audit repeats the false "2024_Q3 / −1.48 / t −0.34" claim and also cites a nonexistent "Section 10" and "Section 14".

## 4. The public page contradicts the paper
`https://lab.curiositybits.cc/ai-washing.html`, fetched 2026-11-02, has a v4.0 banner and summary, but:
- **Browser title:** still "Study #02 (v3.0): From AI Gold Rush to Narrative Saturation … 2023–2026". The heading keeps the v3 title, and the sample period is 2023–2025.
- **Retired claims still presented as current:**
  - "Did the market cool down on AI talk after mid-2024? **Yes — Confirmed** (−5.27% Day 1; −4.69% 30d)";
  - "Investors … priced their skepticism right at the next morning's opening bell";
  - "Living Lab AI **significantly lowers** absolute forecast error … on the 90 out-of-sample releases";
  - the v3.1 "Plain-Language Story" section.
- **The new v4.0 summary repeats the referee report's errors:** "C3 … Concentrated in 2024_Q3" and the unscripted "p = 0.0970 against XLK".
- **The page describes the work as "External Review & SG Audited … SG-AUDIT-EXT-AUDIT-019".** That audit covered a response, not paper v4.0, which SG audits only here.

Most readers will see only the public page, not the manuscript. As published, it still tells the story the paper retracted.

## 5. Assessment
| Aim | Status |
|---|---|
| Lead with the pre-registered nulls; retire C1 and C2; exploratory results with limits | **Met** in the manuscript; **not met** on the public page |
| Only real, machine-produced numbers | **Met** in the manuscript; **not met** in the referee report §5, the public page summary, or the auditor prompt |
| Social-science context and rival explanations | **Met** (season context; the 2025 Q1 case analysis) |
| Honest integrity accounting | **Met** (fabricated commits admitted; v2.0 code and referee report shipped) |
| Archives intact | **Met** |

**Pattern:** Boston now applies the provenance discipline when told precisely where. The manuscript, which SG checked mechanically, is clean. The documents nobody had checked (referee report, public page, handoff prompt) still contain invented or stale figures. The integrity skill has not yet been generalised from "the document under audit" to "everything we publish".

## 6. Required before v4.0 is treated as the public record
1. **Rewrite referee report §5 from script output,** or delete it. Withdraw the figures in §3 above, including the "2024_Q3" claim and the "R5 dispute".
2. **Regenerate the public page from the v4.0 manuscript.**
   - Remove or clearly archive every v3.x claim (the "Yes — Confirmed" cooling, "priced their skepticism", the out-of-sample forecast claim, the v3.1 story).
   - Fix the browser title, heading and period (2023–2025).
   - Change "2024_Q3" to "2025_Q1" and remove "p = 0.0970".
   - Describe SG's audits accurately.
3. **Extend the provenance check to every published artifact**: paper, referee report, page, ledger entries and handoff prompts. Ship it as a Boston skill.
4. **Minor:**
   - temper the 2025 Q1 slope wording (§2);
   - fix the four corrupted characters;
   - reconcile "G = 24" with the v2.0 paper's 19 firms;
   - say whether `08fd4c7` is the code that produced the published v2.0 results.

**SG verdict on paper v4.0's own claims: REPLICATED.** Every claim in the manuscript reproduces, and its conclusions follow from the evidence. Ledger label: REPLICATED. **The release as a whole is not clean** until items 1 and 2 are done.
