archive script: back off and retry on 429, stop when throttled

@@ -0,0 +1,36 @@
+# Italy's 1934 narcotics decree — 20 October 2026
+
+Added one unscored event and an article on Royal Decree-Law 151, filling the
+gap between the 1923 and 1954 entries.
+
+## Evidence and limits
+
+- Read the complete decree in the 19 February 1934 official gazette, printed
+  pp. 838–841 (PDF pp. 22–25). Visually inspected printed pp. 839–841 against
+  extracted text, including the Indian hemp clause, penalties, confinement,
+  repeal and commencement provisions. OCR errors in headings and dates were
+  resolved from the images.
+- Article 1's possession clause concerns clandestine or fraudulent commerce.
+  Article 2 covers specified gatherings; Article 7 has setting and condition
+  requirements. None was recast as a general personal-possession prohibition.
+  Article 12's confinement power is discretionary, not automatic for all users.
+- Article 21 expressly sets commencement on publication, 19 February. The
+  instrument's 15 January date is separately retained in the description.
+- Read the complete sole article and signature of Law 1145, printed p. 3394
+  of the 25 July gazette (PDF p. 2), and visually checked that page. It converts
+  the decree into law; 7 June is its instrument date, not the decree's original
+  commencement. No redundant conversion event was added.
+- Read the official 15 March corrigendum's text. Its corrections concern the
+  poppy wording in Article 1 and a cross-reference in Article 11; neither was
+  used to alter the cannabis or confinement account.
+- Article 19's colonial application and local implementing powers need separate
+  research before adding country-specific events. No colonial score is inferred.
+- Original PDFs and page renders are in ignored `tmp/pdfs/italy-2026-10-20/`.
+  Sources support the legal text, not estimates of enforcement or confinement
+  frequency. No profiles, scores or score baselines were changed.
+
+## Verification
+
+Baseline: `tmp/history-italy-1934-2026-10-20/changes-before.yaml` (ignored).
+Check preservation of existing records and score vectors, article metadata,
+word count and internal links, production build and generated event output.

@@ -0,0 +1,45 @@
+# Italy's 1923 narcotics framework — 20 October 2026
+
+Added an article to the existing 1923 law event, corrected its date and scope,
+and added one unscored event for commencement of the implementing regulation
+expressly listing Indian hemp and its hydroalcoholic extract.
+
+## Evidence and limits
+
+- Read the complete original Law 396 in the 5 March 1923 official gazette,
+  printed pp. 1530–1531, and visually checked both pages. Article 1 covers
+  holding for sale or administration, not general possession for personal use.
+  Articles 8 and 10 separately punish specified drug-taking gatherings. Article
+  12 provides for a ministerial list. The law itself does not name cannabis.
+- Read the implementing regulation in the 8 December 1923 official gazette,
+  printed pp. 7097–7101. Visually checked Table A: entries 13–14 identify Indian
+  hemp and its hydroalcoholic extract. Articles 1–2 establish scheduling and
+  authorisation, including the public-pharmacy exception; Article 39 concerns
+  transitional declarations. Article 31's cultivation rule concerns poppies;
+  it was not treated as evidence of a cannabis cultivation prohibition.
+- Read Normattiva's metadata for both instruments: law commencement 20 March,
+  decree commencement 23 December. These are distinct from instrument dates
+  and publication dates. The existing law event now uses its instrument date.
+- Original PDFs and page renders remain in ignored
+  `tmp/pdfs/italy-2026-10-20/`. Sources were read, not merely summarised from
+  search results. No empirical enforcement claims were added.
+- Preserved the existing score vector without fresh validation. No profiles,
+  current scores or score baselines were changed. The new event has no `before`.
+- The 1934 replacement framework remains a research lead: secondary accounts
+  and official publication references were located, but its original provisions
+  were not verified in this pass and no 1934 claim was added.
+
+## Verification
+
+Baseline: `tmp/history-italy-2026-10-20/changes-before.yaml` (ignored).
+Compare earlier records except the intended 1923 correction, preserve every
+score vector, check article metadata, word count and links, build the site and
+inspect generated pages and exported dates.
+
+Results: all 2,636 baseline records retained except the intended 1923 date,
+description and source correction, with every score vector unchanged. Article
+checks passed: 421 words, four sources and valid internal event links. Production
+build passed. Both generated event pages, article heading, primary-source links
+and exported dates verified. Staged and unstaged whitespace checks passed.
+Totals at verification: 2,637 events and 572 articles. Unrelated changes to
+`scripts/archive-sources.cjs` were left untouched.

@@ -54,7 +54,20 @@ const write = () => {
   fs.renameSync(FILE + ".tmp", FILE);
 };
+// fetch, waiting and trying again while the Archive answers 429 (too many requests) or 5xx;
+// it waits as long as Retry-After asks, otherwise 30 s, 60 s, 120 s, 240 s
+class Throttled extends Error {}
+async function politeFetch(address, options) {
+  for (let attempt = 0; ; attempt++) {
+    const r = await fetch(address, options);
+    if (r.status !== 429 && r.status < 500) return r;
+    if (attempt === 4) throw new (r.status === 429 ? Throttled : Error)(`${r.status} after ${attempt + 1} tries`);
+    const wait = (+r.headers.get("retry-after") || 30 * 2 ** attempt) * 1000;
+    console.log(`  ${r.status}, waiting ${wait / 1000} s`);
+    await sleep(wait);
+  }
+}
 // the newest copy the Internet Archive holds, or null
 async function lookup(url) {
-  const r = await fetch(`https://archive.org/wayback/available?url=${encodeURIComponent(url)}`);
+  const r = await politeFetch(`https://archive.org/wayback/available?url=${encodeURIComponent(url)}`);
   if (!r.ok) throw new Error(`availability ${r.status}`);
   const snap = (await r.json()).archived_snapshots?.closest;
@@ -73,5 +86,5 @@ async function lookup(url) {
       if (!copy && what === "save") {
         // Save Page Now: without an account the Archive accepts only a few requests a minute
-        const r = await fetch(`https://web.archive.org/save/${url}`, { redirect: "follow" });
+        const r = await politeFetch(`https://web.archive.org/save/${url}`, { redirect: "follow" });
         console.log(`  save ${r.status} ${url}`);
         await sleep(15000);
@@ -83,7 +96,12 @@ async function lookup(url) {
       failed++;
       console.log(`  failed ${url}: ${e.message}`);
+      // still throttled after the long waits: stop instead of failing every address that is left
+      if (e instanceof Throttled) {
+        console.log("  the Archive keeps refusing requests; stopping. Run again later.");
+        break;
+      }
     }
     write();
-    await sleep(what === "check" ? 1200 : 1000);
+    await sleep(what === "check" ? 3000 : 1000);
   }
   console.log(`${what}: handled ${todo.length} | copies recorded ${found} | failed ${failed} | still to do ${all.filter(what === "check" ? stale : (u) => !archive[u]?.archived).length}`);