feat: Phase 1 graceful degradation for WAF-blocked links
- Backend: emit X-MSDL-Link-Status (fresh/cached/stale) and X-MSDL-Link-Expires headers on every /proxy response; expose both via Access-Control-Expose-Headers - Frontend: read headers in handleGetLinks/handleRefreshLinks; show amber stale warning when backend is WAF-blocked - Frontend: new OfficialFallback component (collapsible) with step-by-step guide to Microsoft's official download page, pre-filled with the user's selected language; shown on stale - Plan: add Redis/Valkey persistence section to Phase 3 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Shekhar Vaidya
parent
5d1d402715
commit
77056cb50f
4 changed files with 486 additions and 19 deletions
284
IMPLEMENTATION_PLAN.md
Normal file
284
IMPLEMENTATION_PLAN.md
Normal file
|
|
@ -0,0 +1,284 @@
|
|||
# MSDL Resilience & CLI Implementation Plan
|
||||
|
||||
> **Status:** Proposed — 2026-06-17
|
||||
> **Context:** Microsoft's Azure Sentinel WAF is blocking data-center IPs (Hetzner, Cloudflare Workers) on the `GetProductDownloadLinksBySku` endpoint at the ASN level. Two senior engineers independently concluded that server-side calls from hosting IPs are not sustainable, and that moving the Microsoft call onto the end-user's machine (the Rufus/Fido model) is the only durable fix.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem Statement
|
||||
|
||||
MSDL fetches signed Windows ISO download URLs from Microsoft's internal download API. That endpoint is now protected by Azure Sentinel WAF, which blocks by **ASN reputation** — data-center ranges are cleanly distinguishable from residential/commercial ISPs, so Microsoft can throttle hosting traffic with zero impact on real browser users.
|
||||
|
||||
**Observed escalation:**
|
||||
- 2026-06-15: ~2 Sentinel rejections/day, intermittent
|
||||
- 2026-06-16: ~15 rejections clustered in the evening, consistently hitting Win11 25H2 (products 3262, 3321)
|
||||
- Windows 10 (2618) and eval builds still succeed; newer/high-traffic products are targeted first
|
||||
|
||||
**Current blast radius:** The caching layer (95% hit rate) absorbs most of it. The failure mode is: cache expires → backend refresh is WAF-blocked → stale-on-failure serves the old URL → eventually the URL passes its real `se` expiry → **user gets a dead Microsoft CDN link.**
|
||||
|
||||
**Confirmed not fixable server-side:** IP rotation buys weeks; CF Workers already blocked; residential proxies cost money and add a volatile dependency unjustifiable for a free tool. This is Microsoft policy, not a glitch.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture Decision
|
||||
|
||||
**Invert the network footprint.** Today: `backend (Hetzner IP) → Microsoft → users`. The hard call originates from a blocked ASN.
|
||||
|
||||
New model: **`users (residential IP) → Microsoft → backend cache → all users`**
|
||||
|
||||
Three coordinated changes:
|
||||
|
||||
1. **CLI calls Microsoft directly** from the user's own machine. Their residential/commercial IP is one Microsoft cannot block without breaking downloads for real customers. This is exactly why Rufus/Fido work.
|
||||
2. **Crowdsourced cache warming.** When a CLI user fetches a fresh link, the CLI optionally contributes that URL back to the MSDL backend. Microsoft's signed URLs are *not* user-specific (the `se` param is just an expiry timestamp — MSDL already serves one cached URL to many users), so a link fetched by one user is valid for everyone. The backend's Microsoft API calls trend toward zero; Sentinel becomes irrelevant.
|
||||
3. **Web graceful degradation.** When the cache is dead and the backend is blocked, the web stops silently serving dead links — it flags expiry honestly and offers both the official Microsoft fallback and the web-to-CLI handoff.
|
||||
|
||||
**What stays unchanged:** The Go backend keeps serving the React frontend. Eval builds (separate Eval Center endpoint, unaffected) remain a first-class, reliable offering. The two-layer caching system is retained and becomes the distribution layer for crowdsourced links.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ User visits msdl.tech-latest.com │
|
||||
└───────────────────┬───────────────────────┘
|
||||
│
|
||||
Cached link valid & fresh?
|
||||
──────────┬──────────────┬──────────
|
||||
(Yes) │ │ (No / expired / WAF-blocked)
|
||||
▼ ▼
|
||||
Serve signed link Graceful degradation:
|
||||
1. Show "likely expired" flag
|
||||
2. Official Microsoft fallback (guided)
|
||||
3. Web-to-CLI handoff command
|
||||
│
|
||||
▼
|
||||
User runs: msdl --id 3262 --lang "..."
|
||||
│
|
||||
┌────────────────────────┼────────────────────────┐
|
||||
▼ ▼ ▼
|
||||
CLI fetches from MS Prints URL to user POST /contribute
|
||||
(their residential IP) (immediate result) (warms shared cache)
|
||||
│
|
||||
▼
|
||||
Next web visitor: cache hit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Phasing Overview
|
||||
|
||||
| Phase | Deliverable | Why this order |
|
||||
|-------|-------------|----------------|
|
||||
| **1** | Web graceful degradation (expired flag + official fallback) | Immediate — stops users hitting silent dead links today. Pure frontend + small backend signal. No dependency on the CLI. |
|
||||
| **2** | CLI rework: direct-to-Microsoft | The core durable fix. Replaces the backend-routed CLI plan. |
|
||||
| **3** | Crowdsourced cache contribution (`/contribute` + CLI flag) | Depends on Phase 2 CLI existing. Turns every CLI run into cache warming. |
|
||||
| **4** | Web-to-CLI handoff UX | Depends on Phases 2–3. Wires the website's expired state to the CLI install/command. |
|
||||
| **5** | Distribution & winget | Ships the CLI to users (covered in detail in the existing TDD plan). |
|
||||
|
||||
> The granular, TDD-level, step-by-step build instructions for the CLI binary itself (module scaffold, catalog, picker, flags, release workflow, winget manifests) already live in
|
||||
> [`docs/superpowers/plans/2026-06-15-msdl-cli.md`](docs/superpowers/plans/2026-06-15-msdl-cli.md).
|
||||
> **That plan must be amended for Phase 2** (it currently routes through `api.msdl.tech-latest.com`; it must instead port the Microsoft session flow into the binary). This document is the strategic master; that document is the implementation detail.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Web Graceful Degradation (Immediate)
|
||||
|
||||
**Goal:** No user is ever silently handed a dead Microsoft CDN link. When the link is at-risk or unavailable, say so and give a guaranteed path forward.
|
||||
|
||||
### 1.1 Backend: expose link freshness in the proxy response
|
||||
|
||||
**Files:** `backend/main.go` (`handleProxy`, ~line 817; `linkCacheEntry` line 105)
|
||||
|
||||
The frontend already parses `se` for the countdown, but it can't distinguish "fresh from Microsoft" from "stale-on-failure served past a failed refresh." Surface that explicitly.
|
||||
|
||||
- Add a response header on `/proxy` responses indicating provenance and confidence:
|
||||
- `X-MSDL-Link-Status: fresh` — fetched live from Microsoft this request
|
||||
- `X-MSDL-Link-Status: cached` — served from a valid (not-yet-soft-expired) cache entry
|
||||
- `X-MSDL-Link-Status: stale` — served via stale-on-failure after a WAF block; **may be expired**
|
||||
- The `stale` branch is the existing `serving stale` path (the `fetch failed (...), serving stale` log line). Set the header there.
|
||||
- Also emit `X-MSDL-Link-Expires: <RFC3339>` (the `se`-derived expiry) so the frontend has authoritative metadata for the countdown without re-parsing the URL itself. *(Engg-2 suggestion.)*
|
||||
- Ensure `enableCORS` (line 708) exposes the headers: add `Access-Control-Expose-Headers: X-MSDL-Link-Status, X-MSDL-Link-Expires`.
|
||||
|
||||
### 1.2 Frontend: "likely expired" flag
|
||||
|
||||
**Files:** `frontend/src/pages/ProductDetailPage.tsx`
|
||||
|
||||
- Read `X-MSDL-Link-Status` from the `/proxy` fetch response.
|
||||
- When `stale`, render an amber inline warning above the download button:
|
||||
> ⚠️ This link is cached and may have expired. Microsoft is currently limiting automated requests. Use the options below to get a guaranteed fresh download.
|
||||
- When `fresh`/`cached`, keep the existing expiry countdown unchanged.
|
||||
|
||||
### 1.3 Frontend: official Microsoft fallback (guided)
|
||||
|
||||
**Files:** `frontend/src/pages/ProductDetailPage.tsx`, new `frontend/src/components/OfficialFallback.tsx`
|
||||
|
||||
- Below the download card, a collapsible "Get it directly from Microsoft" section that:
|
||||
- Links to the official page (`https://www.microsoft.com/software-download/windows11`, or the matching page per product family).
|
||||
- Lists the exact dropdown selections the user must make (edition + the language matching their chosen SKU).
|
||||
- **Data-driven, not hardcoded** *(Engg-2 suggestion):* derive the official-page URL and edition label from the product catalog (`frontend/public/data/products.json`) keyed by product family, and use the SKU's own language string for the language step — so adding a product doesn't require touching the fallback component.
|
||||
- This keeps MSDL useful as an **information directory** even when automation is fully blocked.
|
||||
|
||||
### 1.4 Acceptance
|
||||
|
||||
- Trigger a stale serve (point the backend at a WAF-blocked product or mock the stale path); confirm the amber warning + official fallback both render.
|
||||
- Confirm `fresh`/`cached` responses show no warning and the countdown still works.
|
||||
- `git commit -m "feat: graceful degradation for WAF-blocked links (expiry flag + official fallback)"`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — CLI Direct-to-Microsoft (Core Durable Fix)
|
||||
|
||||
**Goal:** A standalone `msdl` binary that performs the **entire** Microsoft flow locally — session setup, SKU lookup, link fetch — so the request originates from the user's residential IP.
|
||||
|
||||
### 2.1 Amend the existing CLI plan
|
||||
|
||||
The detailed plan at [`docs/superpowers/plans/2026-06-15-msdl-cli.md`](docs/superpowers/plans/2026-06-15-msdl-cli.md) currently has `cli/api.go` calling `api.msdl.tech-latest.com`. **Replace that approach:** the CLI must own the Microsoft logic. Keep from that plan: module scaffold (Task 1), product catalog (Task 2), interactive picker (Task 4), main flow/flags (Task 5), release workflow + winget (Tasks 6–7). **Rewrite the API layer (Task 3).**
|
||||
|
||||
### 2.2 Port the Microsoft session flow into the CLI
|
||||
|
||||
**Source (backend):** these functions are the reference implementation to port —
|
||||
- `setupSession() (string, *simpleCookieJar, error)` — `backend/main.go:394` (vlscppe → ov-df fingerprinting chain + cookie accumulation)
|
||||
- `simpleCookieJar` — custom `http.CookieJar` with domain scoping disabled
|
||||
- `fetchSkuInfoFromMS(productID string) ([]byte, int, error)` — `backend/main.go:470`
|
||||
- `fetchDownloadLinksFromMS(productID, skuID string) ([]byte, error)` — `backend/main.go:566`
|
||||
|
||||
**Target (CLI):** create `cli/microsoft.go` with the ported equivalents. Strip server concerns (no shared session map — the CLI creates a fresh session per invocation, which is fine and more browser-like). Reuse the catalog and picker from the existing plan.
|
||||
|
||||
> **Shared package vs. copy** *(Engg-2 raised `pkg/microsoft/`):* the CLI is a separate Go module (`cli/go.mod`), so sharing a package across the backend and CLI modules requires a Go workspace or `replace` directives — added friction for ~250 lines of stable code. **Decision: copy/port for v1** (the logic changes rarely; divergence risk is low). Revisit a shared module only if the fingerprinting flow starts changing frequently and the two copies drift.
|
||||
|
||||
**New `cli/microsoft.go` responsibilities:**
|
||||
- `newSession() (*http.Client, string)` — build the `simpleCookieJar`-backed client, run the fingerprinting chain, return the client + session ID.
|
||||
- `fetchLanguages(client, sessionID, productID) ([]Language, error)` — GET SKU info, parse `Skus`.
|
||||
- `fetchDownloadLinks(client, sessionID, productID, skuID) ([]DownloadLink, error)` — link fetch (with SKU-info warmup), parse `ProductDownloadOptions`.
|
||||
- `fetchEvalLinks(evalURL) ([]EvalLink, error)` — scrape the Eval Center page directly and resolve fwlinks.
|
||||
|
||||
**Eval builds run directly too.** For consistency and future-proofing, the CLI scrapes the Microsoft Eval Center itself rather than proxying evals through the backend. The Eval Center isn't WAF-blocked today, but "today" is exactly the assumption that just broke for the consumer endpoint — keeping the CLI fully backend-independent means it never inherits a future Eval Center block.
|
||||
|
||||
### 2.3 TLS fingerprint hardening (note, not blocking)
|
||||
|
||||
Go's default `crypto/tls` ClientHello produces a JA3 fingerprint unlike any real browser. On user residential IPs this is a secondary signal (IP diversity is the primary protection), so ship with stdlib first. **If Sentinel starts fingerprinting the CLI,** swap `net/http`'s transport for [`utls`](https://github.com/refraction-networking/utls) with a Chrome ClientHello profile. Track as a follow-up, not a launch blocker.
|
||||
|
||||
### 2.4 Acceptance
|
||||
|
||||
- On a residential connection: `msdl --id 3262 --lang "English (United States)"` prints a valid signed URL with no backend involved.
|
||||
- `msdl --id 3321` (a product the backend currently gets WAF-blocked on) succeeds from a home IP — this is the proof the architecture works.
|
||||
- Eval builds: `msdl --eval server-2025` scrapes the Eval Center directly and prints the resolved `.iso` URLs — no backend involved.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Crowdsourced Cache Contribution
|
||||
|
||||
**Goal:** Every CLI link fetch optionally warms the shared backend cache, so web visitors get hits without the backend ever calling Microsoft.
|
||||
|
||||
### 3.0 Redis/Valkey cache persistence (prerequisite to 3.1)
|
||||
|
||||
**Why:** Without persistence, a backend restart wipes the link cache. If the backend is also WAF-blocked it cannot rebuild from Microsoft — crowdsourced contributions would be the only source of fresh links, so losing them on restart defeats Phase 3 entirely.
|
||||
|
||||
**Stack:** Redis/Valkey on Coolify (same Hetzner instance). TTL-native, Go has a first-class client (`github.com/redis/go-redis/v9`), zero added ops cost. Preferred over PostgreSQL (overkill for key-value TTL cache) and CF KV (requires routing through the Worker from Go, awkward).
|
||||
|
||||
**Architecture:** Two-layer cache — Redis as L2 persistent store; the existing in-memory `linkCache` map stays as L1 for hot reads.
|
||||
|
||||
**Files:** `backend/main.go`, `backend/go.mod`
|
||||
|
||||
- Add `REDIS_URL` env var (e.g. `redis://localhost:6379`). If unset, skip Redis silently — backend runs in memory-only mode for local dev.
|
||||
- Key format: `msdl:link:<productID>:<skuID>` (namespaced).
|
||||
- On startup: seed in-memory cache from Redis (so a restart recovers immediately).
|
||||
- On link cache write (fresh fetch or contribution): `SET key value EX <ttl_seconds>` using `time.Until(entry.ExpiresAt)`.
|
||||
- On in-memory cache miss in `handleProxy`: check Redis before calling Microsoft.
|
||||
- If Redis is unavailable at runtime: log a warning and fall through to in-memory-only (keeps the existing failure behaviour, no hard dependency).
|
||||
|
||||
**Deploy order:** Provision a Valkey service on Coolify and set `REDIS_URL` before Phase 3 goes to production. Add `github.com/redis/go-redis/v9` to `backend/go.mod`.
|
||||
|
||||
---
|
||||
|
||||
### 3.1 Backend: `/contribute` endpoint
|
||||
|
||||
**Files:** `backend/main.go` (register in `main()` ~line 1147; cache write mirrors `handleProxy` line 922)
|
||||
|
||||
- `POST /contribute` accepting JSON: `{ "product_id": "3262", "sku_id": "19675", "raw_json": <the exact GetProductDownloadLinksBySku response bytes> }`.
|
||||
- **Validation (reject untrusted contributions):**
|
||||
1. `product_id` must be in the known catalog (reuse the allow-list logic the proxy uses).
|
||||
2. Parse `raw_json` with the existing `parseLinkExpiry(rawJSON []byte) time.Time` (line 208). Reject if expiry is in the past or < 1 hour out.
|
||||
3. Extract the URL(s) from `ProductDownloadOptions`; reject unless the host matches the Microsoft CDN pattern (e.g. `*.download.prss.microsoft.com` / `software-static.download.prss.microsoft.com`).
|
||||
4. **No `HEAD` validation by default** *(both engineers):* parsing + host-allow-list + expiry are the real defense, and an extra request to Microsoft's CDN on every contribution adds traffic Microsoft may be sensitive to. Rely on parsing first; only add a `HEAD` check later if junk slips through.
|
||||
- On success, write to the shared cache exactly as `handleProxy` does:
|
||||
```go
|
||||
cacheKey := productID + ":" + skuID
|
||||
expiresAt := parseLinkExpiry(raw)
|
||||
linkCacheMu.Lock()
|
||||
linkCache[cacheKey] = linkCacheEntry{RawJSON: raw, ExpiresAt: expiresAt, FetchedAt: time.Now()}
|
||||
linkCacheMu.Unlock()
|
||||
```
|
||||
- Respond `204 No Content` on accept, `400`/`422` on validation failure.
|
||||
- **Logging** *(Engg-2):* log accepts as `contribute: product_id=%s sku_id=%s -> accepted, cached until %s` AND rejections clearly as `contribute: product_id=%s sku_id=%s -> rejected (<reason>)` so contribution patterns/abuse are visible in ops logs from day one.
|
||||
- **Abuse protection:** since this writes to a cache served to all users —
|
||||
- **Per-IP rate limit** on `/contribute`: ~5/min *(Engg-2's concrete number)*. Build it as small middleware (a per-IP token bucket in a `map[string]…` with mutex) — there's no existing rate-limit middleware to reuse.
|
||||
- **`CONTRIBUTE_SECRET`** env var, baked into released CLI builds, checked as a request header. Cheap obfuscation that blocks casual abuse; not real security.
|
||||
- The host allow-list + expiry validation remain the real defense: the worst a bad actor can do is submit a *valid Microsoft link*, which is harmless.
|
||||
|
||||
### 3.2 CLI: contribution flag
|
||||
|
||||
**Files:** `cli/main.go`, `cli/microsoft.go`
|
||||
|
||||
- After a successful link fetch, POST the raw response to `https://api.msdl.tech-latest.com/contribute` in the background. **Truly non-blocking** *(Engg-2):* fire it in a goroutine with a `context.WithTimeout(ctx, 5*time.Second)`, send the `CONTRIBUTE_SECRET` as a header, ignore all errors. The user already has their URL printed before this runs — and `main` must wait on it only briefly (or the process may exit first; spawn it before printing the result and `wg.Wait()` with the same short timeout, or simply accept best-effort).
|
||||
- **Opt-out, transparent by default:** print to stderr `✓ Shared with msdl.tech cache to help other users (disable with --no-contribute)`. Add `--no-contribute` flag. Honor a `MSDL_NO_CONTRIBUTE=1` env var too.
|
||||
|
||||
### 3.3 Acceptance
|
||||
|
||||
- Run the CLI for a product whose backend cache is empty/expired; confirm `/contribute` logs an accept and a subsequent web request to `/proxy` for that product is a **cache hit** with `X-MSDL-Link-Status: cached`.
|
||||
- Submit a junk/non-Microsoft URL to `/contribute`; confirm rejection.
|
||||
- Submit a near-expired link; confirm rejection.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Web-to-CLI Handoff
|
||||
|
||||
**Goal:** When the web cache is dead and the backend is WAF-blocked, the site offers the CLI as the guaranteed path — and that CLI run heals the cache for the next visitor (Phase 3).
|
||||
|
||||
### 4.1 Frontend handoff UI
|
||||
|
||||
**Files:** `frontend/src/pages/ProductDetailPage.tsx`, `frontend/src/components/CliHandoff.tsx`
|
||||
|
||||
- In the `stale` state (from Phase 1.1), in addition to the official fallback, show:
|
||||
> **Get a fresh link instantly with the MSDL CLI** — runs from your computer, always works.
|
||||
- Install one-liner per platform (winget for Windows, curl|sh for Linux/macOS — final commands from the release in Phase 5).
|
||||
- The exact command pre-filled for this product: `msdl --id 3262 --lang "English (United States)"`
|
||||
- One-line explanation: "Runs the download lookup from your own connection, so Microsoft's rate limits don't apply. It also shares the fresh link back to help other visitors."
|
||||
- Copy-to-clipboard button (reuse the existing CLI command tab component pattern).
|
||||
|
||||
### 4.2 Acceptance
|
||||
|
||||
- Force a stale state; confirm the handoff block shows the correct install command + pre-filled product command, and copy works.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Distribution
|
||||
|
||||
Covered in full by [`docs/superpowers/plans/2026-06-15-msdl-cli.md`](docs/superpowers/plans/2026-06-15-msdl-cli.md) Tasks 6–7:
|
||||
- GitHub Actions release workflow (4-platform binaries, `cli/v*` tag trigger)
|
||||
- Winget publishing (`starkSV.msdl`) with `wingetcreate` auto-PR on future releases
|
||||
- `curl | sh` install script for Linux/macOS
|
||||
|
||||
No changes needed here beyond ensuring the `--no-contribute` flag and contribute endpoint URL are baked into release builds.
|
||||
|
||||
---
|
||||
|
||||
## 4. Cleanup / Catalog Note (do ASAP — independent of the above)
|
||||
|
||||
- **Product 48 (Windows 8.1 Single Language)** returns `ERROR [502]: no download links found for this SKU` on every attempt — this is a Microsoft-side removal, not a WAF block. **Both reviewers flagged this as do-ASAP.** Remove it from `frontend/public/data/products.json` now (and from `cli/catalog.go` when the CLI is built). This is a one-line config change → straight to main.
|
||||
|
||||
---
|
||||
|
||||
## 5. Risk & Open Questions
|
||||
|
||||
- **Contribution trust model:** the CDN-host allow-list + expiry validation make malicious contributions essentially harmless (worst case: a valid Microsoft link). Revisit if abuse appears.
|
||||
- **Eval Center longevity:** the CLI scrapes Eval Center directly (decided). If Microsoft changes that page's markup, `fwlinkRe`/`isoLangRe` in `cli/microsoft.go` (and the backend) need updating — low risk, easy fix.
|
||||
- **TLS fingerprinting (`utls`):** only if Sentinel begins fingerprinting the CLI on residential IPs. Not a launch blocker.
|
||||
- **Backend's own Microsoft calls:** once crowdsourced contributions are flowing, consider whether the backend should still attempt live fetches at all, or become contribution-only + stale-serving. Likely keep live fetch as a fallback for cold products with no contributor yet.
|
||||
|
||||
---
|
||||
|
||||
## 6. Sequencing Recommendation
|
||||
|
||||
1. **Phase 1 now** — independent, immediate user benefit, low risk.
|
||||
2. **Catalog cleanup (§4)** — trivial, do alongside Phase 1.
|
||||
3. **Phase 2** — amend & execute the CLI TDD plan with the direct-to-Microsoft rewrite.
|
||||
4. **Phase 3** — once the CLI fetches links, add contribution.
|
||||
5. **Phase 4 + 5** — wire the web handoff and ship distribution together.
|
||||
|
|
@ -245,6 +245,40 @@ func parseLinkExpiry(rawJSON []byte) time.Time {
|
|||
return time.Now().Add(jitter(ttl, 5*time.Minute))
|
||||
}
|
||||
|
||||
// extractRawExpiry returns the raw `se` signed-expiry query param from the
|
||||
// first URL in a GetProductDownloadLinksBySku response (RFC3339 string).
|
||||
// Returns "" if absent, unparseable, or not present in the JSON.
|
||||
func extractRawExpiry(rawJSON []byte) string {
|
||||
var data map[string]interface{}
|
||||
if err := json.Unmarshal(rawJSON, &data); err != nil {
|
||||
return ""
|
||||
}
|
||||
opts, ok := data["ProductDownloadOptions"].([]interface{})
|
||||
if !ok || len(opts) == 0 {
|
||||
return ""
|
||||
}
|
||||
first, ok := opts[0].(map[string]interface{})
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
uri, ok := first["Uri"].(string)
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
u, err := url.Parse(uri)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
se := u.Query().Get("se")
|
||||
if se == "" {
|
||||
return ""
|
||||
}
|
||||
if _, err := time.Parse(time.RFC3339, se); err != nil {
|
||||
return ""
|
||||
}
|
||||
return se
|
||||
}
|
||||
|
||||
// rateLimitError marks a Microsoft rate-limit response so callers can choose
|
||||
// to serve stale data instead of propagating the error.
|
||||
type rateLimitError struct{ message string }
|
||||
|
|
@ -713,6 +747,7 @@ func enableCORS(next http.Handler) http.Handler {
|
|||
}
|
||||
w.Header().Set("Access-Control-Allow-Methods", "GET, OPTIONS")
|
||||
w.Header().Set("Access-Control-Allow-Headers", "Content-Type")
|
||||
w.Header().Set("Access-Control-Expose-Headers", "X-MSDL-Link-Status, X-MSDL-Link-Expires")
|
||||
if r.Method == "OPTIONS" {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
return
|
||||
|
|
@ -845,6 +880,10 @@ func handleProxy(w http.ResponseWriter, r *http.Request) {
|
|||
if !forceRefresh && hasCached && time.Now().Before(cached.ExpiresAt) {
|
||||
atomic.AddInt64(&mLinkCacheHits, 1)
|
||||
log.Printf("/proxy: product_id=%s sku_id=%s -> cache hit\n", productID, skuID)
|
||||
w.Header().Set("X-MSDL-Link-Status", "cached")
|
||||
if exp := extractRawExpiry(cached.RawJSON); exp != "" {
|
||||
w.Header().Set("X-MSDL-Link-Expires", exp)
|
||||
}
|
||||
w.Write(cached.RawJSON)
|
||||
return
|
||||
}
|
||||
|
|
@ -894,6 +933,10 @@ func handleProxy(w http.ResponseWriter, r *http.Request) {
|
|||
return nil, nil
|
||||
})
|
||||
}()
|
||||
w.Header().Set("X-MSDL-Link-Status", "stale")
|
||||
if exp := extractRawExpiry(cached.RawJSON); exp != "" {
|
||||
w.Header().Set("X-MSDL-Link-Expires", exp)
|
||||
}
|
||||
w.Write(cached.RawJSON)
|
||||
return
|
||||
}
|
||||
|
|
@ -933,6 +976,10 @@ func handleProxy(w http.ResponseWriter, r *http.Request) {
|
|||
} else {
|
||||
log.Printf("/proxy: product_id=%s sku_id=%s -> fetched and cached until %s\n", productID, skuID, expiresAt.Format(time.RFC3339))
|
||||
}
|
||||
w.Header().Set("X-MSDL-Link-Status", "fresh")
|
||||
if exp := extractRawExpiry(raw); exp != "" {
|
||||
w.Header().Set("X-MSDL-Link-Expires", exp)
|
||||
}
|
||||
w.Write(raw)
|
||||
}
|
||||
|
||||
|
|
|
|||
97
frontend/src/components/OfficialFallback.tsx
Normal file
97
frontend/src/components/OfficialFallback.tsx
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
import { useState } from 'react'
|
||||
import { motion, AnimatePresence } from 'motion/react'
|
||||
import { ExternalLink, ChevronRight } from 'lucide-react'
|
||||
|
||||
interface Props {
|
||||
productId: string
|
||||
languageName: string
|
||||
}
|
||||
|
||||
function officialUrl(productId: string): string {
|
||||
const id = Number(productId)
|
||||
if (id >= 3113) return 'https://www.microsoft.com/software-download/windows11'
|
||||
if (id >= 2378) return 'https://www.microsoft.com/software-download/windows10ISO'
|
||||
return 'https://www.microsoft.com/software-download/windows8ISO'
|
||||
}
|
||||
|
||||
function editionLabel(productId: string): string {
|
||||
const id = Number(productId)
|
||||
if (id >= 3113) return 'Windows 11'
|
||||
if (id >= 2378) return 'Windows 10'
|
||||
return 'Windows 8.1'
|
||||
}
|
||||
|
||||
export default function OfficialFallback({ productId, languageName }: Props) {
|
||||
const [open, setOpen] = useState(false)
|
||||
const url = officialUrl(productId)
|
||||
const edition = editionLabel(productId)
|
||||
|
||||
return (
|
||||
<div className="mt-3">
|
||||
<button
|
||||
onClick={() => setOpen(!open)}
|
||||
className="flex items-center gap-2 text-[12px] text-zinc-600 hover:text-zinc-400 transition-colors py-1"
|
||||
>
|
||||
<ExternalLink size={13} />
|
||||
<span>Get it directly from Microsoft</span>
|
||||
<motion.span
|
||||
animate={{ rotate: open ? 90 : 0 }}
|
||||
transition={{ type: 'spring', stiffness: 400, damping: 30 }}
|
||||
>
|
||||
<ChevronRight size={13} />
|
||||
</motion.span>
|
||||
</button>
|
||||
|
||||
<AnimatePresence initial={false}>
|
||||
{open && (
|
||||
<motion.div
|
||||
key="official-fallback"
|
||||
initial={{ height: 0, opacity: 0 }}
|
||||
animate={{ height: 'auto', opacity: 1 }}
|
||||
exit={{ height: 0, opacity: 0 }}
|
||||
transition={{ type: 'spring', stiffness: 300, damping: 30 }}
|
||||
className="overflow-hidden"
|
||||
>
|
||||
<div className="mt-2 rounded-xl border border-white/7 bg-[#0d0d0f] p-4 space-y-3">
|
||||
<p className="text-[12px] text-zinc-500 leading-relaxed">
|
||||
Microsoft's official page runs the same lookup from your browser — it always works.
|
||||
</p>
|
||||
<ol className="space-y-2.5">
|
||||
<li className="flex items-start gap-2.5">
|
||||
<span className="flex-shrink-0 w-5 h-5 rounded-full bg-white/5 border border-white/10 flex items-center justify-center text-[10px] font-mono text-zinc-500 mt-0.5">1</span>
|
||||
<span className="text-[12px] text-zinc-400">
|
||||
Open{' '}
|
||||
<a
|
||||
href={url}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="text-blue-400 hover:text-blue-300 underline underline-offset-2"
|
||||
>
|
||||
Microsoft's official {edition} download page
|
||||
</a>
|
||||
</span>
|
||||
</li>
|
||||
<li className="flex items-start gap-2.5">
|
||||
<span className="flex-shrink-0 w-5 h-5 rounded-full bg-white/5 border border-white/10 flex items-center justify-center text-[10px] font-mono text-zinc-500 mt-0.5">2</span>
|
||||
<span className="text-[12px] text-zinc-400">
|
||||
Select edition: <span className="font-mono text-zinc-300">{edition}</span>, then click Confirm
|
||||
</span>
|
||||
</li>
|
||||
<li className="flex items-start gap-2.5">
|
||||
<span className="flex-shrink-0 w-5 h-5 rounded-full bg-white/5 border border-white/10 flex items-center justify-center text-[10px] font-mono text-zinc-500 mt-0.5">3</span>
|
||||
<span className="text-[12px] text-zinc-400">
|
||||
Select language: <span className="font-mono text-zinc-300">{languageName}</span>, then click Confirm
|
||||
</span>
|
||||
</li>
|
||||
<li className="flex items-start gap-2.5">
|
||||
<span className="flex-shrink-0 w-5 h-5 rounded-full bg-white/5 border border-white/10 flex items-center justify-center text-[10px] font-mono text-zinc-500 mt-0.5">4</span>
|
||||
<span className="text-[12px] text-zinc-400">Click the download link for your architecture</span>
|
||||
</li>
|
||||
</ol>
|
||||
</div>
|
||||
</motion.div>
|
||||
)}
|
||||
</AnimatePresence>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
|
@ -8,6 +8,7 @@ import type { Sku, DownloadOption } from '../types'
|
|||
import SystemRequirements from '../components/SystemRequirements'
|
||||
import CliCommand from '../components/CliCommand'
|
||||
import RelatedReleases from '../components/RelatedReleases'
|
||||
import OfficialFallback from '../components/OfficialFallback'
|
||||
import { addRecentEntry, updateRecentExpiry } from '../components/RecentlyViewed'
|
||||
|
||||
const API_BASE = import.meta.env.VITE_API_URL ?? 'http://localhost:3002'
|
||||
|
|
@ -61,6 +62,7 @@ export default function ProductDetailPage() {
|
|||
const [, setLinkExpiry] = useState<Date | null>(null)
|
||||
const [expiryRemaining, setExpiryRemaining] = useState<number | null>(null)
|
||||
const [isRefreshing, setIsRefreshing] = useState(false)
|
||||
const [linkStatus, setLinkStatus] = useState<'fresh' | 'cached' | 'stale' | null>(null)
|
||||
const expiryTimerRef = useRef<ReturnType<typeof setInterval> | null>(null)
|
||||
|
||||
const [meta, setMeta] = useState<{ badge: string; archs: string[]; active: boolean }>({ badge: '', archs: [], active: false })
|
||||
|
|
@ -195,26 +197,36 @@ export default function ProductDetailPage() {
|
|||
setDownloadLinks([])
|
||||
setLinkExpiry(null)
|
||||
setExpiryRemaining(null)
|
||||
setLinkStatus(null)
|
||||
if (expiryTimerRef.current) clearInterval(expiryTimerRef.current)
|
||||
|
||||
fetch(`${API_BASE}/proxy?product_id=${productId}&sku_id=${selectedSku.Id}`)
|
||||
.then(r => r.json())
|
||||
.then(data => {
|
||||
if (data.error) throw new Error(data.error)
|
||||
if (!data.ProductDownloadOptions?.length) throw new Error('No download links returned.')
|
||||
setDownloadLinks(data.ProductDownloadOptions)
|
||||
const expiry = parseSeExpiry(data.ProductDownloadOptions[0].Uri)
|
||||
setLinkExpiry(expiry)
|
||||
if (expiry) {
|
||||
startExpiryCountdown(expiry)
|
||||
if (productId) updateRecentExpiry(productId, expiry.getTime())
|
||||
}
|
||||
})
|
||||
.catch(e => {
|
||||
setError(e.message)
|
||||
toast.error(e.message)
|
||||
})
|
||||
.finally(() => setIsFetching(false))
|
||||
try {
|
||||
const r = await fetch(`${API_BASE}/proxy?product_id=${productId}&sku_id=${selectedSku.Id}`)
|
||||
const status = r.headers.get('X-MSDL-Link-Status') as 'fresh' | 'cached' | 'stale' | null
|
||||
const expiresHeader = r.headers.get('X-MSDL-Link-Expires')
|
||||
const data = await r.json()
|
||||
if (data.error) throw new Error(data.error)
|
||||
if (!data.ProductDownloadOptions?.length) throw new Error('No download links returned.')
|
||||
setDownloadLinks(data.ProductDownloadOptions)
|
||||
setLinkStatus(status)
|
||||
let expiry: Date | null = null
|
||||
if (expiresHeader) {
|
||||
const d = new Date(expiresHeader)
|
||||
if (!isNaN(d.getTime())) expiry = d
|
||||
}
|
||||
if (!expiry) expiry = parseSeExpiry(data.ProductDownloadOptions[0].Uri)
|
||||
setLinkExpiry(expiry)
|
||||
if (expiry) {
|
||||
startExpiryCountdown(expiry)
|
||||
if (productId) updateRecentExpiry(productId, expiry.getTime())
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
const msg = e instanceof Error ? e.message : 'Failed to fetch links'
|
||||
setError(msg)
|
||||
toast.error(msg)
|
||||
} finally {
|
||||
setIsFetching(false)
|
||||
}
|
||||
}
|
||||
|
||||
function handleCopy(uri: string) {
|
||||
|
|
@ -256,11 +268,19 @@ export default function ProductDetailPage() {
|
|||
setError(null)
|
||||
try {
|
||||
const r = await fetch(`${API_BASE}/proxy?product_id=${productId}&sku_id=${selectedSku.Id}&force=true`)
|
||||
const status = r.headers.get('X-MSDL-Link-Status') as 'fresh' | 'cached' | 'stale' | null
|
||||
const expiresHeader = r.headers.get('X-MSDL-Link-Expires')
|
||||
const data = await r.json()
|
||||
if (data.error) throw new Error(data.error)
|
||||
if (!data.ProductDownloadOptions?.length) throw new Error('No download links returned.')
|
||||
setDownloadLinks(data.ProductDownloadOptions)
|
||||
const expiry = parseSeExpiry(data.ProductDownloadOptions[0].Uri)
|
||||
setLinkStatus(status)
|
||||
let expiry: Date | null = null
|
||||
if (expiresHeader) {
|
||||
const d = new Date(expiresHeader)
|
||||
if (!isNaN(d.getTime())) expiry = d
|
||||
}
|
||||
if (!expiry) expiry = parseSeExpiry(data.ProductDownloadOptions[0].Uri)
|
||||
setLinkExpiry(expiry)
|
||||
if (expiry) {
|
||||
startExpiryCountdown(expiry)
|
||||
|
|
@ -439,6 +459,7 @@ export default function ProductDetailPage() {
|
|||
if (sku) setSelectedSku(sku)
|
||||
setDownloadLinks([])
|
||||
setError(null)
|
||||
setLinkStatus(null)
|
||||
}}
|
||||
>
|
||||
<Select.Trigger className="w-full flex items-center justify-between px-4 py-3 rounded-xl bg-white/5 border border-white/7 text-sm text-white hover:bg-white/7 hover:border-white/12 focus:outline-none transition-all cursor-pointer">
|
||||
|
|
@ -523,6 +544,16 @@ export default function ProductDetailPage() {
|
|||
transition={{ type: 'spring', stiffness: 300, damping: 28 }}
|
||||
>
|
||||
<div className="pt-4 border-t border-white/6 space-y-3">
|
||||
{/* Stale warning */}
|
||||
{linkStatus === 'stale' && (
|
||||
<div className="flex items-start gap-3 p-3.5 rounded-xl bg-amber-500/8 border border-amber-500/20 text-amber-400">
|
||||
<AlertTriangle size={14} className="flex-shrink-0 mt-0.5" />
|
||||
<p className="text-[12px] leading-relaxed">
|
||||
This link is cached and may have expired — Microsoft is currently limiting automated requests. Use the options below for a guaranteed fresh download.
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Expiry row */}
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<div className={`flex items-center gap-2 p-3 rounded-lg border text-[11px] flex-1 ${
|
||||
|
|
@ -606,6 +637,14 @@ export default function ProductDetailPage() {
|
|||
|
||||
{(!isNotFound && !hasCatalogError) && (
|
||||
<>
|
||||
{/* Official fallback — shown when backend is WAF-blocked (stale) */}
|
||||
{meta.active && linkStatus === 'stale' && selectedSku && (
|
||||
<OfficialFallback
|
||||
productId={productId!}
|
||||
languageName={selectedSku.LocalizedLanguage}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* System requirements */}
|
||||
<SystemRequirements isWin11={isWin11(productId!)} />
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue