From 3a007662be662a80676a1dab6e1100db01b749cf Mon Sep 17 00:00:00 2001 From: Shekhar Vaidya Date: Sat, 20 Jun 2026 14:32:12 +0530 Subject: [PATCH] docs: update README with CLI tool, Redis cache, /contribute endpoint - Add CLI Tool section: install table, usage examples, crowdsourced cache opt-out - Update project structure to include cli/ directory - Add /contribute API reference - Update caching section to mention Redis/Valkey L2 - Add CLI and Redis to tech stack table - Bump winget manifests from 0.1.0 to 0.2.0 with updated SHA256 Co-Authored-By: Claude Sonnet 4.6 --- README.md | 75 ++++++++++++++++++- winget/manifests/starkSV.msdl.installer.yaml | 6 +- .../manifests/starkSV.msdl.locale.en-US.yaml | 2 +- winget/manifests/starkSV.msdl.yaml | 2 +- 4 files changed, 78 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index e94367b..f429467 100644 --- a/README.md +++ b/README.md @@ -34,12 +34,59 @@ MSDL replicates the session flow that Microsoft uses to serve ISO download links --- +## CLI Tool + +`msdl` is a standalone command-line tool that fetches Windows ISO links directly from your machine — no browser, no server in the middle. + +```bash +# Interactive — pick product and language +msdl + +# Direct with flags +msdl --id 3262 --lang "English" + +# Evaluation / Server ISOs +msdl --eval server-2025 + +# Pipe directly into a downloader +msdl --id 3262 --lang "English" | wget -i - +msdl --id 3262 --lang "English" | xargs aria2c -x 16 -s 16 + +# List all available products +msdl --list +``` + +### Install + +Download the latest binary from [GitHub Releases](https://github.com/starkSV/windows-iso-downloader/releases/latest) and rename it: + +| Platform | File | Rename to | +|---|---|---| +| Windows | `msdl-windows-amd64.exe` | `msdl.exe` | +| macOS (Apple Silicon) | `msdl-darwin-arm64` | `msdl` | +| macOS (Intel) | `msdl-darwin-amd64` | `msdl` | +| Linux | `msdl-linux-amd64` | `msdl` | + +winget support is in progress: `winget install starkSV.msdl` (pending package review). + +### Crowdsourced cache + +By default, each successful fetch is contributed back to the web app's cache — so the next visitor gets a cached link instead of hitting Microsoft cold. Contribution is a background POST to `/contribute`. No personal data is sent — only the product ID, SKU ID, and the raw Microsoft JSON response. To opt out: + +```bash +msdl --id 3262 --lang "English" --no-contribute +# or: MSDL_NO_CONTRIBUTE=1 +``` + +--- + ## Project Structure ``` windows-iso-downloader/ ├── frontend/ # React 19 + TypeScript + Vite + Tailwind v4 ├── backend/ # Go proxy server (recommended for production) +├── cli/ # Go CLI — direct ISO fetching from user's machine ├── cloudflare-worker/ # Optional CF Worker for distributed IP routing └── README.md ``` @@ -82,7 +129,7 @@ Additional resilience mechanisms: - **Jitter** — ±5 min random offset on TTLs prevents synchronized mass-expiry spikes. - **Cache eviction** — a background goroutine evicts expired entries every 30 minutes. -All caches are in-memory only and reset on restart. Monitor them via the [`/metrics` endpoint](#get-metricssecret). +All caches are in-memory and reset on restart. If `REDIS_URL` is set, a Redis/Valkey L2 cache supplements the in-memory layer: on startup the in-memory cache is seeded from Redis (restart-safe), and every fresh Microsoft fetch is written through to Redis. If Redis is unavailable the backend falls back to memory-only mode silently. Monitor the cache via the [`/metrics` endpoint](#get-metricssecret). --- @@ -173,6 +220,20 @@ Valid slugs: `server-2025`, `server-2022`, `server-2019`, `server-2016`, `win11- } ``` +### `POST /contribute` + +Accepts a crowdsourced link from the CLI tool to warm the cache. Requires `Authorization: Bearer ` header. + +```json +{ + "product_id": "3262", + "sku_id": "0x0409", + "response": { /* raw Microsoft JSON */ } +} +``` + +Returns `200 OK` on acceptance, `400` on validation failure (expired link, unknown product), `429` on rate limit (~5 req/min per IP). + ### `GET /metrics?secret=` Returns real-time cache statistics for the running instance. Auth via `?secret=` query param or `Authorization: Bearer ` header. Requires `METRICS_SECRET` env var to be set. @@ -241,10 +302,20 @@ Returns real-time cache statistics for the running instance. Auth via `?secret=` |---|---| | Language | Go 1.25+ | | HTTP | `net/http` (stdlib, no framework) | -| Caching | 4-layer in-memory · `sync.RWMutex` · dynamic TTL | +| Caching | 4-layer in-memory · `sync.RWMutex` · dynamic TTL · Redis/Valkey L2 | | Concurrency | `golang.org/x/sync/singleflight` | | UUID | `github.com/google/uuid` | | MDT parsing | `regexp` (stdlib) | +| Redis client | `github.com/redis/go-redis/v9` | + +### CLI (`cli/`) + +| | | +|---|---| +| Language | Go 1.25+ | +| Interactive UI | `github.com/charmbracelet/huh` | +| HTTP | `net/http` (stdlib) | +| Releases | GitHub Actions · cross-compiled for Windows / macOS / Linux | --- diff --git a/winget/manifests/starkSV.msdl.installer.yaml b/winget/manifests/starkSV.msdl.installer.yaml index 933956f..98840d0 100644 --- a/winget/manifests/starkSV.msdl.installer.yaml +++ b/winget/manifests/starkSV.msdl.installer.yaml @@ -1,5 +1,5 @@ PackageIdentifier: starkSV.msdl -PackageVersion: 0.1.0 +PackageVersion: 0.2.0 Platform: - Windows.Desktop MinimumOSVersion: 10.0.0.0 @@ -8,7 +8,7 @@ Commands: - msdl Installers: - Architecture: x64 - InstallerUrl: https://github.com/starkSV/windows-iso-downloader/releases/download/cli%2Fv0.1.0/msdl-windows-amd64.exe - InstallerSha256: 7FE06D224364D400C7A896048B6B3E8DEA7AA19DCA304C6D986A2EDF92DA72B9 + InstallerUrl: https://github.com/starkSV/windows-iso-downloader/releases/download/cli%2Fv0.2.0/msdl-windows-amd64.exe + InstallerSha256: 4AF1C4FBE16F39657893235298DAF00BF402DBC15227B30C0077A27D435DF3FA ManifestType: installer ManifestVersion: 1.6.0 diff --git a/winget/manifests/starkSV.msdl.locale.en-US.yaml b/winget/manifests/starkSV.msdl.locale.en-US.yaml index eb99b67..eea48af 100644 --- a/winget/manifests/starkSV.msdl.locale.en-US.yaml +++ b/winget/manifests/starkSV.msdl.locale.en-US.yaml @@ -1,5 +1,5 @@ PackageIdentifier: starkSV.msdl -PackageVersion: 0.1.0 +PackageVersion: 0.2.0 PackageLocale: en-US Publisher: starkSV PublisherUrl: https://github.com/starkSV diff --git a/winget/manifests/starkSV.msdl.yaml b/winget/manifests/starkSV.msdl.yaml index 21d5884..59667d0 100644 --- a/winget/manifests/starkSV.msdl.yaml +++ b/winget/manifests/starkSV.msdl.yaml @@ -1,5 +1,5 @@ PackageIdentifier: starkSV.msdl -PackageVersion: 0.1.0 +PackageVersion: 0.2.0 DefaultLocale: en-US ManifestType: version ManifestVersion: 1.6.0