TLS fingerprint fix (v0.3.7) checked in 17 days post-release: aggregate
CLI Sentinel-rejection rate still ~21%, unchanged -- inconclusive given
mixed-version population, revisit once 0.3.7+ dominates usage.
New backlog item: per-language SHA256 checksums on product pages.
Confirmed live that Microsoft's own download pages publish a static
per-locale hash table for the current build, not reachable through any
API call MSDL/CLI already makes. Since it only changes when a product
ID is replaced, no scraper needed -- just copy it by hand at the same
time a new product gets added to the catalog.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Homepage's featured card said "Security support until October 2025" --
that date has already passed, so it read as a future deadline that
wasn't. Updated to reflect actual status (mainstream support ended,
consumer ESU runs through Oct 2026) -- kept tight to match the other
three cards' description length so the grid stays even (the first
draft ran long enough to wrap to 3 lines and grow that one card's
height past its siblings).
Also found products.json still tagged both Windows 10 22H2 variants
(2618, 2378) as "EOL SOON" -- inconsistent with the homepage's own
"EOL"/"END OF LIFE" badge for the same product, and equally stale
for the same reason. products.json's badge field renders verbatim
on the product detail page, so this was live and user-facing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Now that msdl-bin is actually published on AUR, future releases'
auto-generated "What's New" -> Install section should mention it
too, matching what's already in README.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
README still said the AUR package was "prepared but not yet
published" -- it went live today (msdl-bin 0.3.7-1, now that AUR
registration reopened). Added it as a proper install method alongside
winget/Homebrew, matching their existing formatting.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
.SRCINFO regenerated with real makepkg this time (Docker was running),
confirming the same values as the manual field-mapping used previously.
AUR disabled new account registrations 2026-06-15 after a malware
campaign; reopened as of today. Updated the README note accordingly --
this package is ready to actually publish once an account + SSH key
are set up, which is a manual step only the maintainer can do.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Version bump: backend/main.go latestCLIVersion and the local
winget/manifests/*.yaml mirror to 0.3.7, with the real
InstallerUrl/InstallerSha256 for the cli/v0.3.7 release asset.
CI fix: the first real winget-submission attempt (v0.3.7, now that
WINGET_TOKEN is finally set) failed immediately -- wingetcreate isn't
published as a NuGet/dotnet-tool package at all, it's a native Windows
binary requiring .NET 6 + VC++ Redistributable. `dotnet tool install
--global wingetcreate` on ubuntu-latest could never have worked; this
path was just never exercised before since the empty-token check
always short-circuited it first. Split winget submission into its own
job on windows-latest, downloading wingetcreate.exe directly from its
GitHub release instead of dotnet tool install.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PROGRESS.md and CLAUDE.md now document the full investigation: three
weekly checkpoints confirming the backend's direct link-fetch is
100% dead, the crowdsourced CLI-contribution cache carrying the real
load (286 accepted, 0 rejected), the two merged backend fixes
(lockdown TTL, /proxy product_id validation), and the CLI TLS
fingerprint hardening in this branch (functionally verified, not yet
validated against the Sentinel-rejection-rate trend).
Removed two plan+spec pairs under docs/superpowers/ (the original CLI
build plan and the telemetry/update-check plan+design) -- both fully
shipped, redundant with PROGRESS.md's own record of what happened.
Committed docs/superpowers/specs/2026-07-13-needs-warming-design.md
for the first time -- it's been sitting untracked in the working tree
for weeks. Unlike the two removed docs, this one is a live, unbuilt
proposal (a public page surfacing which products are currently
Sentinel-locked with no cached fallback), not completed work, so it's
kept and now tracked in the backlog instead of discarded.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three checkpoints over 17 days confirmed a stable ~20% Sentinel
rejection rate on the CLI's own requests, despite running from
residential IPs specifically to avoid ASN-based blocking. That
persistence points at TLS ClientHello fingerprinting as an additional
signal: Go's stdlib crypto/tls produces a handshake that looks nothing
like a real browser, independent of IP or headers.
Swaps the CLI's HTTP transport (session setup, SKU lookup, download
link fetch, eval link resolution) from net/http to
github.com/bogdanfinn/tls-client, which wraps utls with a maintained
Chrome browser profile (TLS + HTTP2 fingerprint together, not just
TLS -- a browser-consistent one without the other is its own tell).
Bumped msUA's claimed Chrome version to 133 to match the chosen
profile, since a mismatched UA vs. TLS fingerprint is itself
detectable.
The custom simpleCookieJar is gone -- tls-client provides its own
cookie jar. fetchEvalLinks got the same treatment for consistency,
even though the Eval Center path isn't currently blocked.
Verified: go build/vet/test all pass, and a live end-to-end run
(msdl --id 3262 --lang English) returned a real signed download link
and successfully contributed it back to the shared cache. That proves
the flow still works functionally through the new client -- it does
NOT prove the fingerprint theory, since the old client already
succeeded ~80% of the time. The real test is watching the Sentinel-
rejection-rate telemetry over a comparable multi-day window after
this ships.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Log analysis over 17 days showed 181 Sentinel-block retry attempts on
/proxy, all 181 failed (0% success). Retrying every 90 minutes yields
zero value right now and just adds more blocked-request noise against
our own IP for nothing in return. Bumped lockdownTTL to 5h so it backs
off harder while still recovering same-day if Microsoft's block ever
eases.
Also found /proxy never validated product_id against the known
catalog, so a request for a nonexistent ID (e.g. product_id=2861,
never a real product) still burned a full outbound Microsoft session
attempt and a Sentinel-block hit for something that could never
succeed anyway. Now rejected with 404 before any Microsoft call,
reusing the existing validContributeProducts allow-list.
Verified locally: unknown product_id -> 404 (no MS fetch attempted in
logs), missing params -> 400 unchanged, known product_id -> proceeds
to a real MS session attempt as before.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cloudflare's deploy log has been flagging the SPA fallback rule
(/* /index.html 200) as a false-positive infinite loop and ignoring
it on every deploy. Investigated live: direct navigation to /about, a
dynamic route, and a genuinely invalid path all return 200 with the
correct content already, since Cloudflare Pages' own default fallback
serves index.html for any unmatched path. The rule was dead weight --
removing it to stop the recurring warning noise. Closes the tracked
issue in CLAUDE.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CliHandoff: the visible curl install command was a hardcoded truncated
placeholder ("...install.sh"), so the copy button worked but manually
selecting/copying the text gave a broken command. Now shows the real URL
and lets CSS `truncate` handle the visual clipping.
- Dock: adds a full-width backdrop-blur shelf under the floating desktop
dock so it doesn't visually blend into scrolled content (page bg is
near-black, so plain color gradients were invisible -- blur is what
actually reads). Desktop only; the mobile dock is already a flush bar.
- HomePage: hero now mentions "Open source" (linked, dotted underline) --
previously absent anywhere on the landing page.
- StatsBar: "releases available" count now includes eval/enterprise
editions, not just the consumer catalog (17 -> combined total).
- About/Privacy/Disclaimer: these hadn't been updated since the CLI
shipped. Privacy Policy now discloses the CLI's anonymous telemetry and
crowdsourced link-contribution behavior (previously undisclosed). About
page's stale hardcoded product list replaced with a link to the catalog,
plus a new section explaining the CLI. Disclaimer now covers the CLI
binary under the same terms.
- Fido/Rufus attribution was scattered with duplicate linked mentions on
the same pages; consolidated to one canonical linked credit per surface
(About's Credits section, README's intro) instead of repeating it.
- FAQAccordion: fixed a factual error ("ported to Go and Node.js" -- no
Node.js backend exists), and linked the actual repo in the open-source
FAQ answer (previously just asserted it with no link).
- sitemap.xml: removed a dead entry for product 48, which was already
removed from the catalog (Microsoft returns 502 for it).
- README: fixed three inaccuracies in the /contribute API doc (wrong auth
header, wrong body field name, wrong status codes) against the actual
backend/main.go implementation, and added the missing telemetry/sentinel
fields to the /metrics example response.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Local mirror only -- AUR account registration is still disabled
(since 2026-06-15), so this isn't pushed to aur.archlinux.org yet.
Kept in sync so it's ready to publish the moment registration
reopens.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
IMPLEMENTATION_PLAN.md (Phases 1-5: CLI direct-to-Microsoft, Redis L2
cache, /contribute crowdsourcing, web graceful degradation, CLI
handoff UI, distribution) had shipped in full but still read "Status:
Proposed" -- folded a summary of what actually landed into
PROGRESS.md instead.
issues.md's one drafted issue (inactive products still triggering
backend requests) is already fixed -- ProductDetailPage.tsx gates the
SKU-info effect on meta.active.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prior commit staged an intermediate version of this file. Completes
the Method 1 / Method 2 writeup with usage examples, the comparison
table, and the field-name bug/session-permit history.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Moves msdls_v3.py from repo root into scripts/check-new-releases/,
alongside a new Go tool, documented as Method 1 (Python brute-force
range scan, can write results to a catalog-shaped JSON) and Method 2
(Go auto-discovery via page-scrape + bounded adjacent-ID probe, no
range-guessing needed). Only Windows 11 is checked -- 8.1 is fully
frozen and Windows 10 is past end-of-life, so neither will produce a
new consumer ISO again.
Fixed msdls_v3.py's release-name extraction: it checked EditionName/
ReleaseName/FriendlyName, none of which exist in the real API
response -- the actual field is ProductDisplayName, confirmed live.
Also added the vlscppe session-permit call to setup_session(), which
was skipped entirely; higher-risk gap for this script's brute-force
range-scan usage pattern than for a single lookup.
Both tools' live runs surfaced a real catalog gap: product IDs
3322/3323/3325/3326 (Home/Pro China variants of the 25H2 "V2" refresh)
exist on Microsoft's side but weren't in our catalog. Added them to
cli/catalog.go, products.json, validContributeProducts, and
sitemap.xml, and renamed 3321/3324 from our own guessed "(Updated
Oct)" label to Microsoft's actual name for this refresh, "(V2)"
(their internal name is "Windows 11 25H2__V2"; cleaned up the
double-underscore for user-facing display).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The msdl-cli vs msdl naming footnote is useful context in README/release
notes, but unnecessary clutter for a casual visitor on the web CLI page --
the "recommended" badge and the command itself are enough.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
homebrew/core already has an unrelated package literally named "msdl"
(a streaming-protocol downloader). brew install msdl silently
resolved to that instead of our tap, since Homebrew always prefers
core for a bare-name collision -- confirmed in production when a
user's `brew install msdl` pulled homebrew-core's msdl 1.2.7-r2
instead of ours. The tap's formula is renamed to msdl-cli
(starkSV/homebrew-msdl, separate repo) to fix this permanently; the
installed command itself is unaffected, still just `msdl`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a GET /install.sh endpoint on the backend (embedded via Go's
embed package, so the script stays a real, shellcheck-able .sh file
rather than a Go string literal). Auto-detects OS/arch (including
Termux on Android via $PREFIX) and always resolves the latest GitHub
release via the /releases/latest/download/ redirect, so unlike the
winget/brew/AUR manifests it needs no per-release maintenance.
Verified end-to-end in an ephemeral Ubuntu container (both the normal
/usr/local/bin path and the Termux $PREFIX path), and locally against
a running backend instance.
Documented alongside the existing winget/Homebrew instructions in
README.md, the release notes template, the web CLI page, and the
per-product CliHandoff card.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Arch Linux disabled new AUR account registrations on 2026-06-15 after
a malware campaign compromised 1,500+ AUR packages. There's no
announced reopening date, so aur/msdl-bin/README.md now says so
clearly instead of implying publishing is just a pending step.
Adds the Homebrew tap install command (brew tap starkSV/msdl && brew
install msdl) to every place winget is already documented: README.md,
the release workflow's notes, the web CLI page, and the per-product
CliHandoff card. Also adds the missing linux-arm64 row to the manual
download tables.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
.SRCINFO generated with the real makepkg tool inside an ephemeral
archlinux container -- no VM or native Arch install needed, and no
risk of hand-written formatting drift from the makepkg-canonical
output.
Also adds .gitattributes forcing LF for PKGBUILD/.SRCINFO regardless
of platform checkout settings -- both are parsed by shell/makepkg
tooling that breaks on CRLF.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tracks the AUR PKGBUILD in-repo for version history, mirroring how
winget/manifests/ already works. Also corrects CONTRIBUTING.md's claim
that winget auto-updates via CI -- WINGET_TOKEN isn't set, so that
step always no-ops; package manager updates are all submitted
manually by the maintainer after each release.
Homebrew tap published separately at starkSV/homebrew-msdl.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Matches the shape actually asked for and how every other telemetry
counter in this file already works (all-time cumulative via HIncrBy,
no expiry, no per-day key rotation): /metrics now exposes flat
sentinel_errors and sentinel_distinct_sources_est fields instead of a
nested, date-scoped sentinel_today object.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CLI telemetry showed a growing count of Sentinel rejections but no way
to tell whether they come from one user retrying repeatedly or many
distinct users independently getting blocked.
Adds a per-day Redis HyperLogLog (msdl:telemetry:sentinel_hll:<date>)
seeded from the reporting IP whenever a CLI error contains "Sentinel",
plus a parallel daily counter. /metrics now exposes sentinel_today:
{date, errors_today, distinct_sources_est}. No raw IPs are stored --
HLL is a one-way cardinality estimator, and both keys expire after 48h.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The bare `msdl` invocation showed all 22 products (17 consumer + 5
eval) as a flat numbered list. Replaces it with a curated landing
screen: 5 popular products (picked from real usage telemetry) plus a
2-item eval shortlist, with "list" as an escape hatch to today's full
picker and free text routed through the existing search flow.
Also fixes a search bug: querying "windows 10" incorrectly matched
Windows 11 24H2 because "10" is a substring of its build number
(26100.1742). Query words now must start at a word boundary rather
than matching anywhere, so digit fragments inside unrelated numbers
can no longer collide -- while "arm" still fuzzy-matches "ARM64".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Consumer Windows products list their English option as "English" (or
"English International"), not "English (United States)" -- the exact
example this used, which is why it was such a common source of --lang
mismatch errors. Applies to --help, the --lang flag description, and
the release workflow's usage example.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs combined to make v0.3.4's changelog show the previous
release's version-bump commit instead of the actual change:
1. The path filter (cli/ backend/ frontend/) excluded workflow-only
changes, so a CI-only addition (like linux-arm64) never appeared.
2. The "bump version + winget manifest" commit is created *after* its
own tag (it needs the release's SHA256, which doesn't exist until
after tagging) -- so it always lands in the *next* release's diff
range instead of its own, showing up as stale noise.
Adds .github/workflows/cli-release.yml to the path filter, filters
out "chore(release): bump" commits as noise, and falls back to a
generic line if a release genuinely has nothing else to show.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Moves the README's inline contributing notes into a dedicated file
and expands them with dev setup, PR conventions, and how CLI releases
work, so contributors have one place to look.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Telemetry showed a growing number of Termux (Android) users running
the CLI, but the release workflow only builds linux-amd64 -- those
users had to build from source since Termux runs on ARM64. Adds a
linux-arm64 binary and install line so they can just curl it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Real-world telemetry showed --lang "English (United States)" failing
for several consumer products that only list "English International"
-- the exact example used in --help. The mismatch error now lists the
languages actually available, and the help text notes names vary by
product.
Also fixes truncateError to keep both head and tail of long error
strings (previously kept only the head, which for wrapped errors with
a long URL cut off the actual root cause at the end).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
