shekhar/windows-iso-downloader
1
0
Fork 0
Code Issues Pull requests Projects Releases 15 Packages Wiki Activity Actions

feat(cli): TLS/HTTP2 fingerprint hardening via tls-client

Browse source 
Three checkpoints over 17 days confirmed a stable ~20% Sentinel
rejection rate on the CLI's own requests, despite running from
residential IPs specifically to avoid ASN-based blocking. That
persistence points at TLS ClientHello fingerprinting as an additional
signal: Go's stdlib crypto/tls produces a handshake that looks nothing
like a real browser, independent of IP or headers.

Swaps the CLI's HTTP transport (session setup, SKU lookup, download
link fetch, eval link resolution) from net/http to
github.com/bogdanfinn/tls-client, which wraps utls with a maintained
Chrome browser profile (TLS + HTTP2 fingerprint together, not just
TLS -- a browser-consistent one without the other is its own tell).
Bumped msUA's claimed Chrome version to 133 to match the chosen
profile, since a mismatched UA vs. TLS fingerprint is itself
detectable.

The custom simpleCookieJar is gone -- tls-client provides its own
cookie jar. fetchEvalLinks got the same treatment for consistency,
even though the Eval Center path isn't currently blocked.

Verified: go build/vet/test all pass, and a live end-to-end run
(msdl --id 3262 --lang English) returned a real signed download link
and successfully contributed it back to the shared cache. That proves
the flow still works functionally through the new client -- it does
NOT prove the fingerprint theory, since the old client already
succeeded ~80% of the time. The real test is watching the Sentinel-
rejection-rate telemetry over a comparable multi-day window after
this ships.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Shekhar Vaidya 2026-07-31 12:02:44 +05:30
parent 6ace2ceb19
commit 9e22cd8bf8
3 changed files with 99 additions and 41 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
cli/go.mod
View file

@ -1,3 +1,24 @@
module github.com/starkSV/msdl-cli
go 1.21
go 1.24.1
require (
github.com/bogdanfinn/fhttp v0.6.8
github.com/bogdanfinn/tls-client v1.15.1
)
require (
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/bdandy/go-errors v1.2.2 // indirect
github.com/bdandy/go-socks4 v1.2.3 // indirect
github.com/bogdanfinn/quic-go-utls v1.0.9-utls // indirect
github.com/bogdanfinn/utls v1.7.7-barnius // indirect
github.com/bogdanfinn/websocket v1.5.5-barnius // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 // indirect
golang.org/x/crypto v0.46.0 // indirect
golang.org/x/net v0.48.0 // indirect
golang.org/x/sys v0.39.0 // indirect
golang.org/x/text v0.32.0 // indirect
)

48
cli/go.sum Normal file
View file

@ -0,0 +1,48 @@
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/bdandy/go-errors v1.2.2 h1:WdFv/oukjTJCLa79UfkGmwX7ZxONAihKu4V0mLIs11Q=
github.com/bdandy/go-errors v1.2.2/go.mod h1:NkYHl4Fey9oRRdbB1CoC6e84tuqQHiqrOcZpqFEkBxM=
github.com/bdandy/go-socks4 v1.2.3 h1:Q6Y2heY1GRjCtHbmlKfnwrKVU/k81LS8mRGLRlmDlic=
github.com/bdandy/go-socks4 v1.2.3/go.mod h1:98kiVFgpdogR8aIGLWLvjDVZ8XcKPsSI/ypGrO+bqHI=
github.com/bogdanfinn/fhttp v0.6.8 h1:LiQyHOY3i0QoxxNB7nq27/nGNNbtPj0fuBPozhR7Ws4=
github.com/bogdanfinn/fhttp v0.6.8/go.mod h1:A+EKDzMx2hb4IUbMx4TlkoHnaJEiLl8r/1Ss1Y+5e5M=
github.com/bogdanfinn/quic-go-utls v1.0.9-utls h1:tV6eDEiRbRCcepALSzxR94JUVD3N3ACIiRLgyc2Ep8s=
github.com/bogdanfinn/quic-go-utls v1.0.9-utls/go.mod h1:aHph9B9H9yPOt5xnhWKSOum27DJAqpiHzwX+gjvaXcg=
github.com/bogdanfinn/tls-client v1.15.1 h1:KiFAlED55DJ8Fcocn+/1nX6PrDFcttIHAf/GDkV6KN8=
github.com/bogdanfinn/tls-client v1.15.1/go.mod h1:LsU6mXVn8MOFDwTkyRfI7V1BZM1p0wf2ZfZsICW/1fM=
github.com/bogdanfinn/utls v1.7.7-barnius h1:OuJ497cc7F3yKNVHRsYPQdGggmk5x6+V5ZlrCR7fOLU=
github.com/bogdanfinn/utls v1.7.7-barnius/go.mod h1:aAK1VZQlpKZClF1WEQeq6kyclbkPq4hz6xTbB5xSlmg=
github.com/bogdanfinn/websocket v1.5.5-barnius h1:bY+qnxpai1qe7Jmjx+Sds/cmOSpuuLoR8x61rWltjOI=
github.com/bogdanfinn/websocket v1.5.5-barnius/go.mod h1:gvvEw6pTKHb7yOiFvIfAFTStQWyrm25BMVCTj5wRSsI=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 h1:YqAladjX7xpA6BM04leXMWAEjS0mTZ5kUU9KRBriQJc=
github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5/go.mod h1:2JjD2zLQYH5HO74y5+aE3remJQvl6q4Sn6aWA2wD1Ng=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=
go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=
golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=
golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=
golang.org/x/net v0.0.0-20211104170005-ce137452f963/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=
golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk=
golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=
golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

69
cli/microsoft.go
View file

@ -7,17 +7,25 @@ import (
"fmt"
"html"
"io"
"net/http"
"net/url"
"regexp"
"strconv"
"strings"
"sync"
"time"
http "github.com/bogdanfinn/fhttp"
tls_client "github.com/bogdanfinn/tls-client"
"github.com/bogdanfinn/tls-client/profiles"
)
// msTLSProfile is the browser TLS/HTTP2 fingerprint the CLI presents to Microsoft.
// Kept in step with msUA's claimed Chrome version -- a mismatched UA vs. TLS
// fingerprint is itself a detectable signal, so if one changes, change both.
var msTLSProfile = profiles.Chrome_133
const (
msUA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
msUA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
msProfile = "606624d44113"
msLocale = "en-US"
msOrgID = "y6jn8c31"
@ -47,37 +55,6 @@ type EvalLink struct {
URL string
}
type simpleCookieJar struct {
mu sync.Mutex
cookies []*http.Cookie
}
func (j *simpleCookieJar) SetCookies(_ *url.URL, cookies []*http.Cookie) {
j.mu.Lock()
defer j.mu.Unlock()
for _, c := range cookies {
found := false
for i, existing := range j.cookies {
if existing.Name == c.Name {
j.cookies[i] = c
found = true
break
}
}
if !found {
j.cookies = append(j.cookies, c)
}
}
}
func (j *simpleCookieJar) Cookies(_ *url.URL) []*http.Cookie {
j.mu.Lock()
defer j.mu.Unlock()
out := make([]*http.Cookie, len(j.cookies))
copy(out, j.cookies)
return out
}
func newSessionID() string {
b := make([]byte, 16)
rand.Read(b)
@ -116,10 +93,16 @@ func mapDownloadType(n int) string {
}
}
func newSession() (*http.Client, string) {
func newSession() (tls_client.HttpClient, string) {
sessionID := newSessionID()
jar := &simpleCookieJar{}
client := &http.Client{Timeout: 15 * time.Second, Jar: jar}
client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(),
tls_client.WithTimeoutSeconds(15),
tls_client.WithClientProfile(msTLSProfile),
tls_client.WithCookieJar(tls_client.NewCookieJar()),
)
if err != nil {
return nil, sessionID
}
q1 := url.Values{}
q1.Set("org_id", msOrgID)
@ -158,7 +141,7 @@ func newSession() (*http.Client, string) {
return client, sessionID
}
func msGet(client *http.Client, reqURL, productID string) ([]byte, error) {
func msGet(client tls_client.HttpClient, reqURL, productID string) ([]byte, error) {
req, _ := http.NewRequest("GET", reqURL, nil)
req.Header.Set("User-Agent", msUA)
req.Header.Set("Referer", referer(productID))
@ -249,7 +232,7 @@ func parseDownloadLinks(raw []byte) ([]DownloadLink, error) {
return links, nil
}
func fetchLanguages(client *http.Client, sessionID, productID string) ([]Language, error) {
func fetchLanguages(client tls_client.HttpClient, sessionID, productID string) ([]Language, error) {
q := url.Values{}
q.Set("profile", msProfile)
q.Set("productEditionId", productID)
@ -267,7 +250,7 @@ func fetchLanguages(client *http.Client, sessionID, productID string) ([]Languag
}
// fetchDownloadLinks returns parsed links and the raw Microsoft JSON (for cache contribution).
func fetchDownloadLinks(client *http.Client, sessionID, productID, skuID string) ([]DownloadLink, []byte, error) {
func fetchDownloadLinks(client tls_client.HttpClient, sessionID, productID, skuID string) ([]DownloadLink, []byte, error) {
wq := url.Values{}
wq.Set("profile", msProfile)
wq.Set("productEditionId", productID)
@ -335,7 +318,13 @@ func detectLang(rawURL string) string {
}
func fetchEvalLinks(evalURL string) ([]EvalLink, error) {
client := &http.Client{Timeout: 20 * time.Second}
client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(),
tls_client.WithTimeoutSeconds(20),
tls_client.WithClientProfile(msTLSProfile),
)
if err != nil {
return nil, fmt.Errorf("creating http client: %w", err)
}
req, _ := http.NewRequest("GET", evalURL, nil)
req.Header.Set("User-Agent", msUA)
req.Header.Set("Accept", "text/html,application/xhtml+xml")