feat: two-layer in-memory caching for backend (closes #12) #14

Merged

starkSV merged 1 commit from feat/caching-layer into main

2026-05-18 22:49:34 +05:30

starkSV commented

2026-05-18 22:49:18 +05:30

(Migrated from github.com)

Summary

Reduces Microsoft API traffic from thousands/day to ~50–100/day, eliminating the 715-123130 rate-limit block under real traffic.

What's implemented

#	Sub-task	Detail
1	Singleflight	`golang.org/x/sync/singleflight` wraps all MS fetches — 500 concurrent cache misses → 1 API hit
2	SKU info cache	7-day TTL keyed by `product_id` — language lists are stable
3	Download link cache	Dynamic TTL keyed by `product_id:sku_id`
4	Negative cache	60s TTL on 429/715-123130 — stops retry storms from worsening a block
5	Dynamic TTL	Parses `se` (signed expiry) from CDN URL: `(se - now) - 30min`, falls back to 22h
6	Stale-on-failure	Serves expired cache entry if refresh fails — no global outage on transient rate-limit
7	Jitter	±5min random offset on all TTLs — prevents synchronised mass-expiry spikes

Verified locally

2026/05/18 22:45:50 eval warm: server-2025 -> 8 links cached
2026/05/18 22:45:50 MS fetch /skuinfo: product_id=3113 -> 38 languages
2026/05/18 22:45:50 /skuinfo: product_id=3113 -> fetched and cached
2026/05/18 22:45:50 /skuinfo: product_id=3113 -> cache hit
2026/05/18 22:45:50 /skuinfo: product_id=3113 -> cache hit
2026/05/18 22:45:52 /proxy: product_id=3113 sku_id=18480 -> fetched and cached until 2026-05-19T20:48:14
2026/05/18 22:45:52 /proxy: product_id=3113 sku_id=18480 -> cache hit
2026/05/18 22:46:07 /evallinks: product=server-2019 -> cache hit (8 links)

Test plan

Deploy to Coolify, watch logs for cache hit/miss lines under real traffic
Confirm cached until <timestamp> shows dynamic TTL (~22h from fetch time)
Confirm eval warm lines fire at startup
Monitor for absence of 715-123130 errors under load

Summary by CodeRabbit

Release Notes

New Features
- Intelligent multi-layer caching reduces response times and server load
- Fallback mechanism serves cached data during service disruptions
- Improved handling of concurrent requests for better efficiency
- Enhanced rate-limit detection and error recovery
Chores
- Updated runtime version for improved stability and performance

## Summary Reduces Microsoft API traffic from thousands/day to ~50–100/day, eliminating the 715-123130 rate-limit block under real traffic. ## What's implemented | # | Sub-task | Detail | |---|---|---| | 1 | **Singleflight** | `golang.org/x/sync/singleflight` wraps all MS fetches — 500 concurrent cache misses → 1 API hit | | 2 | **SKU info cache** | 7-day TTL keyed by `product_id` — language lists are stable | | 3 | **Download link cache** | Dynamic TTL keyed by `product_id:sku_id` | | 4 | **Negative cache** | 60s TTL on 429/715-123130 — stops retry storms from worsening a block | | 5 | **Dynamic TTL** | Parses `se` (signed expiry) from CDN URL: `(se - now) - 30min`, falls back to 22h | | 6 | **Stale-on-failure** | Serves expired cache entry if refresh fails — no global outage on transient rate-limit | | 7 | **Jitter** | ±5min random offset on all TTLs — prevents synchronised mass-expiry spikes | ## Verified locally 2026/05/18 22:45:50 eval warm: server-2025 -> 8 links cached 2026/05/18 22:45:50 MS fetch /skuinfo: product_id=3113 -> 38 languages 2026/05/18 22:45:50 /skuinfo: product_id=3113 -> fetched and cached 2026/05/18 22:45:50 /skuinfo: product_id=3113 -> cache hit 2026/05/18 22:45:50 /skuinfo: product_id=3113 -> cache hit 2026/05/18 22:45:52 /proxy: product_id=3113 sku_id=18480 -> fetched and cached until 2026-05-19T20:48:14 2026/05/18 22:45:52 /proxy: product_id=3113 sku_id=18480 -> cache hit 2026/05/18 22:46:07 /evallinks: product=server-2019 -> cache hit (8 links) ## Test plan - [ ] Deploy to Coolify, watch logs for cache hit/miss lines under real traffic - [ ] Confirm `cached until <timestamp>` shows dynamic TTL (~22h from fetch time) - [ ] Confirm eval warm lines fire at startup - [ ] Monitor for absence of 715-123130 errors under load  ## Summary by CodeRabbit ## Release Notes * **New Features** * Intelligent multi-layer caching reduces response times and server load * Fallback mechanism serves cached data during service disruptions * Improved handling of concurrent requests for better efficiency * Enhanced rate-limit detection and error recovery * **Chores** * Updated runtime version for improved stability and performance  [![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/starkSV/windows-iso-downloader/pull/14?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

coderabbitai[bot] commented

2026-05-18 22:49:27 +05:30

(Migrated from github.com)

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 28a1604d-fdea-4d32-8296-30f3122fbe04

📥 Commits

Reviewing files that changed from the base of the PR and between ddf96a873d and c6d8615cf1.

⛔ Files ignored due to path filters (1)

backend/go.sum is excluded by !**/*.sum

📒 Files selected for processing (2)

backend/go.mod
backend/main.go

📝 Walkthrough

Walkthrough

Refactored Microsoft API interactions in backend/main.go to add multi-layer caching (SKU info, download links, negative cache, eval links), singleflight deduplication for concurrent requests, structured rate-limit error classification, jittered cache TTLs, and stale-on-failure serving. Dependency added: golang.org/x/sync for singleflight.

Changes

Microsoft Caching and Concurrency

Layer / File(s)	Summary
Dependency and import setup `backend/go.mod`, `backend/main.go` (imports)	Go version updated to 1.25.0; added `golang.org/x/sync v0.20.0` dependency; imported `math/rand` and `golang.org/x/sync/singleflight` for jitter and request deduplication.
Cache and synchronization infrastructure `backend/main.go` (cache globals, TTL constants)	Added SKU info cache (7-day TTL), download link cache (dynamic expiry), negative cache (60-second rate-limit protection), session cache, and global `singleflight.Group` for per-key deduplication of concurrent Microsoft fetches.
Core helper utilities and error types `backend/main.go` (helpers, types, constants)	Introduced `jitter()` for cache TTL variance, `parseLinkExpiry()` to extract expiry from Microsoft signed-URL `se` parameter, `rateLimitError` type to classify rate-limit signals (Type 9 / code `715-123130`), regex patterns for link/error extraction, and eval cache storage with TTL.
Session setup and Microsoft fetch implementations `backend/main.go` (setupSession, fetchSKUInfo, fetchDownloadLinksFromMS, helpers)	Reworked `setupSession()` to handle fingerprint-trigger requests and JS extraction; refactored `fetchSKUInfo()` to return structured errors and raw bytes for caching; implemented `fetchDownloadLinksFromMS()` to map TooManyRequests and non-OK responses to `rateLimitError`; added worker-secret header setter; refactored eval-links extraction with deduplication and language-first sorting.
Endpoint handlers with caching strategies `backend/main.go` (/skuinfo, /proxy, /evallinks handlers)	`/skuinfo` checks negative cache, then SKU cache, singleflights fetch, stores negative entries on rate-limit, caches success with jitter, returns raw JSON. `/proxy` checks link cache, conditionally checks negative cache (only if no stale), singleflights per-product fetch, serves stale on failure, caches with dynamic expiry. `/evallinks` serves cached results when fetch fails or yields zero links.
Routing and server setup `backend/main.go` (main function)	Updated `main()` to explicitly register `/skuinfo`, `/proxy`, `/evallinks`, and `/health` handlers; warmup and session cleanup logic retained.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

starkSV/windows-iso-downloader#12: This PR directly implements the two-tier SKU + download-link caching, singleflight deduplication, dynamic TTL parsing from Microsoft signed URLs, stale-on-failure behavior, and negative-cache rate-limit protection described in the issue.

starkSV/windows-iso-downloader#7: Both PRs modify the /skuinfo and /proxy Microsoft-fetch paths in backend/main.go; #7 wraps requests via Cloudflare proxy/X-Worker-Secret header, while this PR adds caching, singleflight, and structured error handling around those same fetch flows.

Poem

🐰 The caches leap and singleflight calls,
No more shall Microsoft rate-limits frustrate halls,
With jitter and stale, when fetches all fail,
The Windows ISO downloader prevails! ✨

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/caching-layer

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

> [!CAUTION] > ## Review failed > > The pull request is closed.  <details> <summary>ℹ️ Recent review info</summary> <details> <summary>⚙️ Run configuration</summary> **Configuration used**: defaults **Review profile**: CHILL **Plan**: Pro Plus **Run ID**: `28a1604d-fdea-4d32-8296-30f3122fbe04` </details> <details> <summary>📥 Commits</summary> Reviewing files that changed from the base of the PR and between ddf96a873df3ce10e30495206b7b6a5400de0a9a and c6d8615cf18d21c1e8e1e460f0fe163e9319a58d. </details> <details> <summary>⛔ Files ignored due to path filters (1)</summary> * `backend/go.sum` is excluded by `!**/*.sum` </details> <details> <summary>📒 Files selected for processing (2)</summary> * `backend/go.mod` * `backend/main.go` </details> </details> ---  <details> <summary>📝 Walkthrough</summary> ## Walkthrough Refactored Microsoft API interactions in `backend/main.go` to add multi-layer caching (SKU info, download links, negative cache, eval links), singleflight deduplication for concurrent requests, structured rate-limit error classification, jittered cache TTLs, and stale-on-failure serving. Dependency added: `golang.org/x/sync` for singleflight. ## Changes **Microsoft Caching and Concurrency** |Layer / File(s)|Summary| |---|---| |**Dependency and import setup** `backend/go.mod`, `backend/main.go` (imports)|Go version updated to 1.25.0; added `golang.org/x/sync v0.20.0` dependency; imported `math/rand` and `golang.org/x/sync/singleflight` for jitter and request deduplication.| |**Cache and synchronization infrastructure** `backend/main.go` (cache globals, TTL constants)|Added SKU info cache (7-day TTL), download link cache (dynamic expiry), negative cache (60-second rate-limit protection), session cache, and global `singleflight.Group` for per-key deduplication of concurrent Microsoft fetches.| |**Core helper utilities and error types** `backend/main.go` (helpers, types, constants)|Introduced `jitter()` for cache TTL variance, `parseLinkExpiry()` to extract expiry from Microsoft signed-URL `se` parameter, `rateLimitError` type to classify rate-limit signals (Type 9 / code `715-123130`), regex patterns for link/error extraction, and eval cache storage with TTL.| |**Session setup and Microsoft fetch implementations** `backend/main.go` (setupSession, fetchSKUInfo, fetchDownloadLinksFromMS, helpers)|Reworked `setupSession()` to handle fingerprint-trigger requests and JS extraction; refactored `fetchSKUInfo()` to return structured errors and raw bytes for caching; implemented `fetchDownloadLinksFromMS()` to map TooManyRequests and non-OK responses to `rateLimitError`; added worker-secret header setter; refactored eval-links extraction with deduplication and language-first sorting.| |**Endpoint handlers with caching strategies** `backend/main.go` (/skuinfo, /proxy, /evallinks handlers)|`/skuinfo` checks negative cache, then SKU cache, singleflights fetch, stores negative entries on rate-limit, caches success with jitter, returns raw JSON. `/proxy` checks link cache, conditionally checks negative cache (only if no stale), singleflights per-product fetch, serves stale on failure, caches with dynamic expiry. `/evallinks` serves cached results when fetch fails or yields zero links.| |**Routing and server setup** `backend/main.go` (main function)|Updated `main()` to explicitly register `/skuinfo`, `/proxy`, `/evallinks`, and `/health` handlers; warmup and session cleanup logic retained.| ## Estimated code review effort 🎯 4 (Complex) | ⏱️ ~50 minutes ## Possibly related issues - [starkSV/windows-iso-downloader#12](https://github.com/starkSV/windows-iso-downloader/issues/12): This PR directly implements the two-tier SKU + download-link caching, singleflight deduplication, dynamic TTL parsing from Microsoft signed URLs, stale-on-failure behavior, and negative-cache rate-limit protection described in the issue. ## Possibly related PRs - [starkSV/windows-iso-downloader#7](https://github.com/starkSV/windows-iso-downloader/pull/7): Both PRs modify the `/skuinfo` and `/proxy` Microsoft-fetch paths in `backend/main.go`; `#7` wraps requests via Cloudflare proxy/X-Worker-Secret header, while this PR adds caching, singleflight, and structured error handling around those same fetch flows. ## Poem > 🐰 *The caches leap and singleflight calls,* > *No more shall Microsoft rate-limits frustrate halls,* > *With jitter and stale, when fetches all fail,* > *The Windows ISO downloader prevails!* ✨ </details>   <details> <summary>✨ Finishing Touches</summary> <details> <summary>📝 Generate docstrings</summary> - [ ]  Create stacked PR - [ ]  Commit on current branch </details> <details> <summary>🧪 Generate unit tests (beta)</summary> - [ ]  Create PR with unit tests - [ ]  Commit unit tests in branch `feat/caching-layer` </details> </details>   > [!WARNING] > There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. > > <details> > <summary>🔧 golangci-lint (2.12.2)</summary> > > level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" > > > > > </details>   --- Comment `@coderabbitai help` to get the list of available commands and usage tips.