Squid Web Cache wiki

Squid Web Cache documentation

🔗 Feature: Support caching of partial responses

🔗 Details

(from the bug report): When range_offset_limit is set to -1, Squid tries to fetch the entire object in response to an HTTP range request.

The proper fix for this is to add caching of partial responses, eleminating the need for range_offset_limit entirely.

🔗 Proposal 1: chunked sequencing

Squid should store each object as a series of N-byte chunks. When a range is requested the whole of the chunk containing those bytes should be fetched into the relevat chunk position.

For example; in my cache i would save it in ranges of 1KB which means 40KB file will be 40 chunks of 1KB each.

Suppose my file is called faysal.data

chunk0 = faysal.data,0-1KB
chunk1 = faysal,data,1-2KB
...

Now suppose the file is not cached yet nor requested and I request a range: bytes=2100-3000.

What Squid should do is:

  1. skip the 1024-2048 chunk
  2. fetch the 2049-3072 chunk
  3. optionally skip the 3073+ chunks, or contiue fetching.

Caching the chunks received and marking the response as incomplete.

🔗 Proposal 2: sparse cache file

Squid should open a disk file for the full file size and fill only the bytes received. The rest can be loaded in background requests later.

NP: requires some method of mapping which file bytes have been received and accountig for the variable-length metadata headers in each cache object file.

🔗 Proposal 3: caching by Range

Squid caches Vary responses using additional cache key components. Range responses should be cached using a similar mechanism altering the cache key for the range.

NP: needs logic to represent a list of all stored ranges in the variant meta entry. Vary caching uses X-Vary-Headers. Range would need a payload.

Update: a Store-ID helper can possibly be used to track which URLs are stored incompletely and handle the key alterations.

Categories: WantedFeature

Navigation: Site Search, Site Pages, Categories, 🔼 go up