Add Range / resume support to Receive-RemoteItem and download endpoints

[michaellwest]

Summary

Receive-RemoteItem and the download endpoints under RemoteScriptCall.ashx are full-blob: every retry restarts from byte 0, and the entire response body is buffered in client memory before writing to disk. Add HTTP Range request handling on the server and resume-from-offset logic on the client.

Priority

Low. Typical SPE media items are images and PDFs under 15 MB, so re-downloading the full body on retry costs roughly 2-5 seconds on a normal CM/CI link and the in-memory buffering is harmless at that size. This issue exists for the unusual cases where it does matter (operators who store larger media, slow / metered links, parallel downloads compounding the cost) and as the long-term right answer for the safe-transport-retry classifier in #1487 to handle mid-stream connection resets cleanly. Treat as a "nice to have when there's time" rather than a near-term roadmap item.

Problem

A Receive-RemoteItem call that drops mid-stream has to re-transfer the entire body on retry. For a typical sub-15 MB media item this is annoying but bounded; for an outlier 200 MB asset on a flaky link it wastes the bytes already in flight.

Server (src/Spe/sitecore modules/PowerShell/Services/RemoteScriptCall.ashx.cs):

• AddContentHeaders (line 2435) writes Content-Type, Content-Disposition, Content-Length, Content-Transfer-Encoding. No Accept-Ranges.
• File download: context.Response.TransmitFile(file) (line 1589). Synchronous full-file send via http.sys.
• Media download: WebUtil.TransmitStream(mediaStream, response, StreamBufferSize) (line 1774). Full stream copy from byte 0.
• No Range / If-Range parsing anywhere in the handler.

Client (modules/SPE/Receive-RemoteItem.ps1):

• client.GetAsync($url).Result sends no Range header.
• ReadAsByteArrayAsync().Result (line 248) buffers the entire response into a [byte[]] before writing to disk.
• No offset tracking, no append-on-retry, no resume logic.

Server changes

In AddContentHeaders and the two download paths (file and media):

1. Add Accept-Ranges: bytes to all download responses unconditionally.
2. Parse the request Range: header. Accept the simplest forms: bytes=N- (open-ended) and bytes=N-M (closed). Single-range only; reject multipart/byteranges for v1.
3. On a valid Range:
   • Set status 206 Partial Content.
   • Set Content-Range: bytes N-M/total and Content-Length to the slice size.
   • Stream from byte N to byte M (or to EOF for open-ended).
4. On invalid Range (out-of-bounds, malformed): return 416 Range Not Satisfiable with Content-Range: bytes */total.
5. If-Range validation is out of scope for v1; mention as future work.

Implementation notes:

• TransmitFile does not take an offset/length. Replace with manual FileStream + bounded CopyTo(Response.OutputStream, ...) for the partial path. Keep TransmitFile on the full-body path because it is faster (kernel-mode send).
• WebUtil.TransmitStream does not take an offset. Use manual stream copy with Seek(N, SeekOrigin.Begin) followed by bounded copy.

Client changes

In Receive-RemoteItem.ps1:

1. On retry, check the destination file's existing size. If greater than 0, send Range: bytes=<size>- on the next request.
2. Replace ReadAsByteArrayAsync with streamed write: ResponseContent.CopyToAsync(FileStream). Eliminates the in-memory full-body buffer and lets resume actually save bytes on retry.
3. On 206 Partial Content: append to the destination file.
4. On 200 OK after sending a Range header: truncate the destination and write fresh (server may not support resume, or the resource changed - safest to restart).
5. On 416 Range Not Satisfiable: truncate and restart from byte 0.
6. Validate Content-Length (or Content-Range total) matches what we expect; surface a verbose log on mismatch.

Test plan

Unit:

• Server: Range: bytes=N- returns 206 with correct Content-Range and slice. bytes=N-M returns the closed slice. Out-of-range returns 416. Malformed Range returns 416. Missing Range returns 200 with full body (no regression).
• Client: existing partial destination file + retry sends Range: bytes=<size>-. 206 response appends. 416 response truncates and restarts. 200 response after a Range request truncates and writes fresh.

Integration:

• Upload a known-size media item (e.g. 5 MB), download it, truncate the local file to half, re-run Receive-RemoteItem, verify final bytes match the original.
• Server-side simulated mid-stream close after N bytes followed by -MaxRetries 2, verify resume completes the file.

Compatibility

Path	Behavior
Old client + new server	Client sends no Range, gets full body with 200 OK. Identical to today aside from the new Accept-Ranges: bytes advertising header.
New client + old server	Client sends Range, server ignores, returns full body with 200 OK. New client falls back to truncate-and-write. Correct, slightly wasteful on retry.
New client + new server, no retry	Identical to today (no Range sent on first attempt).
New client + new server, retry mid-stream	Resume from offset; only the missing bytes flow.

No template, config, or auth surface changes.

Out of scope

• Multi-range requests (bytes=0-99,200-299) and multipart/byteranges.
• If-Range ETag / Last-Modified validation.
• Parallel chunk downloads.
• Resume support for Send-RemoteItem (uploads). Different problem - server would need to track partial uploads with content-hash continuation; treat as a separate future issue.

Related

• Jittered backoff and safe-transport retry for remoting client #1487 - jittered backoff and safe-transport retry. Phase 1.5 (or whatever lands next on the retry track) can already auto-retry Receive-RemoteItem on transient failures because the cmdlet is read-only and safe by construction; resume support here is an optimization on top, not a prerequisite. At typical media sizes the wasted re-download cost is small enough that this issue can wait.