feat(selectivity): implement fs-caching

[KuznetsovRoman]

Caching selectivity assets on event "styleSheetAdded" or "scriptParsed"

• we check for existing cache, If it is cached, selectivity just acknowledges cache presence and does nothing extra.
• we check for existing cache, if it is NOT cached, its received via CDP or directly from node.js, then cached, and marked as "cached on fs"
• if source map is inlined, its not cached.

Caching on "stop record"

• If we encounter fs-cached asset, its read from fs

• If its not fs-cached and was not handled during test run, its received via CDP or directly from node.js, then cached and stored in RAM because we will use it right now

• Testplane runtime dependencies are cached as well

• "Testplane runtime dependencies" and "selectivity assets" are stored with different prefix in order to reduce chance of matching hashes

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/gemini-testing/testplane@1204

commit: 665e4df

[KuznetsovRoman]

used to read/write fs cache when multiple threads read/write the same cache
flag file (empty file with '-ready' prefix) is used as mark of successful cache write

[shadowusr]

Great additions overall! 🔥

Another important case I'd consider is to analyze if all cache-related errors are wrapped. Right now it feels like there are definitely places where we can get critical runtime errors, but would be better to just get "cache miss" & refetch. For example:

• JSON parse over here: https://github.com/gemini-testing/testplane/pull/1204/changes#diff-7eb13dc346c366c526687e33bf89aae5439daef0253a6bba001dbfd1a7d4f592R81
• throwing error over here (but probably it's handled, not sure): https://github.com/gemini-testing/testplane/pull/1204/changes#diff-55677838336ba3c4e42ca0361c8af649e5298055cb7bee4e28a93c116d9673f7R173

[shadowusr]

Hmmm this doesn't look correct, because getProtocol looks for :// in string, but data URLs don't have those?

Looking at examples on MDN:

data:text/plain;base64,SGVsbG8sIFdvcmxkIQ==

[shadowusr]

I prefer a more explicit object shapes like

enum AssetStatus {
    CachedOnFs,
    Loaded,
    Errored,
    Missing
}

interface CachedAsset {
    status: AssetStatus.CachedOnFs;
}

interface LoadedAsset {
    status: AssetStatus.Loaded;
    data: string;
}

// ...

type Asset = CachedAsset | LoadedAsset /* ... */;

This usually makes the code more explicit, readable, type-safe and easier to work with. For example, with current shape you've already pretty much exhausted what you can store in such objects, but what if you'll need to add another type of asset state which has string value too, but it's not an actual value? There's no convenient way to do that. With discriminated union, we have infinite options.

Not critical to me

[KuznetsovRoman]

yeah, its kind of quirky, but i wanted to avoid extra heap allocations and memory usage overall
Memory footprint here might seem low, but with large amount of workers and parallel tests and test assets, it adds up.

Without selectivity:

With selectivity:

Its memory usage when only "true" is stored for each asset as they are all fs-cached, and besides it, i dont see where selectivity uses extra RAM)

[shadowusr]

Hmmm one thing is a bit concerning to me over here:

• setCachedSelectivityFile doesn't provide a way to know if a cache write was successful, it can silently fail
• But we mark it as successful here anyways (by writing true)
• This leads to incorrect state: we consider cache to be successfully written on fs, but in fact it doesn't exist. Right now the stop() call would fail in this case with ENOENT when trying to read the source

Can this truly happen or am i missing something?

[KuznetsovRoman]

Hm, if "write" actually fails, then yeah, it can definitely happen.
It probably wont, because other test might check "cache does not exist" and fetch it, but it can.

I will fix it by removing ignoring error in "setCachedSelectivityFile" and handling it here so we would store actual value in RAM if writing to fs-cache fails