WIP: Begin reorganizing the kernel with kernel-space typescript fs

[ElvishJerricco]

The plan here is to move all the syscall implementations into the kernel space. For each syscall the process makes to the kernel, it will postMessage a SharedArrayBuffer containing all the relevant information to the kernel, then Atomics.wait on the buffer. The kernel will then do all the appropriate work and Atomics.wake the process with results.

To do all of this on nodejs, we can use the worker_threads module. It's experimental, but so is everything about WebGHC, so this doesn't bother me :P

This is WIP until the new kernel can actually run wasm programs.

[ElvishJerricco]

/cc @dfordivam

[ElvishJerricco]

Of course, allocating a new SharedArrayBuffer on each syscall is a bad plan. So we should try to share one between syscalls until it needs to grow for whatever reason.

Also, of course not all syscalls will want to jump out to the kernel. Memory allocation, for instance, still needs to be done in the user space.

[ElvishJerricco]

Ideally, the buffer underlying the webassembly memory would be our sharedarraybuffer, but wasm doesn't support that yet.

[dfordivam]

If I understand correctly, we are using SharedArrayBuffer just for Atomics wait and wake. In that case we can decouple the messaging onto a regular buffer / regular Set style messaging, or perhaps if typescript offers some better type safe messaging mechanism then use that.

So I think we need not worry about growing the SharedArrayBuffer here.

Hmm just realized we need to get the data back also.. and currently there is no way to do that.. so will have to use SharedArrayBuffer

Also as the kernel is now a separate worker, it would be able to handle incoming messages (like jsaddle JS).
So ideally there should be a mechanism to execute the relevant handler when we are doing the syscalls.
This will require polling another channel of SharedBufferArray for pending interrupt messages everytime we do a syscall?

[ElvishJerricco]

@dfordivam We are using the SharedArrayBuffer not just for wait and wake, but also as the payload. Rather than sending JSON serializable data over postMessage, we'll just put all the relevant data in the SharedArrayBuffer. This ought to be much faster.

I'm not sure what you mean when you say "handler." The "process" (the worker executing the wasm) will be blocked on at most one syscall at a time, so the kernel can always wake it with whatever is appropriate.

[dfordivam]

As for postMessage we would definitely use the regular messaging style (no SharedBufferArray).
We dont have to serialize the actual payload, that can go as a buf argument ...

My point was SharedBufferArray is only useful for the return messaging, for doing postMessage we dont need that.

Regarding handler I meant that the sequence of syscalls should be

function syscall{
...
  case XYZ:
     doSyscall_xyz; <- blocking on Atomics.wait
...
  checkForPendingInterrupts(); <- non-blocking, poll a different SharedArrayBuffer written by kernel worker.
}

[ElvishJerricco]

I don't think we'll need any fancy JSON communication like that. Basically the only difference between operating on this SharedArrayBuffer and on the actual wasm memory will be that things need to be copied. The process might have to remember some mappings from the buffer to wasm memory, but the kernel shouldn't have to transmit any info like that. AFAIK, no syscalls start reading and writing to memory that the process didn't explicitly ask for.

As for interrupts: Well, we're not really considering interrupts yet :P But when we do, I think we'll just make it so that each syscall that blocks on the SharedArrayBuffer will just also respond to a special signal from the kernel that says "Hey this is a signal, not what you asked for." Then they'll enter a generic routine for signal handling.