# Usermode Dusk

Usermode Dusk is a wrapper around Dusk OS allowing it to be used as an
applicative platform on modern OSes. This wrapper allows the code to be ran at
*native speed* (that's crazy fast) on the host OS. This wrapper is known to be
able to compile on:

* NetBSD
* FreeBSD
* Linux
* Hurd
* Windows (MinGW)
* Windows (Cygwin)

Because we're talking about native speed, this means that this can only be
ran on CPUs for which there's a support in Dusk OS and for which there is a
usermode kernel. This means:

* `i386` (works on `x86_64`)
* `arm`. Theoretically, Dusk's ARM code runs on `armv5+`, but it has only been
tested on v6+ so far. No `aarch64` for now, but theoretically, it works. We
probably have to fiddle with GCC flags.

Usermode Dusk can run on 64-bit variants of supported CPUs (for example
`x86_64`), but only if the host OS has the capability to run 32-bit executables.

To compile this wrapper on a 64-bit host OS, it often implies additional
tooling. For example, on Debian, you need to install `gcc-multilib` as well as
the 32-bit variants of libraries used by the different wrapper "flavors".

Theoretically, all combinations of hosts/CPU listed above work, but they haven't
been all tested. Moreover, the Makefile is likely broken for some of those
combinations. If you stumble on such combinations, please send patches to fix
the Makefile.

## Flavors

There are 4 "flavors" of Usermode Dusk: raw, stream, grid and graphic.

The "raw" flavor stands apart from the others. Unlike them, it's not really
designed to be used interactively, but rather to be a drop-in, lighting fast
replacement for the POSIX VM. Unlike the 3 other flavors, it has an embedded,
read-only tar FS (like the POSIX VM). Its payload and kernels are also embedded
in the executable.

Then come the "interactive" flavors, with kernel, payloads and disk image living
outside the executable for maximum debuggability. "disk.img", the external disk
image, is a read/write FAT.

The "stream" flavor is the lowest common denominator and has a serial interface
which is plugged in the host OS' standard input and output for programs. The
name of its target is `dusk-stream`.

The "grid" flavor allows a Text User Interface (TUI) through Dusks' Grid and
piggy backs on the `curses` library on POSIX hosts and `conio` on Windows. The
names of its targets are `dusk-curses` and `dusk-conio`.

The "graphic" flavor allows a graphic interface through Dusk's Screen and uses
the [SDL2][sdl2] library. The name of its target is `dusk-sdl`.

## Packages

While Usermode Dusk can be used in "regular" interactive mode, one exciting
possibility that it allows is to package application in compact, fast and
standalone executables for the host OS. These applications are called "Dusk
packages".

To create a package, what you need is a "payload", that is, a stream of Forth
source that will intepreted by the kernel. A Dusk kernel starts from nothing but
the HAL, so that payload is likely to begin with the contents of
`/xcomp/boot.fs`. But afterwards, you place what you want.

In interactive Usermode Dusk, both the kernel and payload live in regular files
named `kernel` and `payload_<flavor>` in the same directory as the executable.
This allows you to fiddle with them for maximum debuggability.

Dusk packages will typically embed both their kernel and payload in the
executable themselves. The most straightforward and portable way to do so is to
generate ".h" file with hardcoded contents in it.  You'll need to generate a
`kernel.h` and a `payload.h` from `usermode/kernel` and from your generated
payload. For this, you can use `embedh.sh` supplied by Dusk.

Then, you wrap all this in a `main()` function that will call `common_setup()`
and `common_exec()`.  Your C source might look like this:

	#include "duskos/usermode/common.h"
	#include "kernel.h"
	#include "payload.h"

	int main(int argc, char *argv[]) {
		int res = common_setup(
			argc, argv, 4*1024*1024 /* 4MB of memory */,
			kernel, sizeof(kernel), /* supplied by kernel.h */
			payload); /* supplied by payload.h */
		if (res) return res;
		common_exec();
		return 0;
	}

Usermode Dusk does some host system probing to figure out proper flags to send
to the C compiler, you're likely to want to use the same. For this, you might
want to use the special `machineflags` target in `usermode/common.mk` which
spits out the machine specific flags for your host.

This will result in a standalone executable that runs your payload!

Confused? The best way to figure out how to build your package is to look at
examples. Take a look at existing Dusk packages:

* [Dusk Examples](http://duskos.org/examples.html)
* [Dusk Invoice](http://duskos.org/invoice.html)
* [Dusk Gopher Daemon](http://duskos.org/gopherd.html)
* [Dusk inet server boilerplate](http://duskos.org/inet.html)

## Build and run

To build Usermode Dusk, you need a compiler that can compile 32-bit code for
your host CPU, which might mean `gcc-multilib` depending on your OS.

Then, it's only a matter of doing `cd usermode && make`. This builds targets
for all flavors. If you're missing a dependency for a flavor, this might result
in an error. In that case, you might be more specific. For example,
`make dusk-stream` will avoid such errors.

Then, you can invoke the executable without argument to begin an interactive
Usermode Dusk session. The "stream" flavor has the same double-echo problem as
the POSIX VM and should be ran with a terminal in "raw" mode. You can run
`make run-stream` as a shortcut.

## Theory of operation

The ability of the Usermode wrapper to run Dusk natively rests on Dusk kernels'
ability to auto-relocate themselves. They aren't quite position independent
because links in the system dictionary are absolute addresses, but the kernels
are built in a way that it's possible (trivial even) for it, at boot time, to
examine itself, know where it's ran from, then modify itself to run properly
from this location.

From that point on, it reads the "boot arguments" supplied by the Usermode
wrapper which gives it enough information to bootstrap itself into whatever its
final purpose is.

That's why the wrapper job at boot time is relatively simple:

1. Create a memory area that is readable, writable and executable.
2. Load the kernel at its first address.
3. Place boot arguments at a fixed address in that memory.
4. Call first address of that memory area.

### interopzone

The Usermode wrapper and Dusk communicate through a structure defined in
`common.h` called `interopzone`. It's through this struct that boot arguments
are passed, but it's also through there that API functions receive their
arguments and yield their results.

This structure lives in Dusk memory at a pre-defined address, a constant we call
`BOOTZONESZ`, which has a value of 8KB. That zone represents the maximum size
that a Dusk kernel (which is quite small) can have. This doesn't include the
payload, which lives outside Dusk memory and is read-only.

The `interopzone` struct lives at the very end of `BOOTZONESZ`, which means that
the actual maximum size of a kernel is rather `BOOTZONESZ-sizeof(interopzone)`.

With such a predefined constant, Dusk Usermode kernels know where to look for
boot arguments, which allows them to bootstrap themselves properly.

### Calling an API function

The Usermode wrapper does more than merely booting Dusk, it also provides it
with an API to the Host system. As previously mentioned, this is done through
the `interopzone` struct. `common.c` exposes a global pointer to it as the `iz`
variable.

`iz->funcs` is a pointer to an array of function pointers, which all have a
`void (*)(void)` signature. Its those pointers that the wrapper API words
described below call. `iz->arg{1,3}` are for arguments passed to and from those
functions.

We don't pass arguments directly to the function because different host OSes
have different calling conventions, so passing arguments reliably is murky,
hence this struct.

## Freezing

Like regular Dusk, Usermode Dusk bootstraps itself from a tiny kernel. It does
so quickly, so boot time is not a concern for executables designed to run more
than a few milliseconds, but for executables that are designed to be short-lived
and repeatedly called, this warm up time becomes a problem.

To mitigate this, Usemode Dusk has "freezing": When Dusk is finished
bootstrapping itself, right before it's ready to rumble, it halts itself and we
dump its memory. Then, we compile a new executable with that contents as a
"kernel" instead. All this kernel does is jump to its compiled "payload word"
(we could call it "main()") and thus fulfill its purpose.

Simple, right? Yeah, but there's a problem: Dusk is the opposite of Position
Independent. It's Very Position Dependent. That the kernel is able to auto-
relocate itself already requires careful threading, but relocating memory of a
bootstrapped Dusk system is way outside the realm of the reasonable.

Therefore, we need something specific from the host OS: the ability to create
a RWX mmap at a fixed address. **Not all OSes allow this, which means not all
host OSes can use Usermode freezing.** If we can manage to have a consistent
mmap address, then we can run that frozen kernel without having to relocate
anything.

### How to freeze

On the Dusk side, there's only one thing we must do: call `freeze ( "name" --
)` at the freezing point. This finds `name` in the dictionary and writes a
call to that word at address 0. Then, it calls the host's `freeze` API (see
below) with proper arguments.

This call will be the last call of your payload.

On the wrapper side, it's more involved. A Usermode wrapper first checks if it
was called with the `_dusk_freeze_` as its last argument. If it was, the
argument is removed from the list before passing them on to Dusk. Then, the
wrapper knows it's in "wants to freeze mode". The global `wantstofreeze`
variable exposed in `common.h` indicates whether we're in this mode.

When we're not in this mode, the `freeze` API function is a noop, which means
that the `freeze ( "name" -- )` word described above will simply call that
word and then shutdown.

When in this mode, when it's time to create the mmap, it does so with the
`MAP_FIXED` option and asks for the `FREEZEADDR` address, which is a constant.

If it manages to get it, good, it continues. Otherwise, it exits with a failure.

Then, it runs the payload normally until its `freeze` API function is called.
When it is, it dumps the frozen kernel to `frozen_kernel` and then exits with
success.

Then, that frozen kernel can be reused to compile a new Usermode package. This
package can behave identically to a regular package, with the peculiarity that
the "kernel" size will be bigger than `BOOTZONESZ`, but that doesn't cause
actual problems because the `interopzone` memory area has been reserved by Dusk
when it bootstrapped itself. All good.

And that's it! Drastically improved speed of short-lived packages! See the Dusk
Examples repo for... examples.

## Usermode API

Each usermode "flavor" exposes an API to the Dusk code so that it can interact
with the host machine.

The API that is described below is the "user visible" API, as wrapped in Forth
code in `api.fs`, `glue.fs`, `grid.fs` and `graphic.fs`.

The actual API as implemented by C wrapper code is a bit different, but it's not
worth documenting unless you're writing API wrapping code or developing a
usermode kernel. At that point, you're better off looking at the code.

### FD API

Usermode Dusk has the exact same `fd*` API as the one described in
`posix/README.md`.

Regular (as in: not Dusk Packages) Usermode builds use this API to open
"disk.img" at startup and thus have full filesystem functonality.

### Common API

These words are present in all flavors and implemented in `api.fs` and
`glue.fs`.

	bye ( -- )
		Shutdown the program. Initially, it is to that word that ABORTPTR is
		hooked.

	freeze ( here -- )
		If in "wants to freeze" mode, freeze the program and shutdown.
		Otherwise, this is a noop.

	breathe ( -- )
		Let the system breathe a little bit (sleep 1ms). This should be called
		in idle loops. Other appropriate "breathe" actions can be taken by the
		wrapper.	

	putchar ( c -- )
		Print character "c" to the console/screen at current position and
		advance that position.

	?getchar ( -- ?c f )
		Check if a key has been pressed by the user. If yes, f=1 and "c" is
		present. Otherwise, f=0 and "c" is absent.

	dbgPrint ( n1 n2 -- )
		Print numbers "n1" and "n2" to the console/screen in a debuggable way.

	sleep ( n -- )
		Sleep for "n" microseconds.

	ticks ( -- n )
		Yield a "tick number" appropriate for sys/timer.

	transferin ( srcname dstdirid dstname -- )
		Open *host* file path "srcname" and read its contents into a file
		"dstname" that will be created in dir "dstdirid". Example:

		"/my/host/path" p"/lib" "foo.fs" transferin
	
	transferout ( srcpath dstname -- )
		Write the contents of "srcpath" into *host* path "dstname". Example:

		"/lib/foo.fs" "/my/host/path" transferout

In addition to the words above, the common API also defines a `StdIO` struct
with its companion `stdio` structbind that works very much like the `Console`
(see [doc/sys/io]) except that it wraps `?getchar` and `putchar` directly
instead of wrapping `in<` and `emit`. This is useful because `in<` is plugged
to the payload stream rather than to `?getchar`. Therefore, `stdio` is the only
straightforward way to access `stdin` and `stdout` through Dusk's I/O semantics.
Example usage:

	"Hello" stdio :puts
    stdio :readline stype

### Grid

In the "Grid" flavor implemented in `grid.fs`, we have a `UMGrid` struct that is
bound to the `grid` structbind [doc/sys/grid]. That grid supplies us with a
console.

### Graphic

In the "Graphic" flavor implemented in `graphic.fs`, we have a `UMScreen` struct
that is bound to the `screen` structbind [doc/sys/screen]. On top of that
screen, we have a Framebuffer Grid [drv/fbgrid] giving us a Grid and thus a
console.

On top of that, we also have a `UMMouse` struct that is bound to the `mouse`
structbind [doc/sys/mouse].

### Extending the API

It's sometimes useful to have your own API calls, for example if you want to
wrap a library on the host OS. You can do so by creating a new C function with
a `void (*)(void)` signature and assign it to a `cbfuncs[]` slot. There are
`APIFUNCCNT` (256) available slots and the first `APIRESERVEDCNT` (32) ones are
reserved for Dusk itself, the rest is yours.

Arguments are passed through the global `iz` (`interopzone`, see above) variable
This structure has `arg1`, `arg2` and `arg3` members which are used both for
input arguments and output results. Thus, for an API word with the signature `(
a1 a2 a3 -- r2 r1 )`, `a1` goes comes from `arg1`, `a2` comes from `arg2`, `a3`
comes from `arg3`. The return values are reversed and `r2` goes in `arg2` and
`r1` goes in `arg1`. Here's a simple example:

	void myadder() { // ( a b -- n )
		iz->arg1 = iz->arg1 + iz->arg2;
	}
	// ... Later in setup code
	cbfuncs[42] = myadder;

On the Forth side, those APIs have to be wrapped with the `syscallback,`
(function with no result) or `syscallbackr,` (function with result). You supply
it with the number of arguments, the number of return values and a cbfuncs slot
ID and it generates a word for you. For example, let's wrap `myadder`:

	2 1 42 syscallbackr, myadder
	12 23 myadder . \ Prints "35"

[sdl2]: https://www.libsdl.org/