Preemption [WIP] [RFC] #90

sargun · 2017-03-16T06:19:29Z

This PR introduces basic preemption support into Gravity. I'd mostly like to get your thoughts on this. I'd also like to make it so that when (certain) native functions are called, it can also trigger preemption.

marcobambini · 2017-03-16T06:30:11Z

May I ask you why you'd like to introduce preemption?

sargun · 2017-03-16T06:34:21Z

I tried to discuss in Gitter. It might be easier there.

sargun · 2017-03-16T06:37:30Z

I'm introducing concurrency to Gravity, with the following semantics:

Every gravity program will belong to a processlet.
Every processlet has a 64-bit integer ID that's derived from incrementing a number.
Processlets do not share memory.
You can communicate between two processlets by doing a send(ProcessletID, Message) OR a receive(timeout)
A processlet can monitor the life of another processlet by using a monitor(ProcessletID). If that foreign processlet fails, a message will be put in the monitoring process's mailbox
A processlet can link is life with another processlet. If the foreign processlet fails, the processlet can either link its lifetime with the foreign processlet, or vice-versa.
A processlet can kill another processlet with a kill signal.

I/O (sockets) will be represented as processlets that take special commands, and are "owned" by a specific processlet.

InusualZ · 2017-03-16T11:00:33Z

src/cli/gravity.c

@@ -158,6 +171,7 @@ int main (int argc, const char* argv[]) {

 	// create VM
 	gravity_vm *vm = gravity_vm_new(&delegate);
+    delegate.preempt_callback = &should_preempt;


InusualZ · 2017-03-16T11:04:32Z

src/runtime/gravity_vm.c

-
-	DEBUG_STACK();
-
+    gravity_preempt_callback    preempt_cb = NULL;          // Callback to cause preemption


InusualZ · 2017-03-16T11:05:16Z

src/runtime/gravity_vm.c

@@ -1407,7 +1460,7 @@ void gravity_vm_loadclosure (gravity_vm *vm, gravity_closure_t *closure) {
 	gravity_fiber_reassign(vm->fiber, closure, 0);

 	// execute $moduleinit in order to initialize VM
-	gravity_vm_exec(vm);
+    gravity_vm_exec(vm, false);


InusualZ · 2017-03-16T11:05:37Z

src/runtime/gravity_vm.h

@@ -21,12 +21,15 @@ typedef void (*vm_transfer_cb) (gravity_vm *vm, gravity_object_t *obj);
 typedef void (*vm_cleanup_cb) (gravity_vm *vm);

 gravity_vm			*gravity_vm_new (gravity_delegate_t *delegate);
+void                gravity_vm_init (gravity_vm *vm, gravity_delegate_t *delegate);


marcobambini · 2017-03-16T13:12:05Z

Have you measured the impact of performing MAYBE_PREEMPT on each instruction?

…itialization)

sargun · 2017-03-17T02:59:44Z

It's negligible, performance wise to invoke a callback which returns false. It's about a 3.1% overhead versus commenting out MAYBE_PREEMPT entirely.

with preemption callback returning false (median of 50 runs): 797.5753 ms
with no preemption code (median of 50 runs): 773.0993 ms

I think over time, we can introduce lighter weight mechanisms of preemption. Perhaps enabling / disabling it per instruction.

sargun · 2017-03-17T05:45:09Z

The other place where need preemption / resumption support is native calls. I have yet to determine the interface here, but when making a call to a native function, it needs to be able to yield up, and resume. I haven't dug in very much, but I was thinking of introducing a return code from native functions, or another function that's bound to them that allows for yielding.

marcobambini · 2017-03-17T09:50:03Z

Why a callback to check if VM must be interrupted and not a simpler:
gravity_vm_interrupt
that internally set a bool flag?

I would image APIs like:

bool gravity_vm_interrupt(gravity_vm *vm, gravity_vm_state *state);
bool gravity_vm_resume(gravity_vm *vm, gravity_vm_state *state);

In this way no extra parameters are needed and the API is much more cleaner.

sargun · 2017-03-17T15:27:08Z

I thought about such an API. It's not a bad approach, I thought about it initially -- especially invoking it from native C calls. I have some questions though:

What calls vm_interrupt / vm_suspend? If it's not being invoked through a callback, it needs to be a thread. Some of the platforms I intend to target do not readily have threads available -- highly sandboxed VMs, and WebASM / Emscripten.
When gravity_vm_interrupt gets called, does it unwind the stack of the VM? If so, how does the VM get resumed? If not, how does the VM yield?
If the boolean is being read locally, and modified by an external thread, we would have to declare it volatile, and I think that introducing synchronization would actually make it slower in some cases -- I can benchmark this if you want.

As a side note, I tried out one other approach -- rather than a run-time callback, doing a compile-time macro. Unfortunately, this had no large speed difference as compared to the callback (<1% faster).

marcobambini · 2017-03-17T16:03:54Z

Something like this should work in my opinion:

bool gravity_vm_interrupt (gravity_vm *vm) {
    // if vm already interrupted return false
    if (vm->interrupted) return false;

    // check for mandatory vm_state callback
    if (!vm->delegate->vm_state) return false;

    // don't think we need to protect this bool flag
    vm->interrupted = true;

    // wait for vm_state callback to be called
    // automatically called by vm when flag is set
    return true;
}

void vm_state (gravity_vm *vm, gravity_vm_state *state) {
    // this callback is automatically called by the vm when it is interrupted
    // state contains info necessary to resume vm (like your vm_state struct)
}

bool gravity_vm_resume(gravity_vm *vm, gravity_vm_state *state) {
    // if vm not interrupted return false
    if (!vm->interrupted) return false;

    // reset interrupted flag 
    vm->interrupted = false;

    // resume execution
    gravity_vm_run_resume_main(...);    

    return true;
}

// your MAYBE_PREEMPT macro is now
#define MAYBE_PREEMPT     if (vm->interrupted) goto DO_PREEMPT;

marcobambini · 2017-03-17T16:04:47Z

The ability to be able to interrupt the VM (and step) will open us the possibility to attach a debugger to it.

Jezza · 2017-03-17T16:40:09Z

If you don't have threads, in which to interrupt the VM, why not look into a yield mechanic, or something similar?
Granted, that doesn't have the exact same functionality, unless you want to introduce a language level yield, but again, you would still differ in functionality.

Why not a "debug/halt" opcode that halts the VM, also potentially reducing back pressure?

InusualZ · 2017-03-17T17:51:43Z

src/runtime/gravity_vm.c

 }

-gravity_vm *gravity_vm_new (gravity_delegate_t *delegate/*, uint32_t context_size, gravity_int_t gcminthreshold, gravity_int_t gcthreshold, gravity_float_t gcratio*/) {
+gravity_vm *gravity_vm_new(
+		gravity_delegate_t *delegate/*, uint32_t context_size, gravity_int_t gcminthreshold, gravity_int_t gcthreshold, gravity_float_t gcratio*/) {


InusualZ · 2017-03-17T17:51:56Z

src/runtime/gravity_vm.c

+	vm->time += millitime(tstart, tend);
+
+	if (!gravity_vm_preempted(vm))
+			PRINT_STATS(vm);


sargun · 2017-03-18T05:07:47Z

The ability to be able to interrupt the VM (and step) will open us the possibility to attach a debugger to it.

The callback based method will allow for this as well, specifically, the runtime based callback.

sargun · 2017-03-18T05:09:21Z

If you don't have threads, in which to interrupt the VM, why not look into a yield mechanic, or something similar?
Granted, that doesn't have the exact same functionality, unless you want to introduce a language level yield, but again, you would still differ in functionality.
Why not a "debug/halt" opcode that halts the VM, also potentially reducing back pressure?

A couple things:

To me, this doesn't feel much better than coroutines.
What would the C API look like? Would it unwind the stack? If so, we would still need the resume API.

sargun · 2017-03-18T05:20:27Z

Is the fear of around the callback mechanism about performance?

One of the answers here might be to have an interpreter loop with preemption enabled, and one without. You can then run performance-sensitive & trusted code without preemption enabled at all. This is similar to the Linux kernel's preemptive scheduler.

sargun · 2017-03-24T07:34:15Z

How about we allow people to define a macro instead of a full-on function. This way, if they want to use the flag, they can have a struct like:

struct wrapper_vm {
    bool preempted;
    gravity_vm vm;
}

And their macro can look like this:

static inline bool maybe_preempt(gravity_vm *vm) {
    struct wrapper_vm *wvm = container_of(vm, struct wrapper_vm, vm);
    return wvm->preempted;
}

#define MAYBE_PREEMPT maybe_preempt(vm)
// It an exercise left to the user to reset the state prior to calling resume.

marcobambini · 2017-03-24T23:26:11Z

@sargun I'd like to have the ability to decide the pre-emption based on a runtime bool value instead of a macro that would require recompilation. I think that my latest proposal could be the best compromise between performance degradation (a bool comparison on each instruction) and the flexibility to interrupt the vm.

For debugging purpose some special bytecode instructions like BREAK and STEP would be the best option.

Please let me know what you think.

sargun · 2017-03-24T23:37:20Z

What if we do something like have a config option for PREEMPT_MODE, and have a function bool mark_preempt(gravity_vm *vm) (where the bool returns true / false depending on if the VM was in a preemptible context) to tell a vm to yield if #define PREEMPT_MODE internal and alternatively, you can have #define PREEMPT_MODE callback and #define PREEMPT_CB ...?

marcobambini · 2017-11-27T07:35:52Z

Hi @sargun I am going to re-open it because I'd like to use some of your ideas to implement a built-in debugger.

sargun · 2017-11-27T07:49:27Z

Ah sounds good, I was just going through and cleaning out my github activity.

sargun force-pushed the preemption branch from 9be16a9 to 23dc8e6 Compare March 16, 2017 06:19

sargun force-pushed the preemption branch from 8d09c00 to 6af2854 Compare March 16, 2017 10:58

InusualZ reviewed Mar 16, 2017

View reviewed changes

sargun added 3 commits March 16, 2017 19:07

Add scaffolding for preemption support

324fdac

Split up gravity_vm_init and gravity_vm_new (memory allocation vs. in…

4097692

…itialization)

Set buffer on dumping closure

4fba1c3

sargun force-pushed the preemption branch from 6af2854 to 4fba1c3 Compare March 17, 2017 02:13

InusualZ reviewed Mar 17, 2017

View reviewed changes

This was referenced Apr 10, 2017

Stack Overflow Vulnerability #126

Closed

Stack Overflow (operator_int_cmp) #130

Closed

sargun closed this Nov 26, 2017

marcobambini reopened this Nov 27, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Preemption [WIP] [RFC] #90

Preemption [WIP] [RFC] #90

sargun commented Mar 16, 2017

marcobambini commented Mar 16, 2017

sargun commented Mar 16, 2017

sargun commented Mar 16, 2017 •

edited

InusualZ Mar 16, 2017

InusualZ Mar 16, 2017

InusualZ Mar 16, 2017

InusualZ Mar 16, 2017

marcobambini commented Mar 16, 2017

sargun commented Mar 17, 2017 •

edited

sargun commented Mar 17, 2017

marcobambini commented Mar 17, 2017

sargun commented Mar 17, 2017

marcobambini commented Mar 17, 2017 •

edited

marcobambini commented Mar 17, 2017

Jezza commented Mar 17, 2017 •

edited

InusualZ Mar 17, 2017

InusualZ Mar 17, 2017

sargun commented Mar 18, 2017

sargun commented Mar 18, 2017

sargun commented Mar 18, 2017 •

edited

sargun commented Mar 24, 2017

marcobambini commented Mar 24, 2017

sargun commented Mar 24, 2017

marcobambini commented Nov 27, 2017

sargun commented Nov 27, 2017


		DEBUG_STACK();

		gravity_preempt_callback preempt_cb = NULL; // Callback to cause preemption

Preemption [WIP] [RFC] #90

Are you sure you want to change the base?

Preemption [WIP] [RFC] #90

Conversation

sargun commented Mar 16, 2017

marcobambini commented Mar 16, 2017

sargun commented Mar 16, 2017

sargun commented Mar 16, 2017 • edited

InusualZ Mar 16, 2017

Choose a reason for hiding this comment

InusualZ Mar 16, 2017

Choose a reason for hiding this comment

InusualZ Mar 16, 2017

Choose a reason for hiding this comment

InusualZ Mar 16, 2017

Choose a reason for hiding this comment

marcobambini commented Mar 16, 2017

sargun commented Mar 17, 2017 • edited

sargun commented Mar 17, 2017

marcobambini commented Mar 17, 2017

sargun commented Mar 17, 2017

marcobambini commented Mar 17, 2017 • edited

marcobambini commented Mar 17, 2017

Jezza commented Mar 17, 2017 • edited

InusualZ Mar 17, 2017

Choose a reason for hiding this comment

InusualZ Mar 17, 2017

Choose a reason for hiding this comment

sargun commented Mar 18, 2017

sargun commented Mar 18, 2017

sargun commented Mar 18, 2017 • edited

sargun commented Mar 24, 2017

marcobambini commented Mar 24, 2017

sargun commented Mar 24, 2017

marcobambini commented Nov 27, 2017

sargun commented Nov 27, 2017

sargun commented Mar 16, 2017 •

edited

sargun commented Mar 17, 2017 •

edited

marcobambini commented Mar 17, 2017 •

edited

Jezza commented Mar 17, 2017 •

edited

sargun commented Mar 18, 2017 •

edited