Initial RISC-V support implemented

[jperkin]

Hi!

I've ported Hubris to RISC-V, specifically the Freedom E310 core on a Sparkfun RED-V board that I have. The repository is available at https://github.com/jperkin/hubris/tree/riscv.

While I've yet to implement interrupts and peripherals, the rest of the kernel is done, and a demo application successfully runs jefe, ping, pong (without LEDs), and idle:

$ cargo xtask gdb app/sparkfun-redv/app.toml app/sparkfun-redv/openocd.gdb

starting: laziness
starting: impatience
starting: hubris
viva el jefe
Task #2 Memory fault at address 0x0
Task #2 Illegal instruction
Task #2 Memory fault at address 0x0
Task #2 Illegal instruction
...

I've also implemented enough that the test suite runs with only 1 expected failure, due to differences between ARM and RISC-V fault handling:

$ cargo xtask gdb test/tests-sparkfun-redv/app.toml app/sparkfun-redv/openocd.gdb

starting: laziness
starting: impatience
starting: hubris
meta
assistant starting
assistant starting
expect 41
case test_send
case test_recv_reply
case test_floating_point_lowregs
case test_floating_point_highregs
case test_floating_point_fault
case test_fault_badmem
case test_fault_stackoverflow
case test_fault_execdata
case test_fault_illop
case test_fault_nullexec
case test_fault_textoob
case test_fault_stackoob
case test_fault_buserror
case test_fault_illinst
case test_fault_divzero
case test_fault_maxstatus
case test_fault_badstatus
case test_fault_maxrestart
case test_fault_badrestart
case test_fault_maxinjection
case test_fault_badinjection
case test_fault_superinjection
case test_fault_selfinjection
case test_panic
case test_restart
case test_restart_taskgen
case test_borrow_info
case test_borrow_read
case test_borrow_write
case test_borrow_without_peer_waiting
case test_supervisor_fault_notification
case test_timer_advance
case test_timer_notify
case test_timer_notify_past
case test_task_status
case test_task_fault_injection
case test_refresh_task_id_basic
case test_refresh_task_id_off_by_one
case test_refresh_task_id_off_by_many
case test_lpc55_flash_write
case test_post
run
assistant starting
start test_send
finish ok test_send
assistant starting
start test_recv_reply
finish ok test_recv_reply
assistant starting
start test_floating_point_lowregs
finish ok test_floating_point_lowregs
assistant starting
start test_floating_point_highregs
finish ok test_floating_point_highregs
assistant starting
start test_floating_point_fault
Task #2 Illegal instruction
finish ok test_floating_point_fault
assistant starting
start test_fault_badmem
Task #2 Memory fault at address 0x5
finish ok test_fault_badmem
assistant starting
start test_fault_stackoverflow
finish ok test_fault_stackoverflow
assistant starting
start test_fault_execdata
Task #2 Illegal text
finish ok test_fault_execdata
assistant starting
start test_fault_illop
task @2 panicked: panicked at 'unexpectedly survived IllegalOperation', test/test-assist/src/main.rs:291:29
Task #2 Panic!
task @1 panicked: panicked at 'expected InvalidOperation; found Panic', test/test-suite/src/main.rs:246:13
Task #1 Panic!
Task #2 Panic!
assistant starting
finish FAIL test_fault_illop
assistant starting
start test_fault_nullexec
Task #2 Illegal text
finish ok test_fault_nullexec
assistant starting
start test_fault_textoob
Task #2 Memory fault at address 0x20040000
finish ok test_fault_textoob
assistant starting
start test_fault_stackoob
Task #2 Memory fault at address 0x80004000
finish ok test_fault_stackoob
assistant starting
start test_fault_buserror
Task #2 Memory fault at address 0xe000ed28
finish ok test_fault_buserror
assistant starting
start test_fault_illinst
Task #2 Illegal instruction
finish ok test_fault_illinst
assistant starting
start test_fault_divzero
finish ok test_fault_divzero
assistant starting
start test_fault_maxstatus
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_fault_maxstatus
assistant starting
start test_fault_badstatus
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_fault_badstatus
assistant starting
start test_fault_maxrestart
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_fault_maxrestart
assistant starting
start test_fault_badrestart
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_fault_badrestart
assistant starting
start test_fault_maxinjection
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_fault_maxinjection
assistant starting
start test_fault_badinjection
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_fault_badinjection
assistant starting
start test_fault_superinjection
Task #2 Bad Syscall Usage IllegalTask
finish ok test_fault_superinjection
assistant starting
start test_fault_selfinjection
Task #2 Bad Syscall Usage IllegalTask
finish ok test_fault_selfinjection
assistant starting
start test_panic
task @2 panicked: panicked at 'wow this blew up, here's my soundcloud', test/test-assist/src/main.rs:24:5
Task #2 Panic!
assistant starting
finish ok test_panic
assistant starting
start test_restart
assistant starting
finish ok test_restart
assistant starting
start test_restart_taskgen
task @2 panicked: panicked at 'wow this blew up, here's my soundcloud', test/test-assist/src/main.rs:24:5
Task #2 Panic!
assistant starting
finish ok test_restart_taskgen
assistant starting
start test_borrow_info
finish ok test_borrow_info
assistant starting
start test_borrow_read
finish ok test_borrow_read
assistant starting
start test_borrow_write
finish ok test_borrow_write
assistant starting
start test_borrow_without_peer_waiting
finish ok test_borrow_without_peer_waiting
assistant starting
start test_supervisor_fault_notification
task @2 panicked: panicked at 'wow this blew up, here's my soundcloud', test/test-assist/src/main.rs:24:5
Task #2 Panic!
finish ok test_supervisor_fault_notification
assistant starting
start test_timer_advance
finish ok test_timer_advance
assistant starting
start test_timer_notify
finish ok test_timer_notify
assistant starting
start test_timer_notify_past
finish ok test_timer_notify_past
assistant starting
start test_task_status
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_task_status
assistant starting
start test_task_fault_injection
Task #2 Fault injected by task #1
finish ok test_task_fault_injection
assistant starting
start test_refresh_task_id_basic
assistant starting
finish ok test_refresh_task_id_basic
assistant starting
start test_refresh_task_id_off_by_one
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_refresh_task_id_off_by_one
assistant starting
start test_refresh_task_id_off_by_many
Task #2 Bad Syscall Usage TaskOutOfRange
finish ok test_refresh_task_id_off_by_many
assistant starting
start test_lpc55_flash_write
finish ok test_lpc55_flash_write
assistant starting
start test_post
finish ok test_post
done FAIL

I figured that now I have a working implementation it'd be a good time to verify that you're happy with the general direction. Some general notes:

• The core I've targeted does not support Supervisor mode, so for now the kernel runs purely in Machine mode with tasks running in User mode.
• I've retained the standard calling convention for system calls. RISC-V doesn't push onto the stack during exceptions so the handling is somewhat simpler, however a full context switch is currently taken each time.
• The current task pointer is stored as a global variable in a similar manner as ARM, but it may be possible to use the mscratch register instead to keep track.
• There's no support in RISC-V for issuing a wfi call from User mode, so instead the idle task spins on a sys_get_timer() call. It might be worth adding a specific system call that implements wfi instead.

This is my first exposure to RISC-V so it's highly likely I've misunderstood things or missed optimisations, but hopefully this will at least provide a starting point for the conversation.

If you'd like me to submit a PR with this work then just let me know, otherwise I'll continue adding support for interrupts, GPIO, etc, as well as looking at RISC-V support in Humility which I've sorely missed during bring-up!

[mx-shift]

This is really cool. Thank you for sharing early! @cbiffle and @bcantrill will need to comment on Hubris kernel and Humility respectively but this looks fantastic so far. As far as how to integrate, I'd like to see a PR for the kernel changes only.

[bcantrill]

This looks great! I really want to get away from ITM entirely; have you been able to get Humility to work? (I ask because I would like to retool the test suite to not use ITM.)

[jperkin]

Not yet, I've briefly looked at it but I think it'll be a reasonable amount of work. Even a simple first step such as querying the RISC-Vmvendorid instead of the ARM CPUID doesn't work as this FE310 core doesn't appear to support it!

For now I'm running the test suite purely over semihosting.

[bcantrill]

Yeah, that makes sense -- the test suite over semihosting is very useful for early bringup, so it's really just ITM that I want to ditch. I know that the ARM'isms in Humility are going to be a pain to factor out -- but it would be good to start with just the ability to stop the target and read its memory. And apologies if it ends up being harder to port the debugger than the operating system!

[cbiffle]

This is pretty slick, @jperkin. I need to pick up one of those E310 boards -- I have older SiFive hardware that lacks the memory protection unit, as we discovered during early Hubris development. (If anyone at SiFive is excited about Hubris, I accept samples. :-) )

Others may differ, of course, but here are my responses to your bullet points.

The core I've targeted does not support Supervisor mode, so for now the kernel runs purely in Machine mode with tasks running in User mode.

Yeah, that's how I'd port it. M/U cores are probably going to be quite common in the microcontroller space, the M/S/U distinction is less important.

I've retained the standard calling convention for system calls. RISC-V doesn't push onto the stack during exceptions so the handling is somewhat simpler, however a full context switch is currently taken each time.

I'll take a look at your specific ABI choices when I get some spare time, but, this sounds reasonable. The trick for avoiding a full context switch is pretty ARMv7-M specific; most processors don't have an equivalent to the PendSV hack. However, perhaps there are things we can optimize.

In general, Hubris was originally designed with the intent of moving to RISC-V eventually, which is part of why we're so register-focused in the calling convention.

The current task pointer is stored as a global variable in a similar manner as ARM, but it may be possible to use the mscratch register instead to keep track.

Yeah, that seems like the right place to start, and you have correctly guessed my intent. :-) The reason why task pointer tracking is left to the arch module, instead of centralized, is because of mscratch. The wee ARMs without coprocessors have no equivalent of it, which is unfortunate.

There's no support in RISC-V for issuing a wfi call from User mode, so instead the idle task spins on a sys_get_timer() call. It might be worth adding a specific system call that implements wfi instead.

wfi should trap, right? We could implement the trap. That's kind of like a syscall. That being said, if wfi requires a trap, that cost might make a really good argument for eliminating the idle task (see #111).

It looks like the spec permits WFI to be exposed to U-mode, but that also smells like the parts of the spec that tend not to get implemented (like U-mode interrupt handling), so I could believe that it's not possible on the E310.

---

Support for interrupts should be straightforward assuming the E310 has a PLIC or something similar. The interrupt model was intended to cover the common RISC-V interrupt controller designs. If it doesn't, I'd sure like to know about it!

Because Hubris winds up sending all application interrupts through a single handler, if you're able to disable vectoring and route all interrupts to a single location, you could save quite a bit of Flash compared to the ARM-M ports where the vector table is mandatory. I seem to recall this capability being included in the privileged spec at one point, but again, I have a hard time predicting which parts of the spec actually get implemented. :-\