Interesting performance characteristics on the Nvidia Jetson

[arduano]

Just thought I'd share...

Ran the tests multiple times to make sure this was not an accident.

[ogxd]

That's indeed very interesting! 128 bytes is the size at which the algorithm starts to proceed with an "high ILP construction" for greater throughput. It seems to underperform on this hardware compared to the <128 bytes processing, I'd be curious to understand why but it might be difficult to put my hands on such hardware.
I'm not familiar with this product. Are you using the "Jetson Orin Nano Developer Kit"? It seems it uses an ARM Cortex-A78AE CPU, but the throughput does not depend on the CPU alone but also on memory bandwidth.

[arduano]

criterion.zip
Here's a zip of the full criterion folder, if needed

[ogxd]

all/gxhash/64 time: [169.12 ps 169.49 ps 169.90 ps]
thrpt: [350.83 GiB/s 351.68 GiB/s 352.43 GiB/s]

😮
Thanks a lot for these benches. I am not sure what is going on. Maybe there is another provider of github runners that proposes hardware on which we can observe similar results.

[arduano]

Just curious (I don't fully understand gxhash), but would it be possible to extend the <128 bytes processing functions to >=128? Just for running a benchmark here

[ogxd]

Simply put, the processing <128 bytes is a manually unrolled loop, like so:

if len >= 16
  hash next 16 bytes
if len >= 32
  hash next 16 bytes
if len >= 48
  hash next 16 bytes
if len >= 128
  loop by 128 bytes
     hash next 16 bytes (8 times)
  loop by 16 bytes (what remains)
     hash next 16 bytes

This manual unroll < 128 bytes is there to avoid the cost of jump instructions from the looping (the algorithm is so fast that the loop itself costs more than the actual block hashing on some platforms).
The manual unroll is great however it has a cost in terms of bytecode size: because the "hash next 16 bytes" is inlined, every "if len >= X then hash next 16 bytes" makes the bytecode larger. The larger the bytecode is, the less likely it is to be inlined within functions that use the hashing algorithm or to entirely fit in the instructions cache. This is why it has to be used sparingly.

That said, this discussion makes me think that it could be possible to get rid of the second loop, I'll run some experiments.
Also, I may have a theory which is that the Nvidia Jetson hardware has quite a high latency on memory access, and the loops make it more difficult to prefetch. There are ways to explicitly prefetch data, and I've seen some algorithms making use of it. I didn't see performance improvements on my laptop, so I did not include it, but maybe it would make a difference on the Nvidia Jetson.

I'll experiment with a few ideas and get back to you. Do you think you'll still be able to run a few benches on the Jetson in a some days ?

[arduano]

Yeah just ping me, I'm on Sydney time, but I get notifications over email usually.