Expanded support for CFUs. #14

tcal-x · 2021-09-30T20:40:16Z

The new Fomu variants are very hacked, so use carefully. They have removed many safety checks (alignment, etc). Also, they have a hard multiplier but no divider, so you would need to compile with "-march=rv32im -mabi=ilp32 -mno-div".

- Now submodule ext/VexRiscv uses tip of `dev` branch - Added ext/SpinalHDL submodule, also `dev` branch - Modified `build.sbt` to bring up to date with current `dev` VexRiscv - Modified `src/main/scala/vexriscv/GenCoreDefault.scala` to add a `--cfu` option, to add the CFU interface - Added `VexRiscv_FullCfu.v` and `VexRiscv_FullCfuDebug.v` (and yamls), and updated `Makefile` to build them - I did **not** rebuild the other .v/.yaml files Signed-off-by: Tim Callahan <tcal@google.com>

Signed-off-by: Tim Callahan <tcal@google.com>

Both L1 caches are changed from 4k direct-mapped to 8k 2-way associative. The change to 2-way was required since each "way" can be 4k max. Signed-off-by: Tim Callahan <tcal@google.com>

Signed-off-by: Tim Callahan <tcal@google.com>

Rebuilt just the CFU-enabled Vex Verilogs. Signed-off-by: Tim Callahan <tcal@google.com>

Signed-off-by: Tim Callahan <tcal@google.com>

…y stages.

Tweak Fomu variant for performance and LCs.

Signed-off-by: Joey Bushagour <jbushagour@google.com>

Require a memory and writeback stage for the CFU plugin.

… variant. Signed-off-by: Joey Bushagour <jbushagour@google.com>

Update the Fomu CPU Variant for performance.

Also increase "Slim" D$ to 4kB, I$ to 2kB. Signed-off-by: Tim Callahan <tcal@google.com>

Signed-off-by: Tim Callahan <tcal@google.com>

Also remove obsolete genWrapper.py script. Signed-off-by: Tim Callahan <tcal@google.com>

Signed-off-by: Tim Callahan <tcal@google.com>

enjoy-digital · 2021-09-30T21:22:00Z

@tcal-x: This create lots of variants very specific to https://github.com/google/CFU-Playground project and I'm not sure this should be integrated here where we try to provide more generic variants. This would probably be better to integrate these variants directly in the CFU project or in a fork, as is doing Betrusted project for example for the specific variants that have been created for it: https://github.com/betrusted-io/pythondata-cpu-vexriscv.

Fomu is also not a good name for a variant (I imagine it has been created to limit resource usage on iCE40) since we would also want to use it on the iCEBreaker, so we could eventually integrate this one, but would probably have to find a proper name.

cc @mithro, @kgugala

mithro · 2021-10-01T00:39:22Z

pythondata_cpu_vexriscv/verilog/README.md

@@ -17,6 +17,16 @@ src/main/scala/vexriscv GenCoreDefault.scala
 - Available in normal an -Debug, with the Debug bus exposed


+## Changes in `tcal-x`'s fork


This needs fixing.

Yes, definitely.

mithro · 2021-10-01T00:40:31Z

pythondata_cpu_vexriscv/verilog/src/main/scala/vexriscv/GenCoreDefault.scala

@@ -6,10 +6,12 @@ import spinal.lib._
 import spinal.lib.sim.Phase


I don't understand why this is changing a scala file? Shouldn't this be part of the source module?

This is the top generator script that handles user options and then builds the VexRiscv accordingly, adding the plugins etc.. It's not part of the VexRiscv "library". It is local to the pythondata_cpu_vexriscv repo.

For example, cache line size is currently hardcoded in this file. If we want to make it an option that can be changed, we'd alter this file to add an user option and then pass the value to where the caches are instantiated.

mithro · 2021-10-01T00:42:47Z

@enjoy-digital - I think we might need a way to provide pythondata-cpu-vexriscv-XXX packages which add more options outside the base package. Having people have their own forks with the same name as the pythondata-cpu-vexriscv package create a huge number of issues.

kgugala · 2021-10-01T06:12:58Z

pythondata_cpu_vexriscv/verilog/VexRiscv_Full.v

@@ -1,6 +1,6 @@
 // Generator : SpinalHDL v1.6.0    git head : 73c8d8e2b86b45646e9d0b2e729291f2b65e6be3
 // Component : VexRiscv
-// Git hash  : 6276bf628be9d0a58c0284dca83137b71ef29098
+// Git hash  : 593e180abf5ba39e7fa33d4f371646453f84496a


we don't change anything in this file, but hash. This looks weird

kgugala · 2021-10-01T06:13:22Z

pythondata_cpu_vexriscv/verilog/VexRiscv_Debug.v

@@ -1,6 +1,6 @@
 // Generator : SpinalHDL v1.6.0    git head : 73c8d8e2b86b45646e9d0b2e729291f2b65e6be3
 // Component : VexRiscv
-// Git hash  : 6276bf628be9d0a58c0284dca83137b71ef29098
+// Git hash  : 593e180abf5ba39e7fa33d4f371646453f84496a


we don't change anything in this file, but hash. This looks weird

kgugala · 2021-10-01T06:13:26Z

pythondata_cpu_vexriscv/verilog/VexRiscv.v

@@ -1,6 +1,6 @@
 // Generator : SpinalHDL v1.6.0    git head : 73c8d8e2b86b45646e9d0b2e729291f2b65e6be3
 // Component : VexRiscv
-// Git hash  : 6276bf628be9d0a58c0284dca83137b71ef29098
+// Git hash  : 593e180abf5ba39e7fa33d4f371646453f84496a


we don't change anything in this file, but hash. This looks weird

kgugala · 2021-10-01T06:13:48Z

pythondata_cpu_vexriscv/verilog/.gitignore

@@ -1,34 +0,0 @@
-*.class


why do you remove gitignore?

kgugala · 2021-10-01T06:15:30Z

pythondata_cpu_vexriscv/verilog/VexRiscv_FullDebug.v

@@ -1,6 +1,6 @@
 // Generator : SpinalHDL v1.6.0    git head : 73c8d8e2b86b45646e9d0b2e729291f2b65e6be3
 // Component : VexRiscv
-// Git hash  : 6276bf628be9d0a58c0284dca83137b71ef29098
+// Git hash  : 593e180abf5ba39e7fa33d4f371646453f84496a


kgugala · 2021-10-01T06:16:01Z

pythondata_cpu_vexriscv/verilog/VexRiscv_IMAC.v

@@ -1,6 +1,6 @@
 // Generator : SpinalHDL v1.6.0    git head : 73c8d8e2b86b45646e9d0b2e729291f2b65e6be3
 // Component : VexRiscv
-// Git hash  : 6276bf628be9d0a58c0284dca83137b71ef29098
+// Git hash  : 593e180abf5ba39e7fa33d4f371646453f84496a


again hash (there are many such updates in this PR)

enjoy-digital · 2021-10-01T07:00:36Z

@enjoy-digital - I think we might need a way to provide pythondata-cpu-vexriscv-XXX packages which add more options outside the base package. Having people have their own forks with the same name as the pythondata-cpu-vexriscv package create a huge number of issues.

@mithro: This could be that yes, people doing specialized variants would also have to ensure they are not reusing existing names to avoid confusion (Betrusted added a Betrusted suffix for example, similarly to whas is done with CFU variants). Another option would also be to do things similarly to VexRiscv-SMP where CPU parameters are directly exposed to the user and the CPU is automatically rebuilt when the variant is not found in the cached ones, but this requires SBT and VexRiscv toolchain installed on the machine. That's what is used currently in Linux-on-LiteX-VexRiscv to avoid too much cached variants, which I'm also trying to avoid here.

enjoy-digital · 2021-10-01T09:20:35Z

@tcal-x, @mithro: It seems the auto-build feature is already in place in CFU playground: google/CFU-Playground#305 so it would probably be better to use it and cache some CPU variants if you want to ease access to users.

tcal-x · 2021-10-04T01:25:46Z

Hello @enjoy-digital , I agree with all of your points. And thank you for keeping the change rate low in pythondata_cpu_vexriscv; it makes it easy to maintain my fork. I guess I was so fed up with the CFU Playground dependency on my personal GitHub fork, which I desperately want to remove, that I didn't think through better alternatives.

One thing I now realize is that this wouldn't be the last update -- we are still iterating on the ideal CPU for our use case, so we continue to need somewhere to commit and share experimental variants.

I've now thought through some of the alternatives.

I could create a new org to host the CFU-specific fork (I hadn't realized that it was so easy to create an org, but I might not understand everything that comes along with that).
Could the "fork" instead be a non-master branch on litex-hub/pythondata_cpu_vexriscv? I don't see that often in practice, but the README does show the possibliity of installing from a branch: pip install --user git+https://github.com/litex-hub/pythondata-cpu-vexriscv.git@<branch>
I could create a static directory inside of CFU-Playground that's essentially the same as the pythondata_cpu_vexriscv/pythong_cpu_vexriscv/verilog directory of my fork -- it wouldn't include the Python packaging stuff, so it couldn't be used that way; the pre-generated Verilogs would only be used directly by CFU-Playground. A user with sbt installed could generate and experiment with new variants, and commit the ones they want to share.
- One benefit is that new CPU variants would be part of the CFU Playground commit stream.
- But, sharing outside of CFU Playground would be more difficult.
- Also, breaking the fork relationship with pythondata_cpu_vexriscv means that it might be necessary to manually copy in upstream changes to build.sbt and GenCoreDefault.scala.
Dynamic generation of variants inside of CFU Playground (simply by requesting a new variant name that encodes the desired parameterization). This would imitate to some degree what's done in pythondata_cpu_vexriscv_smp. But we could have the generation and some pre-generated common variants directly within CFU Playground. We'd need to be able to map all of the parameters from an encoded variant name -- for example, in Vex SMP, there's VexRiscvLitexSmpCluster_Cc1_Iw32Is4096Iy1_Dw32Ds4096Dy1_Ldw128_Cdma_Ood.v. In CFU Playground, our list of VexRiscv parameters is even longer and still growing, but the name would only need to have the parameters that differed from the default.

@kgugala , I've realized that there are a number of problems with the implementation of google/CFU-Playground#305 , so while parts of it might be adapted to one of the last two alternatives above, it won't go in as-is.

tcal-x · 2021-10-04T05:39:21Z

Perhaps google/CFU-Playground#309 is a better place for any continued discussion / suggestions.

enjoy-digital · 2021-10-04T06:53:12Z

@tcal-x: My concern was mainly about the number of pre-generated variants added to the repository. A temporary approach could be to:

Integrate the CFU related changes to build.sbt and GenCoreDefault.scala to avoid having different version of this.
Also make sure to have the eventual CFU related changes integrated in LiteX/VexRiscv CPU wrapper.
Allow dynamic generation of variants in this repository when not already pre-generated (Which also be useful for non-CFU configuration).
While you are still refining the optimal CFU configurations, store the pre-generated variants in a different repository or branch. (Since pre-generated variants are large can pollute git repositories if just temporary).

For 4) if you are able in the end to limit the number of pre-generated versions you want to provide to user, I would be fine having them directly in the main branch. But while this is still experimented, we could have the pre-generated version in a branch (that we could eventually delete later) or in a CFU specific dev fork that would be the main + just a different Makefile + more pre-generated variants.

tcal-x and others added 30 commits November 4, 2020 17:54

Set up wrappers to contain the cpu+cfu.

7374fad

Signed-off-by: Tim Callahan <tcal@google.com>

Add 'clk' input to CFU modules.

ae3d1d7

Signed-off-by: Tim Callahan <tcal@google.com>

Add perf count CSRs; rebuild CFU VexRiscv Verilogs.

07a72e2

Signed-off-by: Tim Callahan <tcal@google.com>

Increase to 8 perf counter CSRs; rebuild.

f022529

Signed-off-by: Tim Callahan <tcal@google.com>

Bump both I$, D$ from 4k to 8k.

59f011d

Both L1 caches are changed from 4k direct-mapped to 8k 2-way associative. The change to 2-way was required since each "way" can be 4k max. Signed-off-by: Tim Callahan <tcal@google.com>

Bump SpinalHdl/VexRiscv submodules; rebuild; widen CFU funcid port.

12ecb67

Signed-off-by: Tim Callahan <tcal@google.com>

Delete SpinalHDL submodule.

9f87086

Signed-off-by: Tim Callahan <tcal@google.com>

Rebuild: VexRiscv master w/ cfu fix, Spinal 1.4.3, wide func_id.

b6724a2

Signed-off-by: Tim Callahan <tcal@google.com>

Update CFU instr. encoding to match spec (update vex submodule too).

767476c

Signed-off-by: Tim Callahan <tcal@google.com>

Match CFU spec; function_id now 10b {funct7,funct3}.

eafff82

Rebuilt just the CFU-enabled Vex Verilogs. Signed-off-by: Tim Callahan <tcal@google.com>

Remove I-format CFU instruction; it impacts timing.

c888198

Signed-off-by: Tim Callahan <tcal@google.com>

Add 'SlimCfu' variants; add missing reset/rst connections.

b31d499

Signed-off-by: Tim Callahan <tcal@google.com>

Slim: reduce Icache size to 1kB.

6db4891

Signed-off-by: Tim Callahan <tcal@google.com>

Rebuild SlimCfu CPUs w/ 1kB Icache.

528360a

Signed-off-by: Tim Callahan <tcal@google.com>

Remove wrapper mechanism for VexRiscv/CFU.

e78ff1c

Signed-off-by: Tim Callahan <tcal@google.com>

Rebuild Verilogs at hash e78ff1c.

aa12b0d

Signed-off-by: Tim Callahan <tcal@google.com>

Add VexRiscv for Fomu -- minimal, plus hard muldiv, plus mcycle.

848042f

Signed-off-by: Tim Callahan <tcal@google.com>

Trim Fomu variants by removing alignment etc. checks.

301554e

Signed-off-by: Tim Callahan <tcal@google.com>

Tweak the Fomu variant to remove division, debug, and writeback/memor…

9188280

…y stages.

Merge pull request #1 from JosephBushagour/jbushagour_fomu_tweaks

43e1317

Tweak Fomu variant for performance and LCs.

Require a memory and writeback stage for the CFU plugin.

d8ec2d8

Signed-off-by: Joey Bushagour <jbushagour@google.com>

Merge pull request #2 from JosephBushagour/fomu-cfu

dc1f9dd

Require a memory and writeback stage for the CFU plugin.

Add icache, single-cycle-shift, and single-cycle multiply to the Fomu…

9face5f

… variant. Signed-off-by: Joey Bushagour <jbushagour@google.com>

Merge pull request #3 from JosephBushagour/fomu-cfu

38f50db

Update the Fomu CPU Variant for performance.

New "perf" variant has perfCSRs.

5bb9114

Also increase "Slim" D$ to 4kB, I$ to 2kB. Signed-off-by: Tim Callahan <tcal@google.com>

Rebuild affected verilogs with correct hash comment (5bb9114).

cbb04d3

Signed-off-by: Tim Callahan <tcal@google.com>

Add slimperf+cfu variant.

7490bb2

Signed-off-by: Tim Callahan <tcal@google.com>

Merge branch 'master' into fomu-cfu-mm2

593e180

Rebuild Verilogs at 593e180.

9f85993

Signed-off-by: Tim Callahan <tcal@google.com>

tcal-x added 4 commits September 29, 2021 14:10

Add 'slim' variant for completeness.

6f2afa7

Also remove obsolete genWrapper.py script. Signed-off-by: Tim Callahan <tcal@google.com>

Rebuild VexRiscv_Slim.v so it has correct hash in comment.

6dfdf25

Signed-off-by: Tim Callahan <tcal@google.com>

Remove performance CSRs from 'FullCfu' variants.

7454d1a

Signed-off-by: Tim Callahan <tcal@google.com>

Rebuild VexRiscv_FullCfu*.v at 7454d1a.

9a22d43

Signed-off-by: Tim Callahan <tcal@google.com>

mithro reviewed Oct 1, 2021

View reviewed changes

kgugala requested changes Oct 1, 2021

View reviewed changes

tcal-x marked this pull request as draft October 7, 2021 21:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expanded support for CFUs. #14

Expanded support for CFUs. #14

tcal-x commented Sep 30, 2021

enjoy-digital commented Sep 30, 2021

mithro Oct 1, 2021

tcal-x Oct 1, 2021

mithro Oct 1, 2021

tcal-x Oct 1, 2021

mithro commented Oct 1, 2021

kgugala Oct 1, 2021

kgugala Oct 1, 2021

kgugala Oct 1, 2021

kgugala Oct 1, 2021

kgugala Oct 1, 2021

kgugala Oct 1, 2021

enjoy-digital commented Oct 1, 2021

enjoy-digital commented Oct 1, 2021

tcal-x commented Oct 4, 2021

tcal-x commented Oct 4, 2021

enjoy-digital commented Oct 4, 2021

		@@ -17,6 +17,16 @@ src/main/scala/vexriscv GenCoreDefault.scala
		- Available in normal an -Debug, with the Debug bus exposed


		## Changes in `tcal-x`'s fork

		@@ -6,10 +6,12 @@ import spinal.lib._
		import spinal.lib.sim.Phase

Expanded support for CFUs. #14

Are you sure you want to change the base?

Expanded support for CFUs. #14

Conversation

tcal-x commented Sep 30, 2021

enjoy-digital commented Sep 30, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mithro commented Oct 1, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enjoy-digital commented Oct 1, 2021

enjoy-digital commented Oct 1, 2021

tcal-x commented Oct 4, 2021

tcal-x commented Oct 4, 2021

enjoy-digital commented Oct 4, 2021