building custom risc-v socs in chipyard · •create a heterogeneous boom and rocket risc-v soc...

Post on 18-Feb-2020

74 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Abraham GonzalezUC Berkeleyabe.gonzalez@berkeley.edu

Building Custom RISC-V SoCs in Chipyard

Goals• Get the basics of modifying a configuration

• Create a heterogeneous BOOM and Rocket RISC-V SoC• Learn how to generate Verilog for an SoC• Learn how to run Verilator RTL simulations• Case Study: Integrating a SHA3 accelerator into a Chipyard SoC!

• Add a SHA3 accelerator to Chipyard• Add the accelerator to a configuration!• Get Verilog and run Verilator simulations

Tutorial RoadmapCustom SoC Configuration

RTL GeneratorsRISC-V Cores

Multi-level Caches

CustomVerilogPeripheralsAccelerators

Software RTL Simulation

VCS Verilator

FireSim FPGA-Accelerated Simulation

Simulation Debugging Networking

Automated VLSI Flow

Hammer Tech-plugins

Tool-plugins

RTL Build ProcessFIRRTL

TransformsFIRRTL IR Verilog

FireMarshalBare-metal &

Linux

Custom Workload

QEMU & Spike

Tutorial RoadmapCustom SoC Configuration

RTL GeneratorsRISC-V Cores

Multi-level Caches

CustomVerilogPeripheralsAccelerators

Software RTL Simulation

VCS Verilator

FireSim FPGA-Accelerated Simulation

Simulation Debugging Networking

Automated VLSI Flow

Hammer Tech-plugins

Tool-plugins

RTL Build ProcessFIRRTL

TransformsFIRRTL IR Verilog

FireMarshalBare-metal &

Linux

Custom Workload

QEMU & Spike

Getting Started

Having trouble? Raise a hand and someone will come help you.

You can find these slides onhttps://fires.im/micro-2019-tutorial/

Interactive Section!

How things will work

Interactive Slide“Follow Along”

Explanation Slide“What’s happening?”

How things will work

Terminal Section Inside-a-File Section

# command 1> echo “Chipyard Rules!”

# command 2> do_this arg1 arg2

// SOME COMMENT HEREclass SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

Interactive

10

# start a tmux session> tmux new –s soc

# return to chipyard> cd ~/chipyard-morning/> ls

Directory Structurechipyard-morning/

generators/rocket-chip/sha3/

sims/verilator/

software/firemarshal/

tools/chisel/firrtl/

build.sbt11

Our library of Chisel generators

Utilities for simulating SoCs

Config file enumerating generators and dependencies

Chisel/FIRRTL

Utilities for building RISC-V software

Build and simulate a heterogeneous BOOM + Rocket SoC

12

Default Chipyard Project• You need a top-level project that combines all the

generators wanted• Want Rocket Chip, BOOM, SHA3 accelerator, etc…

• In our case we are using the `example` project• Located in `generators/example`

• This holds things such as• Test Harness code• Top-level module (matches the top-level of the DUT)• SoC configurations

• Most of the work will be done in this project

13

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

BOOM Integration• BOOM source code already added

• Located in `generators/boom`• Already has default configurations

• Located in `generators/example/src/main/scala/ BoomConfigs.scala`

14

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Interactive

15

# open up the default boom configurations file> cd generators/example/src/main/scala> less BoomConfigs.scala

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Interactive

16

# open up the default boom configurations file> cd generators/example/src/main/scala> less BoomConfigs.scala

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Reminder: use q to quit less

Configuring a SoC with BOOM

17

• Bottom-to-top (right-to-left) config hierarchy• Configs can read/override other configsBoomTilesKey => Seq()RocketTilesKey => Seq()BankedL2Key => Module(new TLBroadcast)BootROMParams => BootROMParams()BuildTop => None

class SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Configuring a SoC with BOOM

18

• Bottom-to-top (right-to-left) config hierarchy• Configs can read/override other configsBoomTilesKey => Seq()RocketTilesKey => Seq()BankedL2Key => Module(new TLBroadcast)BootROMParams => BootROMParams()BuildTop => None

class SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Configuring a SoC with BOOM

19

• Bottom-to-top (right-to-left) config hierarchy• Configs can read/override other configsBoomTilesKey => Seq(BoomParams())RocketTilesKey => Seq()BankedL2Key => Module(new TLBroadcast)BootROMParams => BootROMParams()BuildTop => None

class SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Configuring a SoC with BOOM

20

• Bottom-to-top (right-to-left) config hierarchy• Configs can read/override other configsBoomTilesKey => Seq(BoomParams(Small))RocketTilesKey => Seq()BankedL2Key => Module(new TLBroadcast)BootROMParams => BootROMParams()BuildTop => None

class SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Configuring a SoC with BOOM

21

• Bottom-to-top (right-to-left) config hierarchy• Configs can read/override other configsBoomTilesKey => Seq(BoomParams(Small))RocketTilesKey => Seq()BankedL2Key => Module(new InclusiveCache)BootROMParams => BootROMParams()BuildTop => None

class SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Configuring a SoC with BOOM

22

• Bottom-to-top (right-to-left) config hierarchy• Configs can read/override other configsBoomTilesKey => Seq(BoomParams(Small))RocketTilesKey => Seq()BankedL2Key => Module(new InclusiveCache)BootROMParams => BootROMParams(“../bootrom.rv64.img”)BuildTop => None

class SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Configuring a SoC with BOOM

23

• Bottom-to-top (right-to-left) config hierarchy• Configs can read/override other configsBoomTilesKey => Seq(BoomParams(Small))RocketTilesKey => Seq()BankedL2Key => Module(new InclusiveCache)BootROMParams => BootROMParams(“../bootrom.rv64.img”)BuildTop => Module(new Top)

class SmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

chipyard-morning/generators/example/src/main/scala/BoomConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Interactive

24

# open heterogeneous configs file> vim HeteroConfigs.scala

chipyard-morning/generators/example/src/main/scala/HeteroConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class TutorialSmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

Feel free to use another editor! These slides assume vim.

Interactive

25

# open heterogeneous configs file> vim HeteroConfigs.scala

chipyard-morning/generators/example/src/main/scala/HeteroConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class TutorialSmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

Modify

Interactive

26

# open heterogeneous configs file> vim HeteroConfigs.scala

chipyard-morning/generators/example/src/main/scala/HeteroConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class TutorialSmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

class SmallBoomAndRocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithRenumberHarts ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

Modify and add

VIM Reminders: use i to insert textuse ESC to exit inserting text

Interactive

27

# open heterogeneous configs file> vim HeteroConfigs.scala

chipyard-morning/generators/example/src/main/scala/HeteroConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class TutorialSmallBoomConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.system.BaseConfig)

class SmallBoomAndRocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithRenumberHarts ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

Modify and add

VIM Reminders: use :q to quituse :w to saveuse :wq to save and quit

Interactive

28

# navigate to the verilator> cd ~/chipyard-morning/sims/verilator

# start the verilator rtl simulator build> make CONFIG=SmallBoomAndRocketConfig –j16

# this will take a while!

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Behind the Scenes: Config

29

chipyard-morning/generators/example/src/main/scala/HeteroConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class SmallBoomAndRocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithRenumberHarts ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

Adds a “Big” Rocket core to the SoC

Behind the Scenes: Config

30

chipyard-morning/generators/example/src/main/scala/HeteroConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class SmallBoomAndRocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new boom.common.WithRenumberHarts ++new boom.common.WithSmallBooms ++new boom.common.WithNBoomCores(1) ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

Renumbers the core ID’s so that there are no duplicates

Adds a “Big” Rocket core to the SoC

Behind the Scenes: Make• `make` is the top-level build system to combine everything

• Invokes the Scala Build Tool (sbt)• Used in Chisel, FIRRTL, and other Chipyard tools

• Invokes all of the simulator builds (VCS and Verilator)• Automatically keeps track of file dependencies for you!

• Powerful `make` commands• Can just `make verilog` for Verilog only• Can run pre-added or unique tests

• `make CONFIG=<YOUR CONFIG> BINARY=<YOUR BINARY> run-binary`• `make CONFIG=<YOUR CONFIG> run-bmark-tests`

• Keeps all outputs organized based of a unique name of the SoC• `<PROJECT>.<DUT MODULE>.<CONFIG>`• Ex. `example.TestHarness.SmallBoomAndRocketConfig`

31

Building a SW Simulator

32

Rocket

BOOM

BootROM

FIRRTL Verilog SimulatorConfig

L2 Cache

1. Configs parameterize Chisel generators 2. Chisel elaborates into

FIRRTL 3. FIRRTL elaborates into Verilog 4. Verilator compiles Verilog

into simulator binary

Building a SW Simulator

33

Rocket

BOOM

BootROM

Config

L2 Cache

1. Configs parameterize Chisel generators

In this case `SmallBoomAndRocketConfig` creates a BOOM + Rocket SoC

Building a SW Simulator

34

Rocket

BOOM

BootROM

FIRRTLConfig

L2 Cache

1. Configs parameterize Chisel generators 2. Chisel elaborates into

FIRRTL

Emits a `*.fir` file

Building a SW Simulator

35

Rocket

BOOM

BootROM

FIRRTL VerilogConfig

L2 Cache

1. Configs parameterize Chisel generators 2. Chisel elaborates into

FIRRTL 3. FIRRTL elaborates into Verilog

Emits a `*.top.v` and `*.harness.v` file

Why `*.top.v` and `*.harness.v`

36

TestHarnessTop

Tile 0BOOM

L1I$ L1D$

3-w BOOM

SysBus

MemBus

BootROM

L2

GPIOs

Tile 1

L1I$ L1D$

BOOMRocket

SimBlockDevice SimAXIMem

JTAG

37

TestHarness

TopTile 0

L1I$ L1D$

3-w BOOM

SysBus

MemBus

BootROM

L2

GPIOs

Tile 1

L1I$ L1D$

Rocket

SimBlockDevice SimAXIMem

JTAG

Why `*.top.v` and `*.harness.v`

38

TopTile 0

L1I$ L1D$

3-w BOOM

SysBus

MemBus

BootROM

L2

GPIOs

Tile 1

L1I$ L1D$

Rocket

JTAGPass to your VLSI flow!

Why `*.top.v` and `*.harness.v`

Building a SW Simulator

39

Rocket

BOOM

BootROM

FIRRTL Verilog SimulatorConfig

L2 Cache

1. Configs parameterize Chisel generators 2. Chisel elaborates into

FIRRTL 3. FIRRTL elaborates into Verilog 4. Verilator compiles Verilog

into simulator binary

Emits a `simulator-*` executable that represents the `SmallBoomAndRocketConfig`

Where is my Verilog!• Once completed the build outputs are located here

40

> ls generated-src/*/

generated-src/*/*.harness.v*.top.v*.dts*.json...

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/generated-src/*/

tools/chisel/firrtl/

tests/build.sbt

Verilog used for testing

Main Verilog of the design

Device Tree used for SW

MMIO files (where and what is connected)

Interactive

41

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/generated-src/*/

tools/chisel/firrtl/

tests/build.sbt

# navigate to the output sources> cd generated-src/*/> ls

# take a look at the following files> less example.TestHarness.*.top.v> less example.TestHarness.*.harness.v

How does the Verilog look?

42

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/generated-src/*/

tools/chisel/firrtl/

tests/build.sbt

// Rocket Tile (includes a Rocket Core + L1$’s)module RocketTile( // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@347233.2]input clock, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@347234.4]input reset, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@347235.4]

. . . // BOOM Tile (includes a BOOM Core + L1$’s)module BoomTile( // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@293433.2]input clock, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@293434.4]input reset, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@293435.4]

. . . // Top-level DUT that includes the BOOM and Rocket Tilesmodule Top( // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@384841.2]input clock, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@384842.4]input reset, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@384843.4]

example.TestHarness.SmallBoomAndRocketConfig.top.v

Interactive

43

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

# navigate to the output sources> cd ../..> ls

# run a multi-threaded vector-vector add> make CONFIG=SmallBoomAndRocketConfig BINARY=mt-vvadd.riscv run-binary

# take a look at the output file

> less mt-vvadd.*.out

Output file

44

using random seed 1570590524This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.Listening on port 38117

C1: 0 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162C1: 1 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162C1: 2 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162

C1: 3 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162C1: 4 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162C1: 5 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162

. . .C1: 63210 [1] pc=[000000008000010e] W[r 0=0000000000000000][0] R[r10=0000000000000001] R[r11=0000000000000001] inst=[00b57063] bgeu a0, a1, pc + 0C1: 63211 [0] pc=[000000008000010e] W[r 0=0000000000000000][0] R[r10=0000000000000001] R[r11=0000000000000001] inst=[00b57063] bgeu a0, a1, pc + 0

*** PASSED *** Completed after 107267 cycles

Commit log lines emitted from the Rocket CoreTest passed

mt-vvadd.example.TestHarness.SmallBoomAndRocketConfig.top.v

Case Study: Create and test a SHA3-accelerated SoC!

45

Why do the case study?• “We are in a golden age of computer architecture”• Domain Specific Accelerators are extremely popular

• Vector Accelerators• ML Accelerators

• Training• Inference

• Compression Accelerators• More!

• Goal is to answer: “But how do I add my own accelerator?”

46

SHA3 accelerator?• SHA3 accelerator is a pre-implemented Chisel accelerator• Implements the Secure Hash Algorithm 3 (SHA3)

• Rough specification in 2012, released in late 2015 • Uses variable length messages with a sponge function

• Designed to be more efficient when implemented in HW!• Want to improve the hashes/sec and hashes/Watt

47

Let’s begin adding it!

Interactive

48

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

# add sha3 to chipyard> cd ~/chipyard-morning/generators> git submodule add https://github.com/ucb-bar/sha3.git

# open the dependencies file> cd ..> vim build.sbt

Interactive

49

//lazy val sha3 = (project in file("generators/sha3"))

// .dependsOn(rocketchip, chisel_testers)

// .settings(commonSettings)

lazy val example = (project in file("generators/example"))

.dependsOn(boom, hwacha, sifive_blocks, sifive_cache, utilities)

.settings(commonSettings)

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Interactive

50

//lazy val sha3 = (project in file("generators/sha3"))

// .dependsOn(rocketchip, chisel_testers)

// .settings(commonSettings)

. . .

lazy val example = (project in file("generators/example"))

.dependsOn(boom, hwacha, sifive_blocks, sifive_cache, utilities)

.settings(commonSettings)

lazy val sha3 = (project in file("generators/sha3"))

.dependsOn(rocketchip, chisel_testers)

.settings(commonSettings)

. . .

lazy val example = (project in file("generators/example"))

.dependsOn(sha3, boom, hwacha, sifive_blocks, sifive_cache, utilities)

.settings(commonSettings)

Uncomment

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Add sha3

What happened?• There are three steps to add an accelerator to Chipyard

1. Add the source code to `generators`2. Add the accelerator project to the build system in `build.sbt`

• Need to tell the build system it exists• Need to add it to a top-level project (in our case `example`)

3. Create a new configuration that adds it!

51

What happened?• There are three steps to add an accelerator to Chipyard

1. Add the source code to `generators`2. Add the accelerator project to the build system in `build.sbt`

• Need to tell the build system it exists• Need to add it to a top-level project (in our case `example`)

3. Create a new configuration that adds it!

52

What happened?• There are three steps to add an accelerator to Chipyard

1. Add the source code to `generators`2. Add the accelerator project to the build system in `build.sbt`

• Need to tell the build system it exists• Need to add it to a top-level project (in our case `example`)

3. Create a new configuration that adds it!

53

First a quick-background on RoCC accelerators

RoCC Accelerators• RoCC: Rocket Chip Coprocessor• Execute custom RISC-V instructions

for a custom extension

• Examples of RoCC accelerators• Hwacha vector accelerators• Memcpy accelerator• Machine-learning accelerators• Java GC accelerator

Tile

BOOM/Rocket

L1I$ L1D$

PTWTLBsDecoupled

RoCCAccelerator

(SHA3)

L2

SystemBus

Core Complex Peripherals

cmd req

cmd resp

chipyard-morning/generators/example/rocket-chip/sha3/src/main/scala/sha3.scala

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Understanding the SHA3 Accelerator• SHA3 is a minimal example of a RoCC-based

accelerator• Executes custom “sha3” instructions sent by the

Rocket or BOOM core• sha3.scala

• Note the `WithSha3Accel` mixin, which plugs into the Rocket Chip config system

• `Sha3AccelImp` implements the Chisel-based accelerator

55

class WithSha3Accel extends Config ((site, here, up) => {case Sha3WidthP => 64case Sha3Stages => 1case Sha3FastMem => truecase Sha3BufferSram => falsecase BuildRoCC => Seq(

(p: Parameters) => {val sha3 = LazyModule.apply(new Sha3Accel(OpcodeSet.custom2)(p)

)sha3

})

})

The SHA3 Accelerator• The SHA3 mixin

56

chipyard-morning/generators/example/rocket-chip/sha3/src/main/scala/sha3.scala

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Rocket Chip uses the “BuildRoCC” key to figure out which accelerator to build

SHA3 Parameters

The SHA3 Accelerator• RoCC Command Stream

57

Tile

BOOM/Rocket

SHA3: Decoupled RoCC Accelerator

cmd.valid

cmd.bits.inst.funct[1:0]

cmd.bits.rs1[63:0]cmd.bits.rs2[63:0]cmd.bits.inst.rd[4:0]busy

cmd.ready

mem.req.ready

mem.req.addr[63:0]mem.req.cmd[2:0]mem.req.size[1:0]

mem.req.valid

L1DCachemem.req.data[63:0]mem.resp.validmem.resp.data[63:0]

chipyard-morning/generators/example/rocket-chip/sha3/src/main/scala/sha3.scala

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Interactive

58

# open rocket configs file> cd generators/example/src/main/scala> vim RocketConfigs.scala

chipyard-morning/generators/example/src/main/scala/RocketConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class TutorialRocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

Interactive

59

# open rocket configs file> cd generators/example/src/main/scala> vim RocketConfigs.scala

chipyard-morning/generators/example/src/main/scala/RocketConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class TutorialRocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

Modify

Interactive

60

# open rocket configs file> cd generators/example/src/main/scala> vim RocketConfigs.scala

chipyard-morning/generators/example/src/main/scala/RocketConfigs.scala

rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

class TutorialRocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

class Sha3RocketConfig extends Config(new WithTop ++new WithBootROM ++new freechips.rocketchip.subsystem.WithInclusiveCache ++new sha3.WithSha3Accel ++new freechips.rocketchip.subsystem.WithNBigCores(1) ++new freechips.rocketchip.system.BaseConfig)

Modify and add

Interactive

61

# navigate to the verilator> cd ~/chipyard-morning/sims/verilator

# start the verilator rtl simulator build> make CONFIG=Sha3RocketConfig debug –j16

# this will take a while!

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Reminder: Where is my Verilog!• Once completed the build outputs are located here

62

> ls generated-src/example.TestHarness.Sha3RocketConfig/

generated-src/example.TestHarness.Sha3RocketConfig/*.harness.v*.top.v*.dts*.json...

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/generated-src/*/

tools/chisel/firrtl/

tests/build.sbt

Verilog used for testing

Main Verilog of the design

Device Tree used for SW

MMIO files (where and what is connected)

Coffee Break until 11am

Coming up…

Running customized software on the SHA3 accelerated SoC

63

https://fires.im/micro-2019-tutorial/Slides here:

Interactive

64

# re-enter the tmux session (only if you exited it)> tmux attach –t soc

# navigate to firemarshal> cd ~/chipyard-morning/software/firemarshal

# view sha3 accelerated program

> vim workloads/sha3/benchmarks/src/sha3-rocc.c

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/

tools/chisel/firrtl/

tests/build.sbt

Accelerator Software

65

printf("Start basic test 1.\n");// BASIC TEST 1 - 150 zero bytes

start = rdcycle();

asm volatile ("fence");

// setup accelerator with addresses of input and outputROCC_INSTRUCTION_SS(2, &input, &output, 0);

// Set length and compute hashROCC_INSTRUCTION_S(2, sizeof(input), 1);

asm volatile ("fence" ::: "memory");

end = rdcycle();

sha3-rocc.c chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/workloads/sha3/benchmarks/src/sha3-rocc.c

tools/chisel/firrtl/

tests/build.sbt

Accelerator Software

66

printf("Start basic test 1.\n");// BASIC TEST 1 - 150 zero bytes

start = rdcycle();

asm volatile ("fence");

// setup accelerator with addresses of input and outputROCC_INSTRUCTION_SS(2, &input, &output, 0);

// Set length and compute hashROCC_INSTRUCTION_S(2, sizeof(input), 1);

asm volatile ("fence" ::: "memory");

end = rdcycle();

sha3-rocc.c

opcode rd rs1 rs2 functasm volatile ("custom2 x0, %[msg_addr], %[hash_addr], 0"

:: [msg_addr] "r" (&input), [hash_addr] "r" (&output));

Expanded C macro

Accelerator Software

67

printf("Start basic test 1.\n");// BASIC TEST 1 - 150 zero bytes

start = rdcycle();

asm volatile ("fence");

// setup accelerator with addresses of input and outputROCC_INSTRUCTION_SS(2, &input, &output, 0);

// Set length and compute hashROCC_INSTRUCTION_S(2, sizeof(input), 1);

asm volatile ("fence" ::: "memory");

end = rdcycle();

sha3-rocc.c

opcode rd rs1 rs2 functasm volatile ("custom2 x0, %[msg_addr], %[hash_addr], 0"

:: [msg_addr] "r" (&input), [hash_addr] "r" (&output));

Expanded C macro

Setup and run the accelerator

Understanding SHA3 Software• Send necessary information to the accelerator

• Source(s), Destination, and what function to run• Accelerator returns

• Busy… a.k.a. “Is the accelerator done?”

68

opcode rd rs1 rs2 functasm volatile ("custom2 x0, %[msg_addr], %[hash_addr], 0"

:: [msg_addr] "r" (&input), [hash_addr] "r" (&output));

BOOM/Rocket

cmd.valid

cmd.bits.inst.funct[1:0]

cmd.bits.rs1[63:0]cmd.bits.rs2[63:0]cmd.bits.inst.rd[4:0]busy

cmd.ready

SHA3 Ctrl

Interactive

69

# view sw implementation of SHA3 program> vim workloads/sha3/benchmarks/src/sha3-sw.c

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/

tools/chisel/firrtl/

tests/build.sbt

Software SHA3

70

printf("Start basic test 1.\n");// BASIC TEST 1 - 150 zero bytes

start = rdcycle();

// run sw to compute the SHA3 hashsha3ONE(input, sizeof(input), output);

end = rdcycle();

sha3-rocc.c chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/workloads/sha3/benchmarks/src/sha3-sw.c

tools/chisel/firrtl/

tests/build.sbt

Software SHA3

71

printf("Start basic test 1.\n");// BASIC TEST 1 - 150 zero bytes

start = rdcycle();

// run sw to compute the SHA3 hashsha3ONE(input, sizeof(input), output);

end = rdcycle();

sha3-rocc.c chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/workloads/sha3/benchmarks/src/sha3-sw.c

tools/chisel/firrtl/

tests/build.sbt

SW implementation of SHA3

Interactive

72

# make sure you are in firemarshal directory> cd ~/chipyard-morning/software/firemarshal

# build both binaries> ./marshal build workloads/sha3-bare-*.json

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/

tools/chisel/firrtl/

tests/build.sbt

Building SHA3 Software• Used the FireMarshal utility to build the binaries

• Tool that takes in `.json` description of build and emits the `.riscv` binary• More in-depth view after lunch

• What was done… built two binaries• `sha3-sw.riscv` - software version of SHA3 computation• `sha3-rocc.riscv` - sends SHA3 computation to the accelerator

• Both binaries created in• `workloads/sha3/benchmarks/bare/sha3-*.riscv`

73

Interactive

74

# navigate to the verilator directory> cd ~/chipyard-morning/sims/verilator

# run accelerated program> make CONFIG=Sha3RocketConfig run-binary-debug BINARY=../../software/firemarshal/workloads/sha3/benchmarks/bare/sha3-rocc.riscv

# run non-accelerated program> make CONFIG=Sha3RocketConfig run-binary-debug BINARY=../../software/firemarshal/workloads/sha3/benchmarks/bare/sha3-sw.riscv

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/

tools/chisel/firrtl/

tests/build.sbt

The non-accelerated version takes longer

Outputs• You should see the following after both tests

75

# you just ran: “make CONFIG=… BINARY=… run-binary-debug”# this is the output:

Start basic test 1.

output[0]:221 ==? results[0]:221

. . .output[31]:238 ==? results[31]:238

Success!SHA execution took M cycles

Compare the cycle time of both runs. Which is faster?

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/

tools/chisel/firrtl/

tests/build.sbt

All Outputs• Once completed the build outputs are located here

76

> ls

verilator/sha3-sw.*.outsha3-sw.*.vcdsha3-rocc.*.outsha3-rocc.*.vcdsimulator-example-Sha3RocketConfig-debug. . .

Rocket Core Commit log

chipyard-morning/generators/example/rocket-chip/boom/sha3/

sims/verilator/

software/firemarshal/

tools/chisel/firrtl/

tests/build.sbt

Verilator RTL Simulator

Waveform file (used with GTKWave, etc)

Interactive

77

# exit the terminalCTRL-D

Delete the tmux session

More Information

• https://chipyard.readthedocs.io/en/latest• Talks about heterogeneous SoCs• Talks about adding Verilog IP• Talks about adding accelerators• . . .

78

Covers what we did in this talk and more!

That’s it!

Coming up… Using Hammer to Speed Up the

VLSI Flow

79

Backup Slides

80

Exploring the Module Hierarchy

example.Sha3RocketConfig.(harness/top).v:

• Generated Verilog for Top and TestHarnessexample.Sha3RocketConfig.core.config:

• Configuration information for core

example.Sha3RocketConfig.dts:

• Device tree string

example.Sha3RocketConfig.fir:

• FIRRTL IR

example.Sha3RocketConfig.graphml:

• Diplomacy graph81

chipyard-morning/generators/example/rocket-chip/sha3/

sims/verilator/generated-src/example.Sha3RocketConfig/

tools/chisel/firrtl/

tests/build.sbt

TileLink PWM Custom Module

• Custom PWM module driving a top-level IO pin

• Changes to base SoC• Additional top-level pin for PWM out• Additional memory-mapped register for

configuring PWM• Additional top-level module to drive PWM

82

chipyard-morning/generators/example/PWM.scalaRocketConfigs.scalaTop.scala

rocket-chip/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

Instantiating a PWM Top

PWMModule:• Module which instantiates the PWM

functionalityPWMTL:• Module which creates a TileLink node, and

inherits functionality of the PWMModuleHasPeripheryPWMTL:• Connects PWM TileLink node to

Diplomacy graph of the BaseSubsystemHasPeripheryPWMTLModuleImp• Adds additional IO pin for PWMOut, and

drives it with the PWMModule 83

chipyard-morning/generators/example/PWM.scalaRocketConfigs.scalaTop.scala

rocket-chip/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

PWM Custom Components

TopWithPWMTL:• Adds the HasPeripheryPWMTL trait to

add the PWM node to the Top’s Diplomacy graph

• Instantiates a TopWithPWMTLModuleTopWithPWMTLModule:• Adds the

HasPeripheryPWMTLModuleImp trait to get the additional PWM IO pin and PWM Module

84

chipyard-morning/generators/example/PWM.scalaRocketConfigs.scalaTop.scala

rocket-chip/sha3/

sims/verilator/

tools/chisel/firrtl/

tests/build.sbt

top related