A 4-channel memory-to-memory DMA in SystemVerilog — designed, verified (simulation + formal),
signed off at 500 MHz in FreePDK45 with Design Compiler and PrimeTime, and characterised with a bandwidth / area trade-off study.
| Design | 4 channels · AXI4-Lite CSRs with maskable W1C interrupts · AXI4 INCR bursts split at 4 KB · any byte alignment on source and destination (2-word realigner + WSTRB head/tail masking) · up to 8 outstanding reads · round-robin arbitration · SLVERR/DECERR handling with write poisoning |
| Verification | Verilator constrained-random testbench + DPI-C C reference model (golden memory + exact burst-sequence predictor) · 3,820 transfers / 525k AXI bursts checked over 8 seeds · 14 AXI SVA properties · 34 / 34 functional cover points · 3 extra parameter configurations |
| Formal | Round-robin arbiter starvation-freedom proved unbounded (SymbiYosys + ABC PDR) for N = 2, 3, 4, 8 |
| Mutation testing | 8 / 8 injected RTL bugs caught — and the first run exposed a real coverage hole (an IRQ event racing a W1C write) that was then closed |
| Sign-off | Synopsys DC Ultra + PrimeTime, OSU gscl45nm (FreePDK45), 500 MHz: setup +16 ps, hold +12 ps, 0 DRV · 52,347 µm² · 30.4 mW (PrimePower, SAIF-annotated) |
| Analysis | Outstanding × burst-length sweep at 32-cycle latency: 2.9 % → 99.1 % bus utilisation; analytical model within ~1 %; 8 × 8-beat reaches 99 % with 4 % less area than 4 × 16 |
- Overview
- Architecture
- Programming model
- Verification
- Implementation & sign-off
- Performance analysis
- Getting started
- Repository layout
- Reproducing every number
- Design decisions & trade-offs
- Limitations & roadmap
- Lessons learned
- Author
A DMA engine moves blocks of memory without the CPU touching every byte. Software programs a source address, a destination address and a length through memory-mapped registers, sets START, and gets an interrupt when the copy is finished (or failed).
This project implements that end to end and goes through the whole front-end flow:
spec ─▶ RTL ─▶ lint ─▶ simulation (random + directed, C reference, SVA, coverage)
└▶ formal (arbiter fairness) ─▶ mutation testing
─▶ synthesis (DC) ─▶ STA + power (PrimeTime) ─▶ performance / area trade-off study
| Feature | Detail |
|---|---|
| Channels | 4 by default (parameter NCH, 2–8), independent register sets, round-robin bus sharing at burst granularity |
| Register interface | AXI4-Lite slave, 12-bit address, 32-bit data; AW and W accepted in any order; byte strobes honoured; unmapped address → SLVERR |
| Memory interface | AXI4 master, 32-bit data (parameter DW), INCR bursts of 1 – BL_MAX beats (runtime-selectable per channel), never crossing a 4 KB boundary |
| Alignment | Source and destination may start at any byte and differ in alignment; length is any number of bytes |
| Outstanding reads | Up to MAX_OUTS (runtime register, ≤ OUTS_MAX) read bursts in flight |
| Interrupts | Per-channel done and error bits, W1C status, per-bit enable, single level irq output |
| Errors | RRESP/BRESP = SLVERR or DECERR → error status + interrupt; channel stops issuing and drains; poisoned writes (WSTRB = 0) guarantee bad read data never reaches memory |
| Shadow registers | The next transfer can be programmed while the current one runs; START while busy is ignored |
flowchart LR
SW([Software])
MEM[(AXI4 memory)]
subgraph DMA[dma_top]
direction LR
CSR["dma_csr<br/>registers · IRQ"]
CH["dma_channel × 4<br/>state + next chunk<br/>(src, dst, n)"]
subgraph ENG[dma_engine]
direction LR
ARB["round-robin<br/>arbiter"]
ARREG["AR issue<br/>register"]
RQ[("rq<br/>realign info")]
AWQ[("awq<br/>AW info")]
BQ[("bq<br/>channel id")]
PK["realigning packer<br/>shift + WSTRB"]
WF[("W buffer")]
end
end
SW -- "AXI4-Lite" --> CSR
CSR -- irq --> SW
CSR -- "start / cfg" --> CH
CH -- "done / error" --> CSR
CH -- req --> ARB --> ARREG
ARREG -- "on AR hs" --> RQ & AWQ
ARREG -- AR --> MEM
MEM -- R --> PK
RQ --> PK --> WF -- W --> MEM
AWQ -- AW --> MEM
AWQ -- "on AW hs" --> BQ
MEM -- B --> BQ
BQ -- "chunk done" --> CH
dma_top top level, parameters, wiring
├── dma_csr u_csr AXI4-Lite slave, register file, IRQ logic
├── dma_channel g_ch[c].u_ch per-channel state, next-chunk computation
└── dma_engine u_eng shared AXI4 master datapath
├── dma_rr_arb u_arb round-robin arbiter
├── dma_fifo u_rq read-return descriptors (realignment control)
├── dma_fifo u_awq write-address descriptors
├── dma_fifo u_bq channel id per outstanding write burst
└── dma_fifo u_wf W data buffer {last, strb, data}
| File | Lines | Role |
|---|---|---|
rtl/dma_top.sv |
176 | Top level |
rtl/dma_csr.sv |
286 | Registers, AXI4-Lite, interrupts |
rtl/dma_channel.sv |
206 | Channel state machine, chunking, address update |
rtl/dma_engine.sv |
342 | AR issue, descriptor queues, realigning packer, AW/W/B |
rtl/dma_rr_arb.sv |
52 | Round-robin arbiter |
rtl/dma_fifo.sv |
54 | Flop-based FWFT FIFO with overflow/underflow assertions |
RTL total 1,116 lines; testbench + C model ≈ 1,500 lines; scripts / formal / analysis ≈ 700 lines.
| Port group | Signals | Notes |
|---|---|---|
| Clock / reset | clk, rst_n |
Single clock; asynchronous active-low reset |
s_axil_* |
awaddr[11:0] awvalid awready wdata[31:0] wstrb[3:0] wvalid wready bresp[1:0] bvalid bready araddr[11:0] arvalid arready rdata[31:0] rresp[1:0] rvalid rready |
CSR slave |
m_axi_ar* / r* |
arid araddr arlen[7:0] arsize arburst arvalid arready · rid rdata rresp rlast rvalid rready |
Read master; ID always 0 |
m_axi_aw* / w* / b* |
awid awaddr awlen awsize awburst awvalid awready · wdata wstrb wlast wvalid wready · bid bresp bvalid bready |
Write master; ID always 0 |
| Interrupt | irq |
Level, registered: |(IRQ_STATUS & IRQ_ENABLE) |
| Parameter | Default | Range | Meaning |
|---|---|---|---|
NCH |
4 | 2 – 8 | Number of channels |
AW |
32 | Address width | |
DW |
32 | 32, 64, … | Data width (bytes per beat BPB = DW/8) |
IDW |
4 | AXI ID width (IDs are driven to 0) | |
BL_MAX |
16 | 2 – 256, power of 2 | Maximum beats per burst |
OUTS_MAX |
4 | ≥ 1 | Maximum outstanding read bursts (descriptor queues are 2·OUTS_MAX deep) |
DBUF_DEPTH |
2·BL_MAX |
power of 2 | W data buffer depth (words) |
sequenceDiagram
participant SW as Software
participant CSR as dma_csr
participant CH as dma_channel
participant ENG as dma_engine
participant MEM as AXI memory
SW->>CSR: write SRC, DST, LEN, CTRL{BURST_LOG2, START}
CSR->>CH: start pulse (copies shadow registers)
loop until all bytes issued
CH->>ENG: req + next chunk (src, dst, n)
ENG->>ENG: round-robin grant, load AR register
ENG->>MEM: AR (one read burst)
ENG->>MEM: AW (matching write burst)
MEM-->>ENG: R beats (in order)
ENG->>ENG: realign bytes, apply WSTRB head/tail mask
ENG->>MEM: W beats
MEM-->>ENG: B
ENG-->>CH: bdone (pend − 1)
end
CH->>CSR: ev_done (or ev_err)
CSR->>SW: irq
SW->>CSR: read IRQ_STATUS, write-1-to-clear
Each channel continuously proposes its next chunk — the largest piece of the remaining transfer that fits in exactly one read burst and one write burst:
n = min( remaining,
4096 − src[11:0], 4096 − dst[11:0], // neither side crosses 4 KB
burst_bytes − src % BPB, burst_bytes − dst % BPB ) // ≤ BL beats on each side
AR = { src & ~(BPB−1), ceil((src % BPB + n) / BPB) − 1 }
AW = { dst & ~(BPB−1), ceil((dst % BPB + n) / BPB) − 1 }
Worked example — src = 0x00FF1, dst = 0x09FFA, len = 200, 16-beat bursts (64 B),
32-bit bus. Both ends are unaligned and both cross a 4 KB page at different points:
| # | src | dst | remaining | n | limited by | AR addr / beats | AW addr / beats |
|---|---|---|---|---|---|---|---|
| 1 | 0x00FF1 | 0x09FFA | 200 | 6 | dst 4 KB | 0x00FF0 / 2 | 0x09FF8 / 2 |
| 2 | 0x00FF7 | 0x0A000 | 194 | 9 | src 4 KB | 0x00FF4 / 3 | 0x0A000 / 3 |
| 3 | 0x01000 | 0x0A009 | 185 | 63 | dst burst length | 0x01000 / 16 | 0x0A008 / 16 |
| 4 | 0x0103F | 0x0A048 | 122 | 61 | src burst length | 0x0103C / 16 | 0x0A048 / 16 |
| 5 | 0x0107C | 0x0A085 | 61 | 61 | remaining | 0x0107C / 16 | 0x0A084 / 16 |
The C reference model computes this sequence independently and the testbench checks every AR/AW on the bus against it.
With source byte offset s and destination offset d (each 0–3), write word j is a 4-byte window of two consecutive read beats:
| Case | Window | First read beat |
|---|---|---|
| s > d | {R[j+1], R[j]} shifted by s − d bytes |
only loads prev, emits nothing |
| s ≤ d | {R[j], R[j−1]} shifted by s − d + 4 bytes |
emits a word |
So the packer keeps one register prev, forms win = {current beat, prev} and takes
win >> (sel × 8). If the tail bytes are already all in prev after the last read
beat, one extra flush word is emitted. Only the first and last write word are
masked: WSTRB[l] = (l ≥ d) on the first, (l ≤ (d + n − 1) % 4) on the last.
sel and the case bit are computed when the AR is issued and travel in the rq
descriptor, so the R → W path is just a byte shifter.
Example — s = 1, d = 3, n = 7 bytes (b0…b6): 2 read beats, 3 write beats, sel = 2.
| Step | Read beat (lanes 3‥0) | Window shifted by 2 B | W word (lanes 3‥0) | WSTRB |
|---|---|---|---|---|
| R0 arrives | b2 b1 b0 · |
{R0, –} |
b0 · – – |
1000 |
| R1 arrives | b6 b5 b4 b3 |
{R1, R0} |
b4 b3 b2 b1 |
1111 |
| flush | — | {0, R1} |
– – b6 b5 |
0011 |
All bursts use ID 0, so AXI guarantees R and B return in issue order. Three small FIFOs stay in lock-step with the bus:
rq— pushed on AR handshake, popped when the packer finishes that burst's dataawq— pushed on AR handshake, popped on AW handshakebq— pushed on AW handshake with the channel id, popped on B → tells the channel one chunk is done
Each channel counts chunks in flight (pend). It finishes when all bytes are issued
(or an error stopped it) and pend == 0 — i.e. every write has its B response.
Global limits: rd_cnt < MAX_OUTS (outstanding reads) and infl_cnt < 2·OUTS_MAX
(chunks between AR issue and B) gate new issues, which also guarantees the descriptor
queues can never overflow (checked by assertions in dma_fifo).
| Event | Action |
|---|---|
RRESP = SLVERR / DECERR |
Channel ERR + RD_ERR + response code; that beat and every later W beat of the channel are sent with WSTRB = 0 |
BRESP = SLVERR / DECERR |
Channel ERR + WR_ERR + response code |
| After an error | Channel stops issuing new chunks, waits for in-flight ones (AXI requires every AW to get its W beats), then raises the error interrupt instead of done |
| Other channels | Unaffected (tested: a healthy channel runs alongside all four error types) |
| Recovery | A new START clears the status and the channel works normally (tested) |
Three changes came out of timing closure (details in §5.6):
// (1) chunk size is registered; a channel skips one request cycle after each grant
always_ff @(posedge clk) begin
n_q <= n_comb; // 5-way min computed from the current state
n_vld <= !(grant || load); // stale for the cycle after a state change
end
assign req = busy_q && n_vld && (rem_q != 0) && !err_q;
// (2) carry-select address update: n < 2^13, so only a 13-bit adder is needed
wire [13:0] src_lo = {1'b0, src_q[12:0]} + {1'b0, n_q};
wire [AW-14:0] src_hi1 = src_q[AW-1:13] + 1'b1; // in parallel
wire [AW-1:0] src_nx = {src_lo[13] ? src_hi1 : src_q[AW-1:13], src_lo[12:0]};
// (3) adder-free burst limit: burst_bytes is a power of two ≥ BPB, offset < BPB
assign mx_off = (s_off > d_off) ? s_off : d_off;
assign lim_bl = (mx_off == '0) ? blb_q : {blbm_q, OFFW'(~mx_off + 1'b1)};| Offset | Name | Access | Bits |
|---|---|---|---|
0x000 |
ID |
RO | 0x444D4131 ("DMA1") |
0x004 |
HWCFG |
RO | [31:24] OUTS_MAX · [23:16] log2(BL_MAX) · [15:8] bytes/beat · [7:0] NCH |
0x008 |
IRQ_STATUS |
W1C | bit 2c = channel c done · bit 2c+1 = channel c error |
0x00C |
IRQ_ENABLE |
RW | same layout; 0 = masked |
0x010 |
MAX_OUTS |
RW | outstanding read bursts; writes clamped to 1 … OUTS_MAX (reset = OUTS_MAX) |
0x100 + 0x20·c |
SRC |
RW | source byte address (any alignment) |
+0x04 |
DST |
RW | destination byte address (any alignment) |
+0x08 |
LEN |
RW | length in bytes; 0 completes immediately |
+0x0C |
CTRL |
RW | [0] START (write 1, self-clearing, reads 0) · [7:4] BURST_LOG2 (beats = 2ⁿ, clamped to BL_MAX) |
+0x10 |
STATUS |
RO | [0] BUSY · [1] DONE · [2] ERR · [3] RD_ERR · [4] WR_ERR · [9:8] RESP of the error |
+0x14 |
REMAIN |
RO | bytes not yet issued |
Any other address returns SLVERR (reads return 0).
#define DMA ((volatile uint32_t *)DMA_BASE)
#define REG(off) DMA[(off) >> 2]
#define CH(c, off) REG(0x100 + 0x20 * (c) + (off))
void dma_init(void) {
REG(0x00C) = 0xFF; // IRQ_ENABLE: done + error for 4 channels
REG(0x010) = 4; // MAX_OUTS
}
void dma_copy(int c, uint32_t src, uint32_t dst, uint32_t len) {
CH(c, 0x00) = src;
CH(c, 0x04) = dst;
CH(c, 0x08) = len;
CH(c, 0x0C) = (4u << 4) | 1u; // 16-beat bursts, START
}
void dma_isr(void) {
uint32_t st = REG(0x008);
REG(0x008) = st; // write-1-to-clear what we are about to handle
for (int c = 0; c < 4; c++) {
if (st & (1u << (2 * c))) on_done(c);
if (st & (2u << (2 * c))) {
uint32_t s = CH(c, 0x10); // STATUS: RD_ERR/WR_ERR, RESP = (s >> 8) & 3
on_error(c, s);
}
}
}SRC/DST/LEN/CTRLare shadow registers: the channel copies them on START, so the next transfer can be programmed while one is running. START while BUSY is ignored.- An event and a W1C in the same cycle: set wins — no interrupt is ever lost (this exact case is covered by a directed test, see §4.4).
- Change
MAX_OUTSonly while all channels are idle. - Overlapping source/destination regions (within or across channels) are undefined.
┌──────────────────────────── tb_top.sv ────────────────────────────┐
directed ─▶ │ AXI4-Lite BFM ┌───────────┐ AXI4 ┌────────────────────┐ │
+ random │ (random AW/W order, ──▶│ dma_top │ ══════▶ │ axi_mem_model.sv │ │
tests │ random delays) └───────────┘ │ random READY, │ │
│ │ │ │ bubbles, latency, │ │
│ axi_checker.sv (SVA)┘ └ dma_cov.sv │ SLVERR / DECERR │ │
│ └────────────────────┘ │
│ every AR/AW ─▶ DPI-C tb/ref/dma_ref.c ◀─ completed transfers │
│ • burst planner: predicted (addr, len) per channel │
│ • golden memory: memmove replay, byte compare │
└────────────────────────────────────────────────────────────────────┘
| Component | What it does |
|---|---|
tb/axi_mem_model.sv |
256 KB AXI4 slave, unlimited outstanding, in-order. Per round randomises ARREADY / AWREADY / WREADY probability (30–100 %), RVALID / BVALID bubbles (0–40 %), read and write latency (1–45 cycles). 0x4000_0000 reads and 0x5000_0000 writes return SLVERR; anything outside memory returns DECERR. Accepts W before its AW. |
tb/ref/dma_ref.c |
C reference over DPI-C, written from the spec independently of the RTL. (1) Predicts each channel's exact AR and AW burst sequence → the testbench matches every burst on the bus. (2) Keeps a golden memory image; each completed transfer is replayed with memmove; the destination is compared after every transfer and the whole 256 KB at the end. |
tb/axi_checker.sv |
AXI protocol assertions (below). |
tb/dma_cov.sv |
34 SVA cover property points (below). |
tb/tb_top.sv |
AXI4-Lite BFM, interrupt-driven completion handling, directed tests, random regression, performance mode. TB drives and samples on the negative edge to avoid races with the DUT. |
tb/sim_main.cpp |
Verilator harness (timing mode) that writes coverage.dat. |
AXI SVA (axi_checker.sv)
| Property | Checks |
|---|---|
ap_ar_hold ap_aw_hold ap_w_hold ap_r_hold ap_b_hold |
VALID stays high and payload stays stable until READY (all five channels) |
ap_ar_4k ap_aw_4k |
No burst crosses a 4 KB boundary |
ap_ar_fmt ap_aw_fmt |
INCR, full-width size, aligned address, len < BL_MAX |
ap_wlast_beat |
WLAST on exactly beat AWLEN (works whether W arrives before or after AW) |
ap_w_maxlen |
No W burst longer than BL_MAX without WLAST |
ap_rlast, ap_r_unexpected |
RLAST matches ARLEN; no R without an outstanding AR |
ap_outs_limit |
Outstanding read bursts never exceed MAX_OUTS |
ap_no_overflow ap_no_underflow (RTL) |
Descriptor / data FIFOs never overflow or underflow |
Per transfer: STATUS bits and RESP code, REMAIN = 0 on success, all planned bursts issued, destination bytes equal the golden image. At the end: whole-memory compare, zero protocol errors seen by the memory model.
Directed tests
| Test | Covers |
|---|---|
t_csr |
ID/HWCFG, byte-strobe writes, unmapped → SLVERR, MAX_OUTS clamp, CTRL read-back |
t_zero_len |
LEN = 0 completes with no bus traffic |
t_basic |
Aligned copy on every channel |
t_4k |
4 KB crossings on src, dst, both, at different points |
t_align_sweep |
4 src offsets × 4 dst offsets × 13 lengths (1…65 B) |
t_burst_sweep |
Every burst length 1 … BL_MAX |
t_errors |
RD SLVERR, RD DECERR, WR SLVERR, WR DECERR alongside a healthy channel; destination untouched on read errors; recovery afterwards |
t_irq_mask |
Pending-while-masked, unmask → irq, W1C → irq low |
t_irq_race |
Completion events landing in the same cycle as W1C writes |
t_shadow |
Reprogram + START while busy (ignored), then run the shadowed transfer |
| random rounds | Random MAX_OUTS and memory behaviour, 1–4 concurrent channels, 1 B – 12 KB, random offsets / burst sizes, ~8 % error transfers |
Regression (make regress): 8 seeds × (directed suite + 60 random rounds), all PASS.
| Total over 8 seeds | |
|---|---|
| Successful transfers | 3,667 |
| Expected-error transfers | 153 |
| AR bursts matched against the C planner | 525,291 |
| Simulated cycles | 5.9 M |
Per-seed results
seed 1: PASS transfers ok=457 expected-errors=22 AR=51470 AW=51470 R beats=108659 W beats=108670
seed 2: PASS transfers ok=463 expected-errors=19 AR=78625 AW=78625 R beats=148135 W beats=148129
seed 3: PASS transfers ok=443 expected-errors=18 AR=52044 AW=52044 R beats=114532 W beats=114527
seed 4: PASS transfers ok=461 expected-errors=19 AR=60719 AW=60719 R beats=136717 W beats=136733
seed 5: PASS transfers ok=461 expected-errors=21 AR=88218 AW=88218 R beats=152454 W beats=152455
seed 6: PASS transfers ok=466 expected-errors=20 AR=51921 AW=51921 R beats=122710 W beats=122709
seed 7: PASS transfers ok=473 expected-errors=17 AR=73290 AW=73290 R beats=161925 W beats=161915
seed 8: PASS transfers ok=443 expected-errors=17 AR=69004 AW=69004 R beats=142505 W beats=142512
(W beats can exceed R beats by the realignment flush words.)
Other parameter configurations (scripts/run_configs.sh, 3 seeds each, all PASS):
| NCH | BL_MAX | OUTS_MAX | Coverage |
|---|---|---|---|
| 2 | 2 | 1 | 27 / 27 (1 point waived: W buffer can never fill — unreachable by construction) |
| 3 | 64 | 8 | 34 / 34 |
| 4 | 256 | 2 | 38 / 38 |
Functional coverage (8 seeds merged — results/coverage_report.txt): 34 / 34
| Group | Cover points | Min hits |
|---|---|---|
| Per channel | cp_done[ch0‥3], cp_error[ch0‥3] |
25 |
| 4 KB split was the limiting factor | cp_split4k |
1,026 |
| Alignment | head / tail / both partial WSTRB, realign s>d, s<d, tail flush, poisoned beat | 415 |
| Burst length | 1 / 2 / 4 / 8 / 16 beats | 11,067 |
| Error responses | RRESP SLVERR / DECERR, BRESP SLVERR / DECERR | 110 |
| Concurrency | all channels busy, outstanding at hardware max | 133,084 |
| Back-pressure | AR / AW / W stalled by slave, R stalled by DMA | 147,127 |
| Interrupts | pending while masked, W1C clear, event racing W1C | 76 |
Coverage only says the scenarios you thought of were exercised. To show the checkers
actually fail on bugs, make mutants injects one bug at a time into a copy of the RTL
and requires the run to fail:
| Mutant | Bug | Caught by | First message |
|---|---|---|---|
no_src_4k_split |
source-side 4 KB limit removed | SVA ap_ar_4k |
u_chk.ap_ar_4k |
wlast_off_by_one |
WLAST missing on 1-beat bursts | memory-model WLAST check / SVA | u_mem … WLAST mismatch |
head_strobe_unmasked |
head bytes written when s = d | C reference memory compare | final memory compare: 65 bytes differ |
no_error_poisoning |
read-error data written to memory | C reference memory compare | final memory compare: 6 bytes differ |
outstanding_off_by_one |
< → <= on the outstanding limit |
SVA ap_outs_limit |
u_chk.ap_outs_limit |
arvalid_dropped |
ARVALID dropped before ARREADY | SVA ap_ar_hold |
u_chk.ap_ar_hold |
irq_event_lost_on_w1c |
event lost when it coincides with W1C | t_irq_race |
irq_race 4: ch0 DONE lost |
arbiter_fixed_priority |
arbiter pointer never advances | formal bounded-wait assertion | arb_fairness.sv:78 |
The first run caught only 7 / 8. The IRQ mutant escaped: random traffic never lined a completion event up with the exact cycle of a W1C write, and no cover point measured it. That was a real coverage hole. Adding
cp_irq_ev_on_w1cand thet_irq_racedirected test (48 short transfers while hammering IRQ_STATUS with W1C writes) hit the race 76 times and the mutant is now caught.
formal/arb_fairness.sv + arb_fairness.sby (SymbiYosys). The environment may drive
req and en arbitrarily, except that a request is held until it is granted.
Liveness ("eventually granted") is replaced by a stronger, provable safety property — bounded wait: while channel i is waiting, at most N − 1 grants go to other channels. An inductive invariant makes the proof go through:
wait_cnt[i] + dist(ptr → i) ≤ N − 1
(every grant to someone else moves the priority pointer at least one step closer to i).
| Task | Engine | Result |
|---|---|---|
prove_n2 prove_n3 prove_n4 prove_n8 |
ABC PDR (unbounded) | PASS |
cover_n4 |
smtbmc z3 | bound N − 1 reached in 5–6 steps → the bound is tight |
bmc_n4 |
smtbmc z3, depth 12 | PASS (and FAIL with a trace for the fixed-priority mutant) |
Also proved: grant is one-hot, only to requesters, and work-conserving.
Step-by-step guide for the USC servers: syn/RUN_DC_PT.md ·
full report: results/DC_PT_RESULTS.md.
| Library | FreePDK45, OSU gscl45nm (typical, 1.1 V); the library's seven max_capacitance : 0 pins are removed |
| Synthesis | Design Compiler Ultra Y-2026.03, compile_ultra -no_autoungroup (hierarchy kept) |
| STA / power | PrimeTime Y-2026.03-SP2, PrimePower (averaged, SAIF) |
| Clock | 2.0 ns (500 MHz), 0.1 ns uncertainty, ideal (pre-CTS) |
| I/O | input / output delay 20 % of the period, INVX1 drive, INVX4 input-pin load |
| Reset | rst_n ideal network + false path |
| Cross-check | Yosys 0.33 + ABC + OpenSTA with identical constraints (make syn) |
| Flow | Setup WNS | Hold WNS | Cell area | Flops | Cells | Power |
|---|---|---|---|---|---|---|
| DC + PrimeTime | +0.016 ns | +0.012 ns | 52,346.7 µm² | 2,810 | 13,956 | 30.4 mW |
| Yosys + ABC + OpenSTA | +0.342 ns | +0.009 ns | 62,773.6 µm² | 2,894 | 16,879 | 39.1 mW (vectorless) |
0 setup / 0 hold violations, 0 max-transition / max-capacitance violations. DC and PT setup slack agree to < 1 ps. With identical constraints DC turns the spare slack into 16.6 % less area (area recovery).
| Block | Area (µm²) | Share |
|---|---|---|
u_eng/u_wf — W data buffer (37 b × 32) |
18,692 | 35.7 % |
u_csr — registers |
9,502 | 18.2 % |
u_eng/u_awq — AW descriptors (38 b × 8) |
4,636 | 8.9 % |
u_eng/u_rq — realign descriptors (18 b × 8) |
2,436 | 4.7 % |
u_eng/u_bq — channel ids (2 b × 8) |
452 | 0.9 % |
u_eng/u_arb — arbiter |
87 | 0.2 % |
g_ch[c].u_ch — each channel (×4) |
3,269 | 6.2 % each |
| Total | 52,347 |
The four flop-based queues are half the design; the W buffer alone is a third.
Activity: a 4-channel concurrent copy at 98 % read-bus utilisation (make vcd),
converted with vcd2saif and annotated through DC's SAIF name map (97.9 % of flops,
100 % of inputs annotated).
| Group | Internal | Switching | Leakage | Total | Share |
|---|---|---|---|---|---|
| Clock network | 27.6 mW | 0 | 0 | 27.6 mW | 90.6 % |
| Registers | 0.545 mW | 0.100 mW | 0.216 mW | 0.861 mW | 2.8 % |
| Combinational | 0.823 mW | 1.132 mW | 0.059 mW | 2.015 mW | 6.6 % |
| Total | 28.9 mW | 1.23 mW | 0.275 mW | 30.4 mW |
By block: u_wf 11.6 mW (38 %), u_csr 6.37 mW (21 %), u_awq 2.48 mW, each channel 1.59 mW.
2,810 un-gated flops toggle their clock pin every cycle, while data-path switching is
only 1.2 mW even with the bus 98 % busy → clock gating is the clear next step.
The testbench clock is 10 ns. PrimeTime derives toggle rates from the dump's total time, so the VCD is re-timed to a 2 ns cycle first (
syn/pt/dma_500m.vcd: every timestamp ÷ 5, value changes identical). The raw 100 MHz dump reports 6.4 mW — ~5× too low.read_vcd -rtlannotates only ~40 % of flops (RTL vs. DC names); SAIF with the name map reaches 97.9 %.
g_ch[2].u_ch/rem_q_reg[13]
→ remain != 0 (15-level NOR3/NAND3 chain after area recovery) 0.00 → 0.85 ns
→ req → round-robin arbiter 0.85 → 1.03 ns
→ grant 1.03 → 1.09 ns
→ next-src mux / carry-select add → g_ch[1].u_ch/src_q_reg[29] 1.09 → 1.81 ns
required 1.83 ns → slack +0.016 ns (MET)
Registering a remain_nz flag alongside rem_q would make req a flop output and take
~0.8 ns off this path — the first step before synthesising at 1.8 ns for P&R.
Open-source flow, 2.0 ns target, before the DC run:
| Step | Change | Setup WNS |
|---|---|---|
| 0 | First netlist | −5.409 ns |
| 1 | Buffer trees on high-fanout flop outputs — ABC leaves the W-buffer read pointer driving ~600 pins (DC fixes this itself); syn/open/fanout_buf.py |
−1.412 ns |
| 2 | Register the chunk size — the 5-way min over 13-bit limits leaves the grant cycle; a channel skips one request cycle after a grant (round-robin fills it) | −0.165 ns |
| 3 | Realignment sel / case precomputed at AR issue; carry-select src/dst ± n (13-bit add + parallel ±1 of the upper bits) |
−0.197 ns * |
| 4 | Adder-free burst limit (blb − max(s_off, d_off) is a bit-splice because blb is a power of two) + balanced min tree |
+0.254 ns |
| 5 | Store only the non-zero bits of blb − BPB |
+0.342 ns |
* the critical path moved to the burst-limit logic, which step 4 removed.
Setup. Fixed 32-cycle read and write latency, no back-pressure, one 32 KB aligned
copy per point, 32-bit bus. Every point is a real RTL simulation with the data checked.
Utilisation = R-channel data beats ÷ cycles from START to completion. make perf.
| outstanding \ burst | 1 beat | 2 beats | 4 beats | 8 beats | 16 beats |
|---|---|---|---|---|---|
| 1 | 2.9 % (3) | 5.6 % (6) | 10.5 % (11) | 19.0 % (19) | 32.0 % (32) |
| 2 | 5.7 % (6) | 11.1 % (11) | 21.0 % (21) | 38.0 % (38) | 63.7 % (64) |
| 4 | 11.4 % (11) | 22.2 % (22) | 42.0 % (42) | 75.8 % (76) | 99.1 % (100) |
| 8 | 22.8 % (23) | 44.3 % (44) | 83.7 % (84) | 99.1 % (100) | 99.1 % (100) |
At 500 MHz the 32-bit bus peaks at 2 GB/s: 57 MB/s (1 × 1) up to 1,983 MB/s.
A read burst occupies the read pipeline for L + BL + c cycles and N of them can be
in flight, so
util = min(1, N · BL / (L + BL + c))
The bus saturates once the bytes in flight cover the latency × bandwidth product. Fitting
c on the N = 1 column gives c = 2.03 cycles (issue register + descriptor release);
the model then reproduces all 20 points within ~1 %.
Latency sensitivity — smallest N reaching ≥ 90 %:
| Latency (cycles) | BL = 4 | BL = 16 |
|---|---|---|
| 8 | N = 4 | N = 2 |
| 16 | N = 8 | N = 2 |
| 32 | > 8 (best 84 %) | N = 4 |
| 64 | > 8 (best 45 %) | N = 8 |
| 128 | > 8 (best 24 %) | > 8 (best 85 %) |
Each hardware configuration synthesised (open-source flow, 500 MHz, all timing-clean) and run at its own limits (N = OUTS_MAX, BL = BL_MAX), latency 32:
Full table (12 configurations)
| OUTS_MAX | BL_MAX | Area (µm²) | Flops | WNS (ns) | Utilisation | MB/s | Pareto |
|---|---|---|---|---|---|---|---|
| 1 | 4 | 39,109 | 1,620 | +0.409 | 10.5 % | 210 | ✔ |
| 2 | 4 | 41,236 | 1,733 | +0.381 | 21.0 % | 421 | ✔ |
| 1 | 8 | 44,728 | 1,928 | +0.409 | 19.0 % | 381 | |
| 4 | 4 | 45,318 | 1,946 | +0.409 | 42.0 % | 840 | ✔ |
| 2 | 8 | 47,096 | 2,047 | +0.409 | 38.0 % | 760 | |
| 4 | 8 | 51,405 | 2,272 | +0.403 | 75.8 % | 1,515 | ✔ |
| 8 | 4 | 52,992 | 2,359 | +0.288 | 83.7 % | 1,673 | ✔ |
| 1 | 16 | 55,823 | 2,532 | +0.407 | 31.9 % | 639 | |
| 2 | 16 | 58,246 | 2,657 | +0.392 | 63.7 % | 1,275 | |
| 8 | 8 | 60,200 | 2,709 | +0.271 | 99.1 % | 1,983 | ✔ |
| 4 | 16 | 62,774 | 2,894 | +0.342 | 99.1 % | 1,983 | |
| 8 | 16 | 71,254 | 3,355 | +0.217 | 99.1 % | 1,983 |
- Outstanding requests buy bandwidth more cheaply than longer bursts. A longer burst doubles the W buffer (2 · BL words × 37 bits); one more outstanding read only adds a few descriptor entries. 8 outstanding × 8-beat bursts reach 99 % with 4 % less area than the default 4 × 16, and 8 × 4 already gives 84 % with 16 % less area.
- Long bursts coarsen arbitration. A 16-beat burst holds the read bus for 16 cycles, so latency-sensitive channels prefer shorter bursts with more outstanding reads.
- Latency moves the knee. At L = 64, BL = 16 needs N = 8; at L = 128 even 8 × 16 only reaches 85 % (the model asks for N ≥ 10).
- Single-channel issue bubble. With 1-beat bursts one channel tops out at 49.9 % (it can be re-granted only every other cycle because its chunk size is registered); with 4 channels round-robin fills the gap: 72.6 %, exactly the model's value.
| Tool | Tested version | Used for |
|---|---|---|
| Verilator | 5.020 (needs --timing) |
lint, simulation, coverage |
| g++ / make | GCC 13 | Verilator C++ build, DPI-C |
| Python 3 + matplotlib | 3.13 / 3.11 | coverage report, sweeps, plots |
| Yosys + ABC | 0.33 | open-source synthesis, formal front-end |
| SymbiYosys | tag yosys-0.33 (must match Yosys) |
formal |
| z3 | 4.8.12 | formal cover / BMC |
| OpenSTA | 3.1.0 (built with CUDD) | open-source STA |
| Synopsys DC / PT | Y-2026.03 / Y-2026.03-SP2 | sign-off (optional) |
On Ubuntu 24.04: apt install verilator yosys z3 python3-matplotlib; SymbiYosys and
OpenSTA from source (see docs/ISSUES_AND_FIXES.md §1).
The gscl45nm library is downloaded automatically by syn/open/run_open.sh.
make lint # Verilator -Wall → "LINT CLEAN"
make sim # one seed (SEED=3 ROUNDS=100 to change)
make regress # 8 seeds + merged coverage reportExpected end of a simulation log:
[T] random regression: 60 rounds
------------------------------------------------------------
transfers ok=457 expected-errors=22 AR=51470 AW=51470 R beats=108659 W beats=108670
max outstanding reads seen = 4 (OUTS_MAX=4)
cycles = 575450
TEST PASSED
------------------------------------------------------------
| Command | What it runs | Output |
|---|---|---|
make lint |
Verilator -Wall on the RTL |
console |
make sim |
one seed, directed + random | logs/sim_<seed>.log, logs/cov_<seed>.dat |
make regress |
8 seeds + coverage merge | logs/coverage_report.txt |
bash scripts/run_configs.sh |
3 other NCH / BL_MAX / OUTS_MAX configs | logs/coverage_<cfg>.txt |
make formal |
all SymbiYosys tasks | formal/arb_fairness_*/ |
make mutants |
8 mutants | logs/mutants.md |
bash scripts/verify_all.sh |
lint → regress → configs → formal → mutants | logs/verify_all.log |
make perf |
utilisation sweep | analysis/results.csv, tradeoff.md, plots |
bash analysis/area_sweep.sh then python3 analysis/area_tradeoff.py |
12 synthesised configs | analysis/area.csv, area_vs_util.png |
make syn |
Yosys + OpenSTA (PERIOD=2.0 NCH=4 BL_MAX=16 OUTS_MAX=4) |
syn/open/out/<tag>/summary.txt |
make vcd |
activity dump for PrimeTime | syn/pt/dma.vcd, syn/pt/dma_500m.vcd |
cd syn && ./run_dc_pt.sh 2.0 |
DC → PT (Synopsys environment) | syn/dc/reports/, syn/pt/reports/ |
Simulation plusargs: +verilator+seed+<n>, +rounds=<n>, +test=perf +lat=<cycles> +bl=<log2 beats> +outs=<n> +plen=<bytes> +pch=<channels>.
.
├── rtl/ synthesizable RTL (6 files)
├── tb/
│ ├── tb_top.sv testbench: BFM, directed tests, random regression, perf mode
│ ├── axi_mem_model.sv randomised AXI4 slave memory
│ ├── axi_checker.sv AXI protocol SVA
│ ├── dma_cov.sv functional coverage (cover properties)
│ ├── sim_main.cpp Verilator harness (coverage output, tracing)
│ └── ref/dma_ref.c DPI-C reference model
├── formal/ arb_fairness.sv + .sby
├── syn/
│ ├── dc/dc_syn.tcl Design Compiler
│ ├── pt/pt_sta.tcl PrimeTime STA + power
│ ├── pt/dma_500m.vcd activity for PrimeTime (re-timed to 2 ns)
│ ├── open/ Yosys + OpenSTA flow, fanout_buf.py
│ ├── run_dc_pt.sh DC → PT wrapper
│ └── RUN_DC_PT.md step-by-step DC/PT guide (zh-TW)
├── analysis/ sweep scripts, CSVs, plots
├── scripts/ coverage report, config regression, mutation testing, verify_all
├── results/ DC/PT report, coverage, regression, formal, mutation, open-flow STA
├── docs/ISSUES_AND_FIXES.md every problem met during the project and its fix (zh-TW)
└── Makefile
| Claim | Command | Where to look |
|---|---|---|
| Lint clean | make lint |
console |
| 3,820 transfers, all PASS | make regress |
logs/sim_*.log, results/regression_summary.txt |
| 34 / 34 coverage | make regress |
results/coverage_report.txt |
| Other configs pass | bash scripts/run_configs.sh |
results/configs_summary.txt |
| Arbiter proofs | make formal |
results/formal_summary.txt |
| 8 / 8 mutants | make mutants |
results/mutants.md |
| Utilisation table, model | make perf |
analysis/tradeoff.md, analysis/results.csv |
| Area vs bandwidth | analysis/area_sweep.sh + area_tradeoff.py |
analysis/area_tradeoff.md |
| Open-source timing / area | make syn |
results/open_syn_summary.txt, results/open_sta.rpt |
| DC / PT sign-off | syn/run_dc_pt.sh |
results/DC_PT_RESULTS.md |
| Decision | Why | Cost |
|---|---|---|
| One AR + one AW per chunk | Read and write sides stay in lock-step; one descriptor per burst; 4 KB and burst limits enforced in one place | With different src/dst offsets, consecutive chunks may re-read one boundary word |
| Single AXI ID (0) | AXI guarantees in-order R/B → three small FIFOs replace a reorder buffer | A slow slave blocks other channels (head-of-line) |
| 2-word window realigner | Any src/dst alignment with one register and a byte shifter | One flush cycle on some tails |
| Write poisoning after errors | Bad read data never lands in memory, AXI still gets its W beats | Bytes written before the error stay written |
| Registered chunk size | Removes the 5-way min from the grant cycle (timing) | Same channel re-granted at most every 2nd cycle (only visible with 1-beat bursts) |
| Round-robin at burst granularity | Provably starvation-free, cheap (87 µm²) | A long burst holds the bus for BL cycles |
| W buffer = 2·BL_MAX | Absorbs a full burst while the next one fills under write back-pressure | 35.7 % of area |
| Flop-based FIFOs | Portable, no memory compiler needed for FreePDK45 | Area and clock power — SRAM macro is the next step |
- Register
remain_nzsoreqis a flop output (~0.8 ns off the critical path) → synthesise at 1.8 ns - Clock gating on FIFO entries and CSR registers (clock network is 90.6 % of power)
- Replace the W buffer with an SRAM / register-file macro
- Multiple AXI IDs + reorder buffer (no head-of-line blocking)
- Keep a carry word between consecutive chunks of the same channel (no boundary re-read)
- Scatter-gather descriptor ring; AxCACHE / AxPROT / QoS outputs
- Generate the VCD at 2 ns directly in the testbench
- Place & route in Innovus (CTS, post-route sign-off)
Every issue met while building this — tool-chain problems, testbench bugs, coverage
gaps, formal set-up, timing closure, DC/PT sign-off — is written up with the error
message, root cause and fix in docs/ISSUES_AND_FIXES.md.
The most instructive ones:
- A DPI function called twice. Every AXI burst was reported as unexpected. The C model
was fine; Verilator was re-evaluating a non-
contextDPI import that has side effects. Fix:import "DPI-C" context function …and store the result before testing it. - 100 % coverage, one bug still escaped. Mutation testing showed an interrupt lost when a completion event coincides with a W1C write — a scenario no cover point measured.
- Tool limits vs. design limits. −5.4 ns was mostly unbuffered fanout that DC fixes on its own; the remaining −1.4 ns was real and needed microarchitecture changes.
- Power numbers are only as good as the activity. A 100 MHz testbench dump made PrimeTime report 5× too little power; SAIF + name mapping raised flop annotation from 40 % to 98 %.
Ian Chen — M.S. Electrical Engineering, University of Southern California · GitHub · LinkedIn


