Design and Evaluation of an Online Action-Value Learning-Based Memory-Management Controller for an 8-Bit Processor
A SystemVerilog hardware-design and verification study assessing how a small processor can select caching, bypassing and prefetching actions, handle delayed feedback and compare against fixed memory-control policies.
BEng final-year projectAcademic work completedFinal results pendingResearch question and approach
Different memory access patterns change the relative benefit of retaining a cache entry, bypassing allocation or prefetching future data. The project implemented an 8-bit multi-cycle processor, a simplified memory hierarchy, and an online controller that stores bounded action preferences and updates them from subsequent memory outcomes.
The controller distinguished a preferred action from a hardware-permitted operation. Delayed-feedback records tracked which earlier decision was responsible for observed outcomes; independent testbenches checked both architectural correctness and update integrity. It is a reward-directed action-value controller, not a conventional Q-learning or SARSA implementation.
Selected experimental outcomes
The reset-isolated campaign evaluated the online design and three separate fixed-policy reference processors under common simulated response-delay settings.
| Memory control | Execution cycles |
|---|---|
| Online action-value controller | 782,241 |
| Fixed caching | 811,765 |
| Fixed bypassing | 953,512 |
| Fixed prefetching | 768,907 |
The online controller required 3.64% fewer cycles than fixed caching and 17.96% fewer than fixed bypassing; however, fixed prefetching was 1.73% faster than the online controller. Reported checks found no architectural, memory-action or feedback-integrity errors under the tested workloads.
Engineering contributions and limits
What was demonstrated
Working RTL integration, legality-qualified memory actions, traceable feedback ownership, independently checked reward-update arithmetic, fixed-policy benchmarks, and generic synthesis.
What was not established
A separate speed improvement attributable only to learned preferences, equivalence after SRAM adaptation, complete physical routing, timing closure, or readiness for fabrication.
This case study summarises the completed academic report without publishing personal student identifiers. The comparisons were made using synthetic workloads and differing reference hardware structures; the figures should not be generalised to commercial processors.
Methods and technologies
SystemVerilog · 8-bit processor architecture · Cache hierarchy · Online action-value updates · Delayed-feedback attribution · Verification testbenches · Yosys synthesis · SKY130 logic mapping · OpenROAD structural feasibility work