Completed academic investigation · University of Sunderland

Design and Evaluation of an Online Action-Value Learning-Based Memory-Management Controller for an 8-Bit Processor

A SystemVerilog hardware-design and verification study assessing how a small processor can select caching, bypassing and prefetching actions, handle delayed feedback and compare against fixed memory-control policies.

BEng final-year projectAcademic work completedFinal results pending

Research question and approach

Different memory access patterns change the relative benefit of retaining a cache entry, bypassing allocation or prefetching future data. The project implemented an 8-bit multi-cycle processor, a simplified memory hierarchy, and an online controller that stores bounded action preferences and updates them from subsequent memory outcomes.

The controller distinguished a preferred action from a hardware-permitted operation. Delayed-feedback records tracked which earlier decision was responsible for observed outcomes; independent testbenches checked both architectural correctness and update integrity. It is a reward-directed action-value controller, not a conventional Q-learning or SARSA implementation.

320,000Instruction-retirement checks per evaluation campaign
207,776Memory-action legality checks per campaign
16Controlled workload families under three execution arrangements

Selected experimental outcomes

The reset-isolated campaign evaluated the online design and three separate fixed-policy reference processors under common simulated response-delay settings.

Memory controlExecution cycles
Online action-value controller782,241
Fixed caching811,765
Fixed bypassing953,512
Fixed prefetching768,907

The online controller required 3.64% fewer cycles than fixed caching and 17.96% fewer than fixed bypassing; however, fixed prefetching was 1.73% faster than the online controller. Reported checks found no architectural, memory-action or feedback-integrity errors under the tested workloads.

Engineering contributions and limits

What was demonstrated

Working RTL integration, legality-qualified memory actions, traceable feedback ownership, independently checked reward-update arithmetic, fixed-policy benchmarks, and generic synthesis.

What was not established

A separate speed improvement attributable only to learned preferences, equivalence after SRAM adaptation, complete physical routing, timing closure, or readiness for fabrication.

This case study summarises the completed academic report without publishing personal student identifiers. The comparisons were made using synthetic workloads and differing reference hardware structures; the figures should not be generalised to commercial processors.

Methods and technologies

SystemVerilog · 8-bit processor architecture · Cache hierarchy · Online action-value updates · Delayed-feedback attribution · Verification testbenches · Yosys synthesis · SKY130 logic mapping · OpenROAD structural feasibility work