Prefetching Using Principles of Hippocampal-Neocortical Interaction
Prefetching memory accesses improves performance across many system layers. It influences the system throuput and the application turnaround times. However, as memory hierarchies and application memory access patterns become more complicated, achieving high prefetch accuracy with low storage and compute overhead becomes more difficult. Additionally, with the emergence of new applications and the need for dynamic execution, memory management techniques that can learn memory access patterns must be developed rather than relying on hard-coded heuristics that cannot adapt to newer patterns. Recent works have used deep learning techniques to improve prefetching accuracy, although with huge compute and storage overheads. 
In this work, we propose drawing inspiration from the human cognitive learning model, specifically the hippocampus and neocortex (known as Complementary Learning System(CLS)), to create resource-efficient, accurate, and adaptable prefetchers. We investigate the overheads of existing deep-learning based prefetchers. We find that existing learning-based prefetchers have high storage and compute overheads and learn primarily offline. Even if we solve the offline learning challenge by using online learning, they still face the fundamental challenge of online learning - catastrophic forgetting. We discuss how CLS principles such as replay can be used to reduce catastrophic forgetting while techniques like hebbian learning can be used to reduce compute overheads. We outline the challenges involved in implementing these CLS-inspired techniques in real-world systems. Paper:
Michael Wu, Ketaki Joshi, Andrew Sheinberg, Guilherme Cox, Anurag Khandelwal, Raghavendra Pradyumna Pothukuchi, and Abhishek Bhattacharjee. 2023. Prefetching Using Principles of Hippocampal-Neocortical Interaction. In Proceedings of the 19th Workshop on Hot Topics in Operating Systems (HOTOS '23). Association for Computing Machinery, New York, NY, USA, 53–60. https://doi.org/10.1145/3593856.3595901