Machine learning agents are becoming more and more advanced, achieving higher levels of performance across a wide range of tasks. However, despite this progress, there still is a wide variety of limitations. One of those limitations is the ability to adapt to changing environments.
In the real world, rules and patterns often change over time, requiring continuous adaptation. Humans naturally cope with such changes, but most machine learning agents struggle to do so. They rely on fixed architectures and predefined hyperparameters, which are determined by the engineer at design time. This means that adapting to changes in their environment requires manual intervention by the engineer.
To address this limitation, this thesis proposes a self-improvement loop that enables agents to monitor their performance autonomously and, if necessary, modify their architecture. The goal is to create reinforcement learning agents that can not only learn from experience but also evolve their internal structure to remain effective in dynamic and unpredictable environments. The system will be evaluated in an evolving environment where new observations and actions are introduced over time.
A meta-controller determines when adaptation is needed and selects architectural modifications such as adding layers or neurons. The main idea behind this is that enabling online architectural adaptation for RL agents will allow the agents to recover more quickly from performance degradation and remain effective in non-stationary environments. By enabling agents to evolve their internal structure while learning, this research aims to take a step toward more robust, adaptable, and self-improving reinforcement learning systems.
Towards Self-Evolving Reinforcement Learning Architectures