Controlling the Model "Residential Air Source Heat Pump" with AI

Hello,

https://de.mathworks.com/help/hydro/ug/residential-air-source-heat-pump.html

I downloaded the Model from MATLAB and made some changes, so i can cnotroll the Heatpump with AI. But the AI only truns the Heatpump all the Time on or Off. With one Agent the Heatpump is all the Time on and with another Agent it is all the Time off. Depending on the Reward function. But i need it to switch dynamically. So in one Simulation it needs to switch the Heat pump onn and off. So this Question is about the Reward-Function or some issues in my adaptations of the model.

[Sam Chak]

Hi @Henri,

I am making an educated guess! Is the heat pump strictly constrained to only turning ON and OFF? Similar to most roller shutters, which typically move at a fixed speed (Roll UP and Roll DOWN), their motors are designed for security rather than speed, using basic limit switches to stop at set points for reliability and cost-effectiveness.

In your residential heating system project, the temperature is not directly controlled by the heat pump, but rather by the centralized hot water tank, which circulates and distributes hot water throughout the housing's thermal network. The heat pump is likely designed similarly to a water heater in a home or hotel room, where its purpose is to heat the water in the tank as quickly as possible, rather than to operate as dynamically as you might wish.

[Henri]

Hi @Sam Chak,

thanks for the answer. Yes i turn the Heatpump only on and off like you described, but my AI only tuns the Heatpump ON in one Simulation and only OFF in another Simulation depending on the Agent and Rewardfunction i used for Training

[Sam Chak]

Hi @Henri

If you want to control the temperature dynamically, you must first remove the actuator’s physical constraints, and second, from a supervisory perspective, instruct the AI to “think” like a dynamic controller. This assumes that the AI in MATLAB understands how such a controller should behave, either stability-focused in the mathematical sense, or minimum-cost-guided in the dynamic-programming sense.