Online Comfort-Constrained HVAC Control via Feature Transfer

This repository contains the code for the paper "Online Comfort-Constrained HVAC Control via Feature Transfer" by Jing Yu, Tianyu Zhang, Omid Ardakanian, and Adam Wierman. The code is based on EnergyPlus 9.3.0 and Python 3.11. The code is tested on Ubuntu 18.04.5 LTS.

Summary

We propose a novel policy transfer method for HVAC systems, aimed at reducing energy consumption while ensuring thermal comfort constraints are met. By adapting nonlinear features learned from source buildings through efficient online learning, our approach significantly reduces thermal comfort violations. Evaluations using EnergyPlus across 19 climate locations show an average reduction of 81.28% in constraint violations compared to baseline policies, with an acceptable 11.23% increase in energy usage.

Prerequisites

Make sure to have EnergyPlus 9.3.0 installed on your computer. The code is not compatible with other version of EnergyPlus.

Repository Structure

The repository is organized as follows:

• eplus_files/: Contains the EnergyPlus files for the two buildings used in the project, and the weather files for the 19 climate locations. These weather files are sourced from the EnergyPlus Weather Data, and the selected cities follow recommendations by the US Department of Energy to cover diverse climate conditions.
• models/: Contains the trained models from the source buildings. We provide the pretrained ICNN models from the source buildings in this folder. Damper control models are not provided. These control policies are from our previous work Mitigating an Adoption Barrier of Reinforcement Learning-based Control Strategies in Buildings.
• algorithm_helper.py: Contains helper functions for the optimization algorithms used in the project.
• experiment_xxx.py: Contains the main code for running the experiments.
• icnn.py: Contains the implementation of the ICNN model.
• local_setting.py: Configuration file where you can set local variables and paths.
• ppo.py: Contains the implementation of the PPO algorithm.
• README.md: This file, providing an overview and instructions for the project.
• requirements.txt: Contains the required Python packages for the project.
• submit_ray.py: Contains the code for submitting Ray jobs to the cluster.
• utils.py: Contains utility functions for the project.

Environment Setup

This project requires Python 3.11. To install the required packages, run the following command:

pip install -r requirements.txt

We utilize Ray to efficiently parallelize our experiments, enabling distributed computing across multiple instances. Detailed instructions and documentation can be found on the Ray documentation page.

For solving optimization problems within our experiments, we employ Mosek and Gurobi. Installation guidelines are available on their respective websites. Please ensure that appropriate licenses for both Mosek and Gurobi are acquired. Additionally, note that using Ray to distribute computational tasks across multiple instances may require obtaining multiple licenses, depending on your specific licensing agreements.

We also leverage COBS to programmatically execute rule-based control scenarios and train our reinforcement learning agents. More information about COBS, including setup and usage instructions, can be accessed through the COBS GitHub repository.

Please ensure all dependencies are correctly installed and licensed prior to run your experiments.

Running Experiments

1. Set the local variables in local_setting.py to match your local environment.
2. Initialize the Ray cluster.
3. Use submit_ray.py to submit the experiments to the Ray cluster. Please review the code and modify the parameters as needed. For specific arguments to set for each experiment, please refer to the help page for the corresponding code. You can find the help page by running python3 experiment_xxx.py -h.
4. Monitor the progress of the experiments using the Ray dashboard.