Abstract: Residential building energy consumption accounts for a significant portion of global energy use, making the development of home energy management systems (HEMS) a crucial pathway to reduce energy consumption. However, existing HEMS methods face notable limitations, i.e., model-based approaches rely on accurate system modeling or parameter prediction, while deep reinforcement learning-based methods, though circumventing such requirements, suffer from low training sample efficiency leading to sub-optimal policy performance. To address these issues, this paper proposes an online energy management method that integrates prediction and learning. First, an optimization problem minimizing operation costs under multi-source uncertainties and indoor thermal comfort constraints is formulated. Subsequently, the problem is reformulated as a Markov decision process (MDP). By embedding a model predictive control (MPC) framework into the HEMS agent training process, an implicit world model is constructed to capture system dynamics, enabling the agent to interact with it in a rolling manner and improve sample efficiency. After finishing the training process, the HEMS agent can realize online decisions without any parameter prediction. Simulation results demonstrate that, while maintaining indoor thermal comfort, the proposed method reduces the operation cost by 20.57% and 7.92% compared to the rule-based energy management method and the deep deterministic policy gradient -based energy management method, respectively.