Soft Actor Critic with State Abstraction for Random Task Offloading Strategy in Edge Computing

To enhance the performance of the mobile edge computing offloading algorithm in terms of computational delay, computational energy consumption, and algorithmic adaptability, we construct task and communication models under an end-edge cooperative architecture and improve the offloading model. We design a novel action function for proportional offloading and optimize the reward function by incorporating a negative penalty mechanism. These enhancements effectively strengthen the model's ability to characterize the real environment, thereby providing a more reasonable objective for deep reinforcement learning algorithms to find the optimal policy through cumulative rewards. Specifically, to address the sparse reward problem of the soft actor critic (SAC) algorithm in the offloading model, we propose a state abstraction-enhanced SAC (SACSA) strategy for proportional task offloading in edge computing. Through simulation experiments, we verify the effectiveness of the proportional offloading action function and the reward function with the negative penalty mechanism. We conduct a comparative performance analysis with SAC, neural episodic control with state abstraction (NECSA), proximal policy optimization (PPO), and twin delayed deep deterministic policy gradient (TD3). The results demonstrate that our proposed SACSA algorithm achieves superior performance across diverse scenarios: it reduces task latency by 1.64%~85.35%, improves the task completion rate by 0.55%~69.64%, and increases the episode reward by 0.53%~75.8%.