Abstract: A device-to-device (D2D) communication-empowered nonorthogonal multiple access system is associated with complex co-channel interference. In this study, we optimize both mode selection and power control in order to maximize the sum proportional bit rate to balance spectral efficiency and user fairness. Accordingly, we propose a hybrid offline-online mechanism to cope with the original mixed-integer non-convex optimization problem. In offline training, variable transformation is used to equivalently transform the power control subproblem into a convex optimization problem. Its global optimum can be readily obtained in milliseconds by using the sophisticated convex optimization toolbox. According to the obtained optimization results, the deep Q-learning algorithm is then applied to build up the mapping relationship from the mode selection scheme and channel state information to the optimal mode adjustment policy. The trained resource allocation mechanism is suitable for online implementation as it involves only simple algebraic operations and a single convex optimization problem. The simulation results show that the proposed mechanism strikes a good balance between performance and operation time. Particularly, it cuts down the average operation time by 94.54% while suffering approximately 10% performance loss compared with the global optimum obtained by the exhausting search.