The texture orientation is an important parameter to ensure the toughness and corrosion resistance of composite materials. Vision-based composite texture orientation inspection methods have been largely investigated because of their non-intrusiveness, low cost, and high accuracy. However, conventional vision-based methods suffer the issues of region-of-interest (ROI) texture region angle inspection failure in the presence of complex backgrounds and the low accuracy and poor inconsistency of inspection results. To address these issues, an innovative composite texture orientation inspection approach is designed based on the Hough neural network. In ROI segmentation, a channel attentional residual fusion network is designed to extract composite texture regions. Afterward, a coarse-to-fine orientation regression model is designed. It collects correct texture candidates and texture orientation regression from the selected candidates based on the Hough transform. The proposed method has been extensively evaluated on the collected composite texture angle inspection datasets. Simulation results and analysis verified the effectiveness of the proposed method.