Abstract: In communication networks with multiuser MIMO(multiple-input multiple-output) amplify-and-forward relay, aiming at the transmission characteristics of two-way relay,a new optimal relay beamforming filter is proposed by applying minimum sum normalized mean square error(SNMSE) criterion.Combining the OFDM(orthogonal frequency division multiplexing) technique with the proposed SDMA(space division multiple access) scheme,the performance of the system can be improved by fully utilizing frequency diversity and space diversity.Simulation results show that the proposed two-way relay scheme outperforms other relay schemes in the performance of system sum rate and bit error ratio.