To investigate the effect of hexogen(RDX)dopants produced during the preparation of octogen(HMX)on properties of HMX, four kinds of HMX models of doping rate as 4.17%, 8.33%, 12.50% and 16.67% were established. Molecular dynamic method was used to calculate the interaction energy of trigger bond, cohesive energy density, solubility parameter, detonation parameters and mechanical parameters of different models, and the results were compared with the related properties of pure HMX. Results show that RDX doping defect leads to the decrease of interaction energy of trigger bond and cohesive energy density, and the reduction amplitude is 9.53-36.36 kJ·mol^-1, 0.028-0.135 kJ·cm^-3, respectively. The difference between the solubility parameters of HMX explosives and F₂₃₁₁ decreases with the influence of RDX doping defect, and the reduction amplitude is 0.51-2.32. The decrease amplitude of density, detonation velocity, and detonation pressure are 1.12% -5.59%, 0.84% -4.19% and 2.27% -11.14%, respectively. The heat of detonation rises slightly, which is almost negligible.The RDX doping defect also leads to the decrease of elastic modulus, bulk modulus, and shear modulus of HMX, but Cauchy pressure, as well as the ratio of bulk modulus to shear modulus rises. The changes range is 1.04-3.63 GPa, 0.58-1.73 GPa, 0.42-1.45 GPa, 0.35-2.69 GPa, and 0.11-0.64, respectively. It reveals that with the increase of the concentration of RDX doping defects, the safety and detonation performance of HMX explosives decrease, the mechanical properties become worse, and the compatibility with F₂₃₁₁ becomes better.