Markov jump processes are widely used to model interacting species in circumstances where discreteness and stochasticity are relevant. Such mod- els have been particularly successful in computational cell biology, and in this case, the interactions are typically first-order. The Chemical Langevin Equation is a stochastic dierential equation that can be regarded as an approximation to the underlying jump process. In particular, the Chemi- cal Langevin Equation allows simulations to be performed more effectively. In this work, we obtain expressions for the first and second moments of the Chemical Langevin Equation for a generic first-order reaction network. Moreover, we show that these moments exactly match those of the under- lying jump process. Hence, in terms of means, variances and correlations, the Chemical Langevin Equation is an excellent proxy for the Chemical Master Equation. Our work assumes that a unique solution exists for the Chemical Langevin Equation. We also show that the moment matching re- sult extends to the case where a gene regulation model of Raser and O'Shea (Science, 2004) is replaced by a hybrid model that mixes elements of the Master and Langevin equations. We nish with numerical experiments on a dimerization model that involves second order reactions, showing that the two regimes continue to give similar results.