Most existing methods for Magnetic Resonance Imaging (MRI) reconstruction with deep learning assume that a high signal-to-noise ratio (SNR), fully sampled sampled dataset exists and use fully supervised training. In many circumstances, however, such a dataset does not exist and may be highly impractical to acquire. Recently, a number of self-supervised methods for MR reconstruction have been proposed, which require a training dataset with sub-sampled k-space data only. However, existing methods do not denoise sampled data, so are only applicable in the high SNR regime. In this work, we propose a method based on Noisier2Noise and Self-Supervised Learning via Data Undersampling (SSDU) that trains a network to reconstruct clean images from sub-sampled, noisy training data. To our knowledge, our approach is the first that simultaneously denoises and reconstructs images in an entirely self-supervised manner. Our method is applicable to any network architecture, has a strong mathematical basis, and is straight-forward to implement. We evaluate our method on the multi-coil fastMRI brain dataset and find that it performs competitively with a network trained on clean, fully sampled data and substantially improves over methods that do not remove measurement noise.