$\textbf{Purpose:}$ To develop a new method for free-breathing 3D extracellular volume (ECV) mapping of the whole heart at 3T. $\textbf{Methods:}$ A free-breathing 3D cardiac ECV mapping method was developed at 3T. T1 mapping was performed before and after contrast agent injection using a free-breathing ECG-gated inversion-recovery sequence with spoiled gradient echo readout. A linear tangent space alignment (LTSA) model-based method was used to reconstruct high-frame-rate dynamic images from (k,t)-space data sparsely sampled along a random stack-of-stars trajectory. Joint T1 and transmit B1 estimation was performed voxel-by-voxel for pre- and post-contrast T1 mapping. To account for the time-varying T1 after contrast agent injection, a linearly time-varying T1 model was introduced for post-contrast T1 mapping. ECV maps were generated by aligning pre- and post-contrast T1 maps through affine transformation. $\textbf{Results:}$ The feasibility of the proposed method was demonstrated using in vivo studies with six healthy volunteers at 3T. We obtained 3D ECV maps at a spatial resolution of 1.9$\times$1.9$\times$4.5 $mm^{3}$ and a FOV of 308$\times$308$\times$144 $mm^{3}$, with a scan time of 10.1$\pm$1.4 and 10.6$\pm$1.6 min before and after contrast agent injection, respectively. The ECV maps and the pre- and post-contrast T1 maps obtained by the proposed method were in good agreement with the 2D MOLLI method both qualitatively and quantitatively. $\textbf{Conclusion:}$ The proposed method allows for free-breathing 3D ECV mapping of the whole heart within a practically feasible imaging time. The estimated ECV values from the proposed method were comparable to those from the existing method. $\textbf{Keywords:}$ cardiac extracellular volume (ECV) mapping, cardiac T1 mapping, linear tangent space alignment (LTSA), manifold learning