Ptychography is an attractive advance of coherent diffraction imaging (CDI), which can provide high lateral resolution and wide field of view. The theoretical resolution of ptychography is dose-limited, therefore making ptychography workable with a broadband source will be highly beneficial. However, broad spectra of light source conflict with the high coherence assumption in CDI that the current reconstruction algorithm were built upon. In this paper, we demonstrated that incorporation of a blind deconvolution in the reconstruction algorithm can improve the image quality of ptychography with broadband source. This broadband reconstruction algorithm can obtain high-quality amplitude and phase images of complex-valued samples requiring no knowledge of the illumination spectrum. Optical experiments using biological samples demonstrate the effectiveness of our method. The significant improvement in low coherence tolerance by our approach can pave the way for implementing ultrafast imaging with femtosecond or attosecond lasers or high-flux ptychographic imaging with laboratory EUV or X-ray sources.