X-ray free-electron lasers (XFELs) as the world`s most brilliant light sources provide ultrashort X-ray pulses with durations typically on the order of femtoseconds. Recently, they have approached and entered the attosecond regime, which holds new promises for single-molecule imaging and studying nonlinear and ultrafast phenomena like localized electron dynamics. The technological evolution of XFELs toward well-controllable light sources for precise metrology of ultrafast processes was, however, hampered by the diagnostic capabilities for characterizing X-ray pulses at the attosecond frontier. In this regard, the spectroscopic technique of photoelectron angular streaking has successfully proven how to non-destructively retrieve the exact time-energy structure of XFEL pulses on a single-shot basis. By using artificial intelligence algorithms, in particular convolutional neural networks, we here show how this technique can be leveraged from its proof-of-principle stage toward routine diagnostics at XFELs, thus enhancing and refining their scientific access in all related disciplines.