We describe the construction of a class of general, end-to-end, image-based physics event classifiers that directly use simulated raw detector data to discriminate signal and background processes in collision events at the LHC. To better understand what such classifiers are able to learn and to address some of the challenges associated with their use, we attempt to distinguish the Standard Model Higgs Boson decaying to two photons from its leading backgrounds using high-fidelity simulated detector data from the 2012 CMS Open Data. We demonstrate the ability of end-to-end classifiers to learn from the angular distribution of the electromagnetic showers, their shape, and the energy scale of their constituent hits, even when the underlying particles are not fully resolved.