Learning algorithms versus automatability of Frege systems

We connect learning algorithms and algorithms automating proof search in propositional proof systems: for every sufficiently strong, well-behaved propositional proof system $P$, we prove that the following statements are equivalent, 1. Provable learning: $P$ proves efficiently that p-size circuits are learnable by subexponential-size circuits over the uniform distribution with membership queries. 2. Provable automatability: $P$ proves efficiently that $P$ is automatable by non-uniform circuits on propositional formulas expressing p-size circuit lower bounds. Here, $P$ is sufficiently strong and well-behaved if I.-III. holds: I. $P$ p-simulates Je\v{r}\'abek's system $WF$ (which strengthens the Extended Frege system $EF$ by a surjective weak pigeonhole principle); II. $P$ satisfies some basic properties of standard proof systems which p-simulate $WF$; III. $P$ proves efficiently for some Boolean function $h$ that $h$ is hard on average for circuits of subexponential size. For example, if III. holds for $P=WF$, then Items 1 and 2 are equivalent for $P=WF$. If there is a function $h\in NE\cap coNE$ which is hard on average for circuits of size $2^{n/4}$, for each sufficiently big $n$, then there is an explicit propositional proof system $P$ satisfying properties I.-III., i.e. the equivalence of Items 1 and 2 holds for $P$.

Learning algorithms from circuit lower bounds

We revisit known constructions of efficient learning algorithms from various notions of constructive circuit lower bounds such as distinguishers breaking pseudorandom generators or efficient witnessing algorithms which find errors of small circuits attempting to compute hard functions. As our main result we prove that if it is possible to find efficiently, in a particular interactive way, errors of many p-size circuits attempting to solve hard problems, then p-size circuits can be PAC learned over the uniform distribution with membership queries by circuits of subexponential size. The opposite implication holds as well. This provides a new characterisation of learning algorithms and extends the natural proofs barrier of Razborov and Rudich. The proof is based on a method of exploiting Nisan-Wigderson generators introduced by Kraj\'{i}\v{c}ek (2010) and used to analyze complexity of circuit lower bounds in bounded arithmetic. An interesting consequence of known constructions of learning algorithms from circuit lower bounds is a learning speedup of Oliveira and Santhanam (2016). We present an alternative proof of this phenomenon and discuss its potential to advance the program of hardness magnification.