Learning, Generalization, and Functional Entropy in Random Automata Networks

It has been shown \citep{broeck90:physicalreview,patarnello87:europhys} that feedforward Boolean networks can learn to perform specific simple tasks and generalize well if only a subset of the learning examples is provided for learning. Here, we extend this body of work and show experimentally that random Boolean networks (RBNs), where both the interconnections and the Boolean transfer functions are chosen at random initially, can be evolved by using a state-topology evolution to solve simple tasks. We measure the learning and generalization performance, investigate the influence of the average node connectivity $K$, the system size $N$, and introduce a new measure that allows to better describe the network's learning and generalization behavior. We show that the connectivity of the maximum entropy networks scales as a power-law of the system size $N$. Our results show that networks with higher average connectivity $K$ (supercritical) achieve higher memorization and partial generalization. However, near critical connectivity, the networks show a higher perfect generalization on the even-odd task.

Emergent Criticality Through Adaptive Information Processing in Boolean Networks

We study information processing in populations of Boolean networks with evolving connectivity and systematically explore the interplay between the learning capability, robustness, the network topology, and the task complexity. We solve a long-standing open question and find computationally that, for large system sizes $N$, adaptive information processing drives the networks to a critical connectivity $K_{c}=2$. For finite size networks, the connectivity approaches the critical value with a power-law of the system size $N$. We show that network learning and generalization are optimized near criticality, given task complexity and the amount of information provided threshold values. Both random and evolved networks exhibit maximal topological diversity near $K_{c}$. We hypothesize that this supports efficient exploration and robustness of solutions. Also reflected in our observation is that the variance of the values is maximal in critical network populations. Finally, we discuss implications of our results for determining the optimal topology of adaptive dynamical networks that solve computational tasks.

Emergent Network Structure, evolvable Robustness and non-linear Effects of Point Mutations in an Artificial Genome Model

Genetic regulation is a key component in development, but a clear understanding of the structure and dynamics of genetic networks is not yet at hand. In this paper we investigate these properties within an artificial genome model originally introduced by Reil (1999). We analyze statistical properties of randomly generated genomes both on the sequence- and network level, and show that this model correctly predicts the frequency of genes in genomes as found in experimental data. Using an evolutionary algorithm based on stabilizing selection for a phenotype, we show that dynamical robustness against single base mutations, as well as against random changes in initial states of regulatory dynamics that mimic stochastic fluctuations in environmental conditions, can emerge in parallel. Point mutations at the sequence level have strongly non-linear effects on network wiring, including as well structurally neutral mutations and simultaneous rewiring of multiple connections, which occasionally lead to strong reorganization of the attractor landscape and metastability of evolutionary dynamics. Evolved genomes exhibit characteristic patterns on both sequence and network level.

Self-organized criticality and adaptation in discrete dynamical networks

It has been proposed that adaptation in complex systems is optimized at the critical boundary between ordered and disordered dynamical regimes. Here, we review models of evolving dynamical networks that lead to self-organization of network topology based on a local coupling between a dynamical order parameter and rewiring of network connectivity, with convergence towards criticality in the limit of large network size $N$. In particular, two adaptive schemes are discussed and compared in the context of Boolean Networks and Threshold Networks: 1) Active nodes loose links, frozen nodes aquire new links, 2) Nodes with correlated activity connect, de-correlated nodes disconnect. These simple local adaptive rules lead to co-evolution of network topology and -dynamics. Adaptive networks are strikingly different from random networks: They evolve inhomogeneous topologies and broad plateaus of homeostatic regulation, dynamical activity exhibits $1/f$ noise and attractor periods obey a scale-free distribution. The proposed co-evolutionary mechanism of topological self-organization is robust against noise and does not depend on the details of dynamical transition rules. Using finite-size scaling, it is shown that networks converge to a self-organized critical state in the thermodynamic limit. Finally, we discuss open questions and directions for future research, and outline possible applications of these models to adaptive systems in diverse areas.

Network Structure and Dynamics, and Emergence of Robustness by Stabilizing Selection in an Artificial Genome

Genetic regulation is a key component in development, but a clear understanding of the structure and dynamics of genetic networks is not yet at hand. In this work we investigate these properties within an artificial genome model originally introduced by Reil. We analyze statistical properties of randomly generated genomes both on the sequence- and network level, and show that this model correctly predicts the frequency of genes in genomes as found in experimental data. Using an evolutionary algorithm based on stabilizing selection for a phenotype, we show that robustness against single base mutations, as well as against random changes in initial network states that mimic stochastic fluctuations in environmental conditions, can emerge in parallel. Evolved genomes exhibit characteristic patterns on both sequence and network level.