- source
- (Reinke et al. 2020)
Summary
The authors address the problem of automated discovery of diverse self-organized patterns in high-dimensional and complex game-of-life types of dynamical systems. They conduct experiments on Lenia.
Their goal is to use an IMGEP algorithm to represent interesting patterns and discover them.
Problem setting
Goal: With a budget of \(N\) experiments, maximize diversity of observations.
Parameter space \(\Theta\) of available parameters \(\theta\). An observation space \(O\) of observations. A single observation (a time series of images from Lenia in the paper) is denoted \(o\). An unknown dynamic \(D\) maps parameters \(\Theta\) to observations \(O\).
A goal space \(\mathcal{T}\) represents relevant features of an observation \(o\). \(\hat{g} = \mathcal{R}(o)\) (e.g. size or form of a pattern here).
Algorithm
For \(N\) timesteps, the algorithm samples a goal \(g\) from the space of goals and infers the corresponding parameters \(\theta\) with a parameter sampling policy \(\Pi = P(\theta; g)\) and simulate the corresponding experiment, and observation \(o\). \((\theta, o, \mathcal{R}(o))\) is then stored in the history.
In the paper, goals are sampled uniformly in a hyper-rectangle of \(\mathcal{T}\).
Goal space
The goal-space is learned online during the procedure with a variational autoencoders. Every \(K\) epochs, the VAE is trained on all the history of observations. Importance sampling (50% from the \(K\) last iterations / 50% from the rest of the history) is used for training of the VAE.
Parameter sampling
Parameters are sampled by selecting from the history which outcome is the closest to the sampled goal.
Parameter space
The mapping between parameters and observations is also approximated by the model. For this, CPPNs are used.
History
The history is initialized with \(N_{init}\) observations and each new observation is added.
Evaluation
Diversity of patterns is measured by the spread of exploration of an analytic behavior space. The authors use a external evaluation space obtained by training a $β$-VAE to learn important features with a dataset of 42500 Lenia patterns and distill it into a 13-dimensional vector.
That 13-dim space is then partitioned into 7 equal bins on each dimension. 5 evaluations:
- Random exploration: \(\theta\), including the initial grid state is sampled randomly
- IMGEP-HGS Goal exploration with hand-defined goal: This uses a goal space with 5 features defined in the Lenia paper
- IMGEP-PGL Goal exploration with pre-trained goal space: 558 Lenia patterns are used to pre-train the $β$-VAE used to encode the goal space;
- IMGEP-OGL Goal exploration with online learning of the goal space.
- IMGEP-RGS Goal exploration with a random goal space (the VAE has random weights)
Results
Goal-based exploration enables better behavior diversity with less parameter diversity compared to random exploration.
Learned goal-space methods seem more effective at finding a more diverse patterns.
Comments
This approach to pattern discovery and exploration is interesting. I like this idea of learning goals and pattern representation jointly to add as few assumptions as possible within the model.
Lenia is a fun model but has a lot of moving part and parameters. It would have liked to see how this method does with “simpler” models such as ECA or 2D Cellular automata.
Bibliography
- Chris Reinke, Mayalen Etcheverry, Pierre-Yves Oudeyer. . "Intrinsically Motivated Discovery of Diverse Patterns in Self-organizing Systems". Arxiv:1908.06663 [cs, Stat]. http://arxiv.org/abs/1908.06663.