- source
- (Reinke, Etcheverry, and Oudeyer 2020)

## Summary

The authors address the problem of automated discovery of diverse self-organized patterns in high-dimensional and complex game-of-life types of dynamical systems. They conduct experiments on Lenia.

Their goal is to use an IMGEP algorithm to represent interesting patterns and discover them.

### Problem setting

Goal: With a budget of \(N\) experiments, maximize diversity of observations.

Parameter space \(\Theta\) of available parameters \(\theta\). An observation space \(O\) of observations. A single observation (a time series of images from Lenia in the paper) is denoted \(o\). An unknown dynamic \(D\) maps parameters \(\Theta\) to observations \(O\).

A goal space \(\mathcal{T}\) represents relevant features of an observation \(o\). \(\hat{g} = \mathcal{R}(o)\) (e.g. size or form of a pattern here).

### Algorithm

For \(N\) timesteps, the algorithm samples a goal \(g\) from the space of goals and infers the corresponding parameters \(\theta\) with a parameter sampling policy \(\Pi = P(\theta; g)\) and simulate the corresponding experiment, and observation \(o\). \((\theta, o, \mathcal{R}(o))\) is then stored in the history.

In the paper, goals are sampled uniformly in a hyper-rectangle of \(\mathcal{T}\).

#### Goal space

The goal-space is learned online during the procedure with a variational autoencoders. Every \(K\) epochs, the VAE is trained on all the history of observations. Importance sampling (50% from the \(K\) last iterations / 50% from the rest of the history) is used for training of the VAE.

#### Parameter sampling

Parameters are sampled by selecting from the history which outcome is the closest to the sampled goal.

#### Parameter space

The mapping between parameters and observations is also approximated by the model. For this, CPPNs are used.

#### History

The history is initialized with \(N_{init}\) observations and each new observation is added.

### Evaluation

Diversity of patterns is measured by the spread of exploration
of an *analytic behavior space*. The authors use a external
evaluation space obtained by training a $β$-VAE to learn important
features with a dataset of 42500 Lenia patterns and distill it into
a 13-dimensional vector.

That 13-dim space is then partitioned into 7 equal bins on each dimension. 5 evaluations:

- Random exploration: \(\theta\), including the initial grid state is sampled randomly
- IMGEP-HGS Goal exploration with hand-defined goal: This uses a goal space with 5 features defined in the Lenia paper
- IMGEP-PGL Goal exploration with pre-trained goal space: 558 Lenia patterns are used to pre-train the $β$-VAE used to encode the goal space;
- IMGEP-OGL Goal exploration with online learning of the goal space.
- IMGEP-RGS Goal exploration with a random goal space (the VAE has random weights)

### Results

Goal-based exploration enables better behavior diversity with less parameter diversity compared to random exploration.

Learned goal-space methods seem more effective at finding a more diverse patterns.

## Comments

This approach to pattern discovery and exploration is interesting. I like this idea of learning goals and pattern representation jointly to add as few assumptions as possible within the model.

Lenia is a fun model but has a lot of moving part and parameters. It would have liked to see how this method does with “simpler” models such as ECA or 2D Cellular automata.

## Bibliography

Reinke, Chris, Mayalen Etcheverry, and
Pierre-Yves Oudeyer. 2020. “Intrinsically Motivated Discovery of
Diverse Patterns in Self-Organizing Systems.” *arXiv:1908.06663
[Cs, Stat]*, February.