# Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems by Reinke, C., Etcheverry, M., & Oudeyer, P. (2020)

source
(Reinke, Etcheverry, and Oudeyer 2020)

## Summary

The authors address the problem of automated discovery of diverse self-organized patterns in high-dimensional and complex game-of-life types of dynamical systems. They conduct experiments on Lenia.

Their goal is to use an IMGEP algorithm to represent interesting patterns and discover them.

### Problem setting

Goal: With a budget of $$N$$ experiments, maximize diversity of observations.

Parameter space $$\Theta$$ of available parameters $$\theta$$. An observation space $$O$$ of observations. A single observation (a time series of images from Lenia in the paper) is denoted $$o$$. An unknown dynamic $$D$$ maps parameters $$\Theta$$ to observations $$O$$.

A goal space $$\mathcal{T}$$ represents relevant features of an observation $$o$$. $$\hat{g} = \mathcal{R}(o)$$ (e.g. size or form of a pattern here).

### Algorithm

For $$N$$ timesteps, the algorithm samples a goal $$g$$ from the space of goals and infers the corresponding parameters $$\theta$$ with a parameter sampling policy $$\Pi = P(\theta; g)$$ and simulate the corresponding experiment, and observation $$o$$. $$(\theta, o, \mathcal{R}(o))$$ is then stored in the history.

In the paper, goals are sampled uniformly in a hyper-rectangle of $$\mathcal{T}$$.

#### Goal space

The goal-space is learned online during the procedure with a variational autoencoders. Every $$K$$ epochs, the VAE is trained on all the history of observations. Importance sampling (50% from the $$K$$ last iterations / 50% from the rest of the history) is used for training of the VAE.

#### Parameter sampling

Parameters are sampled by selecting from the history which outcome is the closest to the sampled goal.

#### Parameter space

The mapping between parameters and observations is also approximated by the model. For this, CPPNs are used.

#### History

The history is initialized with $$N_{init}$$ observations and each new observation is added.

### Evaluation

Diversity of patterns is measured by the spread of exploration of an analytic behavior space. The authors use a external evaluation space obtained by training a $β$-VAE to learn important features with a dataset of 42500 Lenia patterns and distill it into a 13-dimensional vector.

That 13-dim space is then partitioned into 7 equal bins on each dimension. 5 evaluations:

• Random exploration: $$\theta$$, including the initial grid state is sampled randomly
• IMGEP-HGS Goal exploration with hand-defined goal: This uses a goal space with 5 features defined in the Lenia paper
• IMGEP-PGL Goal exploration with pre-trained goal space: 558 Lenia patterns are used to pre-train the $β$-VAE used to encode the goal space;
• IMGEP-OGL Goal exploration with online learning of the goal space.
• IMGEP-RGS Goal exploration with a random goal space (the VAE has random weights)

### Results

Goal-based exploration enables better behavior diversity with less parameter diversity compared to random exploration.

Learned goal-space methods seem more effective at finding a more diverse patterns.