by Joana Ricou, Jesse Chorn and Polo Chau
Thanks to George Davis.
Visitors are welcomed by a flock of butterflies flying around a meadow. As the visitors approach they "cast" a shadow onto the meadow. The butterflies approach the visitors. Slow or non-moving visitors invite the butterflies stay close; quick movements scare them away. A butterfly that collides with the shadow of the visitor repeatedly will eventually die.
Equipment
Video camera and projector
Laptop computer
Input
Presence and movements of visitors
Black Box
Each butterfly learns with reinforcement learning in its lifetime to optimize the amount of human touches that it can get
Output
Butterflies individually learn to stay close to friendly humans (and away from the "hostile" ones)
Lessons Learned/Thoughts
The point of this experiment was to have the flies absorb information from the environment (the speed of motion of the visitors) and visualize this information in their behavior (by changing speed and closeness to visitors) in real-time . This would happen both with a direct response (ie. Shoo-ing a fly will make it move away), and a more subtle response where the fly would change its behavior depending on whether it is exposed more to one speed or another of movement (ie. become more likely to fly away very fast or more likely to try to land). We could not achieve this with re-inforcement learning because:
- In the unlearned state, the behavior of the fly is uninteresting visually to the visitor, as it appears chaotic and random and visitors’ actions appear to have “no effect”. Eventually, the flies learn to stay as close as possible to the visitors.
In this learned state, the visitors are able to interact with the flies (they land and/or can be shoo-ed away) but won’t see a change in their behavior due to the visitor’s behavior (because the learning is too slow).
- If the learning is slow, the flies eventually learn a set of behaviors and have a fluid and understandable dynamic (tending to stay near the visitor). But, one visitor’s behavior is unlikely to change the pattern of behavior of the flies. So the visitor is able to interact with the flies in the moment but is very unlikely to note a change in the way they behave.
- If the learning if faster, the flies’ behavior becomes more erratic and apears random so the visitor has a less beautiful experience and is unable to see immediate responses to their actions.
Frames of flying animation.
What We Learned
- The exploration process of reinforcement learning (i.e., butterflies trying out different actions) created too much randomness to the interaction and lowered the visual quality of the installation.
- The speed rate is an important parameter to tweak, and it is hard to pick the best value. If the rate is too high, the butterflies' actions appear random. If it is too slow, it takes a long time for the butterflies to adapt to new visitors' movement patterns.
We lowered the parameters so butterflier would adapt in about 1-2 minutes, with appropriate training. But this turned out to be too long, as visitors' tend to move on after less than a minute.
- Having multiple visitors of different "types" (e.g., fast moving, slow moving) interacting with our installation at the same time, confuses the butterflies about which actions are optimal (e.g., fly closer, or fly away). The learned behaviors may then look random, or more like an "averaged" version.
Background for interactive installation.
Through this interactive installation, we've learned that giving the visitors immediate feedback was an important "hook" to entice the visitors to stay and explore. We have made design decisions to create these hooks. For example, we initially proposed to let the butterflies to reproduce, and to evolve in their colors and sizes using the genetic algorithm, but later we decided the visible effect was too subtle, so instead we focused on their motions, which were more visually interesting and engaging, and their immediate reactions to people's movements.
Background for interactive installation. 