MUGIC

Introduction

Mugic is created to turn your daily routine space into a magic world. While we are kids, we both want to be wizard or witches, but we wind up being designers and developers. The same thing doesn’t change is that what we are dreaming of and pursuing for today is still the power of transforming and recreating our surrounding world.

With AR technology, we managed to transform your surrounding environment to one of the fantasy worlds that you have magical powers, where you can manipulate things in a supernatural way with music and spells. The design idea is also deeply inspired by a childhood book called The Little Prince. In the story, each of the characters owns a unique lonesome planet. However, they just repeat their life there every day in a different way (fig.2). And our attempt is creating a new world out of routine life, which exactly the opposite approach, but express the same idea. We feel the scene is a good match. So what we design and create in the end is the world Mugic, where the witch can turn her routine environment to the solitary and isolated planet. In Mugic, she can reshape the world with music and spell, and in the meantime, the world is reconstructed by music, and it plays back recreating new music. Repeating the recreation and demolish of the same music piece in a different way is the sole mission for the witch every single day.

Motivation

Looking into reality, our perception of surroundings is always synthetic.

Holding an apple, one not just visually sees it, but also smells its scents, feels its texture and weight. This kind of synthetic perception not only is the foundation of comprehending our world, but also creating it. A painter sometimes listens and adjust the tone on his canvas, a composer sometimes visually draws shining pitches to light up the symphony. There is a rhythm in music, in movies, in paintings, in architecture. The creation process is sensually connected and consolidated. When the Virtual Reality and Augment Reality trying to be more like reality, they tend to mimic this comprehensive perception process.

We think this process can acheive way much more than just feels similar to the reality. With the technology of augmented reality, the way we feel our world can be hyper-perception. “Listeners” will be able to visually see the music, physically touch the music. “Painters” will be able to audibly play the color. The creation of any kinds of artwork today can be done synthetically with a sense other and more than just the traditional one. In other words, our perception of the world can be profound and compositional enhanced through AR technology. And this is what we are experimenting with creating the game Mugic.

Gameplay

Spell #1 “Aparecium”

To trigger “Aparecium”, put on your Hololens and play a song. Pitch stars are spawned at the notes of music, keeping their original pitch and volume. Each time the stars hear their pitches, they light up to a color representing the pitch. The movement of the star system as a whole follows the rhythm of music.

Each star remembers its volume and pitch at the point of creation, which decides its radius and color (ranging from red to blue) respectively. As the music flows, the stars form a dynamic particle system with explicit spring like interactions between particles. The star cloud, in graph data structure, spinning around and attracted by current pitch, flows up and down rhythmically in time with the music.

Spell #2 “Avifors”

To trigger “Avifors”, pick up your magic wand and dance. The tranquil universe suddenly turns crazy - pitch stars reveal their real form, the surrounding world turns into a kaleidoscope. As you dance, the music and the world controlled by your wand and movement.

Through sound analysis and sound synthesis, we metamorphose environment and music. Special Shader effects are developed and mapped to the environment through spatial mapping to trigger the drastic mutation. The explosion of stars is associated with volume and pitch of the music. When this mode is triggered, the pitch and tone of the music piece are adjusted with your position and the wand’s rotation input.

Spell #3 “Ascendio”

To trigger “Ascendio”, you need to wave your wand in a rather fast pace from top to down (fig.4). This spell will cause all pitch stars losing their flying power and hit the surrounding environment. When hitting, the stars will replay the pitches stored in them. This will recompose a new music piece using the same component of the original piece.

In this mode, we add gravity to stars and remove position and velocity constraints. Once a collision is detected, the star replays the pitch inherited in its property and then is destroyed after three collisions.

Spell #4 “Geminio”

"Geminio" magic is an ability gained by frequently practicing Mugic spell. When triggered, the wizards can replay and relight the pitch stars by touching the stars with their body. This also allows the wizard to light the surrounding environment with the color of that pitch, and add new ingredients to the original music piece.

The player is a rigid body that can collide with the stars. Upon collision, the star echoes its tune and reveals its color. Stars with the same pitch property twinkle as well to recompose the macrocosm.

 Collaborated Project with Jiabao Li

Instructor: Panagiotis Michalatos