The Pianist v2

Back in April of 2007 I was inspired to build a virtual piano in VastPark, you can read the super brief post about it here.

At the time, I was pretty happy with the way it turned out and recently decided it was time for it to make a comeback!

How it works

This time around I decided to do things a little differently and shift all of the logic to a single .NET plugin that controls the entire piano experience.

While this has a number of benefits, I was primarily interested in keeping the structure of the environment cleanly separated from the logic.

To keep this separation simple, I made use of a little known feature of IMML called behaviours. Here’s a sample of my IMML:

<IMML Name="The Pianist v2" Camera="camera" xmlns="http://schemas.vastpark.com/2007/imml/">

  <!-- The plugin manages all interaction with the piano via behaviours -->
  <Plugin Name="PianoPlugin" Enabled="True" Source="plugins/Piano.plugin">
    <Parameter Key="HighlightColour" Value="#00FF00" />
    <Parameter Key="RotationWhenPressed" Value="0.03120605, 0, 0" />
  </Plugin>

  <Model Name="black_piano"
         Size="1.193525,1.065062,0.524349"
         Rotation="6.283185,0,0"
         Position="4.76837E-07,1.63393E-08,0.111626"
         Source="models/black_piano.model" />

  <!-- Piano key models -->
  <Model Name="key_b_8"
         Size="0.01918364,0.02875674,0.1308559"
         Rotation="6.283185,0,0"
         Position="-0.4839502,0.8194631,0.2247874"
         Behaviours="piano-key"
         Source="models/white_key1.model" />
  <!-- other models snipped for brevity -->

  <!-- Piano sounds -->
  <Sound Name="note_b_8" 
         Behaviours="piano-key-sound" 
         Source="sounds/pianokey_b_8.mp3" />
  <!-- other sounds snipped for brevity -->

</IMML>

Note the usage of the piano-key and piano-key-sound behaviours.

Also, I’ve only shown one model and one sound, the actual file contains 88 of each named according to a convention so that the appropriate sound can be mapped.

Using Behaviours

Every IMML element has the ability to be marked as having one or more behaviours that can be accessed as a list via the DOM.

This is extremely handy when writing plugins that manipulate elements, as you can simply query the scene for elements that match the desired behaviours like so:

//find all audio and models based on behaviour
var keyModels = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeyModelBehaviour));
var keySounds = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeySoundBehaviour));

if(keyModels.Any())
{
    //build keys for each model/sound combination
}

This works really well during the Load() method and can also be performed on elements dynamically added into the scene by listening to the ParkEngine.ElementLoaded event

In the context of the-pianist, it finds all elements of type piano-key and piano-key-sound and builds a PianoKey instance to manage them so that:

When the mouse is down, the key is visually down, highlighted and the sound plays.
When the mouse is up, the key is visually up.
When the sound is no longer playing, the key is no longer highlighted

Continuum

Next, I wanted to make sure that the work my plugin was doing would be correctly captured by Continuum. By default, Continuum will happily capture any series of changes in IMML state, so this initially didn’t seem like it would be a concern.

However, in my implementation of the Piano plugin I wanted to allow the same key to be pressed while the previous sound for that key was still playing. Just like a real piano.

To do this, I’ve been a little sneaky by going directly to the sound engine, bypassing the IMML change notification infrastructure which means that Continuum doesn’t see that change and cannot record it without some additional help.

You can see the two contrasting approaches below:

//toggle approach
this.Audible.Enabled = true;

//direct approach, bypass the IMML change framework
this.ParkEngine.SoundEngine.Play(this.Audible);

To overcome this limitation, I wrote an implementation of IStateRecorder for my plugin with one very simple method:

/// <summary>
/// Marks the sound as being played directly by the SoundEngine.
/// </summary>
/// <param name="sound">The sound.</param>
public void MarkSoundPlayed(Sound sound)
{
        if (!this.IsStarted)
        {
            return;
        }

        var captureState = new PianoCaptureState(Encoding.ASCII.GetBytes(sound.Name), Constants.Guid, DateTime.UtcNow, 0);
            
        this.Buffer.Enqueue(captureState);
}

It takes the name of the element that was played directly and writes it into the Continuum stream.

Later during playback, that element is resolved and at the appropriate time is played directly by the PianoStateController.

Goodies

I’ve zipped up the full source code to Plugin.Piano, along with the IMML and models you see in the screenshot at the beginning of this post.

Download here: http://tpiv.s3.amazonaws.com/the-pianist-v2.7z

Note that In order to run this properly, you’ll need to be a member of the Closed Beta community on vastpark.org (contact me if you’d like in) and be rocking Player v1.5.2 build 92 or newer.

The Pianist v2

How it works

Using Behaviours

Continuum

Goodies

Comments

Recent Posts