Open Avatar Introductory Video

At the beginning of June we launched Open Avatar, an initiative to create a framework artists could use when building avatars.

By adhering to a convention, avatars, or parts of avatars, could become more portable across many different applications, produced by many different vendors.

Recently a video was uploaded to vimeo that gives a nice introduction to OpenAvatar and it’s purpose:

Since the launch, we’ve had some positive feedback and a lot of interesting discussion.

If you’ve tried it out and have had some success (or failure) please let us know!

Presentation: The Technology Behind VastPark

Today I presented at La Cantine in Paris to both a real world audience and a virtual audience inside of VastPark.

The presentation covered the technology that powers VastPark, where to find the open source code and the specifications such as IMML, Continuum, and the recently announced OpenAvatar. I also showed some live demonstrations of VastPark in action (the selective Continuum capture I did can be downloaded from here).

See the slideshare below, there was a lot of talking during the presentation so the slides may seem a little light on content. If you have questions about the slides feel free to comment and I will answer.

The Pianist v2

Back in April of 2007 I was inspired to build a virtual piano in VastPark, you can read the super brief post about it here.

At the time, I was pretty happy with the way it turned out and recently decided it was time for it to make a comeback!

How it works

This time around I decided to do things a little differently and shift all of the logic to a single .NET plugin that controls the entire piano experience.

While this has a number of benefits, I was primarily interested in keeping the structure of the environment cleanly separated from the logic.

To keep this separation simple, I made use of a little known feature of IMML called behaviours. Here’s a sample of my IMML:

<IMML Name="The Pianist v2" Camera="camera" xmlns="http://schemas.vastpark.com/2007/imml/">

  <!-- The plugin manages all interaction with the piano via behaviours -->
  <Plugin Name="PianoPlugin" Enabled="True" Source="plugins/Piano.plugin">
    <Parameter Key="HighlightColour" Value="#00FF00" />
    <Parameter Key="RotationWhenPressed" Value="0.03120605, 0, 0" />
  </Plugin>

  <Model Name="black_piano"
         Size="1.193525,1.065062,0.524349"
         Rotation="6.283185,0,0"
         Position="4.76837E-07,1.63393E-08,0.111626"
         Source="models/black_piano.model" />

  <!-- Piano key models -->
  <Model Name="key_b_8"
         Size="0.01918364,0.02875674,0.1308559"
         Rotation="6.283185,0,0"
         Position="-0.4839502,0.8194631,0.2247874"
         Behaviours="piano-key"
         Source="models/white_key1.model" />
  <!-- other models snipped for brevity -->

  <!-- Piano sounds -->
  <Sound Name="note_b_8" 
         Behaviours="piano-key-sound" 
         Source="sounds/pianokey_b_8.mp3" />
  <!-- other sounds snipped for brevity -->

</IMML>

Note the usage of the piano-key and piano-key-sound behaviours.

Also, I’ve only shown one model and one sound, the actual file contains 88 of each named according to a convention so that the appropriate sound can be mapped.

Using Behaviours

Every IMML element has the ability to be marked as having one or more behaviours that can be accessed as a list via the DOM.

This is extremely handy when writing plugins that manipulate elements, as you can simply query the scene for elements that match the desired behaviours like so:

//find all audio and models based on behaviour
var keyModels = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeyModelBehaviour));
var keySounds = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeySoundBehaviour));

if(keyModels.Any())
{
    //build keys for each model/sound combination
}

This works really well during the Load() method and can also be performed on elements dynamically added into the scene by listening to the ParkEngine.ElementLoaded event

In the context of the-pianist, it finds all elements of type piano-key and piano-key-sound and builds a PianoKey instance to manage them so that:

• When the mouse is down, the key is visually down, highlighted and the sound plays.
• When the mouse is up, the key is visually up.
• When the sound is no longer playing, the key is no longer highlighted

Continuum

Next, I wanted to make sure that the work my plugin was doing would be correctly captured by Continuum. By default, Continuum will happily capture any series of changes in IMML state, so this initially didn’t seem like it would be a concern.

However, in my implementation of the Piano plugin I wanted to allow the same key to be pressed while the previous sound for that key was still playing. Just like a real piano.

To do this, I’ve been a little sneaky by going directly to the sound engine, bypassing the IMML change notification infrastructure which means that Continuum doesn’t see that change and cannot record it without some additional help.

You can see the two contrasting approaches below:

//toggle approach
this.Audible.Enabled = true;

//direct approach, bypass the IMML change framework
this.ParkEngine.SoundEngine.Play(this.Audible);

To overcome this limitation, I wrote an implementation of IStateRecorder for my plugin with one very simple method:

/// <summary>
/// Marks the sound as being played directly by the SoundEngine.
/// </summary>
/// <param name="sound">The sound.</param>
public void MarkSoundPlayed(Sound sound)
{
        if (!this.IsStarted)
        {
            return;
        }

        var captureState = new PianoCaptureState(Encoding.ASCII.GetBytes(sound.Name), Constants.Guid, DateTime.UtcNow, 0);
            
        this.Buffer.Enqueue(captureState);
}

It takes the name of the element that was played directly and writes it into the Continuum stream.

Later during playback, that element is resolved and at the appropriate time is played directly by the PianoStateController.

Goodies

I’ve zipped up the full source code to Plugin.Piano, along with the IMML and models you see in the screenshot at the beginning of this post.

Download here: http://tpiv.s3.amazonaws.com/the-pianist-v2.7z

Note that In order to run this properly, you’ll need to be a member of the Closed Beta community on vastpark.org (contact me if you’d like in) and be rocking Player v1.5.2 build 92 or newer.

Update: A more recent version of the-pianist IMML is available here.

Simply unzip into the hosting root directory of your WorldServer to host.

Continuum v2 – Open Capture Framework

Continuum is an open framework designed to read and write changes in state. It supports forward and backwards playback and random seek and is designed to be streamed over a network. The data is captured in a format that can be reused, replayed, repurposed and investigated in much further detail than the flat view provided by a video capture.

While in VastPark we use Continuum for “deep recording” the scene, it can have other uses outside of virtual worlds also.

I can think of many industries (ie: finance, manufacture, emergency services) where being able to capture a series of events for analysis later is of great benefit.

Continuum v1

In v1, Continuum was limited to captures of IMML states; any time an element was added, removed or updated, a corresponding Continuum entry would be generated and added into a filestream.

This made it possible to record what was going on inside of the virtual world both client and server side, as initially everything being used by world developers was IMML based.

Unfortunately, v1 came with a number of limitations due to it’s ties with IMML:

• Live audio/video were not able to be captured as they aren’t static resources and the IMML just pointed to the stream. Playback of the Continuum file later would end up seeing different video/audio than when it was recorded. Or even worse, the stream no longer exists as is often the case with live events.
• Custom texture manipulation could not be captured as it skips the IMML layer and writes directly to the texture
• Custom animation (LIXA) uses direct bone manipulation which also skips the IMML layer

These limitations and others became quite frustrating, as additional features being added to the worlds platform often didn’t make sense to integrate into IMML and as a result could not be recorded easily.

Continuum v2

With v2, lessons have been learned and some major changes to the way Continuum works have been made.

Most notably:

• API for building custom state types
• Support for an almost infinite (~2 billion) number of arbitrary state types per stream
• Support for compressed and/or encrypted states
• Concurrent playback of multiple streams on a single timeline
• Concurrent record of states into the one stream

The design for v2 is a complete overhaul of v1 and revolves around the concept of state controllers and recorders. These constructs work against capture state instances, which is what a Continuum stream is composed of.

The header of a v2 Continuum stream looks like this:

Of note is the inclusion of the State Allocation Table (or SAT) which provides the glue between the custom state types and a stream. Each state controller has a GUID that the stream stores alongside an internal stream identifier:

The body of a Continuum stream contains sequential blocks of capture states. Each state node is self contained and stores an array of bytes:

Source Code

Update: See https://github.com/craigomatic/Continuum

Find the code under the Continuum and Continuum.Imml folders here.

If you’d like to have a go at building your own capture types, you really only need to play with two interfaces, IStateController and IStateRecorder which are fed into the StateRegistry and the CaptureService respectively.

They look like this:

During a record, the buffer of the IStateRecorder is polled and written into the capture stream. Objects should be monitored for change and inserted into the buffer of the IStateRecorder at that time.

During playback, the Create and Execute methods of IStateController are used to recreate the state and then execute it at the appropriate time.

I’ll blog at a later date showing how to do this in more detail, along with taking Continuum in the other direction and integrating it into your own application outside of VastPark.

Recent changes to the VastPark SVN structure

Recently we decided to do some repository reorganisation to make life easier for new developers wanting to tinker with the open source code for VastPark. This involved quite a bit of thought and a substantial amount of effort to migrate towards (actually, we’re still in the process of migrating some of the code) but we are already seeing the benefits of a cleaner structure.

The structure

As you will notice from the image above, we’ve gone with a multi-repository approach. This has a number of benefits for us internally, such as less duplication of files, as well as for those externally in that we can lock the public repository to specific external revisions to avoid pushing out code which might break compatibility.

Lib repository

Contains mainly third party source and binaries that we make use of in the platform.

Common repository

Contains the public source code for the VastPark libraries as well as VastPark developed specifications such as IMML, Metaforik and Continuum

Public repository

Externals common and lib repositories and sample projects developers can use to understand how the system works. It’s recommended that external developers work off the public trunk

Legacy code

Two quick points on some legacy issues to be aware of:

• There are some orphaned branches in public that we plan to keep around for a little bit longer as we migrate code out of there and into the new structure, please avoid working against these branches as they won’t be around for too much longer!
• There are two legacy folders, legacy-slimdx and legacy that exist in the common repository. These libraries will slowly be dropped in favour of newer code that replaces the functionality within. We plan to be much more sane with the number of projects in future 🙂

Getting the source

Using a subversion client, SVN checkout from:

http://vastpark.svn.cvsdude.com/public/trunk/

Be sure to enable externals or you will find only a fraction of the source code will be downloaded.

Happy coding! I’m interested to hear how simple it is to get up and running with the source code. Comments both positive and negative are welcomed on this.

Flocking algorithm in VastPark

Many months ago during May of 09, after meeting Ian Hughes at FCVW, I came across an interesting post on his blog discussing an implementation of the flocking algorithm in Unity.

I’d not heard of the term flocking or boids, so proceeded to get stuck into some research, which lead me to Conrad Parker’s page on Boids Pseudocode as well as many others who had implemented the algorithm, I figured it would be a fun little plugin to write for VastPark so gave it a try.

Implementing the algorithm

Thinking about what was required, I came up with the following as the plan for the plugin:

• Any IPositionalElement (Model, Primitive, Sound, Camera, Light, etc) would be supported
• Implement Rule 1 – “Boids try to fly towards the centre of mass of neighbouring boids”
• Implement Rule 2 – “Boids try to keep a small distance away from other objects (including other boids)”
• Implement Rule 3 – “Boids try to match velocity with near boids”
• For bonus points I’d implement “Tendency towards a particular place”, “Limiting the speed”, “Bounding the position”, “Perching” and “Scattering the flock”

In terms of the plugin this translates into these properties:

• SmallDistance
• MaxVelocity
• MinVelocity
• EnableBounds
• MaxBounds
• MinBounds
• EnablePerching
• MinPerchTime
• MaxPerchTime
• PerceivedCentre (get only)

The following callbacks:

• ElementPerched
• ElementLeftPerch

The following methods:

• SetPointOfInterest
• AddPlaceToAvoid
• AddPerch
• Scatter

The class diagrams look like this:

How does it work?

Put quite simply, the plugin works like this:

• AddElement(ImmlElement element) creates a boid instance for the positional element passed
• The plugin Update method (called once each update by the VastPark framework) proceeds to calculate the rules, one at a time for each boid to work out the new velocity and position for the element it represents
• Scatter requests made during runtime are dealt with during the next update

Downloads

Here is some sample IMML and the source code to the plugin, enjoy! Let me know if you decide to implement any of the other rules or do something cool with it 🙂

Sample IMML: flocking.imml
Source code to plugin: Plugin.Flocking.zip

Note: The source code will probably complain about some missing references. Install the Player from www.vastpark.com and link against the relevant binaries included with it.

Updated IMML for Stock Market Ticker

I’ve made some adjustments to the stock ticker built as part of my building a stock ticker in vastpark post to take advantage of the new Tooltip Plugin and the Define element in IMML. Rather than going all out, I’ve simply added a tip that shows the volume for the current day when the mouse is over one of the green bars and a Define that lets you more easily choose which stocks should be displayed.

If you haven’t already read the original article and are interested in how it works, be sure to have a read before continuing.

Define Element

The Define element is a declarative way of expressing a variable in IMML. It’s equivalent to doing element:set(‘variableName’, variableValue) but doesn’t require interpretation via the scripting engine which results in much better performance. It’s also a much nicer way to store global variables as they can be placed in more a easily identifiable location in the IMML rather than being buried within a script.

My Define in the sample looks similar to this (truncated for brevity):

<!--Use any Yahoo stock codes in the below define to control the stocks being generated and updated-->
 <Define Key="stocks" Value="IPL.AX,VIL.AX,FMG.AX,RIO.AX,BHP.AX,BOQ.AX" />

To change the ticker to show different stocks, replace any of the entries in the Value of the Define with the appropriate Yahoo ticker code for the relevant stock.

Tooltip Plugin

Note: To use the Tooltip plugin, you’ll need to be running either the most recent Player snapshot or version 0.98+ as it uses the Handle property that was added recently to the IRenderEngine interface.

The tooltip concept has been around for quite a while and is used by just about every modern desktop application. Ever wondered what that big red button does and hovered your mouse to find out? If yes, you’ve seen a tooltip. This plugin extends the concept to allow developers to associate a tip with an element in world.

To do so is quite simple, the following functions are provided with the plugin:

int Add(string text, ImmlElement element)
void Remove(int id)
void SetText(int id, string text)

To make use of these, I’ve added 2 lines to _updateStock in the UpdateStocks script:

--update the volume tooltip	
id = lastTradeVisual:get('tooltipId')
tooltipplugin:settext(id, 'Volume: '..volume)

…and 3 lines to _generateStock in the GenerateStocks script:

--store a define value on the last trade to represent volume for the tooltip
lastTradePrim:set('volume', volume)
id = tooltipplugin:add('Volume: '..volume, lastTradePrim)
lastTradePrim:set('tooltipId', id)

The end result still looks similar to before, but we now get tooltips when hovering:

Download the updated IMML here: stock-ticker.imml

Army of Craig

I’ve always wondered what it would be like to have an army of clones that I could send out to do my bidding…

VastPark OpenGL “Army of Craig” Demo from VastPark on Vimeo.

Yes. It is a little strange.

VastPark VNC Plugin

After some recent improvements to the core VastPark framework and the way textures are made available to the plugin API, I decided to build a VNC plugin for VastPark, basing it upon VncSharp. For those of you know don’t know what VNC is, there’s a good overview of it available on Wikipedia.

In it’s simplest form the IMML for the VNC Plugin will look something like this:

<Plugin Name="VncPlugin" Enabled="True" 
Source="http://content.vastpark.com/VastParkWS/get.vpws?publisher=craigomatic&amp;name=VncPlugin&amp;domain=vastpark&amp;context=park">
	<Element Name="TargetElement"/>
	<Element Name="OnPasswordRequired"/>
	<Parameter Key="Uri" Value="the-vnc-server-address.com"/>
	<Parameter Key="Port" Value="5900"/>
 	<Parameter Key="PasswordRequired" Value="OnPasswordRequired"/>
</Plugin>
<Script Name="OnPasswordRequired">
    function main(obj, args)
        vncplugin:setpassword('pass') --hardcoding password is bad, you should ask the user to enter one!
    end
</Script>
<Primitive Type="Box" Name="TargetElement" Size="1,1,1"/>

Here’s one I prepared earlier (well, Adrian did most of the work…):

Plugin Parameters

Uri

The host address of the VNC server. You should avoid using the scheme at the start of this address (ie: don’t put the http://)

Port

Defaults to the VNC default of 5900

ConnectionFailed

Name of a Script/Timeline/other ITimelineExecutable element to execute when a VNC connection has failed

PasswordRequired

Name of a Script/Timeline/other ITimelineExecutable element to execute when a password is required to connect to the VNC server

UpdateFrequency

The number of milliseconds to wait before automatically refreshing the remote screen when ManualUpdate is false. Defaults to 500.

ManualUpdate

True/False. When true, you need to call the vncplugin:requestupdate(true) for a fullscreen update. Defaults to False

You can download an example IMML document here: basic-vnc.imml and view the source code for the VNC Plugin over here: http://vastpark-svn.cvsdude.com/public/trunk/Plugins/Plugin.VNC/

A good free windows VNC Server is TightVNC

VastPark Source Code Online

Yesterday as part of the many happenings during the Virtual Worlds Down Under 09 event, we made the source code for the core VastPark libraries available to the public

The source is pure C# targeting .NET 3.0, licensed under GPL and is available for anonymous access at this address:

http://vastpark-svn.cvsdude.com/public/

Be sure to check out Sample projects\read first.txt for information in acquiring and setting up 3rd party dependencies to avoid losing hair getting the code to compile 🙂

Many thanks to CVSDude for supporting us with hosting and you can find out more about VastPark’s open source philosophy here.

• 
  

• Recent Posts

  • Open Avatar Introductory Video
  • Presentation: The Technology Behind VastPark
  • Plugin Framework available via NuGet
  • The Pianist v2
  • Continuum v2 – Open Capture Framework