Open Avatar Introductory Video

At the beginning of June we launched Open Avatar, an initiative to create a framework artists could use when building avatars.

By adhering to a convention, avatars, or parts of avatars, could become more portable across many different applications, produced by many different vendors.

Recently a video was uploaded to vimeo that gives a nice introduction to OpenAvatar and it’s purpose:

Since the launch, we’ve had some positive feedback and a lot of interesting discussion.

If you’ve tried it out and have had some success (or failure) please let us know!

Presentation: The Technology Behind VastPark

Today I presented at La Cantine in Paris to both a real world audience and a virtual audience inside of VastPark.

The presentation covered the technology that powers VastPark, where to find the open source code and the specifications such as IMML, Continuum, and the recently announced OpenAvatar. I also showed some live demonstrations of VastPark in action (the selective Continuum capture I did can be downloaded from here).

See the slideshare below, there was a lot of talking during the presentation so the slides may seem a little light on content. If you have questions about the slides feel free to comment and I will answer.

Plugin Framework available via NuGet

After a recent visit to Las Vegas for MIX11 (which was excellent btw) I was taken by the usefulness of NuGet and decided it was time to contribute a package into the main feed.

I’ve been developing a plugin framework, imaginatively named VastPark.PluginFramework which I hope to eventually massage into a really convenient MVC framework.

As it stands today it is a useful time saving library to be familiar with and I’d recommend all VastPark plugin developers make use of it.

1. Install NuGet

This has been covered in detail elsewhere and chances are you already have it installed as it is included in MVC3.

Visit the CodePlex documentation site here for a simple install guide.

2. Create a new plugin project in Visual Studio

As per usual, create a new project in Visual Studio:

Note: It’s best to go with .NET Framework 3.5 as the target framework, the client applications are built against this version of the .NET framework.

3. Add package reference to VastPark.PluginFramework

Right-click References and choose the shiny Add Library Package Reference button:

Next perform a search for ‘VastPark’:

Click install to download the package and have it magically added to your project.

You should now see some additional assemblies referenced and a blueprint.xml file included in your project:

4. Code your plugin!

Sit back for a moment to ponder the awesome of NuGet, then get on with coding your plugin 🙂

See here for the NuGet package page: http://nuget.org/List/Packages/VastPark.PluginFramework

The Pianist v2

Back in April of 2007 I was inspired to build a virtual piano in VastPark, you can read the super brief post about it here.

At the time, I was pretty happy with the way it turned out and recently decided it was time for it to make a comeback!

How it works

This time around I decided to do things a little differently and shift all of the logic to a single .NET plugin that controls the entire piano experience.

While this has a number of benefits, I was primarily interested in keeping the structure of the environment cleanly separated from the logic.

To keep this separation simple, I made use of a little known feature of IMML called behaviours. Here’s a sample of my IMML:

<IMML Name="The Pianist v2" Camera="camera" xmlns="http://schemas.vastpark.com/2007/imml/">

  <!-- The plugin manages all interaction with the piano via behaviours -->
  <Plugin Name="PianoPlugin" Enabled="True" Source="plugins/Piano.plugin">
    <Parameter Key="HighlightColour" Value="#00FF00" />
    <Parameter Key="RotationWhenPressed" Value="0.03120605, 0, 0" />
  </Plugin>

  <Model Name="black_piano"
         Size="1.193525,1.065062,0.524349"
         Rotation="6.283185,0,0"
         Position="4.76837E-07,1.63393E-08,0.111626"
         Source="models/black_piano.model" />

  <!-- Piano key models -->
  <Model Name="key_b_8"
         Size="0.01918364,0.02875674,0.1308559"
         Rotation="6.283185,0,0"
         Position="-0.4839502,0.8194631,0.2247874"
         Behaviours="piano-key"
         Source="models/white_key1.model" />
  <!-- other models snipped for brevity -->

  <!-- Piano sounds -->
  <Sound Name="note_b_8" 
         Behaviours="piano-key-sound" 
         Source="sounds/pianokey_b_8.mp3" />
  <!-- other sounds snipped for brevity -->

</IMML>

Note the usage of the piano-key and piano-key-sound behaviours.

Also, I’ve only shown one model and one sound, the actual file contains 88 of each named according to a convention so that the appropriate sound can be mapped.

Using Behaviours

Every IMML element has the ability to be marked as having one or more behaviours that can be accessed as a list via the DOM.

This is extremely handy when writing plugins that manipulate elements, as you can simply query the scene for elements that match the desired behaviours like so:

//find all audio and models based on behaviour
var keyModels = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeyModelBehaviour));
var keySounds = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeySoundBehaviour));

if(keyModels.Any())
{
    //build keys for each model/sound combination
}

This works really well during the Load() method and can also be performed on elements dynamically added into the scene by listening to the ParkEngine.ElementLoaded event

In the context of the-pianist, it finds all elements of type piano-key and piano-key-sound and builds a PianoKey instance to manage them so that:

Continuum

Next, I wanted to make sure that the work my plugin was doing would be correctly captured by Continuum. By default, Continuum will happily capture any series of changes in IMML state, so this initially didn’t seem like it would be a concern.

However, in my implementation of the Piano plugin I wanted to allow the same key to be pressed while the previous sound for that key was still playing. Just like a real piano.

To do this, I’ve been a little sneaky by going directly to the sound engine, bypassing the IMML change notification infrastructure which means that Continuum doesn’t see that change and cannot record it without some additional help.

You can see the two contrasting approaches below:

//toggle approach
this.Audible.Enabled = true;

//direct approach, bypass the IMML change framework
this.ParkEngine.SoundEngine.Play(this.Audible);

To overcome this limitation, I wrote an implementation of IStateRecorder for my plugin with one very simple method:

/// <summary>
/// Marks the sound as being played directly by the SoundEngine.
/// </summary>
/// <param name="sound">The sound.</param>
public void MarkSoundPlayed(Sound sound)
{
        if (!this.IsStarted)
        {
            return;
        }

        var captureState = new PianoCaptureState(Encoding.ASCII.GetBytes(sound.Name), Constants.Guid, DateTime.UtcNow, 0);
            
        this.Buffer.Enqueue(captureState);
}

It takes the name of the element that was played directly and writes it into the Continuum stream.

Later during playback, that element is resolved and at the appropriate time is played directly by the PianoStateController.

Goodies

I’ve zipped up the full source code to Plugin.Piano, along with the IMML and models you see in the screenshot at the beginning of this post.

Download here: http://tpiv.s3.amazonaws.com/the-pianist-v2.7z

Note that In order to run this properly, you’ll need to be a member of the Closed Beta community on vastpark.org (contact me if you’d like in) and be rocking Player v1.5.2 build 92 or newer.

Update: A more recent version of the-pianist IMML is available here.

Simply unzip into the hosting root directory of your WorldServer to host.

Continuum v2 – Open Capture Framework

Continuum is an open framework designed to read and write changes in state. It supports forward and backwards playback and random seek and is designed to be streamed over a network. The data is captured in a format that can be reused, replayed, repurposed and investigated in much further detail than the flat view provided by a video capture.

While in VastPark we use Continuum for “deep recording” the scene, it can have other uses outside of virtual worlds also.

I can think of many industries (ie: finance, manufacture, emergency services) where being able to capture a series of events for analysis later is of great benefit.

Continuum v1

In v1, Continuum was limited to captures of IMML states; any time an element was added, removed or updated, a corresponding Continuum entry would be generated and added into a filestream.

This made it possible to record what was going on inside of the virtual world both client and server side, as initially everything being used by world developers was IMML based.

Unfortunately, v1 came with a number of limitations due to it’s ties with IMML:

These limitations and others became quite frustrating, as additional features being added to the worlds platform often didn’t make sense to integrate into IMML and as a result could not be recorded easily.

Continuum v2

With v2, lessons have been learned and some major changes to the way Continuum works have been made.

Most notably:

The design for v2 is a complete overhaul of v1 and revolves around the concept of state controllers and recorders. These constructs work against capture state instances, which is what a Continuum stream is composed of.

The header of a v2 Continuum stream looks like this:

Of note is the inclusion of the State Allocation Table (or SAT) which provides the glue between the custom state types and a stream. Each state controller has a GUID that the stream stores alongside an internal stream identifier:

The body of a Continuum stream contains sequential blocks of capture states. Each state node is self contained and stores an array of bytes:

Source Code

Update: See https://github.com/craigomatic/Continuum

Find the code under the Continuum and Continuum.Imml folders here.

If you’d like to have a go at building your own capture types, you really only need to play with two interfaces, IStateController and IStateRecorder which are fed into the StateRegistry and the CaptureService respectively.

They look like this:

During a record, the buffer of the IStateRecorder is polled and written into the capture stream. Objects should be monitored for change and inserted into the buffer of the IStateRecorder at that time.

During playback, the Create and Execute methods of IStateController are used to recreate the state and then execute it at the appropriate time.

I’ll blog at a later date showing how to do this in more detail, along with taking Continuum in the other direction and integrating it into your own application outside of VastPark.

v1.5 Video Tutorials

Some instructional videos for v1.5 of the framework were recently created by one of our talented artists (also responsible for the sanctuary design) and uploaded to Vimeo.

The content pipeline is covered, including:

Exporting Maya models to Collada

Exporting 3dsmax models to Collada

Creator Importing and Publishing

The import/export pipeline is something that has been getting a lot of development attention recently and is quite advanced compared to the legacy import functionality of older versions of the framework.

Both the import and export providers are runtime loaded, meaning that others can build custom modules for the Creator.

By default the Creator ships with Collada import and File System export as shown in the video.

VastPark Pumpkin?

A friend of mine recently sent me a picture of a pumpkin she carved for Halloween that looks uncannily similar to the VastPark logo.

Do you see the resemblance?

VastPark pumpkin?

VastPark logo

On the topic of Halloween, it reminds me of some machinima that was shot using VastPark a couple of years ago:

VastServer Developer Edition RTW

Today marks the release to web of a new flavour of VastServer, known as VastServer Developer Edition.

As the name implies, this version of the server is targeted at developers and aims to provide more information on the framework’s behaviour, a little closer to the metal than has been possible in the past.

The user interface for the server is written with WPF and looks like this:

Click for larger view

Some of the more interesting features are covered in detail below.

Server Usage History

Many of the factors that contribute to server load such as send/receive rates, send/receive queues and client connections, are visible at a glance, with a rolling 60 second view of activity graphed:

It’s likely additional metrics will be made available in future releases, such as dynamic element counts, owned element counts, mutex holders, etc along with the ability to filter and manipulate the usage history graph.

Logging

Each world maintains it’s own log that displays IMML notifications such as those related to scene.ui:writeline, along with any world related notices, errors and warnings that the framework associates with that context.

A system log is also available via the View -> Log menu. It contains log information related to the networking infrastructure.

File Hosting Support

One nice new feature is the ability to host assets along side client IMML, meaning that IMML can be coded with relative URIs.

This supports the use case where a team wants to avoid setting up a web server or using a public storage provider such as Amazon S3 and is a convenient way to consider developing worlds.

For example, the model in this IMML:

<IMML xmlns="http://schemas.vastpark.com/2007/imml/">
  <Model Source="models/ground_path1.model" />
</IMML>

Is resolved to the client folder of the hosted context via the HostingRoot:

In the above example, the HostingRoot is at My Documents\VastPark\Server\ and the hosted context is called “sanctuary” (for some live shots of sanctuary in action, see here) .

Client Metrics

The client tab displays the behaviour of all clients connected to a specific world.

Information on user alias, connection time, framework version, ping, message resends, and time since last message provide the ability to troubleshoot scenarios where certain clients are experiencing difficulty with the system.

Both v1.3 and v1.5 clients are supported by this release of the server, the network stack has remained compatible between these versions of the framework.

Download

VastServer Developer Edition is freely available to developers for download with a limitation of 10 concurrent user connections across all hosted worlds. It’s built for windows and requires .NET Framework 4.0

License keys are available to partners and enterprise customers to remove the concurrent user limitation.

Link: http://www.vastpark.com/resources/downloads.html#server

Portable IMML – Host a virtual world like a PDF

As part of the v1.3 release the Portable IMML (PIMML) format was updated to include some new features (encryption, URI impersonation, additional assets, DRM).

An update was also made to the Creator to expose this new functionality and make packaging up a PIMML file a simpler experience.

Here’s a quick step-by-step guide to create a PIMML file using the Creator (v1.3 required!)

Begin by loading your IMML up into the Creator. For this example I’ll use hello-world-hills.imml. Navigate to the Publish menu and select File.

The first step is to decide how the PIMML file will be used. If you have designed it to be a multi-user environment, choose Offline Use. The difference between Online Use and Offline Use is quite simple; online means that the PIMML can use URI Impersonation to connect to a VastServer whereas offline cannot.

For this example I’ll pretend this file will be used online – in reality the IMML for hello-world.hills hasn’t been designed for online use, so don’t expect anything multi-userish to magically happen if you are following along at home 🙂

Next, we need to instruct the PIMML file where the VastServer that will manage the multi-user data is located. In this case I’m using the fictitious example.org as my address.

The benefit of this is that the PIMML file can be shared via email, served via Apache/IIS/VastServer/other, handed out on a USB stick, etc and still have the ability to be multi user. This is known as URI Impersonation.

By default, the Creator will only PIMML assets that are statically referenced in the original IMML.

In our example IMML the only element that really qualifies for packaging is the model called Mountain Terrain.

Often scripting will dynamically source content, so you may wish to add additional assets into the PIMML file. A good example of this is an animation resource as shown in the below example:

URI: The remote URI to map acquisition requests from
File path: The file to resolve the requests for that URI to

The final step is to check the document name is correct (it gets displayed at the bottom left of the Player UI) and choose an encryption level for the package.

I’ve choosen to encrypt in this example using a really thirst quenching password.

That’s all there is to it, hit Next and the Creator will do the rest:

The summary screen tells you where the PIMML was output to (defaults to Documents\VastPark\Creator\PortableIMML), the file can now be opened in the Player and interacted with as per usual:

Note: The title for this post was inspired by http://twitter.com/jokay/status/20869967909 🙂

VastPark Platform 1.3 Released

Today the Creator, Player and Publisher based off v1.3 of the framework were released to web!

Some of the more notable changes include:

  • New Drawing API
  • New physics engine
  • Support for encrypted file formats
  • Improvements to the Portable IMML format
  • Additions to VastScript related to keyboard/mouse triggers
  • Refined Player UI
  • Many bug fixes and stability improvements

Why the jump from 0.98 to 1.3?

My last post implied that v0.98 would be the final DirectX version of the framework. Instead a decision was made for a few more iterations on DirectX to tie up some loose ends, prior to spending significantly more time working on the OpenGL framework (in actual fact, over the past 6 months we’ve been working on both!).

Since the release of 0.98, the version numbers have been ticking over in line with the growing maturing of the framework, with v1.1 and v1.2 surfacing as developer snapshots. As some fairly major pieces of the framework were being swapped out over this period, the numbering became important – some plugins were needing a minor rewrite to be compatible with the newer modules.

Using the magic of Metaforik and multiple Assets linked to each Item, we were able to publish the same plugin multiple times to maintain backwards compatibility for each framework version.

Portable IMML

The PIMML format has been updated to support some exciting new features:

Combined these features mean that a securely encrypted, portable virtual world can be distributed which is capable of connecting to a live server! Your users will benefit by much faster load times (the content is local) while still maintaining the ability to be in a multi-user space.

As part of the format update, a new wizard interface was introduced in the Creator making it easier to use the new features.

Find the wizard under the Publish to File menu:

Note: The update to the PIMML format is not backwards compatible. If you have some older files which are critical to migrate, please speak up at the forums on vastpark.org

Drawing API

Many customers are looking towards virtual worlds as a solution to collaboratively visualise information.

Whilst this was possible previously in VastPark using Primitive and/or Model elements, the new drawing API allows developers to take this to the next level by making it easier to create graphs, overlay annotations and build more intuitive user experiences.

The drawing API provides the following base types:

Access to these types are exposed via the RenderEngine interface and for convenience within VastScript via plugin (DrawingPlugin). We are considering ways that this can integrate nicely into the IMML spec at a later point.

Here’s a basic sample that uses the DrawingPlugin to demonstrate some of the functionality (right-click, save as): drawing-api-sample.imml

Refined Player UI

We’ve made further improvements to the Player UI to tighten up the look along with a brand new vastpark:home design:

At the same time we’ve changed some of the shortcut keys:

F1 – Help
F4 – Options
F5 – Refresh
F6 – Screenshot
F9 – Chat
F10 – Library
F11 – Fullscreen
F12 – Freelook camera

Also of note is the unified default folder structure for all of the applications:

Documents\VastPark\[application name]\[folder]

New physics engine

BulletPhysics had already been chosen as the solution for future versions of the framework. It was decided that to provide a smoother transition between the v1.3 and v1.5 codebases, we would port it back.

Improvements have been observed in a number of areas such as the stability of avatars and a more realistic, robust simulation.

Alas, the port was not without some minor regressions; GenerateExplosion, SetForce, AddForce, SetImpulse and collision events were temporary casualties. All of these will return again in a future version of the framework.

Final Note

The applications are available for download at the usual place.

If you are upgrading from a version prior to v1.3 I’d strongly advise a quick visit to add/remove to uninstall the older applications prior to upgrading to the newer release. The installers are now using MSI and will only allow overwrite from v1.3 onwards.

Enjoy!

Next Page →