Presentation: The Technology Behind VastPark

Today I presented at La Cantine in Paris to both a real world audience and a virtual audience inside of VastPark.

The presentation covered the technology that powers VastPark, where to find the open source code and the specifications such as IMML, Continuum, and the recently announced OpenAvatar. I also showed some live demonstrations of VastPark in action (the selective Continuum capture I did can be downloaded from here).

See the slideshare below, there was a lot of talking during the presentation so the slides may seem a little light on content. If you have questions about the slides feel free to comment and I will answer.

La Cantine Presentation, Paris, 10th June 2011
June 11, 2011 | Filed Under IMML, Open Source, Plugins, Scripting, VastPark General | Comments Off on Presentation: The Technology Behind VastPark 

The Pianist v2

Back in April of 2007 I was inspired to build a virtual piano in VastPark, you can read the super brief post about it here.

At the time, I was pretty happy with the way it turned out and recently decided it was time for it to make a comeback!

How it works

This time around I decided to do things a little differently and shift all of the logic to a single .NET plugin that controls the entire piano experience.

While this has a number of benefits, I was primarily interested in keeping the structure of the environment cleanly separated from the logic.

To keep this separation simple, I made use of a little known feature of IMML called behaviours. Here’s a sample of my IMML:

<IMML Name="The Pianist v2" Camera="camera" xmlns="http://schemas.vastpark.com/2007/imml/">

  <!-- The plugin manages all interaction with the piano via behaviours -->
  <Plugin Name="PianoPlugin" Enabled="True" Source="plugins/Piano.plugin">
    <Parameter Key="HighlightColour" Value="#00FF00" />
    <Parameter Key="RotationWhenPressed" Value="0.03120605, 0, 0" />
  </Plugin>

  <Model Name="black_piano"
         Size="1.193525,1.065062,0.524349"
         Rotation="6.283185,0,0"
         Position="4.76837E-07,1.63393E-08,0.111626"
         Source="models/black_piano.model" />

  <!-- Piano key models -->
  <Model Name="key_b_8"
         Size="0.01918364,0.02875674,0.1308559"
         Rotation="6.283185,0,0"
         Position="-0.4839502,0.8194631,0.2247874"
         Behaviours="piano-key"
         Source="models/white_key1.model" />
  <!-- other models snipped for brevity -->

  <!-- Piano sounds -->
  <Sound Name="note_b_8" 
         Behaviours="piano-key-sound" 
         Source="sounds/pianokey_b_8.mp3" />
  <!-- other sounds snipped for brevity -->

</IMML>

Note the usage of the piano-key and piano-key-sound behaviours.

Also, I’ve only shown one model and one sound, the actual file contains 88 of each named according to a convention so that the appropriate sound can be mapped.

Using Behaviours

Every IMML element has the ability to be marked as having one or more behaviours that can be accessed as a list via the DOM.

This is extremely handy when writing plugins that manipulate elements, as you can simply query the scene for elements that match the desired behaviours like so:

//find all audio and models based on behaviour
var keyModels = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeyModelBehaviour));
var keySounds = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeySoundBehaviour));

if(keyModels.Any())
{
    //build keys for each model/sound combination
}

This works really well during the Load() method and can also be performed on elements dynamically added into the scene by listening to the ParkEngine.ElementLoaded event

In the context of the-pianist, it finds all elements of type piano-key and piano-key-sound and builds a PianoKey instance to manage them so that:

Continuum

Next, I wanted to make sure that the work my plugin was doing would be correctly captured by Continuum. By default, Continuum will happily capture any series of changes in IMML state, so this initially didn’t seem like it would be a concern.

However, in my implementation of the Piano plugin I wanted to allow the same key to be pressed while the previous sound for that key was still playing. Just like a real piano.

To do this, I’ve been a little sneaky by going directly to the sound engine, bypassing the IMML change notification infrastructure which means that Continuum doesn’t see that change and cannot record it without some additional help.

You can see the two contrasting approaches below:

//toggle approach
this.Audible.Enabled = true;

//direct approach, bypass the IMML change framework
this.ParkEngine.SoundEngine.Play(this.Audible);

To overcome this limitation, I wrote an implementation of IStateRecorder for my plugin with one very simple method:

/// <summary>
/// Marks the sound as being played directly by the SoundEngine.
/// </summary>
/// <param name="sound">The sound.</param>
public void MarkSoundPlayed(Sound sound)
{
        if (!this.IsStarted)
        {
            return;
        }

        var captureState = new PianoCaptureState(Encoding.ASCII.GetBytes(sound.Name), Constants.Guid, DateTime.UtcNow, 0);
            
        this.Buffer.Enqueue(captureState);
}

It takes the name of the element that was played directly and writes it into the Continuum stream.

Later during playback, that element is resolved and at the appropriate time is played directly by the PianoStateController.

Goodies

I’ve zipped up the full source code to Plugin.Piano, along with the IMML and models you see in the screenshot at the beginning of this post.

Download here: http://tpiv.s3.amazonaws.com/the-pianist-v2.7z

Note that In order to run this properly, you’ll need to be a member of the Closed Beta community on vastpark.org (contact me if you’d like in) and be rocking Player v1.5.2 build 92 or newer.

Update: A more recent version of the-pianist IMML is available here.

Simply unzip into the hosting root directory of your WorldServer to host.

April 15, 2011 | Filed Under IMML, Open Source, Plugins | Comments Off on The Pianist v2 

VastServer Developer Edition RTW

Today marks the release to web of a new flavour of VastServer, known as VastServer Developer Edition.

As the name implies, this version of the server is targeted at developers and aims to provide more information on the framework’s behaviour, a little closer to the metal than has been possible in the past.

The user interface for the server is written with WPF and looks like this:

Click for larger view

Some of the more interesting features are covered in detail below.

Server Usage History

Many of the factors that contribute to server load such as send/receive rates, send/receive queues and client connections, are visible at a glance, with a rolling 60 second view of activity graphed:

It’s likely additional metrics will be made available in future releases, such as dynamic element counts, owned element counts, mutex holders, etc along with the ability to filter and manipulate the usage history graph.

Logging

Each world maintains it’s own log that displays IMML notifications such as those related to scene.ui:writeline, along with any world related notices, errors and warnings that the framework associates with that context.

A system log is also available via the View -> Log menu. It contains log information related to the networking infrastructure.

File Hosting Support

One nice new feature is the ability to host assets along side client IMML, meaning that IMML can be coded with relative URIs.

This supports the use case where a team wants to avoid setting up a web server or using a public storage provider such as Amazon S3 and is a convenient way to consider developing worlds.

For example, the model in this IMML:

<IMML xmlns="http://schemas.vastpark.com/2007/imml/">
  <Model Source="models/ground_path1.model" />
</IMML>

Is resolved to the client folder of the hosted context via the HostingRoot:

In the above example, the HostingRoot is at My Documents\VastPark\Server\ and the hosted context is called “sanctuary” (for some live shots of sanctuary in action, see here) .

Client Metrics

The client tab displays the behaviour of all clients connected to a specific world.

Information on user alias, connection time, framework version, ping, message resends, and time since last message provide the ability to troubleshoot scenarios where certain clients are experiencing difficulty with the system.

Both v1.3 and v1.5 clients are supported by this release of the server, the network stack has remained compatible between these versions of the framework.

Download

VastServer Developer Edition is freely available to developers for download with a limitation of 10 concurrent user connections across all hosted worlds. It’s built for windows and requires .NET Framework 4.0

License keys are available to partners and enterprise customers to remove the concurrent user limitation.

Link: http://www.vastpark.com/resources/downloads.html#server
November 6, 2010 | Filed Under Content, IMML, VastPark General, WPF | 3 Comments 

Portable IMML – Host a virtual world like a PDF

As part of the v1.3 release the Portable IMML (PIMML) format was updated to include some new features (encryption, URI impersonation, additional assets, DRM).

An update was also made to the Creator to expose this new functionality and make packaging up a PIMML file a simpler experience.

Here’s a quick step-by-step guide to create a PIMML file using the Creator (v1.3 required!)

Begin by loading your IMML up into the Creator. For this example I’ll use hello-world-hills.imml. Navigate to the Publish menu and select File.

The first step is to decide how the PIMML file will be used. If you have designed it to be a multi-user environment, choose Offline Use. The difference between Online Use and Offline Use is quite simple; online means that the PIMML can use URI Impersonation to connect to a VastServer whereas offline cannot.

For this example I’ll pretend this file will be used online – in reality the IMML for hello-world.hills hasn’t been designed for online use, so don’t expect anything multi-userish to magically happen if you are following along at home 🙂

Next, we need to instruct the PIMML file where the VastServer that will manage the multi-user data is located. In this case I’m using the fictitious example.org as my address.

The benefit of this is that the PIMML file can be shared via email, served via Apache/IIS/VastServer/other, handed out on a USB stick, etc and still have the ability to be multi user. This is known as URI Impersonation.

By default, the Creator will only PIMML assets that are statically referenced in the original IMML.

In our example IMML the only element that really qualifies for packaging is the model called Mountain Terrain.

Often scripting will dynamically source content, so you may wish to add additional assets into the PIMML file. A good example of this is an animation resource as shown in the below example:

URI: The remote URI to map acquisition requests from
File path: The file to resolve the requests for that URI to

The final step is to check the document name is correct (it gets displayed at the bottom left of the Player UI) and choose an encryption level for the package.

I’ve choosen to encrypt in this example using a really thirst quenching password.

That’s all there is to it, hit Next and the Creator will do the rest:

The summary screen tells you where the PIMML was output to (defaults to Documents\VastPark\Creator\PortableIMML), the file can now be opened in the Player and interacted with as per usual:

Note: The title for this post was inspired by http://twitter.com/jokay/status/20869967909 🙂

August 11, 2010 | Filed Under IMML, VastPark General | 4 Comments 

VastPark Developer Introduction – v1.3 Framework

Update: The video was re-dubbed, you now get the pleasure of Liz’s delightful voice 🙂

I recently gave a talk to a team of developers interested in working with the platform who are planning to build some new exciting products on top of it.

One of the audience offered to record the session remotely so the audio is a little scratchy, but I think it’s come across ok.

The talk covers:

Hopefully this is a useful primer if you are interested in developing with VastPark or are curious to know a little more about the way the system functions.

Download the PDF shown in the video here: VastPark Developer Introduction - Framework v1.3

July 30, 2010 | Filed Under IMML, Plugins, Scripting, VastPark General | Comments Off on VastPark Developer Introduction – v1.3 Framework 

FCVW 2010

In May of 2010, VastPark again exhibited at the Federal Consortium for Virtual Worlds in Washington DC.

As with the previous year, we provided a way for people who were unable to attend in person to attend virtually:

We also were involved in the keynote by Dr. Robert Childs, following some very cool work by ImageMetrics as shown in this video:

The VastPark portion of the video begins at 5:08, prior to that you are seeing ImageMetrics in action.

June 4, 2010 | Filed Under IMML, Misc, VastPark General | Comments Off on FCVW 2010 

Flocking algorithm in VastPark

Many months ago during May of 09, after meeting Ian Hughes at FCVW, I came across an interesting post on his blog discussing an implementation of the flocking algorithm in Unity.

I’d not heard of the term flocking or boids, so proceeded to get stuck into some research, which lead me to Conrad Parker’s page on Boids Pseudocode as well as many others who had implemented the algorithm, I figured it would be a fun little plugin to write for VastPark so gave it a try.

Implementing the algorithm

Thinking about what was required, I came up with the following as the plan for the plugin:

In terms of the plugin this translates into these properties:

The following callbacks:

The following methods:

The class diagrams look like this:

How does it work?

Put quite simply, the plugin works like this:

Downloads

Here is some sample IMML and the source code to the plugin, enjoy! Let me know if you decide to implement any of the other rules or do something cool with it 🙂

Sample IMML: flocking.imml
Source code to plugin: Plugin.Flocking.zip

Note: The source code will probably complain about some missing references. Install the Player from www.vastpark.com and link against the relevant binaries included with it.

April 13, 2010 | Filed Under IMML, Open Source, Plugins, Scripting | Comments Off on Flocking algorithm in VastPark 

Drag and Drop Sample

One of the major features of the 0.98 release is the inclusion of drag and drop. Basically, this allows you to drag an Item into the 3D space where the IMML can decide to do something with it.

To try drag and drop out for yourself, first download generic-drag-drop-sample.imml (right-click, save as) and open in the Player.

Next, you’ll need to expand the Items list. Do so by clicking on this icon:

By default you are presented with the list of Items of type Video and Document that the currently logged in account has access to (these can be further restricted by the hosting server’s group ACL).

In this case as we are opening the file directly, it will default to showing the most recently published Items without filtering and should look similar to this:

Click to select one of the items from the list (in my example, I’ve chosen “VastPark Logo”) and drag it onto the 3D view. The large rectangle in the centre of the screen will change colour to blue indicating it is a drop target.

Next, drop the Item onto the blue rectangle, which will cause it to change colour to green. Shortly the VastPark logo will appear on it.

How does this work?

Behind the scenes is a plugin (called MetaforikAdapter) which currently supports adapting Items of type “Document” and “Video” to a Model or Primitive element. The code to do this is quite simple.

First, reference the Plugin:

<Plugin Enabled="True" Source="http://id.vastpark.com/VastParkWS/get.vpws?name=MetaforikAdapter&amp;publisher=craigomatic&amp;domain=vastpark&amp;context=park" Name="MetaforikAdapter">
    <Element Name="InvalidRequestHandler" />
    <Parameter Key="InvalidRequestEvent" Value="InvalidRequestHandler" />    
  </Plugin>   
  
  <Script Name="InvalidRequestHandler">
    function main(obj, args)
      scene.ui:writeline('invalid request')          
    end
  </Script>

Next, handle the appropriate events on the elements you want to be able to drop on. Alternatively, this can be at the document level:
<Trigger Event="DragDrop" Target="OnDragDrop" />
<Trigger Event="DragEnter" Target="OnDragEnter" />
<Trigger Event="DragLeave" Target="OnDragLeave" />

In you script for the drop, pass the Item along with the element you want to adapt it to into the load method on the plugin. You may choose to inspect the Item before the drop to give user feedback as is done in the sample IMML.

<Script Name="OnDragDrop">
    function main(obj, args)            
      --args.data contains the Item which was dragged in
      
      --load returns a controller object
      c = metaforikadapter:load(args.data, obj)      
    
      scene.ui:writeline('item name: '..c.item.name)
      scene.ui:writeline('id: '..c.id)
      scene.ui:writeline('is first: '..tostring(c.isfirst))
      scene.ui:writeline('is last: '..tostring(c.islast))
      scene.ui:writeline('total pages: '..tostring(c.pages))
      scene.ui:writeline('current page: '..tostring(c.currentpage))
    end
</Script>
  
<Script Name="OnDragEnter">
    function main(obj, args)
        scene.ui:writeline('drag enter on: '..obj.name..' of type: '..args.data.assets[0].type)     
    end
</Script>
  
<Script Name="OnDragLeave">
    function main(obj, args)
      scene.ui:writeline('drag leave on: '..obj.name)
    end
</Script>

Working with elements other than Document or Video

One thing to be aware of is that you don’t necessarily have to use the MetaforikAdapter plugin to benefit from drag and drop. If you want to allow for models or sounds to be dropped in, you can use the item.updateuri as the source for the element and add it into the scene, like this:

<Script Name="OnDragDrop">
    function main(obj, args)    
      item = args.data
      if(item.assets[0].type == 'Model') then       
          m = model()
          m.source = args.data.updateuri
      
          scene:add(m)     
      end
    end
</Script>

Final Note

That’s really all there is to it, you may want to craft different UI depending on the state of the controller properties in the MetaforikAdapter plugin sample, ie: to handle a multi-page document differently to a video or single page document, or in the model example to only allow certain users to drop models into a specific region of your space.

February 20, 2010 | Filed Under IMML, Plugins, Scripting | Comments Off on Drag and Drop Sample 

Updated IMML for Stock Market Ticker

I’ve made some adjustments to the stock ticker built as part of my building a stock ticker in vastpark post to take advantage of the new Tooltip Plugin and the Define element in IMML. Rather than going all out, I’ve simply added a tip that shows the volume for the current day when the mouse is over one of the green bars and a Define that lets you more easily choose which stocks should be displayed.

If you haven’t already read the original article and are interested in how it works, be sure to have a read before continuing.

Define Element

The Define element is a declarative way of expressing a variable in IMML. It’s equivalent to doing element:set(‘variableName’, variableValue) but doesn’t require interpretation via the scripting engine which results in much better performance. It’s also a much nicer way to store global variables as they can be placed in more a easily identifiable location in the IMML rather than being buried within a script.

My Define in the sample looks similar to this (truncated for brevity):

<!--Use any Yahoo stock codes in the below define to control the stocks being generated and updated-->
 <Define Key="stocks" Value="IPL.AX,VIL.AX,FMG.AX,RIO.AX,BHP.AX,BOQ.AX" />

To change the ticker to show different stocks, replace any of the entries in the Value of the Define with the appropriate Yahoo ticker code for the relevant stock.

Tooltip Plugin

Note: To use the Tooltip plugin, you’ll need to be running either the most recent Player snapshot or version 0.98+ as it uses the Handle property that was added recently to the IRenderEngine interface.

The tooltip concept has been around for quite a while and is used by just about every modern desktop application. Ever wondered what that big red button does and hovered your mouse to find out? If yes, you’ve seen a tooltip. This plugin extends the concept to allow developers to associate a tip with an element in world.

To do so is quite simple, the following functions are provided with the plugin:

int Add(string text, ImmlElement element)
void Remove(int id)
void SetText(int id, string text)

To make use of these, I’ve added 2 lines to _updateStock in the UpdateStocks script:

--update the volume tooltip	
id = lastTradeVisual:get('tooltipId')
tooltipplugin:settext(id, 'Volume: '..volume)

…and 3 lines to _generateStock in the GenerateStocks script:

--store a define value on the last trade to represent volume for the tooltip
lastTradePrim:set('volume', volume)
id = tooltipplugin:add('Volume: '..volume, lastTradePrim)
lastTradePrim:set('tooltipId', id)

The end result still looks similar to before, but we now get tooltips when hovering:

Download the updated IMML here: stock-ticker.imml

February 17, 2010 | Filed Under IMML, Open Source, Plugins, Scripting | 1 Comment 

VastPark Platform 0.98 Released

Along with a fresh look for www.vastpark.com, the 0.98 versions of the Player, Creator, Publisher and Server have been released. This will be the final DirectX only version of the platform as we move towards 1.0

What’s included in 0.98?

Drag and Drop

Drag and Drop allows the user to drag an Item from the Items list into the 3D space. This Item can then be interrogated via script or plugin and an action can occur based on the developer listening to the DragEnter, DragDrop, DragLeave triggers in IMML.

The benefits of this can be extensive, from dynamic presentations with multiple presenters able to drag out their presentation on demand to injecting new content into the scene.

Note: This Items list is populated by the Metaforik index. You can get content into this index using the Publisher application (you must have a publisher account!) which supports Models, Video, Audio, etc, but currently doesn’t support the type “Document”.

Simplified UI

Feedback was received that the UI for the Player was a little complex, so we’ve introduced a menu and removed much of the clutter from the tray area. All of the functionality previously accessible from the tray is now found under the menu at the top of the application.

The address bar now has back, forward and refresh buttons as well as a new icon that indicates the health of the current document and an icon that appears when content is loading in the background.

The debug dialog has been slightly tweaked and simplified also, note that debug rendering options now appear under the menu View -> Render. By default, the LoadQueue tab is now displayed which is useful in working out what is occurring while you are waiting for a document to complete loading.

More of a focus on capture

Along with the support for Continuum capture, the Player now supports capture of screenshots.

Screenshot and Continuum captures default to being stored in your Documents folder under the subfolders “Continuum” and “Screenshots”. This can be customised by visiting the Options menu (Tools -> Options) and selecting a new capture path.

In a future release video capture support is likely to make an appearance as we aim to provide a variety of ways to capture the experience within the Player.

Search Syntax

Some slightly more advanced searching ability is included in the Player’s Item list and follows this syntax:

[type:itemType] [publisher:nameOfPublisher] [tags:tag1, tag2, etc]

Some valid searches that follow this syntax are:

Crate type:Model publisher:craigomatic
– Finds all Model items published by craigomatic that have a name like “Crate”

Crate
– Finds all items that have a name like “Crate”

type:Document
– Finds all items of the type Document

0.98 Platform Changelog

Download link: http://www.vastpark.com/resources/downloads.html

February 17, 2010 | Filed Under IMML, VastPark General | 2 Comments 

Next Page →