Presentation: The Technology Behind VastPark

Today I presented at La Cantine in Paris to both a real world audience and a virtual audience inside of VastPark.

The presentation covered the technology that powers VastPark, where to find the open source code and the specifications such as IMML, Continuum, and the recently announced OpenAvatar. I also showed some live demonstrations of VastPark in action (the selective Continuum capture I did can be downloaded from here).

See the slideshare below, there was a lot of talking during the presentation so the slides may seem a little light on content. If you have questions about the slides feel free to comment and I will answer.

La Cantine Presentation, Paris, 10th June 2011
June 11, 2011 | Filed Under IMML, Open Source, Plugins, Scripting, VastPark General | Comments Off on Presentation: The Technology Behind VastPark 

Plugin Framework available via NuGet

After a recent visit to Las Vegas for MIX11 (which was excellent btw) I was taken by the usefulness of NuGet and decided it was time to contribute a package into the main feed.

I’ve been developing a plugin framework, imaginatively named VastPark.PluginFramework which I hope to eventually massage into a really convenient MVC framework.

As it stands today it is a useful time saving library to be familiar with and I’d recommend all VastPark plugin developers make use of it.

1. Install NuGet

This has been covered in detail elsewhere and chances are you already have it installed as it is included in MVC3.

Visit the CodePlex documentation site here for a simple install guide.

2. Create a new plugin project in Visual Studio

As per usual, create a new project in Visual Studio:

Note: It’s best to go with .NET Framework 3.5 as the target framework, the client applications are built against this version of the .NET framework.

3. Add package reference to VastPark.PluginFramework

Right-click References and choose the shiny Add Library Package Reference button:

Next perform a search for ‘VastPark’:

Click install to download the package and have it magically added to your project.

You should now see some additional assemblies referenced and a blueprint.xml file included in your project:

4. Code your plugin!

Sit back for a moment to ponder the awesome of NuGet, then get on with coding your plugin 🙂

See here for the NuGet package page: http://nuget.org/List/Packages/VastPark.PluginFramework

April 21, 2011 | Filed Under Plugins | Comments Off on Plugin Framework available via NuGet 

The Pianist v2

Back in April of 2007 I was inspired to build a virtual piano in VastPark, you can read the super brief post about it here.

At the time, I was pretty happy with the way it turned out and recently decided it was time for it to make a comeback!

How it works

This time around I decided to do things a little differently and shift all of the logic to a single .NET plugin that controls the entire piano experience.

While this has a number of benefits, I was primarily interested in keeping the structure of the environment cleanly separated from the logic.

To keep this separation simple, I made use of a little known feature of IMML called behaviours. Here’s a sample of my IMML:

<IMML Name="The Pianist v2" Camera="camera" xmlns="http://schemas.vastpark.com/2007/imml/">

  <!-- The plugin manages all interaction with the piano via behaviours -->
  <Plugin Name="PianoPlugin" Enabled="True" Source="plugins/Piano.plugin">
    <Parameter Key="HighlightColour" Value="#00FF00" />
    <Parameter Key="RotationWhenPressed" Value="0.03120605, 0, 0" />
  </Plugin>

  <Model Name="black_piano"
         Size="1.193525,1.065062,0.524349"
         Rotation="6.283185,0,0"
         Position="4.76837E-07,1.63393E-08,0.111626"
         Source="models/black_piano.model" />

  <!-- Piano key models -->
  <Model Name="key_b_8"
         Size="0.01918364,0.02875674,0.1308559"
         Rotation="6.283185,0,0"
         Position="-0.4839502,0.8194631,0.2247874"
         Behaviours="piano-key"
         Source="models/white_key1.model" />
  <!-- other models snipped for brevity -->

  <!-- Piano sounds -->
  <Sound Name="note_b_8" 
         Behaviours="piano-key-sound" 
         Source="sounds/pianokey_b_8.mp3" />
  <!-- other sounds snipped for brevity -->

</IMML>

Note the usage of the piano-key and piano-key-sound behaviours.

Also, I’ve only shown one model and one sound, the actual file contains 88 of each named according to a convention so that the appropriate sound can be mapped.

Using Behaviours

Every IMML element has the ability to be marked as having one or more behaviours that can be accessed as a list via the DOM.

This is extremely handy when writing plugins that manipulate elements, as you can simply query the scene for elements that match the desired behaviours like so:

//find all audio and models based on behaviour
var keyModels = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeyModelBehaviour));
var keySounds = base.ParkEngine.Context.Elements.Where(e => e.Behaviours.Contains(this.KeySoundBehaviour));

if(keyModels.Any())
{
    //build keys for each model/sound combination
}

This works really well during the Load() method and can also be performed on elements dynamically added into the scene by listening to the ParkEngine.ElementLoaded event

In the context of the-pianist, it finds all elements of type piano-key and piano-key-sound and builds a PianoKey instance to manage them so that:

Continuum

Next, I wanted to make sure that the work my plugin was doing would be correctly captured by Continuum. By default, Continuum will happily capture any series of changes in IMML state, so this initially didn’t seem like it would be a concern.

However, in my implementation of the Piano plugin I wanted to allow the same key to be pressed while the previous sound for that key was still playing. Just like a real piano.

To do this, I’ve been a little sneaky by going directly to the sound engine, bypassing the IMML change notification infrastructure which means that Continuum doesn’t see that change and cannot record it without some additional help.

You can see the two contrasting approaches below:

//toggle approach
this.Audible.Enabled = true;

//direct approach, bypass the IMML change framework
this.ParkEngine.SoundEngine.Play(this.Audible);

To overcome this limitation, I wrote an implementation of IStateRecorder for my plugin with one very simple method:

/// <summary>
/// Marks the sound as being played directly by the SoundEngine.
/// </summary>
/// <param name="sound">The sound.</param>
public void MarkSoundPlayed(Sound sound)
{
        if (!this.IsStarted)
        {
            return;
        }

        var captureState = new PianoCaptureState(Encoding.ASCII.GetBytes(sound.Name), Constants.Guid, DateTime.UtcNow, 0);
            
        this.Buffer.Enqueue(captureState);
}

It takes the name of the element that was played directly and writes it into the Continuum stream.

Later during playback, that element is resolved and at the appropriate time is played directly by the PianoStateController.

Goodies

I’ve zipped up the full source code to Plugin.Piano, along with the IMML and models you see in the screenshot at the beginning of this post.

Download here: http://tpiv.s3.amazonaws.com/the-pianist-v2.7z

Note that In order to run this properly, you’ll need to be a member of the Closed Beta community on vastpark.org (contact me if you’d like in) and be rocking Player v1.5.2 build 92 or newer.

Update: A more recent version of the-pianist IMML is available here.

Simply unzip into the hosting root directory of your WorldServer to host.

April 15, 2011 | Filed Under IMML, Open Source, Plugins | Comments Off on The Pianist v2 

VastPark Developer Introduction – v1.3 Framework

Update: The video was re-dubbed, you now get the pleasure of Liz’s delightful voice 🙂

I recently gave a talk to a team of developers interested in working with the platform who are planning to build some new exciting products on top of it.

One of the audience offered to record the session remotely so the audio is a little scratchy, but I think it’s come across ok.

The talk covers:

Hopefully this is a useful primer if you are interested in developing with VastPark or are curious to know a little more about the way the system functions.

Download the PDF shown in the video here: VastPark Developer Introduction - Framework v1.3

July 30, 2010 | Filed Under IMML, Plugins, Scripting, VastPark General | Comments Off on VastPark Developer Introduction – v1.3 Framework 

Flocking algorithm in VastPark

Many months ago during May of 09, after meeting Ian Hughes at FCVW, I came across an interesting post on his blog discussing an implementation of the flocking algorithm in Unity.

I’d not heard of the term flocking or boids, so proceeded to get stuck into some research, which lead me to Conrad Parker’s page on Boids Pseudocode as well as many others who had implemented the algorithm, I figured it would be a fun little plugin to write for VastPark so gave it a try.

Implementing the algorithm

Thinking about what was required, I came up with the following as the plan for the plugin:

In terms of the plugin this translates into these properties:

The following callbacks:

The following methods:

The class diagrams look like this:

How does it work?

Put quite simply, the plugin works like this:

Downloads

Here is some sample IMML and the source code to the plugin, enjoy! Let me know if you decide to implement any of the other rules or do something cool with it 🙂

Sample IMML: flocking.imml
Source code to plugin: Plugin.Flocking.zip

Note: The source code will probably complain about some missing references. Install the Player from www.vastpark.com and link against the relevant binaries included with it.

April 13, 2010 | Filed Under IMML, Open Source, Plugins, Scripting | Comments Off on Flocking algorithm in VastPark 

Drag and Drop Sample

One of the major features of the 0.98 release is the inclusion of drag and drop. Basically, this allows you to drag an Item into the 3D space where the IMML can decide to do something with it.

To try drag and drop out for yourself, first download generic-drag-drop-sample.imml (right-click, save as) and open in the Player.

Next, you’ll need to expand the Items list. Do so by clicking on this icon:

By default you are presented with the list of Items of type Video and Document that the currently logged in account has access to (these can be further restricted by the hosting server’s group ACL).

In this case as we are opening the file directly, it will default to showing the most recently published Items without filtering and should look similar to this:

Click to select one of the items from the list (in my example, I’ve chosen “VastPark Logo”) and drag it onto the 3D view. The large rectangle in the centre of the screen will change colour to blue indicating it is a drop target.

Next, drop the Item onto the blue rectangle, which will cause it to change colour to green. Shortly the VastPark logo will appear on it.

How does this work?

Behind the scenes is a plugin (called MetaforikAdapter) which currently supports adapting Items of type “Document” and “Video” to a Model or Primitive element. The code to do this is quite simple.

First, reference the Plugin:

<Plugin Enabled="True" Source="http://id.vastpark.com/VastParkWS/get.vpws?name=MetaforikAdapter&amp;publisher=craigomatic&amp;domain=vastpark&amp;context=park" Name="MetaforikAdapter">
    <Element Name="InvalidRequestHandler" />
    <Parameter Key="InvalidRequestEvent" Value="InvalidRequestHandler" />    
  </Plugin>   
  
  <Script Name="InvalidRequestHandler">
    function main(obj, args)
      scene.ui:writeline('invalid request')          
    end
  </Script>

Next, handle the appropriate events on the elements you want to be able to drop on. Alternatively, this can be at the document level:
<Trigger Event="DragDrop" Target="OnDragDrop" />
<Trigger Event="DragEnter" Target="OnDragEnter" />
<Trigger Event="DragLeave" Target="OnDragLeave" />

In you script for the drop, pass the Item along with the element you want to adapt it to into the load method on the plugin. You may choose to inspect the Item before the drop to give user feedback as is done in the sample IMML.

<Script Name="OnDragDrop">
    function main(obj, args)            
      --args.data contains the Item which was dragged in
      
      --load returns a controller object
      c = metaforikadapter:load(args.data, obj)      
    
      scene.ui:writeline('item name: '..c.item.name)
      scene.ui:writeline('id: '..c.id)
      scene.ui:writeline('is first: '..tostring(c.isfirst))
      scene.ui:writeline('is last: '..tostring(c.islast))
      scene.ui:writeline('total pages: '..tostring(c.pages))
      scene.ui:writeline('current page: '..tostring(c.currentpage))
    end
</Script>
  
<Script Name="OnDragEnter">
    function main(obj, args)
        scene.ui:writeline('drag enter on: '..obj.name..' of type: '..args.data.assets[0].type)     
    end
</Script>
  
<Script Name="OnDragLeave">
    function main(obj, args)
      scene.ui:writeline('drag leave on: '..obj.name)
    end
</Script>

Working with elements other than Document or Video

One thing to be aware of is that you don’t necessarily have to use the MetaforikAdapter plugin to benefit from drag and drop. If you want to allow for models or sounds to be dropped in, you can use the item.updateuri as the source for the element and add it into the scene, like this:

<Script Name="OnDragDrop">
    function main(obj, args)    
      item = args.data
      if(item.assets[0].type == 'Model') then       
          m = model()
          m.source = args.data.updateuri
      
          scene:add(m)     
      end
    end
</Script>

Final Note

That’s really all there is to it, you may want to craft different UI depending on the state of the controller properties in the MetaforikAdapter plugin sample, ie: to handle a multi-page document differently to a video or single page document, or in the model example to only allow certain users to drop models into a specific region of your space.

February 20, 2010 | Filed Under IMML, Plugins, Scripting | Comments Off on Drag and Drop Sample 

Updated IMML for Stock Market Ticker

I’ve made some adjustments to the stock ticker built as part of my building a stock ticker in vastpark post to take advantage of the new Tooltip Plugin and the Define element in IMML. Rather than going all out, I’ve simply added a tip that shows the volume for the current day when the mouse is over one of the green bars and a Define that lets you more easily choose which stocks should be displayed.

If you haven’t already read the original article and are interested in how it works, be sure to have a read before continuing.

Define Element

The Define element is a declarative way of expressing a variable in IMML. It’s equivalent to doing element:set(‘variableName’, variableValue) but doesn’t require interpretation via the scripting engine which results in much better performance. It’s also a much nicer way to store global variables as they can be placed in more a easily identifiable location in the IMML rather than being buried within a script.

My Define in the sample looks similar to this (truncated for brevity):

<!--Use any Yahoo stock codes in the below define to control the stocks being generated and updated-->
 <Define Key="stocks" Value="IPL.AX,VIL.AX,FMG.AX,RIO.AX,BHP.AX,BOQ.AX" />

To change the ticker to show different stocks, replace any of the entries in the Value of the Define with the appropriate Yahoo ticker code for the relevant stock.

Tooltip Plugin

Note: To use the Tooltip plugin, you’ll need to be running either the most recent Player snapshot or version 0.98+ as it uses the Handle property that was added recently to the IRenderEngine interface.

The tooltip concept has been around for quite a while and is used by just about every modern desktop application. Ever wondered what that big red button does and hovered your mouse to find out? If yes, you’ve seen a tooltip. This plugin extends the concept to allow developers to associate a tip with an element in world.

To do so is quite simple, the following functions are provided with the plugin:

int Add(string text, ImmlElement element)
void Remove(int id)
void SetText(int id, string text)

To make use of these, I’ve added 2 lines to _updateStock in the UpdateStocks script:

--update the volume tooltip	
id = lastTradeVisual:get('tooltipId')
tooltipplugin:settext(id, 'Volume: '..volume)

…and 3 lines to _generateStock in the GenerateStocks script:

--store a define value on the last trade to represent volume for the tooltip
lastTradePrim:set('volume', volume)
id = tooltipplugin:add('Volume: '..volume, lastTradePrim)
lastTradePrim:set('tooltipId', id)

The end result still looks similar to before, but we now get tooltips when hovering:

Download the updated IMML here: stock-ticker.imml

February 17, 2010 | Filed Under IMML, Open Source, Plugins, Scripting | 1 Comment 

Dem bones, dem bones, dem…digital bones

The creative department have been craving bones for a very long time in VastPark and we’ve finally added support for them, both in scripting and via plugin.

Here’s a teaser:

Vastpark Facial Expressions Demo from VastPark on Vimeo.

September 1, 2009 | Filed Under Content, Misc, Plugins, Scripting | Comments Off on Dem bones, dem bones, dem…digital bones 

VastPark VNC Plugin

After some recent improvements to the core VastPark framework and the way textures are made available to the plugin API, I decided to build a VNC plugin for VastPark, basing it upon VncSharp. For those of you know don’t know what VNC is, there’s a good overview of it available on Wikipedia.

In it’s simplest form the IMML for the VNC Plugin will look something like this:

<Plugin Name="VncPlugin" Enabled="True" 
Source="http://content.vastpark.com/VastParkWS/get.vpws?publisher=craigomatic&amp;name=VncPlugin&amp;domain=vastpark&amp;context=park">
	<Element Name="TargetElement"/>
	<Element Name="OnPasswordRequired"/>
	<Parameter Key="Uri" Value="the-vnc-server-address.com"/>
	<Parameter Key="Port" Value="5900"/>
 	<Parameter Key="PasswordRequired" Value="OnPasswordRequired"/>
</Plugin>
<Script Name="OnPasswordRequired">
    function main(obj, args)
        vncplugin:setpassword('pass') --hardcoding password is bad, you should ask the user to enter one!
    end
</Script>
<Primitive Type="Box" Name="TargetElement" Size="1,1,1"/>

Here’s one I prepared earlier (well, Adrian did most of the work…):

vncplugin-in-action

Plugin Parameters

Uri

The host address of the VNC server. You should avoid using the scheme at the start of this address (ie: don’t put the http://)

Port

Defaults to the VNC default of 5900

ConnectionFailed

Name of a Script/Timeline/other ITimelineExecutable element to execute when a VNC connection has failed

PasswordRequired

Name of a Script/Timeline/other ITimelineExecutable element to execute when a password is required to connect to the VNC server

UpdateFrequency

The number of milliseconds to wait before automatically refreshing the remote screen when ManualUpdate is false. Defaults to 500.

ManualUpdate

True/False. When true, you need to call the vncplugin:requestupdate(true) for a fullscreen update. Defaults to False

You can download an example IMML document here: basic-vnc.imml and view the source code for the VNC Plugin over here: http://vastpark-svn.cvsdude.com/public/trunk/Plugins/Plugin.VNC/

A good free windows VNC Server is TightVNC

April 8, 2009 | Filed Under IMML, Open Source, Plugins | Comments Off on VastPark VNC Plugin