Second Alpha

I released the second alpha of the Splicer library last night… you can grab it from the Splicer Codeplex site

Key changes are:

• It’s been FxCop’d – which in turn has seen some naming issues, misspellings, security issues, incorrect disposal implementations etc being fixed.
• Implemented some features:
• Support for adding System.Drawing.Image’s to a track (using overloads of the AddImage(…) methods.
• Support for shadow copying input files – this will take a copy of a clip file, and use that in the ITimeline, disposing of the copy when the clip is disposed – this is especially useful when dealing with source filters which don’t like reading the same file simultaneously.
• AbstractProgressParticipant class which can be used to generate your own progress participants, 0 or more participants can be registered for each group.  This allows you to do things like perform screen captures on the output of the timeline.
• Windows media renderer will now notify you when the selected rendering profile is unsuitable for your timeline.
• Fixed some bugs
• The clock is now correctly disabled for the filter graph when rendering to a file.  This will speed up rendering in many scenarios.
• When supplying an audio encoder for either WAV or AVI output, the format settings for the encoder were not being applied properly (was using defaults, instead of setting the number of channels, khz & kbps).

Though there are also plenty more, I doubt anyone has actually been using this library so far (as it's really only useful in a rather select set of circumstances) - however just to offer some vague hope of encouragement I’ll include some code samples to get you started – as there is no documentation short of the 150 odd unit tests included with the project... and won't be for some time.

Example 1 

First off, lets imagine you have some web 2.0 website, and you encourage people to upload audio comments, instead of text, yet you don’t know what format they’re going to provide, and you want to keep things standard (say by outputting a certain windows media audio format) that eases pressure on your web server.


Here we are creating a Timeline, this is a container for all the audio and video compositing we are doing, next we create a group for our audio, a group is a top level composition… we then add a track to our audio group, which is a container for our audio clip we wish to add (in this case it’s an mp3, but it could be any audio format that you have a codec installed for).

Once your Timeline is ready to be rendered, we create a renderer, in this case it’s a windows media renderer, which takes your timeline, the name of the output file and the windows media profile we wish to use – profiles are xml strings containing the settings for the windows media encoder – A couple of bundled ones are included, though you would probably want to create your own using the “Windows Media Profile Editor” which is included with Windows Media Encoder 9.

Last of all we invoke the Renderer, generally rendering can take a little while, depening on the size of the content, and the complexity of the encoding process - so I would suggest invoking the Renderer asynchronously (all Renderers support the BeginRender(…) … EndRender(…) methods which should make this trivial).  Our examples will use the blocking Render() method, just because it's more concise.

Example 2

So audio cool, what about video?

Much the same:



This time we create two groups, and two tracks, one of each for the audio from our source file, and another of each for the video.

Note we’ve specified the format of the video a little – so it will be 24 bits per pixel, and 320 high by 240 wide.
 
In this case we use a helpful method on the timeline itself which will Add a video clip to the video group and an audio clip to the audio group at the same time, called AddVideoWithAudio… Again we render it to windows media format, but this time we use a profile which has settings for both audio and video.

The bits per pixel becomes important when using something transitions or effects, as you need an alpha channel for them to work - this is generally achieved but just bumping the BPP from 24 to 32.

Example 3

Right, moving on from here, lets cover some more interesting ideas – first off, lets build a slide show video…



In this case we are producing a slideshow – Note that 25 in the constructor of the timline, this is setting the frames per second, in this case we are saying 25fps…  The renderer has the final say in the “rendered” frames per second, based on it’s settings… but some renderers will use the frame rate they receive, such as the windows renderer (which renders the video into a window on-screen)

The slideshow fades between 4 still images, each lasts for a duration of 2 seconds, we add in a couple of transitions, fading out of one clip and into another – last of all we are using an AudioMixer effect on the audio track to create an Audio envelope, which means it will take one second for the audio to fade in from 0.0 volume to 1.0 volume (0 to 100%) and then play until one second before the end of the clip, at which point it will fade out from 100% volume to 0% volume.

Example 4

Last of all, you can capture rendered frames from a video, this might be handy to display on a web page, with links to the various quality formats beneath – you can grab these frames during encoding, or you can render to “null”, which allows you the grab frames, but wont produce any output video or audio file… Lets take a look at the second idea:


Here we add a video group (we don’t need any audio, we’re not capturing sound samples) and then create an ImageToDiskParticipant (this is just an example class, you would probably want to implement your own derived from the AbstractProgressParticipant class).  In this case the format for the video group will be what you end up with, our ImagesToDiskParticipant implements a simple queue where it picks out the nearest frame to the specified times.. so in this case transitions.wmv is an 8 second video clip, and we will take frame "snapshots" at 1,2,3,4,5,6 and 7 seconds aproximately. 

To make use of our participant we use one of the overloads of NullRenderer, and supply it a single-element array containing the image to disk participant.

The end result is 7 jpeg images on disk (frame0.jpg -> frame6.jpg)

Hopefully this has given you some ideas about how Splicer can be used for dealing width audio and video, next time I do an alpha drop I might include some samples which use different container formats (WAV & AVI) - or maybe I'll use splicer as an example of just how not to write an API you wish to expose to IronPython due to it's bloated number of overloads.