Second Alpha
I released the second alpha of the Splicer library last night… you can grab it from the Splicer Codeplex site
Key changes are:
- It’s been FxCop’d – which in turn has seen some
naming issues, misspellings, security issues, incorrect disposal
implementations etc being fixed.
- Implemented some features:
- Support for adding System.Drawing.Image’s to a track (using overloads of the AddImage(…) methods.
- Support for shadow copying input files –
this will take a copy of a clip file, and use that in the ITimeline,
disposing of the copy when the clip is disposed – this is especially
useful when dealing with source filters which don’t like reading the
same file simultaneously.
- AbstractProgressParticipant class which
can be used to generate your own progress participants, 0 or more
participants can be registered for each group. This allows you to
do things like perform screen captures on the output of the timeline.
- Windows media renderer will now notify you when the selected rendering profile is unsuitable for your timeline.
- Fixed some bugs
- The clock is now correctly disabled for the
filter graph when rendering to a file. This will speed up
rendering in many scenarios.
- When supplying an audio encoder for either WAV
or AVI output, the format settings for the encoder were not being
applied properly (was using defaults, instead of setting the number of
channels, khz & kbps).
Though
there are also plenty more, I doubt anyone has actually been using this
library
so far (as it's really only useful in a rather select set of circumstances) -
however just to offer some vague hope of encouragement I’ll include
some code
samples to get you started – as there is no documentation short of the
150 odd
unit tests included with the project... and won't be for some time.
Example 1
First off, lets imagine you have some web 2.0 website, and you
encourage people to upload audio comments, instead of text, yet you don’t know what
format they’re going to provide, and you want to keep things standard
(say by outputting a certain windows media audio format) that eases pressure on your web server.
using (ITimeline timeline = new DefaultTimeline())
{
IGroup audioGroup = timeline.AddAudioGroup();
ITrack rootTrack = audioGroup.AddTrack();
rootTrack.AddAudio("testinput.mp3");
using (
WindowsMediaRenderer renderer =
new WindowsMediaRenderer(timeline, "output.wma", WindowsMediaProfiles.LowQualityAudio))
{
renderer.Render();
}
}
Here we are creating a
Timeline, this is a container for all the audio
and video compositing we are doing, next we create a group for our
audio, a group is a top level composition… we then add a track to our
audio group, which is a container for our audio clip we wish to add (in
this case it’s an mp3, but it could be any audio format that you have a
codec installed for).
Once your Timeline is ready to be rendered, we create a renderer, in
this case it’s a windows media renderer, which takes your timeline, the
name of the output file and the windows media profile we wish to use –
profiles are xml strings containing the settings for the windows media
encoder – A couple of bundled ones are included, though you would
probably want to create your own using the “Windows Media Profile
Editor” which is included with
Windows Media Encoder 9.
Last of all we invoke the Renderer, generally rendering can take a
little while, depening on the size of the content, and the complexity
of the encoding process - so I would suggest invoking the Renderer
asynchronously
(all Renderers support the BeginRender(…) … EndRender(…) methods which
should make this trivial). Our examples will use the blocking
Render() method, just because it's more concise.
Example 2
So audio cool, what about video?
Much the same:
using (ITimeline timeline = new DefaultTimeline())
{
timeline.AddAudioGroup().AddTrack();
timeline.AddVideoGroup(24, 320, 240).AddTrack();
timeline.AddVideoWithAudio("input.avi");
using (
WindowsMediaRenderer renderer =
new WindowsMediaRenderer(timeline, "output.wmv", WindowsMediaProfiles.LowQualityVideo))
{
renderer.Render();
}
}
This time we create two groups, and two tracks, one of each for the
audio from our source file, and another of each for the video.
Note we’ve specified the format of the video a little – so it will be 24 bits per pixel, and 320 high by 240 wide.
In this case we use a helpful method on the timeline itself which will
Add a video clip to the video group and an audio clip to the audio
group at the same time, called AddVideoWithAudio… Again we render it to
windows media format, but this time we use a profile which has settings
for both audio and video.
The bits per pixel becomes important when using something transitions
or effects, as you need an alpha channel for them to work - this is
generally achieved but just bumping the BPP from 24 to 32.
Example 3
Right, moving on from here, lets cover some more interesting ideas – first off, lets build a slide show video…
using (ITimeline timeline = new DefaultTimeline(25))
{
IGroup group = timeline.AddVideoGroup(32, 160, 100);
ITrack videoTrack = group.AddTrack();
IClip clip1 = videoTrack.AddImage("image1.jpg", 0, 2);
IClip clip2 = videoTrack.AddImage("image2.jpg", 0, 2);
IClip clip3 = videoTrack.AddImage("image3.jpg", 0, 2);
IClip clip4 = videoTrack.AddImage("image4.jpg", 0, 2);
double halfDuration = 0.5;
group.AddTransition(clip2.Offset - halfDuration, halfDuration, StandardTransitions.CreateFade(), true);
group.AddTransition(clip2.Offset, halfDuration, StandardTransitions.CreateFade(), false);
group.AddTransition(clip3.Offset - halfDuration, halfDuration, StandardTransitions.CreateFade(), true);
group.AddTransition(clip3.Offset, halfDuration, StandardTransitions.CreateFade(), false);
group.AddTransition(clip4.Offset - halfDuration, halfDuration, StandardTransitions.CreateFade(), true);
group.AddTransition(clip4.Offset, halfDuration, StandardTransitions.CreateFade(), false);
ITrack audioTrack = timeline.AddAudioGroup().AddTrack();
IClip audio =
audioTrack.AddAudio("soundtrack.wav", 0, videoTrack.Duration);
audioTrack.AddEffect(0, audio.Duration,
StandardEffects.CreateAudioEnvelope(1.0, 1.0, 1.0, audio.Duration));
using (
WindowsMediaRenderer renderer =
new WindowsMediaRenderer(timeline, "output.wmv", WindowsMediaProfiles.HighQualityVideo))
{
renderer.Render();
}
}
In this case we are producing a slideshow – Note that
25 in the constructor of the timline, this is setting the frames per
second, in this case we are saying 25fps… The renderer has the
final say in the “rendered” frames per second, based on it’s settings…
but some renderers will use the frame rate they receive, such as the
windows renderer (which renders the video into a window on-screen)
The slideshow fades between 4 still images, each lasts for a duration
of 2 seconds, we add in a couple of transitions, fading out of one clip
and into another – last of all we are using an AudioMixer effect on the
audio track to create an Audio envelope, which means it will take one
second for the audio to fade in from 0.0 volume to 1.0 volume (0 to
100%) and then play until one second before the end of the clip, at
which point it will fade out from 100% volume to 0% volume.
Example 4
Last of all, you can capture rendered frames from a video, this might
be handy to display on a web page, with links to the various quality
formats beneath – you can grab these frames during encoding, or you can
render to “null”, which allows you the grab frames, but wont produce
any output video or audio file… Lets take a look at the second idea:
using (DefaultTimeline timeline = new DefaultTimeline())
{
timeline.AddVideoGroup(24, 320, 240).AddTrack(); // we want 320x240 24bpp sized images
timeline.AddVideo("transitions.wmv"); // 8 second video clip
ImagesToDiskParticipant participant = new ImagesToDiskParticipant(24, 320, 240, Environment.CurrentDirectory, 1, 2, 3, 4, 5, 6, 7);
using (NullRenderer render = new NullRenderer(timeline, null, new ICallbackParticipant[] { participant }))
{
render.Render();
}
}
Here we add a video group (we don’t need any audio, we’re not capturing
sound samples) and then create an
ImageToDiskParticipant (this is just
an example class, you would probably want to implement your own derived
from the
AbstractProgressParticipant class). In this case the
format for the video group will be what you end up with, our
ImagesToDiskParticipant implements a simple queue where it picks out
the nearest frame to the specified times.. so in this case
transitions.wmv is an 8 second video clip, and we will take frame "snapshots" at 1,2,3,4,5,6 and 7 seconds aproximately.
To make use of our participant we use one of the overloads of
NullRenderer, and supply it a single-element array containing the image
to disk participant.
The end result is 7 jpeg images on disk (frame0.jpg -> frame6.jpg)
Hopefully this has given you some ideas about how Splicer can be used
for dealing width audio and video, next time I do an alpha drop I might include some samples
which use different container formats (WAV & AVI) - or maybe I'll
use splicer as an example of just how not to write an API you wish to
expose to IronPython due to it's bloated number of overloads.