You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
575 lines
21 KiB
575 lines
21 KiB
Westley Documentation
|
|
|
|
Copyright (C) 2004 Ushodaya Enterprised Limited
|
|
Authors: Charles Yates <charles.yates@pandora.be>
|
|
Last Revision: 2004-03-20
|
|
|
|
|
|
WESTLEY
|
|
-------
|
|
|
|
Preamble:
|
|
|
|
Westley is the MLT projects XML serialisation/deserialisation format -
|
|
as such, it closely mirrors the internal structure of the MLT API.
|
|
|
|
If you just want to go straight to the DTD, then see
|
|
mlt/src/modules/westley/westley.dtd, which gets installed at
|
|
$(prefix)/share/mlt/modules/westley.dtd. Currently, the westley parser is
|
|
non-validating.
|
|
|
|
|
|
Introduction:
|
|
|
|
A westley document is essentially a list of 'producers' - a producer is
|
|
an mlt object which generates mlt frames (images and associated audio
|
|
samples).
|
|
|
|
There are 3 types of producer:
|
|
|
|
* Basic Producers - these are typically file or device oriented feeds;
|
|
* Playlists - these are arrangements of multiple producers;
|
|
* Multitracks - these are the fx encapsulators.
|
|
|
|
In the mlt model, producers are created and attached to 'consumers' -
|
|
consumers are software playback components (such as SDL), or wrappers for
|
|
hardware drivers (such as bluefish) or even the westley serialising
|
|
consumer itself (the latter doesn't receive frames - it merely
|
|
interrogates the connected producer for its configuration).
|
|
|
|
Although westley was defined as a serialisation mechanism for instantiated
|
|
MLT components, this document will concentrate on the hand authoring of
|
|
westley documents.
|
|
|
|
|
|
Rules:
|
|
|
|
As shall become apparent through the remainder of this document, the basic
|
|
tenet of westley authoring is to organise the document in the following
|
|
manner:
|
|
|
|
1) create producer elements for each unique media clip in the project;
|
|
2) create playlists for each track;
|
|
3) create a multitrack and specify filters and transitions;
|
|
4) adding global filters.
|
|
|
|
While other uses of westley exist, the approach taken here is to maximise
|
|
efficiency for complex projects.
|
|
|
|
|
|
Basic Producers:
|
|
|
|
The simplest westley document is:
|
|
|
|
<westley>
|
|
<producer id="producer0">
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
</westley>
|
|
|
|
The westley wrapping is of course superfluous here - loading this document
|
|
with MLT is identical to loading the clip directly.
|
|
|
|
Of course, you can specify additional properties. For example, consider an
|
|
MPEG file with multiple soundtracks - you could define a westley document to
|
|
ensure that the second audio track is loaded:
|
|
|
|
<westley>
|
|
<producer id="producer0">
|
|
<property name="resource">clip1.mpeg</property>
|
|
<property name="audio_track">1</property>
|
|
</producer>
|
|
</westley>
|
|
|
|
NB: This relies on the mpeg being handled by the avformat producer, rather
|
|
than the mcmpeg one. See services.txt for more details.
|
|
|
|
A more useful example comes with the pango producer for a text producer.
|
|
|
|
TODO: pango example...
|
|
|
|
Notes:
|
|
|
|
1) It is better not to specify in/out points when defining basic producers
|
|
as these can be specified in the playlists. The reasoning is that in/out
|
|
restricts the amount of the clip available, and could lead to the same clip
|
|
being loaded multiple times if you need different regions of the clip
|
|
elsewhere;
|
|
2) A westley can be specified as a resource, so westleys can naturally
|
|
encapsulate other westleys.
|
|
|
|
|
|
Playlists:
|
|
|
|
Playlists provide a 'collection' structure for producers. These can be used
|
|
to define 'tracks' in the multitrack approach, or simple playlists for
|
|
sequential, single track playout.
|
|
|
|
As an example, the following defines two basic producers and a playlist with 3
|
|
items:
|
|
|
|
<westley>
|
|
<producer id="producer0">
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
<producer id="producer1">
|
|
<property name="resource">clip2.dv</property>
|
|
</producer>
|
|
<playlist id="playlist0">
|
|
<entry producer="producer0" in="0" out="2999"/>
|
|
<entry producer="producer1" in="0" out="999"/>
|
|
<entry producer="producer0" in="3000" out="6999"/>
|
|
</playlist>
|
|
</westley>
|
|
|
|
Here we see how the playlist defines the in/out points of the basic
|
|
producers.
|
|
|
|
Notes:
|
|
|
|
1) All in/out points are absolute frame positions relative to the producer
|
|
being appended to the playlist;
|
|
2) Westley documents are currently authored for a specific normalisation;
|
|
3) The last 'producer' in the document is the default for play out;
|
|
4) Playlists can reference the same producer multiple times. In/out regions
|
|
do not need to be contiguous - duplication and skipping is acceptable.
|
|
|
|
|
|
Interlude - Introducing Multitracks:
|
|
|
|
So far we've defined basic producers and playlists/tracks - the tractor is
|
|
the element that allows us to arrange our tracks and specify filters and
|
|
transitions. Similarly to a playlist, a tractor is a container.
|
|
|
|
Note that MLT doesn't see a filter or a transition as a producer in the
|
|
normal sense - filters and transitions are passive when it comes to seeking.
|
|
Internally, seeks are carried out on the producers. This is an important
|
|
point - MLT does not follow a traditional graph oriented model.
|
|
|
|
Visualising an MLT tractor and it's interaction with the consumer will
|
|
assist here:
|
|
|
|
+----------------------------------------------+
|
|
|tractor |
|
|
| +----------+ +-+ +-+ +-+ +-+ |
|
|
| |multitrack| |f| |f| |t| |t| |
|
|
| | +------+ | |i| |i| |r| |r| |
|
|
| | |track0|-|--->|l|- ->|l|- ->|a|--->|a|\ |
|
|
| | +------+ | |t| |t| |n| |n| \ |
|
|
| | | |e| |e| |s| |s| \ |
|
|
| | +------+ | |r| |r| |i| |i| \ | +--------+
|
|
| | |track1|-|- ->|0|--->|1|--->|t|--->|t|-----|--->|consumer|
|
|
| | +------+ | | | | | |i| |i| / | +--------+
|
|
| | | | | | | |o| |o| / | ^
|
|
| | +------+ | | | | | |n| |n| / | |
|
|
| | |track2|-|- ->| |- ->| |--->|0|- ->|1|/ | |
|
|
| | +------+ | | | | | | | | | | |
|
|
| +----------+ +-+ +-+ +-+ +-+ | |
|
|
+----------------------------------------------+ |
|
|
^ |
|
|
| |
|
|
+-----------+ |
|
|
|APPLICATION|--------------------------------------------+
|
|
+-----------+
|
|
|
|
Internally, all frames from all tracks pass through all the filters and
|
|
transitions - these are told which tracks to deal and which regions of the
|
|
tracks to work on.
|
|
|
|
Note that the application communicates with the producer - it can alter
|
|
playback speed, position, or even which producer is connected to which
|
|
consumer.
|
|
|
|
The consumer receives the first non-blank frame (see below). It has no say
|
|
in the order in which gets them (the sdl consumer when used with inigo might
|
|
appear to be an exception - it isn't - it simply has a route back to the
|
|
application to allow the application to interpret key presses).
|
|
|
|
|
|
Tractors:
|
|
|
|
To create a multitrack westley, we can use two playlists and introduce a
|
|
tractor. For the purposes of demonstration, I'll add a filter here too:
|
|
|
|
<westley>
|
|
<producer id="producer0">
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
<producer id="producer1">
|
|
<property name="resource">clip2.dv</property>
|
|
</producer>
|
|
<playlist id="playlist0">
|
|
<entry producer="producer0" in="0" out="2999"/>
|
|
<blank length="1000"/>
|
|
<entry producer="producer0" in="3000" out="6999"/>
|
|
<playlist id="playlist1">
|
|
<blank length="3000"/>
|
|
<entry producer="producer1" in="0" out="999"/>
|
|
</playlist>
|
|
<tractor id="tractor0">
|
|
<multitrack>
|
|
<track producer="playlist0"/>
|
|
<track producer="playlist1"/>
|
|
</multitrack>
|
|
<filter>
|
|
<property name="track">0</property>
|
|
<property name="mlt_service">greyscale</property>
|
|
</filter>
|
|
</tractor>
|
|
</westley>
|
|
|
|
Here we see that blank frames are inserted into the first playlist and a
|
|
blank is provided at the beginning of the second - this can be visualised in
|
|
the traditional timeline widget as follows:
|
|
|
|
+-------+ +-------------+
|
|
|a | |a |
|
|
+-------+---+-------------+
|
|
|b |
|
|
+---+
|
|
|
|
Adding the filter on the top track, gives us:
|
|
|
|
+-------+ +-------------+
|
|
|a | |a |
|
|
+-------+---+-------------+
|
|
|greyscale |
|
|
--------+---+-------------+
|
|
|b |
|
|
+---+
|
|
|
|
Note that it's only applied to the visible parts of the top track.
|
|
|
|
The requirement to apply a filter to the output, as opposed to a specific
|
|
track leads us to the final item in the Rules section above. As an example,
|
|
let's assume we wish to watermark all output, then we could use the
|
|
following:
|
|
|
|
<westley>
|
|
<producer id="producer0">
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
<producer id="producer1">
|
|
<property name="resource">clip2.dv</property>
|
|
</producer>
|
|
<playlist id="playlist0">
|
|
<entry producer="producer0" in="0" out="2999"/>
|
|
<blank length="1000"/>
|
|
<entry producer="producer0" in="3000" out="6999"/>
|
|
<playlist id="playlist1">
|
|
<blank length="3000"/>
|
|
<entry producer="producer1" in="0" out="999"/>
|
|
</playlist>
|
|
<tractor id="tractor0">
|
|
<multitrack>
|
|
<track producer="playlist0"/>
|
|
<track producer="playlist1"/>
|
|
</multitrack>
|
|
<filter>
|
|
<property name="track">0</property>
|
|
<property name="mlt_service">greyscale</property>
|
|
</filter>
|
|
</tractor>
|
|
<tractor id="tractor1">
|
|
<multitrack>
|
|
<track producer="tractor0"/>
|
|
</multitrack>
|
|
<filter>
|
|
<property name="mlt_service">watermark</property>
|
|
<property name="resource">watermark1.png</property>
|
|
</filter>
|
|
</tractor>
|
|
</westley>
|
|
|
|
Here we employ another tractor and we define a single track (being the
|
|
tractor we previously defined) and apply a watermarking filter there.
|
|
|
|
This is simply provided as an example - the watermarking functionality could
|
|
be better handled at the playout stage itself (ie: as a filter automatically
|
|
placed between all producers and the consumer).
|
|
|
|
Tracks act like "layers" in an image processing program like the GIMP. The
|
|
bottom-most track takes highest priority and higher layers are overlays
|
|
and do not appear unless there are gaps in the lower layers or unless
|
|
a transition is applied that merges the tracks on the specifed region.
|
|
Practically speaking, for A/B video editing it does not mean too much,
|
|
and it will work as expected; however, as a general rule apply any CGI
|
|
(graphic overlays with pixbuf or titles with pango) on tracks higher than
|
|
your video tracks. Also, this means that any audio-only tracks that are
|
|
lower than your video tracks will play rather than the audio from the video
|
|
clip. Remember, nothing is affected like mixing or compositing until one
|
|
applies a transition or appropriate filter.
|
|
|
|
<westley>
|
|
<producer id="producer0">
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
<playlist id="playlist0">
|
|
<entry producer="producer0"/>
|
|
</playlist>
|
|
<producer id="producer1">
|
|
<property name="resource">clip2.mpeg</property>
|
|
</producer>
|
|
<playlist id="playlist1">
|
|
<blank length="50"/>
|
|
<entry producer="producer1"/>
|
|
</playlist>
|
|
<tractor id="tractor0" in="0" out="315">
|
|
<multitrack id="multitrack0">
|
|
<track producer="playlist0"/>
|
|
<track producer="playlist1"/>
|
|
</multitrack>
|
|
<transition id="transition0" in="50" out="74">
|
|
<property name="a_track">0</property>
|
|
<property name="b_track">1</property>
|
|
<property name="mlt_service">luma</property>
|
|
</transition>
|
|
<transition id="transition1" in="50" out="74">
|
|
<property name="a_track">0</property>
|
|
<property name="b_track">1</property>
|
|
<property name="mlt_service">mix</property>
|
|
<property name="start">0.0</property>
|
|
<property name="end">1.0</property>
|
|
</transition>
|
|
</tractor>
|
|
</westley>
|
|
|
|
A "luma" transition is a video wipe processor that takes a greyscale bitmap
|
|
for the wipe definition. When one does not specify a bitmap, luma performs
|
|
a dissolve. The "mix" transition does an audio mix, but it interpolates
|
|
between the gain scaling factors between the start and end properties -
|
|
in this example, from 0.0 (none of track B) to 1.0 (all of track B).
|
|
Because the bottom track starts out with a gap specified using the <blank>
|
|
element, the upper track appears during the blank segment. See the demos and
|
|
services.txt to get an idea of the capabilities of the included transitions.
|
|
|
|
Flexibility:
|
|
|
|
The information presented above is considered the MLT Westley "normal"
|
|
form. This is the output generated by the westley consumer, for example,
|
|
when used with inigo. It is the output generated when you use the
|
|
"Westley to File" consumer in the demo script, which beginners will find
|
|
most useful for learning to use westley XML. This section describes
|
|
alternative forms the westley producer accepts.
|
|
|
|
First of all, the normal form is more of a linear format with producers
|
|
and playlists defined prior to their usage in a multitrack. Westley
|
|
also accepts a hierarchical format with producers as children of tracks
|
|
or playlist entries and with playlists as children of tracks:
|
|
|
|
<westley>
|
|
<tractor>
|
|
<multitrack>
|
|
<track>
|
|
<playlist>
|
|
<entry>
|
|
<producer>
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
</entry>
|
|
</playlist>
|
|
</track>
|
|
</multitrack>
|
|
</tractor>
|
|
</westley>
|
|
|
|
Obviously, this example is meant to demonstrate hierarchy and not effective
|
|
use of playlist or multitrack!
|
|
|
|
Secondly, as part of error handling, westley is forgiving if you fail to
|
|
supply <tractor>, <track>, and <entry> where one can be understood. This
|
|
affords an abbreviated syntax that is less verbose and perhaps less
|
|
intimidating for a human to read and understand. One can simplify the
|
|
above example as:
|
|
|
|
<westley>
|
|
<multitrack>
|
|
<playlist>
|
|
<producer>
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
</playlist>
|
|
</multitrack>
|
|
</westley>
|
|
|
|
Yes, filters and transitions can be added to the above example after the
|
|
closing multitrack tag (</multitrack>) because it is still enclosed within
|
|
the westley body tags.
|
|
|
|
If you specify in and out on a producer and it has been enclosed within
|
|
an <entry> or <playlist>, then the edit points apply to the playlist
|
|
entry and not to the producer itself. This facilitates re-use of media:
|
|
|
|
<playlist>
|
|
<producer id="clip1" in="25" out="78">
|
|
<property name="resource">clip1.dv</property>
|
|
</producer>
|
|
<entry producer="clip1" in="119" out="347"/>
|
|
</playlist>
|
|
|
|
In the above example, the producer attribute of the entry element is
|
|
a reference to the preceding producer. All references must follow the
|
|
definition. The edit points supplied on the producer above will not affect
|
|
the entry that references it below because westley knows the clip is a
|
|
playlist entry and optimises this situation. The advantage is that one
|
|
does not need to determine every clip to be included ahead of time
|
|
and specify them outside the context of the mutlitrack timeline.
|
|
|
|
This form of authoring will be easier for many to visualise as a non-linear
|
|
editor's timeline. Here is a more complex example:
|
|
|
|
<westley>
|
|
<multitrack>
|
|
<playlist>
|
|
<producer id="foo" in="100" out="149">
|
|
<property name="resource">clip2.mpeg</property>
|
|
</producer>
|
|
<blank length="25"/>
|
|
<entry producer="foo" in="10" out="59"/>
|
|
</playlist>
|
|
<playlist>
|
|
<blank length="25"/>
|
|
<producer id="bar" in="100" out="199">
|
|
<property name="resource">clip3.mpeg</property>
|
|
</producer>
|
|
<entry out="99" producer="bar"/>
|
|
</playlist>
|
|
</multitrack>
|
|
<filter mlt_service="greyscale" track="0"/>
|
|
<transition mlt_service="luma" in="25" out="49" a_track="0" b_track="1"/>
|
|
<transition mlt_service="luma" in="75" out="99" a_track="0" b_track="1">
|
|
<property name="reverse" value="1"/>
|
|
</transition>
|
|
</westley>
|
|
|
|
Did you notice something different in the last example? Properties can be
|
|
expressed using XML attributes on the element as well. However, only
|
|
non-service-specific properties are supported in this way. For example,
|
|
"mlt_service" is available to any producer, filter, or transition. However,
|
|
"resource" is actually service-specific. Notice the syntax of the last
|
|
property, on the last transition. Westley accepts property values using
|
|
the "value" attribute as well as using element text.
|
|
|
|
We have seen a few different ways of expressing property values. There are
|
|
a couple more for properties that can accept XML data. For example, the
|
|
GDK pixbuf producer with librsvg can handle embedded SVG, and the Pango
|
|
producer can handle embedded Pango markup. You can enclose the embedded XML
|
|
using a CDATA section:
|
|
|
|
<property name="resource"><![CDATA[ <svg>...</svg> ]]></property>
|
|
|
|
Please ensure the opening CDATA tag immediately follows the opening
|
|
property tag and that the section closing tag immediately precedes the
|
|
closing property tag.
|
|
|
|
However, westley can also accept inline embedded XML:
|
|
|
|
<property name="resource">
|
|
<svg>
|
|
</svg>
|
|
</property>
|
|
|
|
Currently, there is no namespace handling so a conflict will occur only on
|
|
any embedded XML that contains an element named "property" because
|
|
westley collects embedded XML until it reaches a closing property tag.
|
|
|
|
|
|
Entities and Parameterisation:
|
|
|
|
The westley producer parser supports XML entities. An example:
|
|
|
|
<?xml version="1.0"?>
|
|
<!DOCTYPE westley [
|
|
<!ENTITY msg "Hello world!">
|
|
]>
|
|
<westley>
|
|
<producer id="producer0">
|
|
<property name="mlt_service">pango</property>
|
|
<property name="text">&msg;</property>
|
|
</producer>
|
|
</westley>
|
|
|
|
If you are embedding another XML document into a property value not using
|
|
a CNODE section, then any DOCTYPE section must be relocated before any of
|
|
the xml elements to be well-formed. See demo/svg.westley for an example.
|
|
|
|
Entities can be used to parameterise westley! Using the above example, the
|
|
entity declared serves as the default value for &msg;. The entity content
|
|
can be overridden from the resource property supplied to the westley
|
|
producer. The syntax is the familiar, url-encoded query string used with
|
|
HTTP, e.g.: file?name=value&name=value...
|
|
|
|
There are a couple of rules of usage. The Miracle LOAD command and inigo
|
|
command line tool require you to preface the URL with "westley:" because
|
|
the query string destroys the filename extension matching peformed by
|
|
Fezzik. Also, inigo looks for '=' to tokenise property settings. Therefore,
|
|
one uses ':' between name and value instead of '='. Finally, since inigo
|
|
is run from the shell, one must enclose the URL within single quotes to
|
|
prevent shell filename expansion, or similar.
|
|
|
|
Needless to say, the ability to parameterise westley XML compositions is
|
|
an extremely powerful tool. An example for you to play with is available in
|
|
demo/entity.westley. Try overriding the name from inigo:
|
|
inigo 'westley:entity.westley?name:Charlie'
|
|
|
|
Technically, the entity declaration is not needed in the head of the XML
|
|
document if you always supply the parameter. However, you run the risk
|
|
of unpredictable behviour without one. Therefore, it is safest and a best
|
|
practice to always supply an entity declaration. It is improves the
|
|
readability as one does not need to search for the entity references to
|
|
see what parameters are available.
|
|
|
|
|
|
Tips and Technique:
|
|
|
|
If one finds the above hierarchical, abbreviated format intuitive,
|
|
start with a simple template and fill and extend as needed:
|
|
<westley>
|
|
<multitrack>
|
|
<playlist>
|
|
</playlist>
|
|
...add a playlist for each track...
|
|
</multitrack>
|
|
...add filters and transitions...
|
|
</westley>
|
|
|
|
By using a playlist for each track, it is easier to iteratively add new
|
|
clips and blank regions as you develop the project. You will not have to
|
|
use <track> or later add <playlist> when necessary.
|
|
|
|
A more advanced template that allows sequencing multitracks is:
|
|
<playlist>
|
|
<entry>
|
|
<multitrack>
|
|
<playlist>
|
|
</playlist>
|
|
...add a playlist for each track...
|
|
</multitrack>
|
|
...add filters and transitions...
|
|
</entry>
|
|
|
|
<entry>
|
|
<multitrack>
|
|
<playlist>
|
|
</playlist>
|
|
...add a playlist for each track...
|
|
</multitrack>
|
|
...add filters and transitions...
|
|
</entry>
|
|
</playlist>
|
|
|
|
If you end up making a collection of templates for various situations, then
|
|
consider using XML Entities to make the template more effective by moving
|
|
anything that should parameterised into an entity.
|
|
|
|
If you want to have a silent, black background for audio and video fades,
|
|
then make the top track simply <producer mlt_service="colour"/>. Then,
|
|
use composite and volume effects. See the "Fade from/to black/silence"
|
|
demo for an example (demo/mlt_fade_black).
|
|
|
|
If you apply the reverse=1 property to a transition like "luma," then
|
|
be careful because it also inherently swaps the roles of A and B tracks.
|
|
Therefore, you need to might need to swap the a_track and b_track values
|
|
if it did not turn out the way you expected. See the "Clock in and out"
|
|
for an example (demo/mlt_clock_in_and_out).
|