DVEdit: Tutorial

Introduction

Prerequisites

DVedit requires at least the following software:

A *nix-style operating system, such as Linux or *BSD. Attempts to build DVEdit on Windows (native or Cygwin) so far have been a world of pain. But it's very possible that it'll build fine on Mac OSX.
Python 2.5 or later
Cython compiler - you must use the current Hg repository; versions installed from distro feeds such as Gentoo, Ubuntu etc will most likely not work! For your convenience, the '3rdparty' directory in the tarball contains a version of Cython that's working fine for me.
A gcc-based C compiler
libdv
ffmpeg - LATEST SVN VERSION (versions built/installed with Ubuntu don't seem to work)
Python Imaging Library (PIL) (available with most linux distros) - only needed if you want to operate on individual frames

Also highly recommended is a decent command-line friendly media player such as MPlayer, with all available codecs installed.

If you want to use the clip viewer, you'll need to install PyFLTK (SVN version) and libFLTK version 1.1. Most Linux feeds have libfltk-dev 1.1, so you may not need to install/build it from source.

Meeting these requirements in Linux or BSD environments is trivial. However, for Windows and other environments, dependencies such as libdv, ffmpeg (and to a lesser extent) mplayer could prove tricky; if you have any trouble building this framework on Windows, then give some consideration to building it in a MinGW and/or Cygwin environment.

Building/Installing DVEdit

Providing you have all the dependencies installed, building/installing DVEdit should be a simple matter of invoking the Python distutils script setup.py:

     $ python setup.py build
     $ sudo python setup.py install
     Password: 
     $

One common cause of build failure is an out-of-date or otherwise incompatible version of Cython. One thing you can try is to remove Cython from your system, and install the one enclosed with this package:

     $ cd 3rdparty/cython/cython-package
     $ python setup.py build
     $ sudo python setup.py install
     Password: 
     $ cd ../../..
     $

then try the DVEdit build again.

Basic Concepts

The core Clip class

The basic building block of DVEdit is the extension type dvedit.Clip

This class has some of the look and feel of a normal python list:

You can index it: frame = myclip[12]
You can iterate over it: for frame in myclip
You can determine its length: nframes = len(myclip)
Some clips are writable: myclip.append(newframe) or myclip[400] = newframe
You can contatenate clips: bigclip = clip1 + clip2
You can take slices: excerpt = clip1[100:325]

But in addition, every instance of Clip and its subclasses has a method .render(), which will render the video out to file of any chosen format (within the limitations of ffmpeg).

Most of the classes in DVEdit (apart from the Frame class, the filter classes and a couple of other utility classes) are subclassed from Clip.

You can add your own functionality to DVEdit by writing your own Pyrex/Cython extension types subclassed from Clip.

Quick tour of DVEdit classes

The following classes in DVEdit are derived from Clip:

BlankClip - a clip of arbitrary length which contains only blank frames
File - gives read/write access to video files
Slice - an excerpt of another clip, the result of taking a slice of the clip
Sum - two clips spliced together, the result of adding two clips together with the '+' operator
Transition - one clip fading to another; by implementing subclasses of Transition, different transition effects can be created
Sequence - zero or more clips arranged in a timeline, analogous to a 'track' in conventional video editors
Stack - zero or more clips arranged as layers in a vertical stack; when rendering, frames will be taken from the first layer which gives a non-blank frame

All Clip (or subclass) objects represent a set of audio/video frames. Indexing a clip will return a Frame object, which contains either the raw audio/video data, or the encoded DV data for this, or both.

In addition to this, DVEdit has a simple plugin architecture which allows for the easy creation of effects, known here as filters. To create a filter, you just subclass the Filter extension type.

This class hierarchy allows considerable flexibility and unrestricted nesting of objects. For example, the classes Slice, Sum, Transition, Sequence and Stack are all based on containing other Clip objects, so it's quite OK for example to have a Stack containing a Sequence which contains a Stack which contains a Sequence which contains a Slice which contains a Stack which contains a File and a Sum and so on. Any class which honours the basic Clip interface may be used as a component of these containing classes.

The purpose of this is to give you a great degree of power and leverage in managing the structure of your video projects, and transcend the limitations of the simple Project/Track/Clip hierarchy of other editors.

Structure of a Clip

Clip objects, as a rule, do not actually contain Frame objects. Instead, Frame objects are created on demand and returned whenever such Clip objects are indexed.

For example, a File object possesses only the ability to open a video file, seek to a given position, and read or write a block of encoded DV data.

Colour Models

DVEdit allows you to work in either or both of two colour models:

YCbCr 4:2:2
RGB

This is especially significant in areas where frame contents are modified, such as filters and transitions.

Individual filters or transitions which need to modify frame video data may ask for the raw video data to be available in the desired format, via the following method calls:

frame.ensureDecodedAsYuv() - causes the frame to, if necessary, convert its internal video representation to a YCbCr 4:2:2 called YUY2, in which case the video buffer contains a sequence of pixel pairs, where each pixel pair comprises the four-byte sequence:

Y0 Cb Y1 Cr

where the first pixel in each pair is defined by (Y0,Cb,Cr), and the second pixel is defined by (Y1,Cb,Cr)
frame.ensureDecodedAsRgb() - causes the frame to, if necessary, convert its internal video representation to RGB, in which case the video buffer contains a sequence of single pixels, where each pixel consists of the three-byte sequence:

R G B

The reason for the somewhat awkward 'YUY2' YCbCr format is speed - this is the format made available by the underlying libdv codec. If you want planar YCbCr, it would be easy to write functions to convert YUY2 buffers to/from sets of 3 Y, Cb, Cr buffers. If you hassle me enough, I might be inclined to integrate planar YCbCr availability into the Frame class.

Usage of DVEdit

Since DVEdit has no GUI (yet), the main way to use DVEdit is to write Python scripts to import video clips, arrange these into a structure, then render them out.

Simple 'Hello, World'

The following Python program takes 5 seconds of the PAL video file raw.vob and renders it out as a medium-quality Quicktime4Linux video myvid.mov:

  #!/usr/bin/env python

  from dvedit.core import *

  # open the input video
  clip = File("raw.vob")

  # 125 frames @ 25fps = 5 secs
  excerpt = clip[:125]

  # now render it out as an AVI with MPEG4 video, MP3 audio
  excerpt.render("myvid.avi",
      "-f avi -vcodec mpeg4 -acodec libmp3lame -b 5000000")

No processing has been done on the video, apart from cutting a 5 second excerpt and saving it out as an excerpt.

Extending DVEdit

There are two main ways of extending DVEdit:

Implementing new filters (the most common way)
Creating new Transitions
Creating new Clip types

All these options require a good understanding of programming in Cython. Pyrex programmers shouldn't have too much trouble migrating to Cython.

If you haven't done any Cython or Pyrex programming, you might like to spend some time studying the Pyrex and Cython documentation. Believe me, your learning effort will be abundantly rewarded, since you will get the best of both worlds - the rapid development and power of Python, together with the speed and efficiency of C.

Most of the time, you will only want to create new custom filters and transitions. However, you might see value in creating a new basic Clip type. We will discuss all three extension methods below.

Clip structure

The basic workhorse of DVEdit is the extension type dvedit.core.Clip. This class allows video files to be accessed in a manner similar to Python lists.

However, for performance reasons, we have not provided the means of extending Clip with pure python code. You will have to write your subclass in Cython.

The most important methods and attributes of Clip are:

.numFrames - integer - declares the length of the clip in frames. In your subclass, you must keep this attribute up to date.
getFrame(frameno) - C method - takes an integer argument frameno and returns a Frame object. In your subclass, you will typically override this method.

Creating new Transition types

To implement a new transition type, you need to create a subclass of dvedit.transitions.transition.Transition and provide the methods init() and blendFrames(). Study the source files dvedit/transitions/transition.pyx and dvedit/transitions/crossfade.pyx.

transition.pyx contains the Transition base class, whose docstring describes the requirements for subclassing.

crossfade.pyx contains an implementation for a simple cross-dissolve transition. You can use this as a model for creating your own transitions.

Creating filters

Creating new filter types for DVEdit should prove fairly easy, once you get familiar with the filter architecture.

Basic principles of DVEdit filters:

Filters can only be applied to Sequence objects
'pull' mechanism - when frames are accessed from a Sequence, the filter stack is checked to see which filters are in force at the required point in time (frame number). The frame is requested from the bottom-most filter, which requests a frame from the next-to-bottom filter, and so on, until a raw frame is finally fetched from the sequence itself.
A filter is created in respect of a portion of the Sequence's timeline. Within the filter's time window, a filter may present any set of frames it so chooses. These frames may be derived from the Sequence, or they could be created from scratch. If created from the Sequence, the frames may have a 1:1 correspondence with Sequence (eg frame 11 as presented by the filter may be based on frame 11 of the raw Sequence), or the filter may alter the timing - for example, the 'reverse' filter.

To implement a custom filter, create a new .pyx file in dvedit/filters, and in that file, create a subclass of Filter.

Here's the code for the 'inverse' filter, which you can use as a starting point for writing your own custom filters:

  from dvedit.core cimport Filter, Frame, COLOR_RGB

  cdef class InverseFilter(Filter):

      cdef Frame getFrame(self, int idx):

          cdef int i, nbytes
          cdef Frame frame

          # we just fetch from the sequence (or further up the filter stack)
          # a frame at the same point in time
          frame = self.getSourceFrame(idx)

          # make sure the frame is decoded, and set it so it needs re-encoding
          frame.ensureDecoded(COLOR_RGB)
          frame.hasEncoded = 0

          # ohh, hardwired assumption of PAL in RGB :(
          nbytes = 720 * 576 * 3
          for i from 0 <= i < nbytes:
              frame.videoBuf[i] = 255 - frame.videoBuf[i]

          # all done
          return frame

A couple of notes here:

The filter works mainly by implementing the method getFrame()
The filter pulls frames from further up the effect stack via the convenience method getSourceFrame(). This method takes care of figuring out whether there are any filters that are active at the same point in time, and pulling the needed frames from those filters (or if no other filters, from the Sequence itself).

Now, let's look at a minimal application script which uses this filter:

  from dvedit.core import *
  from dvedit.filters.inverse import InverseFilter
  from dvedit.filters.flip import FlipFilter
  from dvedit.filters.reverse import ReverseFilter

  clip1 = File("clip1.mpg")

  # create sequence, add clip
  seq = Sequence()
  seq.add(clip1)

  # add some filters
  seq.filter(ReverseFilter) # this will apply for the full duration of the sequence
  seq.filter(InverseFilter, 50, 100) # applies at frame 50 for 100 frames
  seq.filter(FlipFilter, 100, 100) # applies at frame 100 for 100 frames

  seq.render("filtertest.dv")