Documenting the research and production of interactive network based media. By David Wolf
Click image to play (28Kb Quartz Composition in QuickTime wrappper, requires MacOS 10.4)
MPEG 4 - h264 version (1.9Mb) Requires Quicktime 7 or alternative player such as VLC.
Quartz Composition file (64Kb) - Open in Quartz Composer to see how it works or move to ~/Library/Screen Savers/ to use as a screen saver.
Technorati Tags: Quartz Composer, vidget
mDimension Technology have developed a plugin for Safari which lets you not only view Quartz Compositions in a web page, but also interact with them via JavaScript. This is very neat!
So far I’ve been experimenting with creating custom applications (Vidgets) in Xcode which control Quartz Compositions, this plugin appears to give me most of the same functionality within a web page. To control and manipulate the composition, first you must follow the same methods of ‘publishing’ inputs and outputs from a composition as described here.
View source on these pages to see how to send and receive information to and from these inputs via JavaScript. I’m still working it out myself!
Here is a demo page from mDimension with a Quartz Composition embedded.
Here is a quick demo I made up based on the above page. It allows you to load an image by entering its url into a field and clicking outside of the field (don’t need to press enter), and change the spinning text similarly.
Technorati Tags: Quartz Composer, vidget
This is the next version of my live video performance (VJ) application. It is an updated version of the program I used for my recent performances at MIAF: Remains To Be Seen and Liquid Architecture 6.
Like the previous version, Bikelights: a Quartz vidget, MIAF-LA6 features one window with controls on the left hand side and a display on the right. It is designed so that I can have the controls on my laptop screen while sending the video from my second monitor output at 640 * 480 to a projector by arranging my monitors like so and dragging the window across.
At some stage I’ll work out how to do two separate windows but this works well for the moment 
Where the previous version only allowed the display of one source image at a time, this version lets the user layer two separate images through a number of effects and outputs. The left hand side of the control area is split two (Source Clip A and Source Clip B). Each area contains four empty text fields where Quicktime movies may be dragged and dropped from the Finder (click in the field to activate it, drag and drop, then hit Enter to set).
Below the four movie path fields is a fifth path field which allows the user to specify a local path to a folder of still images. Again folder paths may also be either typed in manually or dragged from the Finder. After pressing Enter the “# of Images” field should update, letting you know how many images are in the folder you specified. To the left of this field is the “Sequence Duration”, this lets you specify how long (in seconds) it should take to cycle through all of the images in the folder. For example, if you have 8 images and want them to play through at four frames per second second you should set the duration to 2 seconds.
To the right of these fields is a large vertical slider, this lets the user select any one of the four Quicktime Movies or the folder image sequence to process and display.
To control the playback speed of the Quicktime movies, check the “Clip Speed Control” box and move the horizontal slider below. If unchecked the movie will play out at normal speed. If checked the speed will be controlled by a combination of the slider position and live audio analysis (if you have a microphone set up). The far left of the slider is 0 * original speed, the far right is 5 * original speed. If a microphone is connected, the video will jump up to one second forward through the clip with the loudest audio peak.
Below the speed controls are the image controls. These controls allow the user to adjust the saturation, brightness, contrast and colour angle of the clips or image sequences. To leave the Quicktime movie unaffected uncheck the “Image Controls” box.
On the right hand side of the control area is the output stage, the “Renderers”. Each renderer may be switched on and off via a check box and be used to display either source A or source B (or a combination of the two in the case of the “Billboard” and “Inset Image” renderers).
Download miaf2vidget.zip (Mac OS 10.4 required)
Download miaf2quartz.zip (Quartz Composer file)
Download miaf2xcode.zip (Xcode project)
This work is distributed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 licence.
Technorati Tags: Quartz Composer, vidget
One of the coolest things I’ve discovered about Quartz Composer lately is the fact that you can use it (in combination with Xcode) create stand alone applications with custom interfaces all without having to know any ‘proper’ programming. So, here’s one of my first experiments….
This vidget is a simplified version of a larger Quartz Composer patch I have been using to perform with recently. It allows the user to import a number of Quicktime files or still images and apply a number of effects to them.
On the left hand side of the window are five text fields which store the local path to each movie file. To add your own clips, simply click in the field and then drag the movie’s icon from the finder into the field and press enter.
Just to the right of these text fields is a selection slider, which is used to select which source clip is to be effected. The arrow will snap to each of the clips as you slide or click it.
Below the and to the left are more sliders which control the clip’s playback speed, saturation, brightness, contrast, and hue angle. All of these work in real time to adjust the display of the clip.
To the right of these image controls are check boxes for the different rendering effects. They are ordered from top to bottom in terms of the layers onto which they are drawn. The top three have optional black ‘clear’ backgrounds which block out the layers below.
Expanding audio renders the clip onto three layers, red, blue and green and uses live audio input to resize the three layers to produce a coloured motion blur effect.
Particle system renders the clip onto hundreds of small layers which explode out from a randomly selected point on the screen.
Slow moving layers renders the clip onto four layers which crop the image and oscillate in 3d space based on a combination of audio input and low frequency oscillators.
Inset Image produces a ‘picture in picture’ effect which is manipulated by audio input.
Download (requires Mac OS X 10.4).
BikeLights Clips (these tiny lo-fi clips of my bike lights were shot with my phone and are the reason for the name, try them out)
Quartz Composer file
Xcode Project
This work is distributed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 licence.
Technorati Tags: Quartz Composer, vidget
I’ve been having a good play with Quartz Composer over the last couple of weeks. It is very exciting and a bit scary at this late stage of my MA research: “Does it make what I’ve been doing for the past two years redundant?”; “Should I drop the whole interactive QuickTime thing and start from scratch in this new environment?”; “Should I ignore it for now and continue with QT because it is cross platform and more accessible?”.
In many ways it lets me do what I have been doing, experimenting with, and wanting to do (real time interactive online video) much more quickly and with exciting new visual results. In some ways it makes basic QT redundant but it is quite a different beast.
QuickTime excels on the network. Child movies can be sourced from anywhere, XML and QTlists while a pain to set up sometimes are very powerful and I’ve only really scratched the surface of their potential when combined with server side scripting such as php. Quartz Composer is much more at home on the desktop. It can import still image files from a URL but not movies. It can read RSS very easily, but is designed for human readable text and requires custom scripting to deal with generic XML files and attributes. I have had some success getting QC to load movies from the network via a local QuickTime link file pointing to a URL, but the targeting it is local, relative to the QC composition. It seems this link is lost if the composition is exported to a .mov file.
Here is a quick example (requires Mac OS 10.4). Apologies for the cheesy kaleidoscope imagery once downloaded and unzipped, the .qtz file should play in Quartz Composer, importing link.mov which points to a video file on my server. The zip file is about 4k.
The cool thing is, in many ways this (Quartz Composer), builds upon what I have been doing in QuickTime and is mostly playable by both the QT player and plugin. While the linking to movie files online is problematic at this point, surprisingly live video and audio inputs are supported even in the QuickTime browser plugin! Here is an example which takes a live feed from a FireWire camera, layers it over itself on 3 differently coloured layers (red green and blue) and scales in real time based upon audio input from the computer’s built in microphone. Link to livergb.mov. This has been tested in Safari with a Sony HandyCam and my PowerBook’s built in microphone. Here’s the source .qtz file. While live video input into a movie playing in a browser is pretty exciting, unfortunately more simple things like keyboard and mouse input are missing.
Stay tuned for more examples as I play more…
This is the latest version of my Quicktime vj/image search app. It now works as two separate movies which talk to eachother. Click on each of the images above to load in Quicktime Player.
More details, instructions etc soon (i’ve gotta race to the library to return some very overdue books!).
I recently completed work on an experimental internet radio program with Hannah Miller and Kate Eccles. Hannah and Kate are final year Media students at RMIT majoring in Radio and TV production. My role in the production was to take various pieces of audio, video, still images and text, and create an interface which would allow the user to mix and match the elements in an exploratory, non-linear way.
The result of this work is a program called “Inspiration”, which features interviews, live footage, sound recordings and lyrics from Reset://0 a Japanese influenced Melbourne band.
The program was authored in LiveStage Pro and is a Quicktime file that consists of a sprite track, several movie tracks and a text track which features lyrics. The above image shows the partially completed work as I was assigning sounds to various non-square shaped roll-over buttons. The idea was that rather than presenting the user with a list of options, or even a grid of non-labelled options, the work should encourage the user to explore the screen space with the cursor, almost like they are feeling their way in the dark. To give the users some feedback, and a little direction as to where may be a good place to explore, I used Hanna’s fire twirling image as a guide. I placed invisible sprites over the background image which reacted to the “MouseEnter” event, triggering sounds which played in specific movie tracks, and changing the sprite image for the background so that different parts of the fire twirling would be illuminated and hi-lighted.
You can view the completed work in context on the interadio site. Or, to go straight to Inspiration(requires Quicktime, a fairly recent computer and a decent broadband connection - 15Mb)
This is the latest version of my interactive networked video project.
Click on the image to load Vidget 3 in Quicktime Player. (It is quite small but very processor intensive - especially as it first loads)
This version is a mix between the my first vidget which featured a text based interface for mixing up to three video clips on top of eachother, and my Quicktime Flickr photo viewer which let you search for and view images based on a search word.
The interface has been redesigned and now features a grid of 25 draggable images which represent video clips. These may be dragged and dropped onto three coloured ‘layers’. The blue layer is the uppermost with green below and red at the bottom. Each of these layers has a number of ‘graphics modes’. Like Photoshop layers, these may be combined in a number of modes, ranging from fully transparent to fully opaque. Each of these layers also has a number of playback controls which allow the user to play the clip faster or slower, forwards or backwards and step through frame by frame.
To the right of the three colour layers and their controls is a small white text field. This allows the user to search for images from Flickr. The ten most recent pictures tagged with the search word entered are loaded as thumbnails below. These thumbnails may be dragged and dropped onto any of the layers and combined with other moving and still images.
I have resized the output movie area so that everything fits on one screen.
Behind the scenes, the vidget has also been greatly updated. Rather than being limited to a set number of video clips determined at the time of authoring, this version dynamically loads all content including thumbnails. The names of these files are drawn from an XML file. This file may be updated with a simple text editor to add or delete more clips. The movie automatically loads the first 25 thumbnails from the XML list as it initially loads but may load the next 25, and the following 25 via the 1, 2 and 3 buttons at the top right of the controls.
At the moment the whole movie pauses whenever thumbnails are loaded, either via a Flickr search or by skipping to the next 25 thumbnails of video clips. I am working on ways around this.
The LiveStage Pro source files may be downloaded here: vidget3.zip
Click image to load in Quicktime Player (it seems to be a little funky in a browser)
This is a little Quicktime movie that lets you view photos from Flickr, a photo sharing and social networking site. When users upload their images they associate them with tags, so a picture of a tree may have the tags ‘tree’, ‘green’, ‘eucalyptus’ etc. The entire database of photos is searchable by tag, by author, by series and other organisations. Each of these has a RSS or ATOM feed associated with it. This is what I am using here to access the 10 most recent uploads for a given tag.
To search the site and view the photos, just type a tag, say “cow” (no quotes) into the small white text field and press return. If you press return without entering any text you get the 10 most recent uploads from any category. Once the thumbnails load you can click to view at a larger size on the right.
The movie uses Quicktime’s ability to read and parse XML files such as RSS feeds and access files from the network.
Source files and more detailed explanation soon…
Its been a bit quiet around here for a while and this is why. I’ve been working pretty solidly on this piece for the past couple of weeks leading up to a gig I co-organised last week. Segmentation Fault is a semi-regular experimental music & visual night we put on every couple of months and proved to be a good motivation (ie. deadline) to get a work together for. in my research I am mostly interested in applying VJ aesthetics and methods to the desktop environment where the user becomes the performer, but its always fun to perform in front of an audience of humans in a room.
Now its time to release this draft to the world and see what people think. Click on the images below to load the two parts in Apple - QuickTime Player.
The scrambled looking black, white and green image will load the ‘output’ movie. This is the movie to be projected on a screen or viewed on a second monitor. It is designed to run at full PAL resolution (720 * 576) to suit the TV output of my laptop. If you want to try it out on a single monitor setup, you can load the movie and select ‘Half Size’ from the Movie menu in Quicktime Player. This movie is really just a kind of holder for up to three other movies. To load different clips into the ‘output’ movie you will need to use the ‘interface’ movie below.
This movie controls which video loop is loaded in which layer of the output movie. Along the top of the window are the numbers 1, 2 and 3. These represent the three layers with 3 being the ‘highest’, 1 the lowest and 2 in between. Next to each of the numbers are playback controls for each layer. Once clips have loaded they may be played forwards and backward, in slow and fast motion and stepped through frame by frame. Next to the playback controls are the graphics mode controls. These control the ways in which each of the layers are blended.
‘Blend 0′ means the clip is completely transparent. It is probably a good idea to switch to this setting if you are going to load a big clip as it will take a while to load and display a still image whilst it is doing so. ‘Blend 100′ means the clip is 100% opaque so any other clip below it will not be seen.
‘Add Max’ adds the bright portions of the clips image over the clips below, leaving the dark areas transparent. ‘Add Min’ adds the dark portions of the clips image over the clips below, leaving the lighter areas transparent. ‘Sub Pin Blk’ subtracts the bright areas of the clips image from the ones below so white snow on a black background will result in black snow on a transparent background. ‘Inverse Or’, ‘Exclusive Or’ and ‘Inverse Exclusive Or’ produce other effects but to be honest I’m still not sure exactly how they work :-).
These graphics modes probably won’t be much fun to play with until some different clips are loaded into each of the layers. To do this I have designed two different patch loaders. If you click on the 1 or 2 with red # symbols next to them the # will change to a *, telling you which loader is active. The first thing to do is select which layer or ‘channel’ to load the clip into. These are selected by clicking the large 1, 2 or 3 at the top. Next a clip may be selected from the list at the bottom half of the controls. The clip’s name and id number are displayed and when the ‘Do It!’ button is pressed the clip will start to load. (If you are wondering why it is called ‘Do It!’ go see Starsky and Hutch :-))
Note: there may be a couple missing - such as the ‘live input’ at the bottom right, so if you get a ‘broken movie’ image just try another clip.
The Livestage Pro project files can be found here: interfaceproject.zip outputproject.zip
Twitter last.fm