Documenting the research and production of interactive network based media. By David Wolf
It’s been done for a couple of months, but now that I’ve done the ceremony with the funny hats and received my piece of paper it’s time to show the world what I’ve been working on for the last five years!Â
Vidgets: The Development and Use of Interactive, Network Based Video Works by David Wolf
Big thanks to family, friends and collaborators and especially big thanks to my supervisor Adrian Miles.
Last year an update to QuickTime meant that Quartz Compositions could no longer access RSS feeds when played within the QuickTime Player application. This change was made to stop maliciously created compositions from sending information to remote sources without the user’s knowledge and was made in the name of security. It was very annoying as I was using Quartz Composer to create ‘network aware’ works which accessed RSS feeds, bringing in text and photos from various sources online.
Now with the latest Mac OS software update another of my favourite features has been taken away from Quartz Composer. With 10.4.7 a Quartz Composition cannot use the ‘Image with movie’ node to nest compositions within compositions. Previously I could create a composition, save it in a .mov wrapper and then import it back into another composition as I would a normal movie file. This would let me quickly build up complex interactive compositions, layer and manipulate them in custom Vidgets.
I came across this problem on the Quartz Composer Developer List a few days ago but it was only when I went to play some of my more recent compositions last night that I realised how much of a problem it is. I had come to rely on this feature.
Pierre-Olivier Latour writes that they had to disable this feature so as to prevent people from creating compositions which reference themselves, crashing the computer (another potential security risk). I understand the need to make the software as stable and secure as possible but it seems like there should be another solution which prevents the crashing without ruling out a whole set of features.
It should also be noted that similar features such as network access (including the parsing of remote XML files) and the nesting of movies within movies have been key features of the QuickTime architecture for years without any such restrictions.
“This is brush, a small Max/MSP/Jitter patch that I’ve compiled as a standalone application. It’s aimed at visualists who are just starting out and looking for software to play with. Programmatically, it’s very simple. Video from a live camera (or a movie file) is fed back on itself so that light stays on the canvas (screen). Thus, you can paint with the light in the room you’re in. Decay (fade time), tolerance (lower luma threshold) and color inversion are adjustable so you can adapt your performance to any lighting conditions.”
Dan Winckler’s blog - brush 0.1: a tool for visual performers
This is a great little piece of software, what I would call a Vidget. A small scale application which lets you manipulate digital media in real time for improvised performance. It is very easy to use and entertaining to play with.
Today I played live visuals for Legs For Fish at Thornbury High school. Kent Macpherson from the band is a music teacher there and organised a performance / presentation for the kids from music, media and IT classes. We played three improvised AV sets and the kids seemed not to hate it 
As Paul from the band said, it makes you acutely aware of how self indulgent this sort of improvised performance is when are doing it in front of a room full of kids who aren’t necessarily there by choice. Some of the kids seemed to dig it and some asked some good questions. “Why do you do this?” “What does it mean? The pictures just looked like colours and shapes.”
I used the Quartz Vidget I prepared for the Liquid Architecture gig in July as it works fairly well and made it easy for me to show the kids what I was doing without the confusion of showing a huge messy Quartz Composer patch. After each set we took questions. I showed how I can grab any two video files or sequences of still images and mix them together, manipulating colour, brightness, contrast etc. as well as layering multiple copies over each other. After the second performance I showed very briefly how Quartz Composer works by dropping in a video file, connecting it to a Billboard and then running it through a couple of effects.
Looking back at the QC presentation I gave at Electrofringe after this one I realise I should have shown heaps more examples and started much more slowly in Newcastle. Assuming no prior knowledge meant I explained things much more clearly, and I introduced things in a much more logical order (having slept the night before also helped!).
It’s coming up on that time of year again. Electrofringe is a new media arts festival held in Newcastle as part of the This Is Not Art arts festival. Lots of good stuff on and definitely worth the trip.
This year I’m presenting / performing the following:
Realtime Video Manipulation using Isadora and Quartz Composer
“Introduction to the use of two software platforms which allow the creation of realtime video installations which can respond to the audience or other available data.”
This will be a introductory ‘tips and tricks’ panel presentation with Luke Toop (Adelaide), Steve Huon (Melbourne), Khalid (Melbourne) and myself. I’m probably going to focus on the basics of how create a Quartz Composer patch and then turn it into a stand alone application with Xcode and a little bit on network access and RSS etc.
Spac{v}e dpwolf vs Doktorb Robotnik
A live improvised AV performance with my frequent collaborator Doktorb Robotnik. Audio and video feedback crossed with data pulled from the network in real time. Followed by a discussion.
Re - Imagining (live) Video - Narrative
A panel presentation / discussion with Jean Poole and Anna Helme.
From the blurb:
“Audio has been easily sampled, processed and manipulated live for decades. Although hardware and software now allow video to become just as malleable, it is used in a limited number of ways. How can current-day video tools be used to composite video at live events differently? How can theatre and storytelling better integrate live video? What storytelling possibilities lay beyond recreating cinema, music videos or ‘wallpaper’? How do live video and sound work best together? What video is most worth having live?”
Technorati Tags: Electrofringe, Quartz Composer
Hmm, nothing like retro-promotion… this happened a few weeks ago.
Friday 26th of August: A night of live AV performances with dpwolf and David Sevo, Bits of Clay and Canvas City at Glitch Bar & Cinema in North Fitzroy.
David played an ambient set of prerecorded tracks mixed with musique concrete style found sounds and samples and I projected very abstract, fluid, generative images created using analog to digital feedback through realtime effects in Quartz Composer.
I had just been down to see the excellent white noise exhibition at ACMI and was inspired by the ideas of abstraction on display. Curator Mike Stubbs’ essay on the exhibition is definitely worth a read: white noise : a leap into the light.
Here is a quick diagram which shows how my setup was plugged together:
The video output of the laptop was connected to both the analogue input of the MiniDV camera and the projector. The FireWire output of the video camera was connected back into the computer. Displaying the DV signal from the camera through colour controls and halftone line filters, back out through the computer’s video output I created a feedback loop. Rather than using mouse based on screen controls as I have with previous Vidget setups, for this performance I chose to use a MIDI controller with 8 knobs and 8 faders. This gave me much needed ‘hands on’ control so I was able to manipulate the feedback by adjusting various patch variables.
Click image to play.
Here is a fairly large (29 Mb), long (8 min) video created playing around with the setup in preparation for the gig which should give some idea of how it looked. This kind of play is an important (and fun) part of my research.
The video is released under an Attribution-NonCommercial-ShareAlike 2.5 license however the music is copyright .
Technorati Tags: Quartz Composer, vidget
The main tools of trade when using Quartz Composer are patches. Each of these modular objects has a particular function when plugged together to create flows of information. For example a patch may import or download an image, calculate a mathematical operation, affect an image or function as a switch, selecting between other patches and signals. This flow ends up rendered on screen by a renderer patch.
While you can plug these patches together in almost any way, until recently you could only play with the patches Apple has included in the program by default. You could not create your own. This meant that while QC talks directly to the graphics card and renders all images as OpenGL surfaces, users were limited to rendering onto flat surfaces, spheres, cubes and a teapot shape.
Recently ClockSkew worked out how to enable disabled patches and write your own custom patch. With this information ?? at Quartzcomposer.jp developed a custom patch which allows you to load 3D Studio Max objects into Quartz Composer and render images onto their surfaces.
Via Roger Bolton: Quartonian » Custom patches for Quartz Composer now a reality. Follow the above links for all the downloads how-tos.
I’ve been playing with rendering onto 3D Studio Max .3ds objects in QC for the past week or so but have had mixed results. The fighter jet model included by ?? works well but most other objects I’ve found online have either crashed QC or rendered incorrectly. I’m sure these issues will be worked out down the track.
Here is a quick demo Quicktime movie showing the fighter jet model in QC with cloud effects generated via particle systems based on this smoke effect by Noise Industries.
Click image to play (1.9Mb, 30sec Quicktime).
Technorati Tags: Quartz Composer
mDimension Technology have developed a plugin for Safari which lets you not only view Quartz Compositions in a web page, but also interact with them via JavaScript. This is very neat!
So far I’ve been experimenting with creating custom applications (Vidgets) in Xcode which control Quartz Compositions, this plugin appears to give me most of the same functionality within a web page. To control and manipulate the composition, first you must follow the same methods of ‘publishing’ inputs and outputs from a composition as described here.
View source on these pages to see how to send and receive information to and from these inputs via JavaScript. I’m still working it out myself!
Here is a demo page from mDimension with a Quartz Composition embedded.
Here is a quick demo I made up based on the above page. It allows you to load an image by entering its url into a field and clicking outside of the field (don’t need to press enter), and change the spinning text similarly.
Technorati Tags: Quartz Composer, vidget
Ok, back to the blog after a month…
I’ve been busy working full time and finishing a paper for DAC_05 - Digital Arts & Culture. A conference held in Copenhagen, Denmark in December this year. In 2003 DAC was held here at RMIT in Melbourne. Check out the melbourneDAC site and all the papers.
My paper focusses on the processes of play common to both the creation and use of Vidgets. Rather than looking at my work in terms of its relationship to theatre, literature or cimema (as is often the case when looking at new media art), I draw parallels with the history of sound art practice. In particular I compare the idea of the all data being explored in digital media to the all sound explored by sound artists.
I hope they like it
QuickTime 7.0.1: Security enhancements
This sucks. QT 7.0.1 no longer lets you access remote web locations from within a Quartz Composition when played using Quicktime Player. No more playing such compositions from the Quicktime Plugin in a web page either. This means no RSS feeds unless you run the composition in Quartz Composer itself or in a custom app.
This is the direction I’m heading anyway (making custom apps) but it means you can no longer distribute a .qtz file or a composition in a .mov which accesses network data and expect it to work on machine running Tiger unless the user has Quartz Composer installed.
These were some of the coolest features of Quartz which I was only just starting to explore before the plug was pulled. Hopefully these features will return some time soon.
Update 18/7/05
I don’t think I made this clear enough but the change was made in response to the potential security issue whereby network access from within a composition wrapped in a .mov file to be used to leak information to a malicious third party when used in combination with Quartz Composer’s ability to access sensitive information about the host computer (computer name, local ip address, current username, results of spotlight searches etc). See the original security report from David Remahl here:
Full-Disclosure: [Full-disclosure] [DR018] Quartz Composer / QuickTime 7 information leakage
Regular (non Quartz Composer) wired Quicktime movies have had the potential to expose some information about the host computer via network access for many years, however it was never to this level and I guess never seen as a security issue. As much as I hope to see things like RSS access return to Quartz Compositions when wrapped in .movs I don’t think it is likely.
It would be virtually impossible to modify Quicktime’s handling of QCs to differentiate between allowing nice, friendly information to be sent (get me links to all the recent images of “x” from flickr.com) and preventing nasty information such as (here is my username and ip address, please start trying to hack into my machine).
Using Quartz Compositions as screen savers expose the same potential risks but I guess in this case you are making a conscious decision to install a piece of software, as opposed to playing a .mov file unaware of its hidden code.
Technorati Tags: Quartz Composer, Quicktime
Twitter last.fm