Archive for the 'Tech' Category

Page 3 of 4

Universal Access, Core Image and remixing the desktop

Published on August 15, 2004 in Tech and Video. Closed

Universal Access

Built into Mac OS X are some nifty features for manipulating the on screen display of any content. While they are designed to assist people who have difficulty viewing the screen, they also let you ‘remix’ the visual output of your computer.

The Universal Access system preferences pane allows you to zoom in on a part of the screen, invert the image so that text becomes white on a black background and increase the contrast. The best things about these functions are that they don’t seem to use too much additional CPU power, they are controllable via keyboard shortcuts and they are quite responsive. Any video or image may be effected as these apply to the entire display output. Invert a DVD video, increase the contrast of a Quicktime movie to the point that it looks posterised, zoom in on any image…

To increase contrast press: control+option+command+.
To decrease contrast press: control+option+command+,

To switch zoom on or off press: option+command+8
To zoom in press: option+command+=
To zoom out press: option+command+-

To switch to and from white on black (invert) press: control+option+command+8
Note: this also sets the screen to greyscale when called via keyboard shortcut, for best effects I recommend using the actual preference pane to switch to invert in colour.

These effects are very basic in comparison to those coming soon in Mac OS 10.4 – Tiger with Core Image. Using the graphics card to handle image processing at a lower level, this looks like it will produce some exciting effects in real time.

If you’ve got about an hour and a half check out Steve Jobs’WWDC 2004 keynote presentation, which shows a good demo of the new Core Image / Core Video features in action.

Interactive Andy Warhol Marilyn Prints

Published on August 13, 2004 in Design, Modes of analyzing new media, Tech, The role of the producer and The role of the user. Closed


This is a great example of an ‘ergodic’ interactive work with a very clever, but simple concept produced well. This site allows you to create your own ‘Marilyn’ prints in real time on screen with an embedded flash file.

Andy Warhol’s Marilyn prints

Quicktime, MIDI & Audio

Published on July 27, 2004 in Music and Tech. Closed

Since Version 2.0 Quicktime has had the capability to play MIDI files using either a built in sound synthesiser package or any hardware MIDI device. As I am interested in creating audio-visual works and programs work with existing digital interfaces and networks I recently began to investigate the possibility of triggering MIDI events from a Quicktime Movie. I could create a ‘click’ track to run in synch with a video track and trigger live sounds and effects through MIDI to excellent audio programs like Ableton Live with each clip. This was all looking very exciting except that the ability to send these MIDI signals seems to have been lost.

A ‘music’ track can trigger sounds from;
1. The built in sound synthesiser (licenced from Roland); or
2. Another collection of samples or ‘sound font’ if it is installed on the user’s computer; or
3. Sound samples embedded in the music track itself.

One of the cool things about dealing with sounds this way is that the pitch and tempo of sounds can be controlled relatively independently. For example with a drum track composed of short samples arranged in synch with a video loop, the player could slow down the speed of the clip and the drums would follow. Or, the drum sounds themselves could be pitched up or down without effecting the video or going out of synch.

While this provides a lot of potential options for music output from Quicktime I’d really like to be able to interface with other MIDI hardware and software.

After a long search for a solution to this problem I still haven’t achieved this, but I’ve found some extremely cool little audio/MIDI utilities which I’m already finding useful:

Audio Hijack Pro Logo

Audio Hijack Pro from Rogue Amoeba allows the audio recording of any Mac OS X application’s output. It can be used to record internet radio stations in any format, live performances and even game soundtracks to MP3 or AIFF. Another cool feature is the ability to tap into Core Audio‘s ability to process VST and Audio Units real time audio effects. This means you can add reverb, equalisation, distortion, delay etc to any audio in real time and record it to hard disk.

SoundFlower from Cycling ’74 is another useful little utility that gives you an additional ‘virtual’ audio input and output for routing sound within OS X. Similar to the way Propellerhead’s ReWire lets you plug some audio applications into each other, SoundFlower also allows any audio application to pass sound to another for further processing. This means I can trigger sounds in a Quicktime movie and process them in Live. This rocks.

Jack Tools Logo

Jack Tools is another utility which does the same sort of thing but I haven’t had a chance to test it out yet. SoundFlower seems better at first glance but I’m sure they are quite similar.

FingaMIDI

FingaMIDI is yet another cool audio utility I found as part of my search. This one is definitely going to be used in future live performances. When activated via a Sytem Preferences pane, FingaMIDI turns the trackpad of any recent Powerbook or iBook into a three-dimensional MIDI interface. Just like a Korg Kaoss Pad, the trackpad outputs the absolute X and Y co-ordinates of the user’s finger as well as Z pressure.

Codecs

Published on July 26, 2004 in Tech, Video and Web. Closed

One of the most important discoveries I made in working on Vidget Version 1 was the H.263 Quicktime Codec. For this sort of work it was very important to use a codec which didn’t require a huge amount of CPU power, provided fairly good quality images, at low data rates and which could be played both forward and backwards. In the case of most codecs the rule seems to be: “fast, small, good, pick two”.

Sorenson 3 video features excellent images at small file sizes but requires quite a bit of CPU (I’ve found it gets a bit choppy when I try to do too many things at once but others such as jeanpoole
swear by it) and is no good at playing backwards.

I used to think that the Photo-JPEG Codec was good because it was able to play well backwards or forwards but it only looked decent at high bitrates and was quite CPU intensive.

3ivx D4 4.5 is a 3rd party MPEG 4 codec which produces better images than the standard Apple MPEG 4 codec at quite low bitrates. This is largely due to the fact that it features two pass variable or constant bitrate encoder. I definitely plan on using this codec for future linear works but it is a bit too CPU intensive for this project.

H.263 will soon be joined by H.264, which has designed to be scalable from limited bandwidth applications like video phones right through to HD. I can’t wait.

About Widgets

Published on July 12, 2004 in Tech, The role of the producer, The role of the user, Video, Vidget - Discussion and Web. Closed

I was familiar with the term ‘widget’ as both a kind of gadget to be used in examples (I always picture being in a year seven information processing and management class doing a graph with the title ‘Widget Sales up 17%) and an element of an interface (eg. a play/pause button widget), but recently the term has become quite a trendy buzzword and is used to describe a class of applications or desktop accessories which sit somewhere in between being ‘content’ like a basic web page, and ‘application’ like a word processor or design tool.

For the past couple of years a program called Konfabulator! has been providing Mac OS users with a suite of functional widgets and the tools to create their own. Check out their site and download the free version. Some great examples of widgets developed by the makers of Konfabulator are weather forecasts which automatically update themselves drawing on weather feeds, photo viewers, calculators and iTunes remote controls which display song information in real time. I had heard of Konfabulator ages ago but I wasn’t reminded of it until I watched the Apple WWDC Keynote presentation. In a very long presentation focussing on the the next version of Mac OS X, Tiger , amongst a number of very cool new features was one called Dashboard.

From the website:
“One Dashboard, Many Widgets
The Dashboard is home to a new kind of application called Widgets. Widgets are mini-applications written in JavaScript that are designed for fun as well as function. They keep you up to date with timely information from the Internet such as stock quotes or the latest view from your favorite Web cam. They can also provide quick, simple access to frequently-used applications such as a calculator, a playback controller for iTunes and a contact look-up for Address Book.”

One of the best things about widgets such as the ones developed by Konfabulator and copied by Apple is that they know their place on the desktop and in the network. What I mean by this is that each widget has a single function and does it well, and that it is able to interface with other programs and files on local and remote systems. A good example would be the standard calculator which comes with the current version of OS X. While a basic calculator is a useful tool, making a calculator which is ‘aware’ that it is on a computer which is connected to the network means the user no longer has to look up currency conversion rates manually because the calculator program can automatically update rates from a remote service.

I don’t think this ‘new kind of application’ is all that new, but it is definitely an interesting one to be explored in art practice.

Vidget 1: an interactive networked VJ application for Quicktime

Published on July 12, 2004 in Design, Tech, The role of the producer, The role of the user, Video, Vidget - Discussion and Vidget - Download. 2 Comments

Its been a bit quiet around here for a while and this is why. I’ve been working pretty solidly on this piece for the past couple of weeks leading up to a gig I co-organised last week. Segmentation Fault is a semi-regular experimental music & visual night we put on every couple of months and proved to be a good motivation (ie. deadline) to get a work together for. in my research I am mostly interested in applying VJ aesthetics and methods to the desktop environment where the user becomes the performer, but its always fun to perform in front of an audience of humans in a room.

Now its time to release this draft to the world and see what people think. Click on the images below to load the two parts in Apple – QuickTime Player.

The scrambled looking black, white and green image will load the ‘output’ movie. This is the movie to be projected on a screen or viewed on a second monitor. It is designed to run at full PAL resolution (720 * 576) to suit the TV output of my laptop. If you want to try it out on a single monitor setup, you can load the movie and select ‘Half Size’ from the Movie menu in Quicktime Player. This movie is really just a kind of holder for up to three other movies. To load different clips into the ‘output’ movie you will need to use the ‘interface’ movie below.

This movie controls which video loop is loaded in which layer of the output movie. Along the top of the window are the numbers 1, 2 and 3. These represent the three layers with 3 being the ‘highest’, 1 the lowest and 2 in between. Next to each of the numbers are playback controls for each layer. Once clips have loaded they may be played forwards and backward, in slow and fast motion and stepped through frame by frame. Next to the playback controls are the graphics mode controls. These control the ways in which each of the layers are blended.

‘Blend 0′ means the clip is completely transparent. It is probably a good idea to switch to this setting if you are going to load a big clip as it will take a while to load and display a still image whilst it is doing so. ‘Blend 100′ means the clip is 100% opaque so any other clip below it will not be seen.
‘Add Max’ adds the bright portions of the clips image over the clips below, leaving the dark areas transparent. ‘Add Min’ adds the dark portions of the clips image over the clips below, leaving the lighter areas transparent. ‘Sub Pin Blk’ subtracts the bright areas of the clips image from the ones below so white snow on a black background will result in black snow on a transparent background. ‘Inverse Or’, ‘Exclusive Or’ and ‘Inverse Exclusive Or’ produce other effects but to be honest I’m still not sure exactly how they work :-) .

These graphics modes probably won’t be much fun to play with until some different clips are loaded into each of the layers. To do this I have designed two different patch loaders. If you click on the 1 or 2 with red # symbols next to them the # will change to a *, telling you which loader is active. The first thing to do is select which layer or ‘channel’ to load the clip into. These are selected by clicking the large 1, 2 or 3 at the top. Next a clip may be selected from the list at the bottom half of the controls. The clip’s name and id number are displayed and when the ‘Do It!’ button is pressed the clip will start to load. (If you are wondering why it is called ‘Do It!’ go see Starsky and Hutch :-) )

Note: there may be a couple missing – such as the ‘live input’ at the bottom right, so if you get a ‘broken movie’ image just try another clip.

The Livestage Pro project files can be found here: interfaceproject.zip outputproject.zip

Digitally Obsessed Software – QT HTML

Published on June 24, 2004 in Tech. 2 Comments

QT HTML is a very cool little AppleScript program which generates the HTML needed to embed a Quicktime movie in a web page. It lets you control the way the movie is embedded with a fairly easy to use interface. For example it lets you choose if you want the movie to autoplay, show a controller or not, link to another movie (ie poster movies) etc.

No more need to remember that <OBJECT CLASSID=”clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B”…

You can download it from digitallyOBSESSED! Software where they seem to have quite a few similar useful little tools.

Found via QuickTiming.Org

Wired Quicktime Examples

Published on June 15, 2004 in Tech and Web. Closed

Wired QuickTime Examples

Quicktime Blogroll

Published on June 9, 2004 in Tech, Vidget - Discussion, Vidget - Download and Web. 1 Comment

Using the same techniques as before, this movie accesses my BlogRolling blogroll RSS feed. More on this later – time for bed.

Random Site – An XML Reading Quicktime Linker Movie

Published on June 7, 2004 in Tech, The role of the producer, Vidget - Discussion, Vidget - Download and Web. 2 Comments

I’ve finally got my head around getting Quicktime to read XML files!

This has been a long and frustrating process as for a while any mention of XML would cause LiveStage Pro to unexpectedly quit. I’m still not sure what was going wrong there but I think the key to making everything work smoothly was using lots of strings to remember the paths to different information stored in the XML.

Here it is:

Continue reading ‘Random Site – An XML Reading Quicktime Linker Movie’

Text to String

Published on May 24, 2004 in Tech, Video, Vidget - Discussion and Vidget - Download. Closed

This is a little movie I made to test how Quicktime deals with real time text input. It is also the first step in a project to dynamically source video material from any URL. In future examples I will use this same method to paste a URL and load it into another window.

Click the poster movie to load the real one. Click above the line and enter some text. The movie copies the contents of the top text track to a string and adds it to the start of the bottom text track. I initially had the movie execute this action only when you clicked. In a purely ‘functional’ interface this is how I would do it but I wanted to stress the real time aspect of the process so I have the movie add text on the ‘Idle’ event handler. This means the text will start cascading down the second track as the user types without waiting for them to finish.

The LiveStage Pro source files can be downloaded here.

Remote control 0.2

Published on May 4, 2004 in Design, Tech, Video, Vidget - Discussion and Vidget - Download. Closed
CODEBASE="http://www.apple.com/qtactivex/qtplugin.cab">



 CODEBASE="http://www.apple.com/qtactivex/qtplugin.cab">

Click the poster movies above to load the real ones. The second one will take a little while to load (3.9Mb)

Alternatively (recommended) download the two files, ControlTOBE.mov (Controller) and TOBE.mov (Player) and open them both in Quicktime Player.

Ok, this is a half finished draft of a basic Quicktime vision mixer. Like the inter-movie text communication movies I posted earlier , this project comes in two parts: one movie to control another. I actually created the text sender movies to troubleshoot when I was making this movie.

The ‘player’ movie contains three video tracks. The ‘controller’ movie features a number of clickable sprites which tell the ‘player’ movie what to do. By clicking on the green or red buttons on the left hand side of the ‘controller’ the tracks in the ‘player’ movie may be enabled or disabled. In the middle of the ‘controller’ movie there are two gradients with a percentage number, these control the opacity of each of the two ‘upper’ video tracks. By clicking on the gradients at various different places along the horizontal the numbers should change and the tracks should fade in or out. On the far right of the ‘controller’ movie is the ‘invert’ button. This inverts the top video track so that white is black and black is white etc.. The other button at the bottom of the ‘controller’ is the ‘add’ button. It is the latest addition and it also controls the top movie (I’ll move it up top in the next version). Rather than simply inverting the video track, the ‘add’ button produces an additive effect (like a Photoshop layer) which changes depending on the presence or absence of the underlying two video tracks.

This draft was designed as an experiment in building a VJ instrument in Quicktime with Livestage Pro for a recent Segmentation Fault gig I was organising. The ‘player’ movie is designed to automatically go to full screen on the video output of my laptop while the ‘controller’ sits on my screen out of sight. I built a more complicated version with about 20 different video tracks for the gig but unfortunately most of the effects such as fading in and out and layering decided not to work on the night. I think I’ve worked out the problem so I’ll post a new version soon.

The buttons usually take a couple of clicks to get going but they should work after that.

devshed.com

Published on April 28, 2004 in Tech. Closed

Dev Shed looks like a great resource for all sorts of programming. I’m sure I will be visiting it for more information on CSS, XML, Java etc.

Inter-Movie Text Communication

Published on April 20, 2004 in Tech, Vidget - Discussion and Vidget - Download. Closed

This is an experiment in inter-movie communication. With Quicktime a movie can send information from one movie to another. The movie files can communicate when they are either embedded into one web page (like this page) or when they are open in separate windows in the Quicktime Player application (you can download the two movies to test this).

In this case the orange movie on the left sends some text across to the white movie on the right. While this may not seem all that amazing, it could be sending much more interesting information and the movie receiving the information could be processing or interpreting it in interesting ways. I’ve chosen to send simple text just because it it is a very handy way of testing if a script is working.

CODEBASE="http://www.apple.com/qtactivex/qtplugin.cab">

The first movie xtexttestsender.mov is made up of a sprite track and a text track.

The sprite track contains one sprite, which has the following script assigned to the “Mouse Click” event.

MovieNamed(“xtexttest.mov”).TrackNamed(“Text 1″).SetTextEditstate(kscriptediting)
MovieNamed(“xtexttest.mov”).TrackNamed(“Text 1″).ReplaceText(“BANG”, 0, 0)

The second movie xtexttest.mov is simply a single text track. The important things are that the movie is called “xtexttest.mov” and that the text track is named “Text 1” so that the above script finds its target. The first line of the script is telling the track named “Text 1” in the movie named “xtexttest.mov” to make the text field editable by a script, setting it to “kscriptediting“. The second line is telling the movie to insert the text “BANG” into the same text track. The ““BANG”, 0, 0” tells the movie not to overwrite the text which was previously in the track. If we had changed the first number to 2 the first two letters of “Here” would have been removed, if the second number was two the last two letters would have been removed.

I have included the LiveStage Pro .lsd files for the two movies here: xtexttestsender.lsd xtexttest.lsd

So anyone with a full copy of LSP or even a demo can have a play with my scripts. In this case the script is very simple, but I’m starting to post all my .lsd files so you should be able see the scripting getting more complex as I learn more.

[The User]

Published on April 11, 2004 in Music and Tech. Closed

[The User] are a Canadian artist collective, formed by Thomas McIntosh and Emmanuel Madan. As they write on their website, “[The User] inhabits the trailing edge of technology”, focussing on alternate uses for obsolete technology. Their most famous work Symphony for Dot Matrix Printers was recently sampled by Radiohead in their excellent radiohead.tv. The work involves the sequencing of multiple dot matrix printers to produce rhythmic patterns of noises. Video and MP3 samples are available on the website.

Their most recent work Silophone opens interaction up to users anywhere in the world through a web interface. ‘Users’ can upload sound files to a server and trigger their playback into a giant empty silo. The resulting sounds are then streamed back to the user via realaudio.

Flash interactives

Published on April 10, 2004 in Design, Tech and Web. 1 Comment

ClownStaples is a source of many interactive flash animations. My favourite (while not strictly interactive) is a pun on the loading of flash animated splash screens. It proceeds in an increasingly complex series of loading screens without end. Unfortunately since the site his hosted by Geocities it is often unavailable due to limited data transfer.

Video Content Management Sytems

Published on March 30, 2004 in Design, Film, Tech, Video and Web. Closed

A while ago Adrian Miles’ posted Video Blogs, Vidblogs and Vogs, presenting an ongoing discussion about the nature and definitions of video weblogging.

He writes:

“At the moment all video blogs are video inside text orientated CMS [Content Management System] engines. But here’s a simple idea (more complex backend), you make a movie that has a sprite and a text track. The text track is there to show a number. The sprite reads an external XML file which simply indicates how many trackbacks that video has.”

So I set about looking for examples of alternative content management systems which deal natively with video rather than text. I’m still working out how to get Quicktime movies to read and write to my own XML databases using Livestage Pro.

WaterCooler provides a very slick looking and functional interface for their content management system in a small 265k host movie.

Navicast provide another ‘aqua’ styled interface to their CMS, this time with more controls such as three levels of compression quality and playback size. The selection and organisation of clips is, however, not as well executed as WaterCooler (for example the first movie loads by itself – slowing down access on a slow connection before the user has made a choice).

The two sites provide good examples of what is possible using the Quicktime Player as a front end for content management, accessing online clips and data. While both feature linear movie clips, a similar approach could be used to deal with interactive and dynamic ‘hyper’ media. This is a direction I am looking to explore as I learn more about the tools.

Brick style

Published on March 15, 2004 in Tech and Vidget - Download. Closed

OK, this is a very basic example of what you can do with a little bit of scripting in Livestage Pro:

CODEBASE="http://www.apple.com/qtactivex/qtplugin.cab">




The whole movie file is only 17k and thats mostly taken up by the background image. The light grey rectangle (or paddle) should follow the cursor and the little red square (ball) will bounce around the square. If the ball hits the ground its game over, but you can get it going again by clicking on the paddle.

Inexpensive Panoramic QTVR

Published on March 15, 2004 in Tech. Closed

I don’t actually remember signing up for it but every day since the end of February I’ve been receiving email from the Livestage Pro discussion/mailing list. I’m definitely not complaining! It’s great to eavesdrop on Quicktime interactive developers as they push the envelope.

A great example of the amazing work that is going on was a post from Bill Meikle. He linked to his Inexpensive Panoramic Video pages which document the development of a system for recording 360 degree panoramas in real time using cheap firewire webcams. The most impressive part of the project is that you can see exactly how it progressed from a basic idea to a series of well executed examples and experiments of the courese of only four weeks.

To subscribe to the Livestage Pro Talk list try emailing: requests@lists.totallyhip.com

Parsing and Writing QuickTime Files in Java

Published on March 9, 2004 in Tech. Closed

ONJava.com: Parsing and Writing QuickTime Files in Java [Feb. 19, 2003]