Archive for the 'Tech' Category

Send to Receive

Send to Receive is a WebGL particle animation that responds to SMS.

screen shot

You can see it running live here (you’ll need a WebGL capable browser such as Chrome).

It uses text messages to generate pseudo-random combinations of particle colour, background colour, size and speed to change the animation. It also uses a dictionary of colour words found via XKCD to find matching words in the text messages and use them to colour the particles.

The animation is based on a THREE.js demo project by Paul Lewis / @aerotwist.

DSC_0291

Send to Receive was created over two days at The Creators Project / Eyebeam #arthack hackathon. Read more about the other arthack projects and winners here.

By David Wolf with Twilio SMS integration & Sinatra backend by Jonathan Vingiano based on an idea by Crystal Butler.

You can download the source and fork it on GitHub here

HTML5 Audio Bubble Shooter

screenshot

Click on the image to play (requires the latest Chrome Beta)

This game/demo was created in an afternoon at the Google HTML5 Hackathon NYC on Monday August 1st 2011 by Du Hoang (design), Michael Yin (game programming) and me (audio api integration).

Using the new HTML5 Audio API, the game does real-time audio analysis and beat detection on an mp3 file playing in the browser, triggering music bubbles on the beat and changing the game background gradients.

It’s pretty crazy what you can achieve in a few hours of hacking! It was lots of fun.

It requires the latest Chrome Beta. You will also need to enalble the Web Audio API by going to about:flags. See instructions and more examples here.

Based on http://www.propulsionjs.com/ and the HTML5wow GAL 9000 demo.

Audio: Show My Ass by Dominique Young Unique (via Pitchfork)

TechCrunch Disrupt NYC Hackathon

TechCrunch logo
Hackathon Day Two: The Apps Are In! (TCTV) In the early hours of Hackathon day two, bleary-eyed hax0rz roamed the conference hall, checking out each others’ apps. TCTV caught up with a few who were still standing to see how they fared after twenty-four hours, heads down in the code and design… (video)

Last weekend I participated in the TechCrunch Disrupt NYC Hackathon. Starting at 12.30pm on Saturday and working through until 9.30am on Sunday I worked with Jonathan Dahan on a prototype of Soundscape. We were interviewed by TechCrunch at that special hour of the morning when we were totally manic and couldn’t stand still! See the video here.

The mobile app we built lets you tag and identify songs when you’re out at a café, a bar or a club. It links the song information to the venue as well as whether you like or dislike the song.

Soundscape

The idea is to build up a musical map of what is being played where, so you can choose where to go based on your musical tastes and give feedback to the venue owners and DJs.

The prototype app was built using HTML5/JS/CSS through Phonegap with a server running Node.js and MongoDB. We used The Echonest for track information and foursquare for venue information.

The app was designed by Courtney Christopher.

Another Quartz Composer feature broken with Mac OS 10.4.7


Last year an update to QuickTime meant that Quartz Compositions could no longer access RSS feeds when played within the QuickTime Player application. This change was made to stop maliciously created compositions from sending information to remote sources without the user’s knowledge and was made in the name of security. It was very annoying as I was using Quartz Composer to create ‘network aware’ works which accessed RSS feeds, bringing in text and photos from various sources online.

Now with the latest Mac OS software update another of my favourite features has been taken away from Quartz Composer. With 10.4.7 a Quartz Composition cannot use the ‘Image with movie’ node to nest compositions within compositions. Previously I could create a composition, save it in a .mov wrapper and then import it back into another composition as I would a normal movie file. This would let me quickly build up complex interactive compositions, layer and manipulate them in custom Vidgets.

I came across this problem on the Quartz Composer Developer List a few days ago but it was only when I went to play some of my more recent compositions last night that I realised how much of a problem it is. I had come to rely on this feature.

Pierre-Olivier Latour writes that they had to disable this feature so as to prevent people from creating compositions which reference themselves, crashing the computer (another potential security risk). I understand the need to make the software as stable and secure as possible but it seems like there should be another solution which prevents the crashing without ruling out a whole set of features.

It should also be noted that similar features such as network access (including the parsing of remote XML files) and the nesting of movies within movies have been key features of the QuickTime architecture for years without any such restrictions.

del.icio.us daily links and RSS media feeds

del.icio.us/dpwolf by David Wolf

I’ve set up del.icio.us, a clever social bookmarking service to automatically post my daily links in the blog. I figure this place could do with a bit more activity and by adding a couple of lines of description to each link I can share the sites which catch my attention with a bit more clarity without having to sit down and write a proper post. I use del.icio.us quite a lot as I often move between different computers and locations.

While the daily post service is great, the interface for setting it up is a little clunky and cryptic unless you know what you’re doing. Thankfully Kevin Lim has written a helpful little guide to setting up del.icio.us with WordPress. It basically involves entering a username and password for your blog, the address of the xmlrpc.php file on your server, what time you’d like the post to occur in GMT and which category you’d like the post to appear in.

iTunes downloading video from a del.icio.us RSS feed

Another cool del.icio.us feature I’ve been meaning to post about for ages is the addition of RSS feeds for particular media types. Traditionally del.icio.us has provided RSS feeds of links for each tag, for example the RSS feed for links tagged “example” http://del.icio.us/tag/example is http://del.icio.us/rss/tag/example. Now, (since about halfway through last year actually) you can also search by media file type and generate RSS feeds for these complete with enclosures for use in podcasting applications like iTunes or FireAnt. So it can generate a feed of all the mp3 files people link to directly, or all the QuickTime movies or JPEG or MPEG files. Additionally, you can add tags to generate feeds of a particular media type about a particular topic, for example http://del.icio.us/tag/system:media:video+80s give you a collection of 1980s themed video files (mpeg, wmv, mov etc).

While I use iTunes as a media RSS client at the moment to harvest source clips which may be manipulated in live performance eventually it would be great to add a built in video RSS reader to my experimental video appliations (vidgets) and do live video searches.

[A disclaimer: subscribing to a feed of all mpeg or wmv files linked to by random strangers on the internet can lead to the automatic downloading of porn or other things you may not want on your hard drive (especially on a work or uni computer) so be careful eh ;-) .]

Technorati Tags:

Glitcharama

A few months ago I began playing with manually corrupting jpeg files to see what kinds of artefacts I could create. I selected an image, compressed it down to a very small size (so I could easily manipulate large chunks of the data), opened it in a text editor (I like SubEthaEdit and TextMate) and added random text, copied pasted and generally shuffled the data, occasionally saving as new files.

Above is a QuickTime movie which animates through 12 of the resulting jpeg files. Click it to stop if it’s giving you a headache ;-) I re-compressed the jpegs just to be sure they wouldn’t crash QuickTime Player. Manually introducing errors and noise into files and then playing them is one of those “make sure you save any important files you have open” situations as things can grind to a halt.

I was playing with these images at Plug N Play at Kent St on Thursday night. I mentioned that I was planning on writing a php script which would similarly screw with jpeg images online and Sean told me about glitchbrowser.com.

From the site:

Computers are not allowed to make mistakes.

The glitch browser represents a deliberate attempt to subvert the
usual course of conformity and signal perfection. Information packets
which are communicated with integrity are intentionally lost in
transit or otherwise misplaced and rearranged. The consequences of
such subversion are seen in the surprisingly beautiful readymade
visual glitches provoked by the glitch browser and displayed through
our forgiving and unsuspecting web browsers.

This work was produced for New Langton Arts Packets programme,
by Dimitre Lima, Tony Scott and Iman Moradi.

Glitch Browser lets you enter a website’s URL and it will show you the page with all of the images randomly glitchified. For example, here’s the most interesting photos from Flickr through the Glitch Browser. Great stuff!

Technorati Tags: , ,

Brush 0.1 by Dan Winckler

Brush 0.1 by Dan Winckler

“This is brush, a small Max/MSP/Jitter patch that I’ve compiled as a standalone application. It’s aimed at visualists who are just starting out and looking for software to play with. Programmatically, it’s very simple. Video from a live camera (or a movie file) is fed back on itself so that light stays on the canvas (screen). Thus, you can paint with the light in the room you’re in. Decay (fade time), tolerance (lower luma threshold) and color inversion are adjustable so you can adapt your performance to any lighting conditions.”

Dan Winckler’s blog – brush 0.1: a tool for visual performers

This is a great little piece of software, what I would call a Vidget. A small scale application which lets you manipulate digital media in real time for improvised performance. It is very easy to use and entertaining to play with.

VJ Bertranol

VJ Bertranol

VJ Bertranol aka Bertrand Gondouin is a Sweden based VJ, video blogger and video software developer.

Through his company Mixnbrew he has developed a number of interesting bits of software including Symtonic A Flash based online video mixer application, Patchouli a video blogging uploader / content management system, and Scramble a granular video synthesis engine.

Good work and good to see the whole video blog / vj / network video crossover thing happening.

Technorati Tags:

Electrofringe 2005

Electrofringe 2005

It’s coming up on that time of year again. Electrofringe is a new media arts festival held in Newcastle as part of the This Is Not Art arts festival. Lots of good stuff on and definitely worth the trip.

This year I’m presenting / performing the following:

Realtime Video Manipulation using Isadora and Quartz Composer

“Introduction to the use of two software platforms which allow the creation of realtime video installations which can respond to the audience or other available data.”

This will be a introductory ‘tips and tricks’ panel presentation with Luke Toop (Adelaide), Steve Huon (Melbourne), Khalid (Melbourne) and myself. I’m probably going to focus on the basics of how create a Quartz Composer patch and then turn it into a stand alone application with Xcode and a little bit on network access and RSS etc.

Spac{v}e dpwolf vs Doktorb Robotnik

A live improvised AV performance with my frequent collaborator Doktorb Robotnik. Audio and video feedback crossed with data pulled from the network in real time. Followed by a discussion.

Re – Imagining (live) Video – Narrative

A panel presentation / discussion with Jean Poole and Anna Helme.

From the blurb:

“Audio has been easily sampled, processed and manipulated live for decades. Although hardware and software now allow video to become just as malleable, it is used in a limited number of ways. How can current-day video tools be used to composite video at live events differently? How can theatre and storytelling better integrate live video? What storytelling possibilities lay beyond recreating cinema, music videos or ‘wallpaper’? How do live video and sound work best together? What video is most worth having live?”

Technorati Tags: ,

The Guild website

Yesterday I finally finished setting up a website for The Guild of Commercial Filmmakers, a film production company where I work.

The site is powered by WordPress and makes extensive use of customised templates, css and custom fields. The design is by Nicole Dominic, sliced up and css/xhtml-ised by me.

One of the main functions of the website is to present an easily update-able show-reel of the company’s work (primarily TV ads). Some of the tricks I discovered whilst making the site may be of interest to the videoblogging crew or others wanting to use WordPress as a content management system for video. The next step is going to be working out how to customise the site’s RSS feeds to include this information.

screenshot of The Guild website

For each of the ads I make a regular post, storing a lot of information in custom fields, such as: the url of a thumbnail image; the url of a poster movie and the url of the movie itself. I store this information here rather than in the actual post text in order to separate content from styling and presentation, allowing me to refer to the same clip in a number of different ways from different areas of the site.

Continue reading ‘The Guild website’

3D objects in Quartz Composer

The main tools of trade when using Quartz Composer are patches. Each of these modular objects has a particular function when plugged together to create flows of information. For example a patch may import or download an image, calculate a mathematical operation, affect an image or function as a switch, selecting between other patches and signals. This flow ends up rendered on screen by a renderer patch.

While you can plug these patches together in almost any way, until recently you could only play with the patches Apple has included in the program by default. You could not create your own. This meant that while QC talks directly to the graphics card and renders all images as OpenGL surfaces, users were limited to rendering onto flat surfaces, spheres, cubes and a teapot shape.

Recently ClockSkew worked out how to enable disabled patches and write your own custom patch. With this information ?? at Quartzcomposer.jp developed a custom patch which allows you to load 3D Studio Max objects into Quartz Composer and render images onto their surfaces.

Via Roger Bolton: Quartonian » Custom patches for Quartz Composer now a reality. Follow the above links for all the downloads how-tos.

I’ve been playing with rendering onto 3D Studio Max .3ds objects in QC for the past week or so but have had mixed results. The fighter jet model included by ?? works well but most other objects I’ve found online have either crashed QC or rendered incorrectly. I’m sure these issues will be worked out down the track.

Here is a quick demo Quicktime movie showing the fighter jet model in QC with cloud effects generated via particle systems based on this smoke effect by Noise Industries.

Click image to play (1.9Mb, 30sec Quicktime).

Technorati Tags:

MPEG Streamclip

Squared 5 – MPEG Streamclip for Mac OS X is a handy little application which converts and demuxes mpeg streams. I found it a few weeks ago at work when I needed to recompress a large file from a client to put up on a website. By itself Quicktime Player will only export the video of a muxed mpeg file. MPEG Streamclip is designed specifically to convert mpeg 1 or 2 files to Quicktimes, or split them out to separate video and audio files (mpgs, m2vs, ac3s and aiffs). Very handy and free!

Interact with Quartz Compositions in Safari through JavaScript

mDimension Technology have developed a plugin for Safari which lets you not only view Quartz Compositions in a web page, but also interact with them via JavaScript. This is very neat!

So far I’ve been experimenting with creating custom applications (Vidgets) in Xcode which control Quartz Compositions, this plugin appears to give me most of the same functionality within a web page. To control and manipulate the composition, first you must follow the same methods of ‘publishing’ inputs and outputs from a composition as described here.

View source on these pages to see how to send and receive information to and from these inputs via JavaScript. I’m still working it out myself!

Here is a demo page from mDimension with a Quartz Composition embedded.

Here is a quick demo I made up based on the above page. It allows you to load an image by entering its url into a field and clicking outside of the field (don’t need to press enter), and change the spinning text similarly.

A Quartz Composition embedded in a webpage, being controlled via JavaScript

Technorati Tags: ,

QuickTime 7.0.1: Security enhancements

QuickTime 7.0.1: Security enhancements

This sucks. QT 7.0.1 no longer lets you access remote web locations from within a Quartz Composition when played using Quicktime Player. No more playing such compositions from the Quicktime Plugin in a web page either. This means no RSS feeds unless you run the composition in Quartz Composer itself or in a custom app.

This is the direction I’m heading anyway (making custom apps) but it means you can no longer distribute a .qtz file or a composition in a .mov which accesses network data and expect it to work on machine running Tiger unless the user has Quartz Composer installed.

These were some of the coolest features of Quartz which I was only just starting to explore before the plug was pulled. Hopefully these features will return some time soon.

Update 18/7/05

I don’t think I made this clear enough but the change was made in response to the potential security issue whereby network access from within a composition wrapped in a .mov file to be used to leak information to a malicious third party when used in combination with Quartz Composer’s ability to access sensitive information about the host computer (computer name, local ip address, current username, results of spotlight searches etc). See the original security report from David Remahl here:

Full-Disclosure: [Full-disclosure] [DR018] Quartz Composer / QuickTime 7 information leakage

Regular (non Quartz Composer) wired Quicktime movies have had the potential to expose some information about the host computer via network access for many years, however it was never to this level and I guess never seen as a security issue. As much as I hope to see things like RSS access return to Quartz Compositions when wrapped in .movs I don’t think it is likely.

It would be virtually impossible to modify Quicktime’s handling of QCs to differentiate between allowing nice, friendly information to be sent (get me links to all the recent images of “x” from flickr.com) and preventing nasty information such as (here is my username and ip address, please start trying to hack into my machine).

Using Quartz Compositions as screen savers expose the same potential risks but I guess in this case you are making a conscious decision to install a piece of software, as opposed to playing a .mov file unaware of its hidden code.

Technorati Tags: ,

Quartonian

Roger Bolton from eskatonia has put together a great Quartz Composer VJ patch complete with on screen previews and keystroke triggering of clips.

Quartonian screen shot

Quartonian is designed to be run in Quartz Composer at full screen and thus has very few on screen controls, in fact with a press of the “~” key you can toggle the whole onscreen display on and off. This makes for a fairly steep initial learning curve for users but, you soon get a feel for using the keyboard and mouse.

Clips (up to 24) must be loaded into ‘Image with movie’ nodes from within a macro patch using Quartz Composer’s editor window to begin. *Hint* it’s the little blue one at the top :-)

Quartonian patch editor screen shot

Also on the eskatonia site is a guide to Quartz Composer for VJ’s with lots of useful links and examples.

Until visiting this page I didn’t realise that you could load .qtz files into ‘Image with movie’ nodes. For example, the ‘Audio Barcode’ .qtz file from the site which reacts to audio input can be loaded into a patch and used as a mask on another clip.

Both Quartonian and the other examples on the site are released under attribution – non commercial – share alike creative commons licenses so users are free to modify and create derivative works.

Technorati Tags:

Quartz Composer TV

QCTV, a sample application and source code from Apple's WWDC 05

Quartz Composer QCTV is an application and related source code which demonstrates how to use QC and Xcode to create a TV news style system complete with bluescreen keying and a scrolling text crawl via a RSS feed. It is really good stuff, allowing the user to display either a live camera feed or Quicktime movie over a background. QCTV also features the ability to record the ‘TV show’ to a movie file or stream it out via DV (not while the live camera feed is in operation though).

Pol and the Quartz Composer people at Apple have done an excellent job with their developers documentation and examples. In particular I have found the Quartz Composer Programming Guide: Introduction to Quartz Composer Programming Guide to be most helpful.

Technorati Tags:

MIAF – LA 6: a Quartz Vidget

Screen shot showing the MIAF - LA6 Vidget controls and output

This is the next version of my live video performance (VJ) application. It is an updated version of the program I used for my recent performances at MIAF: Remains To Be Seen and Liquid Architecture 6.

Like the previous version, Bikelights: a Quartz vidget, MIAF-LA6 features one window with controls on the left hand side and a display on the right. It is designed so that I can have the controls on my laptop screen while sending the video from my second monitor output at 640 * 480 to a projector by arranging my monitors like so and dragging the window across.

Screen shot showing monitor arrangement

At some stage I’ll work out how to do two separate windows but this works well for the moment :-)

Screen shot showing controls

Where the previous version only allowed the display of one source image at a time, this version lets the user layer two separate images through a number of effects and outputs. The left hand side of the control area is split two (Source Clip A and Source Clip B). Each area contains four empty text fields where Quicktime movies may be dragged and dropped from the Finder (click in the field to activate it, drag and drop, then hit Enter to set).

Below the four movie path fields is a fifth path field which allows the user to specify a local path to a folder of still images. Again folder paths may also be either typed in manually or dragged from the Finder. After pressing Enter the “# of Images” field should update, letting you know how many images are in the folder you specified. To the left of this field is the “Sequence Duration”, this lets you specify how long (in seconds) it should take to cycle through all of the images in the folder. For example, if you have 8 images and want them to play through at four frames per second second you should set the duration to 2 seconds.

To the right of these fields is a large vertical slider, this lets the user select any one of the four Quicktime Movies or the folder image sequence to process and display.

To control the playback speed of the Quicktime movies, check the “Clip Speed Control” box and move the horizontal slider below. If unchecked the movie will play out at normal speed. If checked the speed will be controlled by a combination of the slider position and live audio analysis (if you have a microphone set up). The far left of the slider is 0 * original speed, the far right is 5 * original speed. If a microphone is connected, the video will jump up to one second forward through the clip with the loudest audio peak.

Below the speed controls are the image controls. These controls allow the user to adjust the saturation, brightness, contrast and colour angle of the clips or image sequences. To leave the Quicktime movie unaffected uncheck the “Image Controls” box.

On the right hand side of the control area is the output stage, the “Renderers”. Each renderer may be switched on and off via a check box and be used to display either source A or source B (or a combination of the two in the case of the “Billboard” and “Inset Image” renderers).

Download miaf2vidget.zip (Mac OS 10.4 required)

Download miaf2quartz.zip (Quartz Composer file)

Download miaf2xcode.zip (Xcode project)

This work is distributed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 licence.

Technorati Tags: ,

Bikelights: a Quartz vidget

One of the coolest things I’ve discovered about Quartz Composer lately is the fact that you can use it (in combination with Xcode) create stand alone applications with custom interfaces all without having to know any ‘proper’ programming. So, here’s one of my first experiments….

This vidget is a simplified version of a larger Quartz Composer patch I have been using to perform with recently. It allows the user to import a number of Quicktime files or still images and apply a number of effects to them.

Bike Lights Vidget Screen Detail

On the left hand side of the window are five text fields which store the local path to each movie file. To add your own clips, simply click in the field and then drag the movie’s icon from the finder into the field and press enter.

Just to the right of these text fields is a selection slider, which is used to select which source clip is to be effected. The arrow will snap to each of the clips as you slide or click it.

Below the and to the left are more sliders which control the clip’s playback speed, saturation, brightness, contrast, and hue angle. All of these work in real time to adjust the display of the clip.

To the right of these image controls are check boxes for the different rendering effects. They are ordered from top to bottom in terms of the layers onto which they are drawn. The top three have optional black ‘clear’ backgrounds which block out the layers below.

Expanding audio renders the clip onto three layers, red, blue and green and uses live audio input to resize the three layers to produce a coloured motion blur effect.

Particle system
renders the clip onto hundreds of small layers which explode out from a randomly selected point on the screen.

Slow moving layers renders the clip onto four layers which crop the image and oscillate in 3d space based on a combination of audio input and low frequency oscillators.

Inset Image produces a ‘picture in picture’ effect which is manipulated by audio input.

Bike Lights Vidget Screen Shot

Download (requires Mac OS X 10.4).
BikeLights Clips (these tiny lo-fi clips of my bike lights were shot with my phone and are the reason for the name, try them out)
Quartz Composer file
Xcode Project

This work is distributed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 licence.

Technorati Tags: ,

Compostition in a composition

For the past few weeks I’ve been playing with Quartz Composer live at Plug ‘n Play on Thursday nights at the Kent St Café on Smith St in Collingwood (Melbourne). It’s a great place to experiment in a comfortable environment with a projector and sound system and the only place I can imagine that gives you free drinks for sitting in the corner programming!

Last Thursday I attempted to use my QuickTime vj application Vidget 3.5 (I’m actually up to version 3.6 but haven’t got around posting it) to mix Quartz compositions saved as .mov files. This produced some interesting (unstable) results. The way the vidget works is to layer up to three Quicktime movies (movies in a movie) over the top of each other with different transparency/opacity graphics modes – like video Photoshop layers which are rendered in real time by Quicktime. With the standard ‘Dither Copy’ mode my compositions played reasonably well but whenever I switched to some other modes the video flickered very fast and bright and whenever I resized the window to go to full screen Quicktime unexpectedly quit.

Since I couldn’t use the vidget I decided to see what happened when I dropped Quartz compositions saved as .mov files into the composition workspace of a new file, to my surprise rather than appearing as ‘image with movie’ nodes (as QuickTime movies usually do), they were added as ‘macro patches’ meaning I could double click and manipulate their inner workings, excellent!

Here’s an image of my original composition space with the new macro patch highlighted:

A composition in a composition 1

Double clicking on the macro patch reveals the composition within:

A composition in a composition 2

Technorati Tags:

Quartz Composer

For me, the coolest new feature of Mac OS 10.4 “Tiger” is an application called Quartz Composer included with ‘Developers Tools’. It allows the user/developer to create patches for the real time generation and manipulation of images using the new Core Image engine via a visual patch based interface.

Quartz Composer shares much of its interface style and function with its predecessor PixelShox, an OpenGL based real time video application a mate of mine Khalid has been using for vj work for a while now. While it is no longer being actively developed (its developer has taken a full time position in the computer graphics field *cough* at apple *cough*), it’s worth a look if you have a Mac and have not moved to 10.4 yet.

Like PixelShox, Quartz Composer has a wide array of live inputs such as mouse and keyboard tracking, MIDI, audio and video inputs. One new input featured in Quartz Composer which has caught my eye is RSS processing using the new Safari RSS engine. While a more generic XML parser would have been preferable I’m sure I can have a lot of fun with RSS alone :-)

Quartz compositions may be incorporated in ‘real’ programs with Cocoa bindings and interfaces built with the ‘inteface builder’ application, used as screen savers (one of the default screen savers included with 10.4 displays the current Apple News RSS feed in an ‘eye candy’ 3d flythrough) and, drumroll…. played in QuickTime Player in 10.4!

QuartzComps has popped up as a blog for sharing and discussing Quartz Composer patches, as has the Apple Quartzcomposer-dev Info Page mail list (see the Mailing List Archives too).

For more info on Core Image, Core Video and Quicktime 7 see this page of ArsTechnica’s excellent and in-depth review of 10.4 – Mac OS X 10.4 Tiger : Page 16