Action Refraction

A Midsummer Night's Dream

The theatre and digital media departments of York University in Toronto, Canada, collaborated on a groundbreaking motion-capture and digital effects-enhanced stage production of A Midsummer Night's Dream in March 2013.

Director Alison Humphrey created this blog during the research and development phase to share links, ideas and inspiration with the show's extensive Dream team.

Please feel free to check out the left-hand column or the archives to track the early influences on this memorable project, or list below collecting all videos, photos and writeups of the production on a single page.

Enjoy!

Canadian sci-fi cable channel SPACE's flagship show, InnerSPACE, is a daily entertainment talk show that covers film, television, video games, technology, comic books, gadgets, and all things genre. SPACE producer Mark Askwith visited the York University campus a week before opening night, to preview the tech wizardry of A Midsummer Night's Dream:

InnerSPACE: "York University - Digital Dream"

[The above video is not embeddable, so you'll need to follow the link.]

Denise Enriquez of the Dream's digital media team put together this compendium of the many digital effects in the show:

Vanessa Shaver edited a short demo of the donkey head effect:

Another brief donkey head demo, this time from Dynamixyz:

Tiffany Kwan of the publicity team put together a rockin' trailer, as well as a behind-the-scenes feature:

The publicity team also created this Facebook page, where you can find rehearsal and crew photos:

Facebook: "Theatre at York: A Midsummer Night's Dream"

Amy Stewart of York's Faculty of Fine Arts Communications office, created these news articles...

Theatre @ York’s “Midsummer Night’s Dream” offers a groundbreaking adventure in text and technology (March 19, 2013)

Theatre and Digital Media “Dream” team conjures midsummer magic onstage (March 4, 2013)

...and this video, shot during our December tech-testing week in York's Sandra Faire and Ivan Fecan Theatre:

Still Image Galleries:

Photographer John Jacques documented the show's first dress rehearsal on March 20, 2013.

John Lauener photographed the second dress on March 21st, 2013. (A few of these images are also on Flickr, including the full company photo.)

And Michelle Tracey's costume designs are up on Pinterest.

Huge thanks to all who helped to get the word out about this production!

Mr. & Mrs. Dream Have a Baby Holodeck

Cool Wired article on A Virtual Stage That Bends Reality and Pushes Theater’s Boundaries:

"French software engineering company [Dassault SystÈmes] typically uses its virtual reality technology to test, model and simulate products for companies like Boeing.... Dassault's Passion for Innovation program [is] an initiative whose goal is to apply the company’s industrial research and technology skills to the worlds of culture and education. Since 2005, Dassault has worked with architect Jean-Pierre Houdin to simulate the construction of Cheops pyramid, partnered with director Luc Besson to bring 3-D interactivity to movie theaters, and helped cartoon artists turn their cartoons into virtual reality (and these are just a few of their projects). Mr. & Mrs. Dream is Dassault’s first crack at live dance..."

For this touring performance project, Dassault created "a mobile 'magic box,' which is basically a disassemble-able series of four gray screens and six projectors that would be the canvas for the immersive world of Mr. & Mrs. Dream. The box is similar to the virtual reality rooms traditionally used by industrial companies, only instead of testing emergency scenarios and modeling new airplane features, this box is used to motion-track dancers and project computer-generated images. For scenes like the one mentioned above, [Benoît] Marini positioned three Kinect sensors above the stage to track the dancers’ movements. So when the dancers jump, the meteorites bounce, or when the dancers kick, a flurry of leaves float through the air. Most of the other projected dance numbers were motion-captured in the studio and are played back in sync with the music...."

Hat tip to Mark Askwith!

New Video Up!

Amy Stewart, head of communications at York University's Faculty of Fine Arts, has just posted her new video feature on Theatre@York and Digital Media's collaboration on A Midsummer Night's Dream.

So exciting to see our awesome Dream Team captured in action!

We've also just set up a new web address for the show, which gives performance info and a link to the box office. Tickets now on sale - get yours early for best seat selection!

www.dream-remix.com

Link Love

To celebrate Valentine's Day, I'm spreading the link love for our lovely and talented digital media team. In addition to weaving fairy magic, they've been writing blogs as part of their coursework with professor Don Sinclair:

Shianney Zaelani
http://mszaelani.com/

Denise Enriquez
http://moonshineandlion.tumblr.com/

Rhys Mendes
http://wedreamlucidly.tumblr.com

Assaf Gadot
http://midsummerwinter.tumblr.com/

Lalaine Ulit-Destajo
http://inmiddream.tumblr.com/

---

And while we're linking...

Here are the show's preliminary costume sketches, by Michelle Tracey:
https://pinterest.com/michelledtracey/midsummer-costume-sketches/

This is a collection of the show's Pinterest image-inspiration boards (click through to various participants for more of their own individual boards):
http://pinterest.com/midsum/

And this is a brand-new website for Vanessa Shaver, Creative Producer, Lucid Dreaming LiveFX:
http://ldlfx.com/

Cordyceps

Talented 3D artist and animator Stephanie Fong is creating the nervous-system-inspired digital background for our fairy forest, but this isn't the first fun she's had with braaaaaaains...

The Mechanical Heart of the Dream

RSC World Shakespeare Festival Director Deborah Shaw describes A Midsummer Night's Dream (As You Like It), Dmitry Krymov's response to Shakespeare's 'A Midsummer Night's Dream', a 'completely off the wall' experience.

"Amongst the chaos of an auditorium that hasn't finished being built, the mechanicals are rehearsing for their big performance.

Striving to be the best, they are now starting to get nervous. Presenting the story of Pyramus and Thisbe won't be easy – there are songs to rehearse, acrobatics to master and lines to learn.

And they haven't even begun to train their acrobatic dog...

Russian artist and director Dmitry Krymov presents his global premiere and unique take on Shakespeare's greatest comedy. His previous productions are internationally renowned for their innovative staging, dynamic imagery and beautiful costumes and set – which have established him as one of the most original and inventive directors working today."

More Fun with Particles

York Digital Media student Jonathon Asperjan found this video by motion graphic artist Esteban Diácono, following our latest Dream tech lab on Friday, in which we discussed both particle effects and interactive sound and visuals:

Two more videos by Diácono. The first is concept development for a Lionhead game called Fable: The Journey: "The idea was to create a two hand Kinect-based combat system.... They were keen about the idea of moving away from the traditional game 'magic effects'. I designed a two hand combat system in which you mix mechanical and organic elements to engage and kill your enemies. The design didn't fully make into the game, but the whip-and-slash element actually did it, which was a great pleasure. Box of Toys provided an incredible monoaural audio track for the animations."

And this one, for the neoclasical Icelandic musician Ólafur Arnalds (whose music also features in the first video), I just liked for its two-and-a-half-D visual playfulness:

---

Investigating treatments for shadow magic, I also came across this 3D calligraphy from TeamLab (previously featured in the post Dancing with Pixels):

Once more, with goldfish:

And a longer, more surreal exploration of ink-shadows, which kinda recall Kinect Graffiti and My Secret Heart from the original Fun with Particles post:

---

Addendum: following this post, Aaron McLean tipped us to the work of Matthias Müller:

If Trees Had Spines

Jennifer Steinkamp is an installation artist who works with video and new media in order to explore ideas about architectural space, motion, and perception. (She also contributed imagery to the U2 concert tours PopMart and Elevation.) York production/design prof Ian Garrett recommended we have a look at her work with digitally animated trees for the Dream's many forest scenes.

Orbit 2 (2008 -- still images and description, artist's video) video installation at The Armory Show:

Mike Kelley (2008 -- still images and description, artist's video), video installation at Mass MoCA:

Dervish (2005) -- still images and description, artist's video)

A brief video of Steinkamp talking about making the piece Mike Kelley (she starts talking about the software at 1:40):

Here are still images with links to several more pieces for which I couldn't find embeddable video:

The Vanquished (2011) -- still images and description, artist's video)

Eye Catching (2004) -- still images and description, artist's video)

Marie Curie (2011) -- still images and description, artist's video)

Florence Nightingale (2010) -- still images and description, artist's video)

And some earlier works:

Happy Happy (1997)

Smoke Screen (1995)

Naysplatter (1996)

3D Set Projection

In this music video for "Sweater" by the Belgian band Willow, director/editor/3D animator Filip Sterckx creates a universe on a soundstage using just three projectors, two walls and a floor:

The behind-the-scenes shows how much craft and practice it takes to interact with a virtual environment (and stay on the treadmill!):

(Hat tip to Jos Humphrey.)

Light of Hand

What kind of theatre magic can you create with a handheld screen (or seven)...?

"The iPad act was a custom creation, made to be performed in the Stockholm booth at MIPIM trade show in Cannes. It was commissioned by lovely bureau Step2 Communications and the brief was to create a visual, magical and striking presentation about Stockholm using modern technology.

Charlie Caper and Erik Rosales imagined, scripted and built it. We also flew in Ben Seidman from the US to help with all aspects of it and the music was written by Markus Jägerstedt. The video on the iPads is made by Image Research."

Created and written by Charlie Caper and Erik Rosales
Commissioned by Step2 Communication
Music by Markus Jägerstedt
iPad animations by Image Research
Magic Consultant Ben Seidman
Conceptual inspiration Marco Tempest

For more information see http://www.d1gits.com

PBS Weighs in on Kinect and Holograms

On Idea Channel (a PBS show that examines the connections between pop culture, technology and art), Mike Rugnetta asks: Does the Kinect Make Microsoft an Arts Benefactor?

If you haven't had a chance to play with Microsoft's Kinect, you're missing out on some great video games and some amazing art! The Kinect is a crazy awesome piece of XBox 360 hardware that maps your physical movements onto any screen. Artists of all stripes have embraced the Kinect - using the gesture recognition technology to create some pretty amazing interactive artworks and insanely impressive visuals. These works wouldn't have existed without this amazing piece of technology, making the corporate giant Microsoft the 21st century's incarnation of Lorzeno de Medici."

And further to yesterday's post on holograms, here's another Idea Channel muse:

You might have heard about the debut of Hologram Tupac at the music festival Coachella earlier this year. But with Queen's recent announcement of a Hologram Freddie Mercury and TLC's plans for a Hologram Lisa Left Eye Lopes, it looks like Hologram stars are here to stay. Is this hologram mania just nostalgia, or is it part of an art movement called "New Aesthetic" that blends art and technology?

Which of course leads inevitably to the question: "Is Miku Hatsune A More Authentic Pop Star Than Lana Del Rey?"

The Holographic Stage

Ian Garrett, York Theatre's newest Production/Design faculty member, is "involved in an experimental mixed-realty theatre project that's playing with everything from Holograms to Second Life." In this video, his co-producer Patty Rangel talks about holographic systems for avatars:

Here's a demonstration of Second Life avatars and a live guitarist sharing the AV Concepts Holographic Stage in San Diego, California (concept development and integration by Patty Rangel):

And here's AV Concepts' Nick Smith explaining how their Musion Eyeliner holographic stage works:

Fun with Particles

Memo Atken created visuals for My Secret Heart, a music and film installation & performance commissioned by Streetwise Opera with music composed by Mira Calix, and sound design by David Sheppard. Memo says,

"The sensation of performing and recording the visuals was that of actually directing a film with thousands of virtual actors, commanding an army, digital puppetry – an approach I’m sure I will be revisiting in the very near future."

Here are some "initial AI/flocking and behaviour controlling tests with Processing":

Excerpts from the final film:

---

And here's a different application of similar visuals, driven by live motion tracked by the Microsoft Kinect:

Kinect Graffiti is a digital graffiti tool created by Jean-Christophe Naour. "The idea behind this project is to use the kinect to track the motion behind graffiti. Visualizing the body and drawing trough different angles in realtime, understanding surrounding space, pausing time..."