Scott Ellington's Blog

SeeSpeak

Let’s start with this:

http://www.ted.com/talks/john_underkoffler_drive_3d_data_with_a_gesture.html?utm_source=newsletter_weekly_2010-06-02&utm_campaign=newsletter_weekly&utm_medium=email

I noticed that among the dissenting opinions in Comments, some people’s objections to the g-speak interface centered on the strenuousness and stamina needed for keeping one’s arms aloft, which led me to the notion of SeeSpeak — that the gestural language of John Underkoffler might be adapted and significantly refined by building a variation of it on the gestural (and facial) communications of a symphony orchestra conductor.  Which led me to think of the rapid evolution of motion-capture technology that made significant strides at Weta during the translation of Gollum’s character from page to screen.

And if a hundred-or-so musicians can read complex, nuanced direction in the contextually-meaningful gestures of their designated conductor, why couldn’t an operating system manage something similar; contributing visual information to facilitate apprehension by the collective audience?  That visual(/olfactory/kinesthetic/climatic…) information could be provided specifically by artists adept in those alternative media, working in collaboration with any/all agents of the production to create/recreate an augmented human experience as varied and wonderful as the human imagination is capable of concieving.

Well, there are copyright concerns that prevent the visual artists from freely incorporating cinematic elements of John Ford films into a production of Aaron Copland masterpieces, for example.  And who, exactly, would pay for all of this innovative and inconcievably expensive creative collaboration?  I have not a clue.  Maybe he does:

http://www.ted.com/talks/lessig_nyed.html?utm_source=newsletter_weekly_2010-06-02&utm_campaign=newsletter_weekly&utm_medium=email

but I’m inclined to doubt it.  So I started thinking about stuff I read here:

http://brandsplusmusic.blogspot.com/2010/05/rise-of-creative-thing.html

And I continue to ruminate on the idea that any generic renaissance faire is the nucleus of an EraFair in which various temporal points in our collective cultural history draw musicians, painters, poets and dancers together to recreate that particular creative period — at various, proximal locations in a city park — where passersby can become interested, initially, in the reach of the music, then the atmosphere of creation and the invitation to engage and, ultimately, study/play with practitioners of arts formerly-defined by whichever eras interest them.  A physical manifestation of creative peaks in human cultural history, open to every art-creator, student and fantasy-seeker (past/present/fictional…) just makes my mouth water.

Mostly, I’m thinking that g-speak, a newly-invented gestural language that controls a user-interface (and replaces a mouse, trackball or a pen-tablet) probably doesn’t have to be all that brand-spanking new; that conducting a band or orchestra is itself such an ecstatic pleasure that people do it anyway with mock-control over recorded music — so SeeSpeak is my way of theorizing about the interface of new technology with traditional, old, human quirkiness as a step in a more interesting direction than teaching people g-speak (which won’t necessarily make us better acoustic bandleaders when the power cuts out, computers crash and all we’re left with is one another).  I kinda like stuff that’s of, by, for and about people — so long as they’re hypothetical.

03 Jun 10 - Posted by Scott Ellington | Uncategorized