Interaktiivisuus

1968 tapahtui edistysaskel ihmisen ja tietokoneen välisen vuorovaikutuksen (HCI – Human-computer interaction) tutkimuksessa, kun Doug Engelbart esitteli tutkimusryhmänsä työtä (Stanford Research Institute). Augmentation Research Center -tutkimusryhmän esitystä on myöhemmin kutsuttu nimellä "The Mother of All Demos". Esitys rakennettiin aikoinaan rajoja rikkovan videoteknologian varaan ja siinä esiteltiin tutkimusryhmän innovaatioita tietokoneen ja käyttäjän välisen vuorovaikutuksen keinoiksi. Engelbartin unelma oli vapauttaa tietokone numeronmurskaimen roolista ja luoda tietokoneesta kommunikaation ja tiedonhaun työkalu.

Demossa esiteltiin ensimmäistä kertaa mm. hiiri, jolla liikuttiin näytöllä, korostettiin tekstiä ja muokattiin ikkunoiden kokoa. Presentaatiossa esiteltiin ensimmäistä kertaa julkisesti integroitu systeemi ruudulla tapahtuvaan tekstinkäsittelyyn. Idea mahdollisti tekstitiedoston muokkaamisen samanaikaisesti eri paikoista sekä muokkajien välisen telekonferenssin videon ja äänen välityksellä. Demossa esiteltiin myös ajatus hypertekstistä, kun alleviivatun tekstin klikkaaminen avasi uuden dokumentin. Kaikki demossa esitellyt ideat ja toiminnallisuudet eivät olleet vielä olemassa tai silloisella tekniikalla mahdollisia. Internetin ja tietotekniikan kehityksen myötä monista Engelbartin tutkimusryhmän ideoista on tullut arkipäivää. Demon videotallenteet ja lisää informaatiota löytyy osoitteesta http://dougengelbart.org/events/1968-demo-highlights.html

Bill Englishin suunnittelema tietokonehiiren ensimmäinen prototyyppi, SRI International, CC BY-SA 3.0, Wikimedia Commons

Näppäimistö ja hiiri ovat hallinneet ihmisen ja tietokoneen välistä vuorovaikutusta pitkään, mutta 2000-luvulla kosketusnäytöt yleistyivät arkipäivään mobiiliteknologian myötä. Kuluvalla vuosikymmenellä ääniohjaus on tullut niin ikään arkipäiväiseksi koneoppimisteknologian ja mobiililaitteiden suorituskyvyn kehittyessä. Alla olevassa Applen mainosvideossa visioitiin vuonna 1987 ääniohjatusta tekoälyassistentista. Vielä ääniohjaus ja tekoäly eivät ole kehittyneet 80-luvun lopun visioiden tasolle, mutta moniin videon unelmiin vuorovaikutustekniikoista voi samastua vielä tänäkin päivänä.

2010 julkaistu Microsoft Kinect loi paljon odotuksia: laitteen syvyyskamera mahdollisti liikkeen tunnistamisen ja tähän perustuvan vuorovaikutuksen. Ääniohjauksen tapaan ajatuksena on luoda näkymättömiä käyttöliittymiä, jotka mukailevat fyysisen maailman vuorovaikutusta. Tutkimuksessa suuntaa kutsutaan nimellä "Natural user interface". Myo ja Leap Motion ovat muita esimerkkejä eleisiin perustuvasta vuorovaikutuksesta.

Vuorovaikutus digitaalisessa taiteessa

Vuorovaikutus ja katsojien osallistaminen on kiinnostanut taiteilijoita pitkään ja vuorovaikutteisia teoksia toteutetaan myös ei-digitaalisesti. Digitaalisessa taiteessa vuorovaikutuksella voidaan tuoda katsoja ja katsojan toiminta osaksi teosta, kerätä dataa ympäristöstä ja tuoda data ja datan edustama ilmiö tarkasteltavaksi tai vaikkapa tuoda katsojan toiminta ja valinnat tarkastelun kohteeksi. Alla on muutama esimerkkivideo liiketunnistukseen perustuvista vuorovaikutteisista taideteoksista.

Installaatioiden lisäksi on mielenkiintoista tarkastella ohjelmistoja ja web-sovelluksia taidemuotona. Esimerkiksi Mat Jarvisin Silk muistuttaa joitakin Casey Reasin töitä, mutta generatiivisuuden sijasta Silk on työkalu, jolla käyttäjät voivat luoda omia teoksiaan.

Tekstitykset

you I studied a course called interaction design at the Royal College of Art in London which is a really varied course there's people there from you know product design industrial design graphic designers psychology cybernetics all sorts of weird things so I was interested in computer vision at the time I was playing around and this actually grew from a brief which was to design a technological magic trick so that's what this is is a technological magic trick and that's the seed of the whole project I spent a lot of time doing graphic design so I was kind of you know stuck to print and stuck to the screen I didn't you know I was a web designer in front of the while and really wanted to break free of that and make something big that people could interact with and take the work off the screen you know began with you know two pencils and there would be eyes and hair on the pencils and we were just making making stories with it and they kind of grew into the monster that it is today why most is because most of the fun and you know it's it's it's engaging and I think you know people I mean the whole point of shadow play is to sort of transform one thing into another thing and for kids and for grown-up kids so the structure of the installation is there's a camera looking at a participant and you're standing in front of a light box so that's cost me a very nice clean silhouette so that's silhouettes going into a piece of software it's doing something called blob detection which is sort of isolating areas of contour objects and then it's analyzing the curvature of that of that object so it's going through all the points of that curvature and it's saying this bits really bendy this bits really sharp this bits this bits a combination of sharp and you know positive and negative curvature bits you know and then it's trying to make assumptions based on that you know maybe that's a mouth you know maybe maybe that's a head but yeah it's all it's all based on the analysis of that outer form of the shape like someone over there is doing it with a chair right now like it doesn't need to know that it's a chair it's just a shape and and it's gonna try and do its best to sort of create graphic elements that work well with that shape kids are a great participant you know they forget it they're just they're in there straight away I think I get the most pleasure watching in 50 year old businessmen in suits turn into five-year-olds so I think that's that's the place where I get the most enjoyment but but yeah definitely for kids kids are gonna love it you

Tekstitykset

you we call if I stand that side he is so I'm not cheating in the Sun yeah but but thank you I think it looks really stupid like this is these are this is like where I live or got outed by back you know empty gas that's where I live it's my crib the idiot checkers make so much money these days I heard the back of the day they did though oh yeah when I got into it it was everything was beginning to fall apart I think there's probably another way to do it and I've been trying to figure out what that might be I'm really interested in trying to figure out ways that technology as technology evolves it can be integrated into human experiences and can you use technology to produce real human emotion in people a dog is thing with you in there so it's a triptych where the three individual panels represent the process of creative conception this is something that I struggle with constantly so the first panel where your body disintegrates into the birds that represents that initial moment of conception it's the moment of inspiration it's the lightning in a bottle it's the purest moment of the idea that there is so this essentially represents birth now the second panel is representative of the critical response either you know by your own self-doubt outside critical forces or just the impossibilities presented through the process of production this is what it feels like to have your purest expression picked apart by a thousand angry beaks in other words this panel is death the third panel where you sprout the giant wings represents that feeling when somehow you and the idea are able to transcend that death and the panel before the idea sort of transforms through the process of abstraction into something that's larger ultimately than its original so this panel is Transfiguration I first got involved with the projects when Chris Chris came to me with it that's the treatment and this initial idea and the creates project were interested in in putting this together first show San Francisco and my role specifically was kind of working with Chris to help design that and design that experience figure out how it was that we wanted people to interact with it and all the best approach for bringing those birds to life was going to be and how to sort of refine the way they're interacting with your shadows there's a lot of fun Spike's and so this was the walkway can you adjust it again so that the RS patata prey on this project we chose to use the Microsoft SDK and the Kinect for Windows the reason we chose it is the system for skeleton tracking which is what infers your posture when you're standing in front of the Kinect is really really robust it's nice to have this opportunity to recontextualize this kind of interaction in an art installation just show people that this cool new way of interacting with moving images has implications beyond video games probably one of the biggest of technical barriers that we hit was the fact that we were trying to get very realistic-looking shadow for the piece and you know we're using the infrared sensor inside the Kinect to be able to pick up people's forms and get skeletons from that and track their movement we achieved something but it wasn't quite what we initially intended it had slightly more artistic or sort of Illustrated feel to it almost you know the shadows felt like you and responded like you but they didn't look as crisp as if you know you were shining a light behind your hand onto the panel so definitely in future versions of this that's one of the things that we're gonna look to try and prove what's your relationship to birds I feel like that's something that I see a lot at work I guess they bring me back to my childhood to a certain extent like I lived on Long Island and in a pretty wooded area and we were always having Birds fall out of nests I was always trying to mend these birds back to health in fact that I did a Modest Mouse video many ago which is essentially that story only Isaac Brock plays the bird get away from me never work always die I always like wake up in the morning and look in the shoebox team could be empty because my momma through the permit bird in the garbage a flock is an entity made up of smaller entities that are each working under their individual will but under a set set of rules so that there's an emergency cases programmers love to make blocks when we were out there in San Francisco we're looking at some real seagulls after looking at fake seagulls or big crows all week just seeing them move it looked unreal it looked fake they were like sort of real one the real one the real ones were returning in weird ways it's stopping it's slowing down and whiz it we didn't imagine it wasn't really until we got to San Francisco to actually ie install and build out the installation but then at the same time test and refine the code and get it to a point where we're ready to work that you know open exhibit so it was then that you really start to feel all the components come together for first time because you could see three panels sitting next to each other and three projectors running at the same time you know we turned it on and people start going through it and it it just worked which is sort of rare this tay's nothing just works for that long you create a website you put online you might read some some nice tweets or see comments online about it but there's nothing quite as rewarding is actually being present and seeing the smile people's faces and seeing them interact with it it's quite a rewarding experience I can imagine that must been a pretty amazing feeling for like a two-year-old on his dad shoulders like seeing him himself sprouting enormous wings I feel like this is where technology is headed it's becoming like more user friendly it's becoming like more interactive and becoming more like people based I guess instead of it being like a separate entity it makes you feel like you're not really at an art gallery but you're taking on the shape of something else and you kind of lose yourself for a second so we're at the beginning of this new art form this interactive media where we don't know what it will be in a hundred years in the same way that at the beginning of cinema they didn't look at it as this could be color and train shots and close-ups and dialogue and music and it could be the Godfather we're at the same stage with this interactive medium the most interesting part for me is that with this new to a canvas there can be an actual conversation between the work and that you are and my hope is that the art becomes the way in which you speak to the piece as much as it is in the way in which it speaks to you and we walked far

Tekstitykset

my work is I'm always trying to make people get inside my head you know in fact inside of the things I designed the things I create like a 3d video display that pickle gets in so what you sing here are like 6000 144 pixels it can be six million different colors the content is eight different things some reacts to people inside it and some are like random random animations some things are very simple and they just change when people get in but some you know follow the movement of the people that get in and it reacts and it follows its movement and it starts reacting to the visitor inside the installation when you are in front of a 2d monitor that's unusual experience like going to the movies and watching TV you end up you know imagining you that you are inside the moving stuff but with this stuff here you know it's literally you are literally in the action I'm very excited to be part of it it's really cool really cool and being together with so many bright people and exciting projects I don't you don't have the time to to talk to all people but I am being watching my neighbor's here and they're cool and I think is going to be really fun

Luvun sisältö ja tavoitteet

Luvussa pääset tutustumaan hiiren ja äänen käyttöön vuorovaikutuskeinoissa Processingissa. Luvussa kerrataan muuttujien käyttöä ja esitellään ehtolauseet. Ääniesimerkissä asennat ja käytät ulkopuolista Processing-kirjastoa. Luvun lopussa toteutat jälleen oman harjoitustyön, jossa hyödynnät hiiren, näppäimistön ja/tai äänen avulla toteutettua vuorovaikutusta.

Luvun tavoitteet:

Hiiren ja näppäimistön hyödyntäminen Processing-ohjelmissa
Muuttujien käyttö
Ehtolauseet
Ulkopuolisen kirjaston asennus ja käyttö
Äänen vastaanotto ja analysointi Processing-ohjelmissa