IOWA STATE UNIVERSITY Department of Electrical and Computer Engineering Computer Control of Theater Performance Electronics SD May06-18

May06-18 Team Tarun Bhatia Tarun Bhatia Amanda Farniok Amanda Farniok Sheng Ly Sheng Ly Alex Sills Alex Sills

Project Client Iowa State Dance Iowa State Dance –Janice Baker Co-Motion Dance Company Co-Motion Dance Company –Valerie Williams

Project Advisory Board Faculty Advisors Faculty Advisors –Dr. Julie Dickerson –Dr. Gerald Sheble Emeritus Faculty Emeritus Faculty –Dr. David Stephenson

Purpose Provide clients with better control of their performance environment Provide clients with better control of their performance environment

Reason The environment can be manipulated as an interpretive, active portion of the artistic dance performance. With this design, up to four dancers can be identified on the stage by their transmitter and their input will manipulate the environment settings. The environment can be manipulated as an interpretive, active portion of the artistic dance performance. With this design, up to four dancers can be identified on the stage by their transmitter and their input will manipulate the environment settings.

Method Using Isadora Using Isadora –It takes input from “sensors” to “Actors”

Method Contd Use electronic sensors on dancer’s body to transmit information to a computer Use electronic sensors on dancer’s body to transmit information to a computer The computer will take the data and use the information to control settings of video, sound, and lighting predetermined by the performers The computer will take the data and use the information to control settings of video, sound, and lighting predetermined by the performers

Previous Research Project VrJuggler was previously used on a research project to manipulate input from sensors and display 3D video on projectors. VrJuggler was previously used on a research project to manipulate input from sensors and display 3D video on projectors. Certain aspects of previous research was taken into consideration. Certain aspects of previous research was taken into consideration. –The old hardware was reviewed to be used for the initial development of our hardware solution –Software setup to read sensors was managed through past documentation and research papers

Hardware Layout

Hardware Design - Research Analog audio wireless transmission method Analog audio wireless transmission method –Problems: Modulation/compression issues, capacity/channel issues, possible interference problems

Hardware Design - Research Industrial SCADA-type wireless transmission method Industrial SCADA-type wireless transmission method –Problems: Bulky packaging, insufficient channels/capacity, possible software (Isadora) compatibility issues

Hardware Design - XBow Crossbow “Mote” Transmission Method Crossbow “Mote” Transmission Method –Reasons for choice: Compact sizing, 2.4GHz band avoids interference and allows for multiple transmitters, TinyOS accompanying software suited for project goals –Possible problems: TinyOS/Isadora communication, line of sight issues

Hardware Design - XBow

Analog Output Circuit Crossbow motes accept analog inputs Crossbow motes accept analog inputs Flex sensor variable resistors used in a simple analog circuit to give Volt output to mote board Flex sensor variable resistors used in a simple analog circuit to give Volt output to mote board

Circuit Diagram - Four Sensor Output

MDA300 Sensor Board Accepts analog inputs via screw terminals Accepts analog inputs via screw terminals Provides excitation voltages, power supplies for analog circuit Provides excitation voltages, power supplies for analog circuit Converts analog to digital signal for wireless transmission Converts analog to digital signal for wireless transmission

MICAz Transceivers Connects to sensor board and serial interface with 51-pin connector Connects to sensor board and serial interface with 51-pin connector Transmits on 2.4 GHz Zigbee compliant band (good range, multiple channels, minimal interference problems) Transmits on 2.4 GHz Zigbee compliant band (good range, multiple channels, minimal interference problems) Powers sensor board, simple AA battery power supply Powers sensor board, simple AA battery power supply

MIB510 Serial Interface Receives signal from transceiver and transmits to computer through TinyOS software Receives signal from transceiver and transmits to computer through TinyOS software Powered by external AC adapter Powered by external AC adapter Through TinyOS, programs/configures transceivers and sensor board Through TinyOS, programs/configures transceivers and sensor board

Hardware Software Integration TinyOS is a platform bridges the hardware serial input and xmidi. TinyOS is a platform bridges the hardware serial input and xmidi. Xmidi - XML Musical Instrument Digital Interface Xmidi - XML Musical Instrument Digital Interface –XML Representation of MIDI data Isadora recognizes MIDI as an input. Isadora recognizes MIDI as an input.

Isadora SDK Provides overall design needed for working Isadora Plugin/Actor. Provides overall design needed for working Isadora Plugin/Actor. Provides data structures and functions that are needed to implement. Provides data structures and functions that are needed to implement.

Isadora Data Structure Implementation How Do We Do It? Important Data Structures Important Data Structures PluginInfo {// Struct that contains variables unique to this plugin ActorInfo* mActorInforRef.//ptr to our ActorInfor MessageReceiveRef mMessageReceiver//ptr to our msg reciever int sensorInput//input sensor to read from int lowerThreshold//input min limit int maxTheshold//input max limit float triggerThreshold//input threshold for on/off float scaledOutput//output scaled for flex float trigger//output for on/off }

Isadora SDK Function Unique Function Logic Unique Function Logic static void ReceiveMessage( IsadoraParameters*ip,//ptr to MessageMask // Message to listen to PortIndex// applicable to midi const MsgData*// data to read UInt32// length of data for the message long inRefCon//ptr to our Actor info )

Other Data Structures Isadora Actor Group Identification - static const OSTypekActorClass Unique 4 letter id for Isadora actor - static const OSTypekActorID Name for the actor used in Isadora - static const char*kActorName Defines the input and output for the actor in Isadora static const char* - static const char*sPropertyDefinitionString Contains information regarding the input and output and Actor info - const char* sHelpStrings[]

Isadora Functions - Minor Isadora Standard functions with minor data Isadora Standard functions with minor data Called when an input parameter value is changed, just reload with new data - static void HandledPropertyChangeValue() Function that gets the area needed to draw the actor in Isadora, bases on # of input and outputs - static Boolean GetActorDefinedArea()

Isadora Functions Standard Function called when actor is required to be drawn in Isadora - static void DrawActorDefinedArea() Gets Actors class and id and also all the ptrs to all the plugins functions - EXPORT_ void GetActorInfo() Getter function for the Parameter Sting - static const char* GetParameterString() Getter function the Help Strings - static void GetHelpString() Function that instantiates the actor - static void CreateActor() Function that destroys the actor when deleted, scene is closed, or actor is cut - static void DisposeActor() Sets the activate flag true or false depending if the actors scene is activated - static void ActivateActor()

General Flow of the Functions

Isadora Actor Mockup Our Actor Mockup Actor Link Connection

Putting the Pieces Together Integration between hardware and software will be completed using the research and documentation available. Integration between hardware and software will be completed using the research and documentation available. Testing will be done with trial components on lend from the Department of Electrical Engineering to assure products to be purchased will function correctly under all requirements. Testing will be done with trial components on lend from the Department of Electrical Engineering to assure products to be purchased will function correctly under all requirements.

Expected Product Deliverables An alpha version delivered February for use in March. An alpha version delivered February for use in March. Consist of a transmitter/receiver set for one dancer Consist of a transmitter/receiver set for one dancer By the first week of May, a finalized version of software and hardware will be released By the first week of May, a finalized version of software and hardware will be released

Coming Up Next Writing SDK for actors specified by our client Writing SDK for actors specified by our client Find testing software for borrowed hardware Find testing software for borrowed hardware Implementing hardware design for final product demonstration Implementing hardware design for final product demonstration Evaluate using test cases Evaluate using test cases

Any Questions?