Feb-13-06 Trinetra Trinetra Independence for the Blind and Visually Impaired Phase 1: Conceptual Design.

Feb-13-06 Trinetra Trinetra Independence for the Blind and Visually Impaired Phase 1: Conceptual Design

Feb-13-06 Trinetra HCI Rapid Prototyping: Spring 06 Roughly 1 / 2 of the course’s students Devoting one full semester Phase 1: Concept Exploration Phase 2: Feasibility Reassessment Phase 3: Prototype Implementation

Feb-13-06 Trinetra Clients Priya Narasimhan –ECE/CS Faculty –Expertise: embedded systems, distributed systems, fault tolerance, middleware –Passionate about ensuring Trinetra leads to improvement in quality of life Dan Rossi –Oracle system administrator –Key driver and target user for Trinetra –Key contributor in ensuring that the technology stays focused on the real needs and desires of a blind user

Feb-13-06 Trinetra Trinetra – HCI Group Puneet Aggarwal Pablo Quinones Chris DeLeon Matt Lee Jim Lin Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Presentation Order 1. Overview 2. Scenarios 3. System Architecture 4. Subsystems 5. Design Process Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra 1.Overview Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Project Origins Fall 2005Trinetra originates, develops barcode identification solution. Client able to identify products in Entropy by scanning any barcode. Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Original Trinetra Developers Patrick Lanigan Aaron Paulos Andrew Williams Clients Dan Rossi Priya Narasimhan Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Project Development Spring 2006Trinetra redefined broadly as enabling greater independence for the blind, visually impaired. Striving to build upon, extend, and complete existing Trinetra technology. Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra 2.Scenarios Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Baseline Scenario “A Day in the Life” Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Baseline Scenario Morning Has cereal with milk, unable to check exp. date Uncertain if a new morning medicine’s side effects Decides bus is too unreliable, walks to work Grabs same old hamburger for lunch Walks to grocery store Requires assistance due to store reorganization Walks home Orders woodworking supplies online, paying for shipping and needing to wait for delivery Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Visionary Scenario “Living with Trinetra” Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Visionary Scenario 1 of 2 Product Information (Home) –Milk expiration date alert, medicine side effects check Object Identification –Bottles in fridge, cereal boxes in pantry Navigation (Macro) –Knows how long before next shuttle will arrive –Able to notify driver that he’ll be waiting Awareness of Location on Shuttle Ride –Client’s device privately calls out stops in earpiece Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Visionary Scenario 2 of 2 Sign Reading –Goes someplace new, can compare menu options to the posted special. Orders the special. –Identifies bus stop, and which buses are on this route Navigation (shopping) –Navigates grocery store, hardware store Product Information (Shopping) –Identifies prices, safety warnings Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra 3.System Architecture Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra User Artifacts Cell Phone (camera, text-to-speech) Earpiece Barcode Pencil RFID Sensor Bracelet Others to Consider: Cane Leash for seeing-eye dog Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Functional Capabilities Object Information –Product prices, serving size, expiration date, cooking instructions, medication side effects Object Identification –Cereal boxes on shelf, bottles in fridge Querying shuttle status –Knows how long before next will arrive –Able to notify driver that user will be waiting Awareness of location during shuttle ride –Client’s device privately calls out stops Awareness of menu options –In a restaurant, user can compare menu options to the ‘whiteboard special’ Identifying bus signs and information –Which buses stop at this stop? Shopping –Navigates grocery store, hardware store –Shelf “Navigation” Product Identification & Information Macro-navigation Reading Signs Retail-navigation Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Use Cases Product Identification - Home - Store (mapped as Retail Navigation) Product Information - Home - Store Reading Signs Catching/Riding the Shuttle Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Product Identification (Home) Goal Finding the carton of milk in fridge Interaction “Physical browsing” - Reach into fridge and touch - Audio from system names the product Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Product Information (Store) Goal Find price, servings, calories, additional ingredients Interaction - user touches or holds product in hand - user presses button corresponding to a value - user hears the price, number of servings… Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Product Information (Home) Goal Find preparation/use instructions, expiration date Interaction - Points camera at package label - Voice reads out the label via earpiece - In future, information is available on the phone Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Retail Navigation Goal Get from front door to product Interaction (many alternatives) Alt 1. Instructions to locate raisin bran Alt 2. Walks by aisles to browse contents, then high-level categories of products in-aisle Alt 3. User specifies “target product” and receives an audio indication when nearby Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Reading Signs Goal User aware of sign information Interaction - If sign placement known, user points camera and voice reads text into earpiece - Else, camera used to pan environment - User receives an audio cue if sign found - User presses a button on phone to hear text Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Catching/Riding the Shuttle Goal Determine shuttle arrival Interaction - Launches Shuttle Helper app on phone - Prompts, “Which shuttle? Which shuttle stop?” - Audio indication of ETA until next shuttle and shuttle interval - App provides shuttle location during ride Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra 4.Subsystems Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Text/Audio Message Voice Recognition (VoiceSignal) Text-to-Speech Engine (TALKS) Text Recognition (OCR), built-in camera Bluetooth Ethernet GPRS/EDGE USB Transportation Text Recognition & Labeling Shopping Capacity, Location Notification ETA, Location, Notification (Transportation) HTTP Server, Global Database (Shopping) Application Server (Transportation) Bus Shuttle Inventory (Shopping) Notification (Transportation) Barcode Reader (Baracoda Pencil) RFID Reader (Baracoda IDBlue) Trinetra Proxy (CerfCube 255) Bluetooth/IP Gateway (Parani100) RFID Sensor Bracelet Local Database Nokia Cell Phone Earpiece Cell Phone Functions Product Name, Expiration Date, etc Inventory Audio Message Personal Desktop Product Info from OCR or barcode Sync with PC IPAQ (in bus) Capacity IR Sensor Localization GPS 802.11 Barcode Printer Preprinted Barcode Labels Barcode Reader (Baracoda Pencil)

Feb-13-06 Trinetra Trinetra - Shopping Paul Cheong Prateek Goenka Jun Han Rohan Kumar Patrick Lanigan Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Shopping – Hardware Scanner –Baracoda Pencil –Baracoda IDBlue –Intel RFID Bracelet Cell Phone –Nokia 6620 Bluetooth Gateway –Parani100 Proxy –CerfCube 255 Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Barcode/RFID Reader Matrix ModelBaracodaPencilBaracoda IDBlueIntel Research RFID reader & glove design CommunicationBluetooth Class 2 RF Transceiver Communication Range 50 ft Unknown Scan methodBarcode readerRFID reader/encoderRFID reader Scan rangeContact scanning0 – 4 cm0 – 5.8 cm Weight1.48 oz (42 g)1.76 oz (50 g)Unknown Cost$232 - $316$468 - $512Unknown AvaliabilityNow Soon, but manual available Battery Life2000 scansUnknown Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Mobile Device Feature Matrix Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Bluetooth Gateway Parani 100 Multi-port Bluetooth to IP Gateway Connects Bluetooth devices to LAN LAN support: 10/100 BaseTX Ethernet, RJ45 Bluetooth: Interface using Bluetooth v1.1 with the range of 10 ~ 400 m Software: Windows Utility Price: $ 389.00 Availability: Now Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Proxy Intrinsyc CerfCube 255 Intel PXA255 32-bit, 400 MHz CPU 32 MB flash storage 64 MB SDRAM Linux based Ethernet, USB 1.1 connectivity CompactFlash (type I & II) expansion slot Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Shopping – Software Architecture Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra SDK Feature Matrix Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Software Architecture – I/O Receives same product ID irrelevant of scanner ID is sent to request broker Output handler forms the info before sending to UI TALKS Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Software Architecture – Caching Minimizes search time for frequently purchased items Allows for storing shopping list Can search before-hand for items on sale Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Software Architecture - Proxy Provides access to in-store database Database Adapter –hides DB specific details –query language –table format –login information Request Handler –performs queries –passes results back to mobile client –can also restrict access to sensitive data Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Three Tier System (Product Information Retrieval) Request Broker 1.Local Cache2. Local Network3. Global Network Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Trinetra – Shopping (end) Paul Cheong Prateek Goenka Jun Han Rohan Kumar Patrick Lanigan Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Trinetra – Text Recognition Jeff Saretsky Kunal Kapur Tudor Dumitras Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Overview Goals: identifying products and objects –Labeling using barcodes –Reading text labels & signs Subsystems –Cell Phone: Nokia 6620 or 6682 –Barcode printer: Zebra LP 2824 –Custom printed barcode labels –Windows PC –OCR software CamReader ( http://www.camreader.jp/eng/product/ocr.html ) http://www.camreader.jp/eng/product/ocr.html Asprise OCR SDK ( http://www.shareit.com/product.html?productid=300041929 ) http://www.shareit.com/product.html?productid=300041929 GOCR ( http://jocr.sourceforge.net/ ) Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra GPRS/EDGE USB Labeling Using Barcodes Bluetooth Ethernet Text Recognition & Labeling Trinetra Proxy (CerfCube 255) Bluetooth/IP Gateway (Parani100) Local Database Nokia Cell Phone Earpiece Inventory Audio Message Personal Desktop Product Info from OCR or barcode Sync with PC Barcode Printer Preprinted Barcode Labels Barcode Reader (Baracoda Pencil)

Feb-13-06 Trinetra GPRS/EDGE USB Text Recognition Bluetooth Ethernet Nokia Cell Phone OCR Software Earpiece Product Name Expiration Date Etc.

Feb-13-06 Trinetra OCR Feature Matrix OCR SWCamreaderAsprise OCR SDK ScanStore Simple OCR Capture Text GOCR TypeSoftware Package SDKFull Source Code Closed Source Open Source Other Features Barcode, Motion detect. Barcode Recognition Multiple Language N/AHandwriting (limited) PlatformSymbian OSSymbian,Win, UNIX,Palm,etc WindowsWindows XPUNIX/OS2/ Windows File FormatsRGB, YUV, etc. bmp,jpg,pcx,p bm,pnb,tiff,etc tiffbmp, jpg, pdf, word png, jpg, etc. Language??C++ ??C++ Memory?? 1840 kB LicenseCommercialSingle Developer Unlimited Distribution Single UserGPL Cost??$1998$5000$29.95Free

Feb-13-06 Trinetra Trinetra – Text Recognition (end) Jeff Saretsky Kunal Kapur Tudor Dumitras Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Trinetra – Transportation Aaron M Paulos Joe Tadduni Jovan Williams Andrew Williams Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Broad Solution Space Technical evaluation and existing solutions –Campus Escort Service identifying end- users –Subject determining location of Campus Shuttle –Localization of transportation end-users –Safety during navigation and in-home Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Escort and Human Interaction How does a bus driver and a visually impaired person communicate in order to determine presence, locale, and exchange other useful information? 11 Mbps ~1 Mbps Through put Disadvantages/Advantages Integration Network TypeRangeAvailability Technology Very Expensive Hardware / Low Power HardAd-hoc~10m COTS UltraWide Band Power Intensive / Inexpensive & Widely Adopted Unknown Ad-hoc & Infrastructure ~50mCOTS 802.11 Extra gear to carry / InexpensiveHardAd-hoc~70mCOTSZigbee Must trigger sensor / InexpensiveNonePhysical sensor Fixed Location COTS Physical Sensors Must have cell reception & Monthly cost / Cell Phone platform already acquired NoneCell Network Coverage Network COTS GPS/Cell Phone Power Intensive / Inexpensive & cell phone feature EasyAd-hoc Network~10mCOTS Bluetooth 115 kbps ~ 30 kbps

Feb-13-06 Trinetra Localization – Transportation Access Identification of an access point to a service? –Custom fixed access points –Hinge off of existing infrastructure (wireless) Existing Localization Solutions –Intel Seattle labs (Place Lab) –802.11 localization (Jeremy Schaffer, CMU) –Context sensitive tour guides (more 802.11) –Active Badge (IR) –GPS Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Localization – Transportation Access Access to field-test- mode. Nokia has support. Cell phones100s of meters urban. 2-3 km rural Phone plus carrier costs. ~50 monthly. Cingluar or Tmobile GSM – Cell Id Minimal. Custom app development Minimal. Software and Source exist for PC. IntegrationInterface optionsRange OutdoorsCost (End-user)Technology Laptop, handhelds, Cell phone ~10 meters50-100$ depending on solution Fixed Bluetooth Access Points Laptop, handhelds, Add- ons for some cell phones. Accuracy to 10s of meters 200+GPS Laptop, handhelds, custom device, few cell phones 100 meters and up 50$802.11 Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Solution Space Revisited End-user to campus shuttle –Location, capacity, ETA –Ability to receive notifications Campus escort service to end-user –Location and capacity –Ability to schedule pick-up –Ability to identify end-user If we have time… –Access point identification (campus escort server stop) –Handshake between campus escort service access point –Safety Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Hardware Architecture Cell Modem GPS Bluetooth 802.11 USB iPAQ IR Sensor (Capacity) Mobile Device (Passenger) Concurrent TCP Server Localization Bus ID Position Capacity (GPRS/EDGE) Notification of blind passenger (Radio) Internet User registrations, ETA queries ETAs, capacities LAN PC Mobile Device GPRS/ EDGE Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Software Architecture Concurrent TCP Server Shuttle notifications Shuttle/Escort input handler Fleet position tracking ETA calculator Static shuttle schedules Repository for scheduling End-user input handler Access escort scheduling Escort notifications Admin GPRS / EDGE End-User 12 3 3 4 5 1. Unidirectional (bus information) 2. Bidirectional Application protocol 3. SMS, Call, or Application protocol 4. SMS, Dispatch, or Application protocol 5. Configuration Driver dispatch / 802.11 / Cell Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Assumptions / Questions End-user (handheld) –Assume user can do localization/navigation to and from shuttle access points USB limitations – 15’ cable Power supply for bus computing device Capacity mechanism –Custom hardware VS off the shelf mechanism Software questions –Back channel for driver notification subject to meeting with shuttle owners –GPRS cell phone protocol should most likely be unidirectional for scalability Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Trinetra – Transportation (end) Aaron M Paulos Joe Tadduni Jovan Williams Andrew Williams Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra 5.Design Process Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Safety and reliability affect every decision Safety Considerations Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Shopping – Security / Privacy Bluetooth connection –3 security modes (service level & link level) –Trusted devices –Key management Local Database Access –Proxy can shield any sensitive information from a Trinetra user Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Product Identification – Danger Should inform the user if expired food(s) are about to be consumed Critical that its detection method has high accuracy when distinguishing dangerous chemical and safe consumable –Fortunately, bottle shape/size if typically a good indicator for general objects Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra 1.Aim for effortless independence 2.Leverage existing technologies 3.Complement basic tools, don’t complicate 4.Reliability for real-life usability 5.Unobtrusive, low-profile use 6.Make system configurable These are our final guides leading into Phase 2: Feasibility Reassessment Design Guidelines Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Questions Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Functional Capabilities Object Information –Product prices, serving size, expiration date, cooking instructions, medication side effects Object Identification –Cereal boxes on shelf, bottles in fridge Querying shuttle status –Knows how long before next will arrive –Able to notify driver that user will be waiting Awareness of location during shuttle ride –Client’s device privately calls out stops Awareness of menu options –In a restaurant, user can compare menu options to the ‘whiteboard special’ Identifying bus signs and information –Which buses stop at this stop? Shopping –Navigates grocery store, hardware store –Shelf “Navigation” Product Identification & Information Macro-navigation Reading Signs Retail-navigation Overview | Scenarios | System Architecture | Subsystems | Design Process

Feb-13-06 Trinetra Trinetra Independence for the Blind and Visually Impaired Phase 1: Conceptual Design.

Similar presentations

Presentation on theme: "Feb-13-06 Trinetra Trinetra Independence for the Blind and Visually Impaired Phase 1: Conceptual Design."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Feb-13-06 Trinetra Trinetra Independence for the Blind and Visually Impaired Phase 1: Conceptual Design.

Similar presentations

Presentation on theme: "Feb-13-06 Trinetra Trinetra Independence for the Blind and Visually Impaired Phase 1: Conceptual Design."— Presentation transcript:

Similar presentations

About project

Feedback