Measurement Systems and Applications Mariolino De Cecco Antonio Selmo Michele Confalonieri

Force Panel Dopo una breve introduzione … The instrument employed 2

[development of serious games with processing] How its electronics work? [Antonio Selmo] Design and purpose of the FP [a couple of frontal lessons] Measurements for Diagnostics [study and identification of the HTF] Dopo una breve introduzione … Rehabilitation and its quantification (measure of indexes of performance) [development of serious games with processing] Principles of human diagnosis and rehabilitation [doct Guandalini and Tomasi] Achronims: Fprce Panel – FP Human Transfer Function - HTF Logic of the Course 3

Organization and final examination Application: identification of human transfer function for dexterity assessment and serious games for rehabilitation Course: frontal lessons (few) exercitation with matlab building of the application with Processing visit to a clinic Method: study of touch technology study of signal analog elaboration and conversion acquisition of human data signal processing Exam: oral discussion homework Material of the course can be found at http://www.miro.ing.unitn.it Organization and final examination

Project acronym: VERITAS Project number: 247765 Project acronym: VERITAS Project full title: Virtual and Augmented Environments and Realistic User Interactions To achieve Embedded Accessibility DesignS Site: http://www.veritas-project.eu/ Starting date: 1 January 2010 Duration: 48 Months VERITAS is an Integrated Project (IP) within the 7th Framework Programme, Theme FP7-ICT-2009.7.2, Accessible and Assistive ICT VERITAS Project

Design and purpose of the FP - VERITAS Project Virtual and Augmented Environments and Realistic User Interactions To achieve Embedded Accessibility DesignS Method SP1: development of virtual users SP2: development of the simulation platform SP3: validation OBJECTIVE develop virtual humans for accessibility design in virtual simulation of ICT and non ICT products UNITN activities Modeling Measurement Design and purpose of the FP - VERITAS Project

Systems developed: Garment based Motion Capture: Ga-MoCap Multi-axis load cell Force Panel VERITAS Project - measurement systems developed 7

Diagnostics by human dexterity estimation Requirements: Measure position of the touch Measure the applied force Store the above for path and trajectories Accuracy and sampling frequency able to reveal human dexterity Visual feedback Idea: “Force Panel” Display LCD Touch panel Force transducers Punto di partenza del lavoro: volontà di valutare … In riferimento ad un’immagine, che può rappresentare dei pulsanti, dei percorsi o delle traiettorie, si vogliono misurare la posizione e la forza impressa dal dito, a frequenze e accuratezze tali da poter valutare le abilità umane. Da qui è nata l’idea di uno strumento dotato di … , denominato poi “Force Panel” Design and purpose of the FP - requirements 8

Stato dell’arte Touch screen LCD and cameras Graph tablets Tra i sistemi trovati in letteratura sono state analizzate alcune applicazioni che impiegano interfacce grafiche e possibilità di considerare la forza, tra i quali … Design and purpose of the FP - state of the art 9

Stato dell’arte joystick Aptics devices and virtual reality … nulla di ciò misura contemporaneamente forza e posizione in relazione all’accuratezza del tocco. Design and purpose of the FP - state of the art 10

Force Panel Design and purpose of the FP - Design Dopo una breve introduzione … Design and purpose of the FP - Design 11

Schema della struttura (disposizione dei vincoli) Horizontal use Vertical use Sensore Composito VS Sensore Monolitico = perpendicular to LCD = parallel to LCD 3 load cells Design and purpose of the FP - Design 12

Tipologie di vincolo considerate Spherical joints wires 2 solutions ! Design and purpose of the FP - Design 13

Design and purpose of the FP - Design

Variante con teste a snodo mechanical design: spherical joint

mechanical design: wires Dopo una breve introduzione … mechanical design: wires 16

mechanical design: wires constraints

mechanical design: wires constraints

F Requirements: No buckling No finger force through the blues Diameter 1 mm Wires with load cell Length 10 mm (due to physical constraints) wires Length 30 mm (FEM analysis → finger force < 0.3 %) F mechanical design: wires length

Lunghezza dei fili mechanical design: wires length horizontal Vertical Worst case for buckling Worst case for force leakage mechanical design: wires length 20

Errore lettura forza [%] Critical length ≈ 65 mm Force lost: Moments ≈ 10-3 – 10-6 Nm Caso Errore lettura forza [%] Worst case horizontal, l = 10 mm 2.41 Worst case vertical, l = 10 mm 2.50 Worst case for buckling, l = 10 mm 5.32 Worst case for buckling, l = 20 mm 0.73 Worst case for buckling, l = 30 mm 0.22 Worst case for buckling, l = 30 mm, e = 0.5 mm 0.23 Worst case for buckling, l = 30 mm, e = 1 mm 0.27 mechanical design: wires length 21

System design: overall layout Touch panel LCD VGA Force sensors PC Force Conditioning 2 x DIGITAL OUT USB µController 2 x ANALOG IN 3 x ANALOG IN System design: overall layout

Touch position measurement circuit A touch screen is a 2-dimensional sensing device constructed of 2 sheets of material separated slightly by spacers A common construction is a sheet of glass providing a stable bottom layer and a sheet of Polyethylene (PET) as a flexible top layer The 2 sheets are coated with a resistive substrate, usually a metal compound called Indium Tin Oxide (ITO). The ITO is thinly and uniformly sputtered onto both the glass and the PET layer Tiny spacers are placed between the 2 sheets in order to prevent false touch When the PET film is pressed the two resistive surfaces meet. This position can be read as illustrated in next slides Touch position measurement circuit

its electrical conductivity optical transparency Indium tin oxide (ITO, or tin-doped indium oxide) is a solid solution of indium(III) oxide (In2O3) and tin(IV) oxide (SnO2), typically 90% In2O3, 10% SnO2 by weight. It is transparent and colorless in thin layers while in bulk form it is yellowish to grey. In the infrared region of the spectrum it is a metal-like mirror. Indium tin oxide is one of the most widely used transparent conducting oxides because of its two chief properties: its electrical conductivity optical transparency Touch position measurement circuit

Touch position measurement circuit 5 V y+ L VS - sense GND R0 is the total resistance of the substrate RT is the touch resistance Touch position measurement circuit

Touch position measurement circuit Reading the x coordinate is similar Note that in the equivalent circuit there are capacitive effects that lead to a certain delay that usually is less than 10 ms Touch position measurement circuit

Electronics scheme

Pre-Amplificazione e Filtro analogico Alim. cella Output Alim. condiz. (± 15 V DC) Circuito di condizionamento Pre-Amplificazione e Filtro analogico passa basso Bessel V ordine DC / DC Alimentazione (12 V DC) Cella di carico μC Shift del segnale (μC legge solo tensioni positive) Amplificazione Conditioning unit

Conditioning unit

Full Wheatstone bridge Bessel lowpass 5° order Gain and offset adjustment: Arduino reads only positive voltages (0 – 5 V) Conditioning unit

Conditioning unit No sovraelongation 50s Tempo di assestamento 500 s RISPOSTA TEMPORALE AL GRADINO DEL FILTRO PASSA BASSO DEL 5° ORDINE (Risposta di Bessel con frequenza di taglio 1 kHz) Conditioning unit

Attesa assestamento circuito RC (costanti, variabili e pin μC) X Y F1 F2 F3 Alimentazione lungo X Attesa assestamento circuito RC Lettura valore X Alimentazione lungo Y Lettura valore Y Lettura valore F1 Lettura valore F2 Lettura valore F3 PC Inizializzazioni (costanti, variabili e pin μC) Arduino UNO sw design: embedded system

Ricezione dati ed applicazione modello di taratura t [ms] TASK 1 – Arduino TASK 2 – PC TASK 3 – PC PIN per lettura X Gestione delle immagini in relazione alle risposte di chi sta di fronte al pannello t assest. RC 5 Read X 5.1 PIN per lettura Y Read F1, F2, F3 nel t assest. RC 8 10.1 Read Y 10.2 Invio dati via RS-232 15.2 Ricezione dati ed applicazione modello di taratura 2 task paralleli su PC: - uno segue l’Arduino, - uno è indipendente 25.2 sw design: task analysis

Elaboration and graphical output Acquisition Elaboration and graphical output Inizializzazioni Lettura RS-232 Riconoscimento stringa Distinzione segnali Eventuali operazioni di filtraggio Operazioni di taratura Inizializzazioni Attesa inizio acquisizione Calcolo della tara del sistema Elaborazione (gestione delle immagini in relazione all’interazione con l’utente) sw design: two independent tasks on the PC

Touch position (resistive touch) Force (load cell) Touch position (load cell) Calibration

Force Panel - calibration

Linear behaviour on both directions No interactions Measurement model: Linear behaviour on both directions No interactions uncoupled modelling xm [pixel] ym [pixel] xm = f (xa) ym = f (ya) xa [bit] ya [bit] calibration: position

Non-linear behaviour for: Touch position Force FT [bit] Fm [N] Fm[N] = f (Fa , xm , ym) xm [pixel] ym [pixel] Non si leggono bene le scritte sul grafici calibration: force 38

WORK IN CLASS Force Panel - calibration: force For N measurements made in different positions and with different forces it is possible to build the following linear relation Than, with a pseudoinverse computation, the 10 coefficients of the polynomial model are estimated WORK IN CLASS Force Panel - calibration: force

WORK IN CLASS Force Panel - calibration: force The pseudoinverse computation gives the 10 coefficients of the polynomial model: NOTE: ATA is generally a square and full rank matrix WORK IN CLASS Force Panel - calibration: force

WORK IN CLASS Force Panel - calibration: force Files given: read_float.m How to use read_float : jj = 1 ; % numero file file = ['dati_celle_armon_', num2str(jj), '.txt'] ; dati = read_float(file, ',', ':') ; clc Dati = cell2mat(dati); % grammi -> Newton Dati(:,1) = Dati(:,1)*(9.81/1000) ; WORK IN CLASS Force Panel - calibration: force

WORK IN CLASS Force Panel - calibration: force % Data are saved on 9 files “dati_celle_armon_1…9.txt” As follows on columns: % 1 -> calibration load [g] % 2 -> x position [pixel] % 3 -> y position [pixel] % 4 -> load cell 1 (in alto) [N] % 5 -> load cell 2 (in basso a sx) [N] % 6 -> load cell 1 (in basso a dx) [N] % 7 -> load cell sum [N] WORK IN CLASS Force Panel - calibration: force

WORK IN CLASS Force Panel - calibration: force chi2gof Chi-square goodness-of-fit test on the vector e Performs a chi-square goodness-of-fit test for discrete or continuous distributions. The test is performed by grouping the data into bins, calculating the observed and expected counts for those bins, and computing the chi-square test statistic SUM((O-E).^2./E), where O is the observed counts and E is the expected counts. This test statistic has an approximate chi-square distribution when the counts are large. WORK IN CLASS Force Panel - calibration: force

Analyse the residuals and the validation behaviour Procedure: With the pseudoinverse computation estimate the 10 coefficients of the proposed polynomial model taking one data file at your choice Apply the estimated coefficients to the model to predict (validate) the model behaviour using a different data file Analyse the residuals and the validation behaviour Apply the chi2gof test to verify the randomness of the residuals Try to choose different mathematical models and see how the chi2gof test performs WORK IN CLASS Force Panel - calibration: force

Force Panel – calibration position F1a [bit] xm = f (F1a , F2a , F3a) ym = f (F1a , F2a , F3a) F2a [bit] F3a [bit] xm [pixel] ym [pixel] Dire che le formule dei piano appoggiato su 3 punti sballavano nell’intorno dei vincoli planari. 1 2 3 Force Panel – calibration position 45

Nail Finger finger horizontal Resolution Accuracy Position X [mm] < 0.40 ± 1.80 (al 95%) Position Y [mm] < 0.30 Force [N] < 0.05 ± 0.10 (al 95%) Time [ms] 10 ± 5 Nail Finger finger horizontal Yellow: resistive touch Blue: load cell reconstruction Force Panel – calibration position