1. Neuroscientific Modeling of Deception with HD-ERP and fMRI Data: Experimental and Computational Problems Jennifer M. C. Vendemia Michael Jay Schillaci Department of Psychology University of South Carolina

2. Outline Theoretical Framework –An Overview of Deception Research –A Cognitive-Physio Model of Deception Experimental Procedure –Studies Performed & Standard Analysis –Main Results Data –Collection & Problems –Analysis Software –Proprietary & Research –In-House Continuing Research –Current Efforts –Wish List

3. Theoretical Framework Detection of Deception is a Multi-disciplinary Problem

4. MemorySalience Emotion Arousal Decision Workload Motivation Attention Inhibition Person A asks question. Based on intensity and prosody and other stimulus relevant information this question has a certain level of salience Salience of the question is adjusted based on Person B’s motivation, personality characteristics, and learning history. Orienting related to salience results in changes in the peripheral nervous system Attention is directed towards the question, and the truthful information is pulled from memory. A motivation to deceive (gain, defense) can occur either before or after recall of the information. However, in most cases motivation occurs post recall. Changes in arousal affect attention, specifically working Memory. This arousal will either Facilitate or impair deception. At the point a decision is Made, the truthful response must Be inhibited and a deceptive response must be generated. The time is takes to generate This response is determined by how far this response will deviate from the truth and how complex this response is.

5. MemorySalience Emotion Arousal Decision Workload RespGSR HR BP fMRIERPPET Motivation Attention Inhibition CNS MeasuresPhysiological Measures A Cognitive-physio Model of Deception MEG

6. What is testable at the cognitive neuroscience level? 0 250 500 750ms P3a: Early attention component with an anterior distribution and positive deflection. Occurs when one switches tasks such as from telling the truth to telling a lie. N400: A component that occurs when what we’ve heard, said, or seen does not match the contents of our semantic (and possibly) episodic memory. Anterior distribution, negative deflection P3b: A late component that is related to decision making, workload, inhibition, and attention, and context updating. Certain aspects related to deception are visible in the HD-ERP signal –Attention: P3a –Memory: P3b –Deception Complexity: N4 and LPC –Salience: P3a

7. The Directed Lie Paradigm Participants respond truthfully to one color and lie to the other. Response Agree Response Agree Response Disagree Response Disagree 750ms Example of someone in “Blue True” condition.

8. Stimulus 1 (2500ms) Fixation Prompt (750 ms) Stimulus 2 (Response Termination)

9. Research Paradigms Semantic Deception Congruity Predictable Deception Predictable No Prediction Auto biographical Deception Predictable No Prediction Practice Effects Episodic Mock Crime: Multiple Memory Mock Crime: Memory Decay Combinations Semantic Auto biographical Semantic Workload fMRI Semantic HD-ERP fMRI

10. Semantic Paradigm Results: P3a When deception was predictable but congruity was not and when neither were predictable, the PCA component of the early frontally-distributed waveform was greater for deceptive responses than truthful responses F(1, 42) = 4.79, p =.034 and F(1, 27) = 4.44, p =.045.

11. Results: P3b When congruity was predictable but deception was not and when neither were predictable, the P3b component was significantly smaller in the deceptive condition than in the truthful condition, F(1, 42) = 5.37, p =.028 and F(1, 27) = 6.63, p =.028. Topographic distribution of Principal Component scores Exp. 2 Exp. 3 Congruent Incongruent Deceptive Truthful

12. Results: N4 When deception and congruity were predictable, and when only deception was predictable, the mean PCA scores for the N4 were significantly more negative in the incongruent condition than in the congruent condition [F(1, 33) = 22.59, p <.0001, and F(1,42) = 46.75, p <.0001], respectively. However, when neither were predictable the N4 waveform was not observed in the data. The PCA scores for deceptive responding were significantly greater than those for truthful responding when both congruity and deception were predictable F (1, 33) = 5.33, p =.027. The relationship between the conditions was similar when congruity was predictable; however, the effect only appeared as an interaction with the congruity effect. Deceptive Truthful Congruent Incongruent

13. H Lie P3a P3b N4 Left N4N2 HD-ERP Results for Participant 2506-H During Deception

14. Participant 2506 (H): Dipole for P3a corresponds to fMRI Activation in Anterior Cingulate Electrical Activity on Scalp At 304 ms Dipole in Anterior Cingulate fMRI Activation in Anterior Cingulate

15. P3b Participant 2506-H 93%

16. N2 Participant 2506-H 90%

17. N4 Participant 2506-H 86%

18. Standard Waveform Analysis –Averaging and Statistical Comparison –Amplitude and Latency Measures –Topographic Mapping Principal Components Analysis (PCA) –Averaging and Statistical Testing –Spatial and Temporal Components –Topographic Mapping Dipole Source Analysis –Averaging and Amplitude and Latency Measures –PCA, Independent Components Analysis (ICA) –Source Localization Individual Response Analysis –Data Replacement –Independent Components Analysis –Cluster Analysis fMRI Seeded Dipole Analysis –fMRI Parametric Mapping and Cluster Identification –HD-ERP Averaging and Amplitude and Latency Determination –Localization of fMRI Locations at Max Amp/Latency Along ERP Waveform Standard Analysis

19. Data Collection Basic Procedure –Subject Preparation –Data Amplification –Data Acquisition –Data Visualization

20. If All Goes Well!

21. Data Problems Hardware –Amplifier “Buzz”

22. Artifacts –Eye Blinks –QRS (Heart Beat) –Muscle Tension (Skull)

23. Subject Variation –Psychology –Physiology

24. HD-ERP / fMRI Analysis Preliminary Screening Netstation Filtering Segmenting Artifact Screening Data Replacement Averaging ERP Waveform Analysis SAS / SPSS Principal Components Statistical Comparison EMSE Topographic Mapping In House Independent Components Amplitude and Latency Two-State Grand Averaging ERP Dipole Modeling EMSE Source Localization Structural MRI reconciling fMRI Basic Analysis SPM Cluster Identification HD-ERP / fMRI Analysis EMSE MRVIEWER Plot fMRI Plot Dipoles onto Individual fMRI In House Talaraich To MRVIEWER Space MRVIEWER to Localization Space Data Analysis

25. Proprietary Software EMSE (Source Signal Imaging) –An integrated solution for brain electromagnetic source estimation FEATURES –Data Editor –Source Estimator –Source Visualizer –MR Viewer

26. Human Brain Project (LANL) –MEGAN MEG and EEG Analysis & Visualization –MRIVIEW An Interactive Tool for Brain Imaging –FEATURES 2D Mode 3D Mode Source Localization Model Viewer Research Software

27. In-House Software JAVA Based Solutions –Area and Latency Analysis of ERP Data –Phase Space Plots of ERP Data

28. VB Based Solutions –Talairach to MRViewer Coordinate Conversion

29. –Model Development and Enhancement

30. Semantic Information Working Memory Continuing Research

31. Current Efforts –Modeling Deception as a Two State System –Building a Continuous Potential Model –Workload, Salience and Deception

32. Wish List –A “Halo” System (Across Subject) –A Unified Analysis Environment (Across Platform)