1. Discussion & Conclusions Indexes of Internal Consistency
Reliability of Word List Task for a Computerized Neuropsychological Battery: The EMBRACED Project Brooke E. Leonard1; Julia C. Daugherty2; Miguel Perez-Garcia, PhD2; Antonio E. Puente, PhD1, Inmaculada Ibanez-Casas, PhD1,2 (1University of North Carolina Wilmington, 2University of Granada, Spain)
Introduction
Method
Discussion & Conclusions
Word lists are the Golden Standard for verbal memory and learning assessment.
There are several factors that may affect the creation of new word lists; however, no published materials have been found to explain the proper procedure for creating one.
The EMBRACED project, a comprehensive computerized neuropsychological battery of tests, needed a word list task for the best measure of verbal memory.
After methodically developing two word lists (one for learning, one for interference) both in English and Spanish, the reliability of this newly developed task had to be measured.
The exhaustive and systematic manner in which the EMBRACED word list task was developed, proves it to be replicable for future studies.
The quality of this word list task has been supported thus far by the results of our study. We have found statistically significant results for the correlations found based on Benedet & Alejandre’s three indexes of internal consistency.
The results have also supported the EMBRACED word list task as a quality measure of cultural differences in memory assessment.
A systematic literature review determined that successful word list criteria include length, frequency, pan-pronunciation (pronunciation without influence of dialect), semantic categorization, and commonality across cultures.
The Spanish Corpus (LEXESP) and English Corpus (SUBTLEXus) were used to determine word frequencies for statistical comparisons. It was found that there were no significant differences in frequency per million words between the lists and/or between the languages.
Based on the literature the lists of words were integrated into the EMBRACED word list tasks as such:
2 separate word lists (learning, interference)
16 words per list covering 4 semantic categories
5 learning trials, 1 interference trial
Short-term and long-term recall trials (20min separates the long-term)
1 recognition trial
The three indexes that Benedet & Alejandre describe for the best way to measure internal consistency are:
1. Comparing odd vs. even trials. Trials 1+3 with trials 2+4 and 2+4 against 3+5 using the Spearman-Brown formula.
2. Comparing the categories that make up learning list.
3. Comparing all five trials, looking at each of the 16 words in the learning list.
Objective
Future Directions
The aim of this study is to test the quality of the verbal memory task included in the EMBRACED battery, both in English and Spanish.
Finishing data collection will further prove the reliability of this word list task as the best measure of memory for the EMBRACED project.
Although this collection sample was taken from the U.S., the EMBRACED team is currently working on data collection in Spain and is expanding to several other European countries in the near future.
We expect to find that this word list task continues to be a valid and reliable tool for cross-cultural memory assessment.
Descriptives
The target sample for the EMBRACED project is 480 participants, 240 Hispanic and 240 non-Hispanic.
For this study, a total of 101 participants had been tested so far, 85 non-Hispanic and 16 Hispanics, as described in Table 1.
The average age was for both sample groups.
All participants from this sample had at least a high school education. None of the Hispanics in this sample had past an Associates Degree.
Results
Table 2. Means and deviations for the different variables of the Word List for the EMBRACED battery.
Variables
Non-Hispanics
n=85
Hispanics
n=16
Total
n=101
Trial 1
4.25
(3.36)
2.19
(2.61)
3.95
(3.34)
Trial 2
7.55
(4.26)
3.88
(3.87)
6.85
(4.60)
Trial 3
8.94
(4.87)
5.38
(5.25)
8.03
(5.47)
Trial 4
9.88
(4.98)
5.44
(5.34)
8.92
(5.64)
Trial 5
10.14
(5.07)
6.00
(5.91)
9.24
(5.69)
Total 5 Trials
40.76
(20.20)
22.88
(22.16)
36.99
(22.82)
Interference
4.94
(3.13)
2.13
(2.75)
4.49
(3.42)
Short-Term
8.07
(5.50)
4.75
(5.49)
7.12
Long-Term
6.74
(6.00)
(4.99)
7.09
(5.67)
Recognition
14.20
(1.87)
14.82
(.60)
14.29
(1.76)
Table 3. Correlations for each of the three indexes of internal consistency for EMBRACED word list, CVLT, and TAVEC
 Indexes of Internal Consistency
Variables
Measured
EMBRACED
CVLT
TAVEC
Index 1
Trials 1+3 vs. 2+4
Trials 2+4 vs. 3+5
.958
.92
.94
Index 2
Cat 1+3 vs Cat 2+4
.955
.77
.82
Index 3
Total Scoring for all 5 trials (List A)
.956
.70
.81
References
Table 1. Sample sociodemographic descriptives
Non-Hispanics
n= 85
Hispanics
n= 16
Stat
Sig.
Age
(mean, SD)
24.08 (10.95)
25.75 (10.40)
T= -.56
p=.57
Gender (n, %)
Female
37 (43.50%)
11 (68.80%)
χ²= 3.43
P=.06
Male
48 (56.50%)
5 (31.30%)
Education
High School Diploma
64 (75.3%)
11 (68.8%)
χ²=1.61
P= .65
Associates Degree
17 (20%)
5 (31.3%)
Bachelor´s Degree
1 (1.2%)
0 (0%)
Doctorate
3 (3.5%)
Language
English
85 (100%)
8 (50%) -
Spanish
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, Joint Committee (2014). The standards for educational and psychological testing. Washington, DC: AERA.
Benedet MJ, Alejandre MA. Test de aprendizaje verbal España-Complutense. Madrid: TEA Ediciones; 1998
Delis, D., Kramer, J., Kaplan, E., & Ober, B. (1987). California Verbal Learning Test. San Antonio, TX: The Psychological Corporation.
With the exception of Long-Term Recall and Recognition, all of the correlations presented at .73 or higher. There are further differences to be found with the long-term and recognition trials, especially with further data collection.
Table 3 compares the EMBRACED word list with the California Verbal Learning Test (CVLT) and the Test de aprendizaje verbal España-Complutense (TAVEC). Correlations from the EMBRACED Word List are stronger than the correlations found for both the CVLT and TAVEC.
*Chronbach’s α: .85