Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

We just began a large new study on
Autism, ADHD, Anxiety & Depression in 12-17 year-olds.
Read more and learn how you can help!

TitleDevelopment of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.
Publication TypeJournal Article
Year of Publication2016
AuthorsMoore, TM, Reise, SP, Roalf, DR, Satterthwaite, TD, Davatzikos, C, Bilker, WB, Port, AM, Jackson, CT, Ruparel, K, Savitt, AP, Baron, RB, Gur, RE, Gur, RC
JournalPsychol Assess
Volume28
Issue12
Pagination1529-1542
Date Published2016 12
ISSN1939-134X
KeywordsAdolescent, Algorithms, Child, Computers, Efficiency, Female, Humans, Male, Neuroimaging, Neuropsychological Tests, Prospective Studies, Psychometrics, Reaction Time, Reproducibility of Results, Young Adult
Abstract

Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores with simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of 4 different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. (PsycINFO Database Record

DOI10.1037/pas0000284
Alternate JournalPsychol Assess
PubMed ID26866796
PubMed Central IDPMC4981571
Grant ListP50 MH096891 / MH / NIMH NIH HHS / United States
R01 MH107235 / MH / NIMH NIH HHS / United States
RC2 MH089983 / MH / NIMH NIH HHS / United States
T32 MH019112 / MH / NIMH NIH HHS / United States
Comments
Leave a Comment