A sequential process for comparison testing of noninvasive neuroassessment devices is presented. Comparison testing of devices in a clinical population should be preceded by computational research and reliability testing with healthy populations, as opposed to proceeding immediately to testing with clinical participants. A five-step process is outlined as follows: 1. Complete a preliminary literature review identifying candidate measures. 2. Conduct systematic simulation studies to determine the computational properties and data requirements of candidate measures. 3. Establish the test-retest reliability of each measure in a healthy comparison population and the clinical population of interest. 4. Investigate the clinical validity of reliable measures in appropriately defined clinical populations. 5. Complete device usability assessment (weight, simplicity of use, cost effectiveness, ruggedness) only for devices and measures that are promising after steps 1 through 4 are completed. Usability may be considered throughout the device evaluation process but such considerations are subordinate to the higher priorities addressed in steps 1 through 4.
Keywords: assessment; psychophysiology; test–retest reliability; validity qEEG.