On the Reliability and Validity of the CBM-Maze Task: Effects of Distractor Selection and Administration Time

The CBM-Maze task is a text passage in which every seventh word is deleted and replaced with the correct word and two incorrect words (distractors). Although research generally supports the technical adequacy of the CBM-Maze scores, the effects of different distractor selection methods for creating the CBM-Maze and of different administration times have seldom been examined. The present study examines the effects of different distractor selection methods and administration times on the technical adequacy of the CBM-Maze scores. Participants were 61 fifth-grade students. Three versions of the CBM-Maze were created, using a different distractor selection method for each version. In Version 1, both distractors were from the same syntactical category as the correct word. In Version 2, one distractor was from the same syntactical category as the correct word, and the other was from a different syntactical category. In Version 3, both distractors were from a different syntactical category...Show moreThe CBM-Maze task is a text passage in which every seventh word is deleted and replaced with the correct word and two incorrect words (distractors). Although research generally supports the technical adequacy of the CBM-Maze scores, the effects of different distractor selection methods for creating the CBM-Maze and of different administration times have seldom been examined. The present study examines the effects of different distractor selection methods and administration times on the technical adequacy of the CBM-Maze scores. Participants were 61 fifth-grade students. Three versions of the CBM-Maze were created, using a different distractor selection method for each version. In Version 1, both distractors were from the same syntactical category as the correct word. In Version 2, one distractor was from the same syntactical category as the correct word, and the other was from a different syntactical category. In Version 3, both distractors were from a different syntactical category than the correct word. Students were given 2 minutes to silently read each passage, selecting the words that restored meaning to the text. The number of correct choices was calculated for the 1- and 2-minute administration time. The alternate-form reliability, criterion-related validity, and discriminative validity were compared across versions and administration times. Version 3 had the highest alternate-form reliability at 1 minute, but differences between versions diminished at 2 minutes. Across administration times, alternate-form reliability and criterion-related validity were higher for 2 minutes than for 1 minute. However, differences were larger for Versions 1 and 2 than for Version 3. Finally, discriminative validity across versions and administration times was similar. The results indicate that there are generally few differences in reliability and validity between different versions of the CBM-Maze; however, Version 3 has a slight advantage over the other two versions. First, the 1-minute scores on Version 3 demonstrated good alternate-form reliability, which was not observed in the other versions. Second, Version 3 is the easiest version to create because distractors do not have to be from the same syntactic category as the correct word. Lastly, an exploratory small-scale eye-tracking study was conducted with four adults to gain insight into processes underlying the completion of the different versions of the CBMMaze. Results indicated differences in fixation times between versions for (a) the correct words, (b) the difference in fixation times between the correct words and distractors, and (c) combined fixation times across correct words and distractors. In conclusion, eye-tracking studies with the CBM-Maze are feasible and valuable for further exploration.Show less

Leiden University Student Repository

Documents

In Collections

On the Reliability and Validity of the CBM-Maze Task: Effects of Distractor Selection and Administration Time