Click here for this document in PDF format

Click here for a more detailed analysis from Year 1 of the pilot

Click here for a more detailed analysis from Year 2 of the pilot

The Performance of a New Computer-Based Reading Tutor

Abstract

We report here on the use of a computer-based program that replicates the knowledge and skills of a successful speech pathologist. This study covers the use for two years in the second grade of a Title 1 school. The results show it to significantly exceed the results seen in classes using standard curriculum as determined through standardized tests. Those students who completed more My Reading Coach lessons achieved the best results.

In summary, both the quantitative (numeric) data, and the qualitative (anecdotal) data reported by the teachers, indicate that My Reading Coach was effective for all students at this grade level regardless of their previous performance. Virtually all students demonstrated improvement, and the few who did not show significant quantitative gains were reported to show growth in self-esteem and confidence.

For the two-year group, students improved in raw score (8.3 points), percentile standing change (from 11.2% to 27.9%), and stanine growth (1.5 stanines). This is for a population where 46% of the students were classified as SEI or learning disabled.

For the second year group alone, students were able to complete a larger fraction of the lessons, the results are even better: improved raw score (11.9 points), percentile standing change (from 6.2% to 30.6%), and stanine growth (2.2 stanines).

 

Introduction

The techniques and skills of an experienced speech pathologist and teacher of reading have been incorporated into a software program called My Reading Coach (MRC). This program provides an intelligent engine to teach students to read one-on-one. This paper describes the program and reports on two years of experience with second grade students.

Program Basis

The program is based on extensive review of the literature regarding the way children learn to read and the experience of one of the authors (Larrabee), a speech pathologist and teacher of reading for over twenty-five years. His work uses what we judge to be the best of the published research.

During his career, Mr. Larrabee has expanded the use of phonics to provide 97% regularity to the English language. These are all included in My Reading Coach. The program provides thousands of short video clips of Mr. Larrabee, as well as animation, used in stop/start mode, to assess, teach, test, correct, and support the learning process.

My Reading Coach is based on cognitive learning theory. Cognitive learning theory posits that learners develop mental models of instructional content and then employ these mental models to solve new problems. These models are composed of several levels of knowledge: declarative, structural, and procedural. My Reading Coach uses phonics to develop the declarative skills and linguistics to provide the structural and procedural levels.

Program Features

Small elements are taught and immediately tested and corrected as needed. The student is thus protected from incorrectly learning and having to re-learn any idea. These elements are taught with short video clips. Tests are designed so that incorrect answers show the student's difficulty. The program then presents instructive corrections again through video clips to help with the difficulty. A given notion can be presented in many ways to help as needed. In addition, all or part of every clip may be replayed as often as needed to ensure understanding.

All responses by the program are positive: either a complement when work is performed correctly; or explanations to clarify the perceived difficulty. The program has over thirty forms of complements; the student is never "put down".

Headphones are used so that persons nearby are unaware of how the student is performing in order to protect the student's privacy. All instruction is provided using voice, video, and animation; text is not used for instruction.

The program is composed of 47 lessons and it takes between 40 and 60 contact hours to complete. In practice, as will be seen below, only a small percentage of students complete the entire program. Regardless, significant gains are made even by those who complete as few as twenty lessons, and, anecdotally, there is subjective gains made by even the poorest performing students.

Organization

In the following, we describe the location, environment, and the methodology. Then we report the results of two one-year studies and some initial longitudinal data.

 

Study Setup

Location

Myers Elementary is located in Tucson, Arizona in a low socio-economic neighborhood. Myers is a Title I school with a high non-English speaking population. Both the principal and teachers felt that their current second graders were functioning well below average for their grade level (verified by their test scores). In the first year, based on the results of a previously successful pilot at Myers, they elected to use My Reading Coach in all second-grade classrooms. In the second year, again all second grade classrooms were involved.

Environment

A new Windows® computer network was established to enable each student to receive 40 minutes with My Reading Coach daily. This lab was a subsection of an existing lab and physical partitions were added to limit a student's visual access to his or her neighbor. Each computer was equipped with headphones and was attached to a file server that hosted My Reading Coach - Professional Edition (Version 3).

 

Methodology

Teacher Training / Support and Feedback

In each year all second grade teachers agreed to participate in the program and each received a full day of My Reading Coach teacher instruction. There were five classes in the first year, four in the second. The second grade teachers determined the schedule for My Reading Coach use. Two teachers took part in both years using My Reading Coach with their second graders. The first year, one teacher, uncomfortable with computer usage, received an additional day of training. The second year, two teachers, who were new to Myers, received minimal training during an on-site in-service session.

Teachers were encouraged to print the management reports daily so that they could effectively monitor and assist any struggling student. Teachers were encouraged to set a goal that each student should complete every week: three lessons the first year, two lessons the second. Student Progress was reviewed bi-weekly by all second grade teachers.

In the first year, one of the five classes was entirely bilingual and another was entirely ESL. In the second year 25 students were labeled SEI, which included both bilingual and ESL students; these were not in separate classes.

Student Selection and Pilot Period

All second grade students including bilingual, ESL, SEI and LD students were selected to take part in the pilot program. Pre- and post-testing was performed using the DRP®, a standardized reading comprehension test; it was the school's test of preference. Student performance was analyzed for the students who were present for both tests. The first year 64 students were present at the beginning and end of the 12 week program; the second year 76 students were present at the fall and spring of the full year program.

 

Pilot Results

Time Spent on My Reading Coach

No students completed the entire program in the first year. The number of lessons completed by students in this year ranged from 15 to 42 and the average amount of time spent on My Reading Coach during the 12-week period at Myers was 27 hours. Pre-test scores for this population were significantly below a normal second grade level. 41 of the 64 students (65%) who were present for both the pre-test and post-test, had pre-test scores at the lowest level - stanine 1.

Five students completed the entire program during of the second year. The time on task for students to complete the entire 47 lessons ranged from 34 to 55 hours. The average amount of time all students spent on My Reading Coach during both semesters was 50 hours. Pre-test scores for this population were significantly below a normal second grade level. 64 of the 76 students (84%) present for both pre- and post- testing had pre-test scores at the lowest level - stanine 1.

Student Categories

First year: Bilingual Classroom
The bilingual class had 12 students who were present for both the pre-test and post-test. These students were Spanish speakers and were being taught in Spanish. Four students completed 30 or more lessons. Remember that My Reading Coach teaches reading in English.

First year: ESL Classroom
The ESL class had 17 students who were present for the pre-test and post-test. These students spoke English and were being taught in English although their native language was Spanish.

First year: LD
No LD students were identified in the first year's program.

Second year: Bilingual / SEI students
There were 25 SEI (Structured English Immersion) or bilingual students present for pre- and post-testing. On average, these students completed 34 of the 47 lessons. Two of these students completed all 47 lessons. Five of these students were also identified as LD. (See next section.)

Second year: LD students
There were 15 LD students present for pre-testing and post-testing. On average these students completed 21 of the 47 lessons. None of these students completed more than 30 lessons. Five of these students were also identified as SEI.

Both years: Regular education students
There were 76 students who were neither LD nor bilingual/SEI.

Performance Summary: Total Population by Student Category

This comparison was based on the primary DRP® test form for fall grade-two students using published national norms for the years 1999-2000 and 2000-2001. Comprehension level used was P = .75 - K0, K9 note. The data for the two years was combined for the total population note.

It is interesting to note the year-to-year differences. The next two charts display the years separately. Note that the Regular Education students were initially higher performing in the year ending 2001. However, they completed significantly fewer lessons in the program than did the 2002 students. (See section on Reading Comprehension Gains vs. My Reading Coach Lessons Completed below.)

For year 2002, a substantial majority, 84%, began the year reading at the first stanine - virtually non-readers. This chart summarizes the results for these students before and after one school year.

Progress Toward a Normal Distribution

This chart shows the improvement of the second grade students. It compares where they were before the pilot and again after the pilot compared with national norms. Post-test results (green line) show significant gains, particularly given the number of SEI and LD students and the socio-economic background of the neighborhood.

Detailed Performance Summary

2001 — 64 students including 29 SEI (ESL and bilingual) students were present for both pre- and post-testing. Teachers felt that the bilingual students may not have scored as well because of their limited English proficiency; however, the four bilingual students who completed more than 30 lessons had gains equal to their peers.

2002 — 76 students including 25 SEI (ESL and bilingual) students, and 15 LD students. Of these five students were both SEI and LD were present for both pre- and post-testing. On the average students gained 11.9 raw score units over both semesters (8 raw score units increase is equal to 1 year's growth). Students who completed 41 or more of the 47 lessons made the most gains (18.5 raw score units). 75 of the 76 students improved in raw score. The students who completed 31 or more My Reading Coach lessons achieved the best results.

COMBINED, the averages for 140 students in the second grade

• began with a raw score of 6.9 and completed with 15.2 for a change of 8.3;
• began with a percentile rank of 11.2 and completed with 27.9 for a change of 16.7; and
• began with a low 2nd stanine and completed with a mid 3rd stanine for a change of 1.5.

The expectation would have no change in the averages as the students were moving with their class. Thus the group substantially outperformed the norm on all measures.

Note that the difference in performance between the two years correlates with the average time with the program. During the year ending 2001, the average time with My Reading Coach was 27 hours; in year ending 2002, it was 50 hours.

Reading Comprehension Gains VS. My Reading Coach Lessons Completed

Viewed another way, the improvement in raw score points indicate the amount of growth compared to standardized norms. We show this as a function of the number of lessons completed.

Students who completed 41 or more lessons achieved the highest gains — an average of over 17 raw score units. Overall, students gained 8.6 raw score units, more than a full year of growth.

 

Conclusions

A computer-based program based on cognitive learning theory and replicating the knowledge and skills of a successful speech pathologist has been studied. It has been found to be effective in teaching second grade students to read. The results, as determined through standardized tests, show that it exceeds the results seen in classes using standard curriculum.

Both the quantitative (numeric) and the qualitative (anecdotal) data reported by the teachers indicate that My Reading Coach was effective for all students at this grade level regardless of their previous performance. All students but one demonstrated improvement, and the few who did not show significant quantitative gains were reported to show growth in self-esteem and confidence. Overall, students improved in raw score, percentile standing, and stanine growth. Those students who completed more My Reading Coach lessons achieved the best results.