Adaptive Assessments

From WikEd

Description, Definitions

Description

Computer based testing CBT is the ability to deliver an assessment via the computer. This can take place in the computer lab or in the classroom using laptop computers. It can also be administered via the internet right to your home computer.

Computer based testing is capable of providing immediate feedback to educators on assessments. Driven by NCLB and the need to show growth in student achievement educators have found that paper and pencil assessments are too slow in their ability to provide feedback. Several months can pass between the time the test was administered and the receipt of the results. The opportunity for intervention is lost. In addition most of the vendors providing computer based testing also offer analysis tools for understanding and translating the assessment results into the classroom.

One form of computer based testing is adaptive assessments. Adaptive assessments are given using a computer but differ from other computer based assessments because the software dynamically adapts the test questions to a difficulty level specific to the student’s knowledge or ability.

Students ranging from grades 2 through 9 typically take the adaptive test. The test will begin with a question that is appropriate to their grade level. If the student answers correctly the next question will be more difficult. The test will continue to give the student questions in a specific knowledge area until it has determined their skill level based on a standard deviation calculation. The computer draws from a bank of questions targeted at a specific skill or content area. As a result each student sees different questions while taking the same test.

The advances in the last decade both in hardware and software directly support the technology needs of computer based testing. Software advances impacting CBT include advancements in relational database systems and the ability to include videos, sound or images in the test. Hardware advances include increases in the processor speeds and storage capacity of the computer as well as more stable and reliable networks.

Definitions

One definition of Adaptive Assessments was found on the Computer Adaptive Assessment Website. This definition embodies the true capabilities of adaptive assessment. The ability to tailor the difficulty level of each question based on the correctness of the previously answered question.

The Computer Adaptive Assessment website defines adaptive assessment as:

"Computer Adaptive Assessment (CAA) is an innovative, online form of assessment in which items are presented in a sequence that is dependent on the correctness of the examinee’s response to the preceding item. Through this process, each examinee is administered a unique set of items that provides an accurate measure of his or her achievement level."

Application in Educational Settings

Two types of assessments that computer based testing can impact are summative and formative. The summative assessment is typically used for evaluating student progress and to assign grades. The formative assessment is primarily used to tailor the learning experience by providing immediate feedback to the student or teacher. When the formative assessment is delivered via the computer, the computer can provide response level feedback to each question thus changing the assessment into an active learning experience. This model could be adapted to the classroom with curriculum embedded features for discovery methods instruction.

NWEA's MAP test provides a summative assessment which is adaptive and computer delivered. This test is administered in the fall, as a benchmark and then again in the spring to determine growth. For older students a contract may be created between the teacher and the student agreeing on a growth target. This target is based on studies done by NWEA defining typical growth for a student at a specific grade level and achievement level.

The analysis tools available with both adaptive and non-adaptive CBT can be powerful in helping the teacher to identify different skill level groups for differentiated teaching. Many tools provide the ability to drill down into the test result data to determine misconceptions pertaining to the course material presented. This can provide direction to the teacher on what material needs to be reviewed and/or clarified.

An adaptive assessment is tailored to the learner’s proficiency and provides the learner with questions, which depend on his/her previous answers. The estimation of the learner’s performance isn’t based only on the percentage of correct responses, but also on the difficulty level of the questions the learner was able to answer correctlyone knowledge module.

Evidence of Effectiveness

Proponents of adaptive assessments cite the following as advantages:

A more accurate measurement of the proficiency of students who are above or below average
Individual test results available upon completion of assessment allowing immediate intervention
Analysis software provided by vendors to target differentiated teaching or student centered learning
More detailed content can be tested for a more accurate measurement of the student knowledge
Supports the development of Individualized Education Programs with specific data about the students skill levels

General advantages of computer based testing also applicable to adaptive assessments:

No printing or shipping of test booklets and answer sheets
Is more engaging than paper and pencil assessments

In addition to the above advantages to CBT continued technology advances could allow for the testing of more authentic real-world scenarios using simulation software that would measure problem solving skills, reasoning and other higher order skills.

Several states are using computer based testing and many have designed their tests to align with NCLB requirements. The participating states can be found in the Table of State Initiatives.

Critics and their Rationale

Critics cite the fact that adaptive testing inherently tests the child out of grade level. For example a 5th grader who is taking the test may be asked a 6th grade level question or a 4th grade level question as the test adapts to the students ability. NCLB requires the school district to show growth at the students grade level and as such the adaptive test model would not reflect the students true ability at grade level. The concern is this could lead to lower expectations for below average students.

Issues that affect computer based testing are also applicable to adaptive testing environments. Concerns are:

The cost of the technology to support the testing. Most districts do not have enough computers to do this type of testing in mass and do not have the additional funds to purchase the computers needed.

The integrity of the network and the computers come into play when CBT is used. Interruptions resulting from a computer crash or a network problem could be considered a problem when administering the assessment.

Training of the school administrators and faculty to facilitate a computer based assessment.

Loss of school resources during the testing period. In many instances school use there computer labs for the test thus rendering it unavailable for regular educational purposes.

Computer skills are also a concern when administering a computer based assessment. Students who are comfortable with the computer may do better on the assessment than those who have little or no previous experience with the computer.

Security of the data is of concern when the CBT is used as the state assessment.

Personal Testimonies

Computer based assessment will be common place in the future. As mentioned, the almost immediate feedback, the fact that scores will almost always be correct, due to the lack of human error, and the overall effectiveness of technology and today's students will make the testing process much more efficient, and manageable for all involved. During our school's PSAE and ACT review sessions this year we used a web based program to take practice tests and review materials. I was amazed at the way the students particpated in the activities. This program allowed the students to move form one level to the next after correctly completing all the questions. The kids were motivated by being able to see the instant assessment and progress they were making. Many commented that they would like to take the ACT in that same format. M. Hay

Our school also utilizes a PSAE/ACT prep program called KeyTrain. It is an online software that students can access from any computer with the Internet. I have seen both positive and negative affects, as some students are motivated by instant feedback as stated above, and some students are motivated to use the software when they don't have much other work to do. However, being that I am a special education teacher, there are always other strategies that the students should be focusing on as well. That is, the focus each day tends to be on homework completion, seeking assistance, working through homework step-by-step, etc.; all which are fairly time-consuming. Therefore, there isn't always an abundance of opportunities for these students to use the software to practice for these standardized tests. Our school is piloting the requirement for students to use KeyTrain during their resource period, which in turn takes away from the time they can receive guided study. While such preparation may be very advantageous to some students, it does not fit the demands of all students. J. Grady