ETS Publishes “How To” Manual on Setting Cutscores

ETS Publishes “How To” Manual on Setting Cutscores Contact: Tom Ewing +1 (609) 683-2899 mediacontacts@ets.org Princeton, N.J. (November 11, 2008) — At a time when educators nationwide are increasingly relying upon test scores for professional licensure and determining whether student skill levels are basic, proficient or advanced, Educational Testing Service (ETS) has announced the publication of a “how to” manual on setting cutscores. Cutscores: A Manual for Setting Standards of Performance on Educational and Occupational Tests is an update to a similar manual, Passing Scores, published in 1982 by ETS. Passing Scores was considered an essential guide for planning and running a cutscore study. For the new manual, the authors of the 1982 publication — Michael J. Zieky, a senior assessment director, and Samuel A. Livingston, a senior psychometrician, both at ETS — are joined by Marianne Perie, a senior associate at the National Center for the Improvement of Educational Assessment, Inc. “We decided to update the manual because much has changed in the setting and uses of cutscores in the quarter century since it was first published,” Zieky explains. “In the early 1980s, cutscores were used primarily on licensing tests for occupations and on high school exit tests that measured basic skills. Those tests consisted almost entirely of multiple-choice questions, and the cutscore was used only to distinguish between passing and failing scores. “In the last 25 years, new methods of setting cutscores have been refined and have become popular,” Zieky adds. “Some components of procedures for setting cutscores that were once considered optional have grown in importance and have become common practice — sharing normative information on test-taker performance and more rigorous documentation requirements are two examples. While opinions on setting cutscores differ among researchers and practitioners, we believe we are in sufficient agreement on many issues to share our opinions and recommendations on planning and conducting cutscore studies.” “In the K-12 education system, the emphasis has moved from setting a single cutscore that differentiates between pass and fail to setting multiple cutscores that differentiate among several levels of knowledge and skills,” Perie says. “Also, many current tests include essay questions as well as multiple-choice questions, and new methods have been designed to integrate both types of questions when used in the same test.” There are 10 narrative sections in the manual plus suggestions for further reading, a glossary and an appendix. The sections cover: * Introduction – background, purpose and overview of the manual * What You Should Know About Cutscores – definitions, misclassifications and appropriate uses of data * What You Have to Do Before You Set Cutscores – important considerations, defining performance levels, selecting methods and participants and complying with professional standards * Overviews of Methods of Setting Cutscores – 18 of the most widely used methods including advantages and disadvantages of each method * How To Choose a Method – help and advice on selecting the method that is most appropriate for the circumstances * Detailed Procedures for Setting Cutscores – step-by-step procedures for setting cutscores for each of the methods described earlier * What You Have to Do After You Set Cutscores – important things one should do after the study, including reviewing and documenting the study and evaluating cutscores * Tips – tips to help users avoid problems and deal with problems that can't be avoided * Consistency Across Grades – advice on the relatively new problem of obtaining reasonably consistent cutscores across grade levels * Conclusion – final thoughts to keep in mind when setting cutscores “The purpose of this book is to give practical advice to people who need to set cutscores,” says Livingston. “We don't assume that the readers have any knowledge of statistics or educational measurement. We let them know when they will need help from a statistician." Copies of Cutscores: A Manual for Setting Standards of Performance on Educational and Occupational Tests are available at: http://www.amazon.com/Cutscores-Standards-Performance-Educational-Occupational/dp/1438250304/ref=sr_1_1?ie=UTF8&s=books&qid=1221744859&sr=1-1