CEP 502 - Use and Interpretation of Classroom Tests

Spring semester 1998	Dr. Necia A. Black
Wed. 4:10-6:50 PM
Ph 645-3572, 231 Computing Center
Clemens 203	Office Hours - by appointment

TEXTBOOK: Measurement & Evaluation in Education and Psychology

(4th edition, 1991) by Mehrens and Lehmann

PROBABLE SEQUENCE of TOPICS --- Please be Prepared --- ATTENDANCE REQUIRED

Wed.	Jan. 21	General comments and concerns	Chapter
		Introduction	1
Wed.	Jan. 28	Norm- and Criterion- Referenced Testing	2
		Role of Objectives in Educational Evaluation	3
		Format for debates - Beth Lesen
Wed.	Feb. 4	Guest Lecturer - Michael LeFever
		Accountability - Testing & Evaluation	21
		Public Concerns - Future Trends in Evaluation	22
		Sign up for Presentation or Debate
Wed.	Feb. 11	Classroom Testing: The Planning Stage	4
		The Essay Test: Preparing and Grading	5
Wed.	18	Writing- Short-Answer, Matching, T/F	6
		Writing- Multiple Choice, Context Dependent	7
Wed.	25	EXAM #1 (Ch. 1-7) - PLEASE REMEMBER PENCILS
		Guest lecturer - Beth Lesen
Wed.	Mar. 4	Assembling, Reproducing and Analyzing Tests	8
		Other Teacher-Made Evaluation Procedures	9
		PRESENTATION - Alternative Assessment
Wed.	11	Spring Break - NO CLASS
Wed.	18	Describing Education Data	10
		Norms, Scores and Profiles	11
		DEBATE - New Standards
Wed.	25	Reliability	12
		Validity	13
Wed.	Apr. 1	EXAM #2 (Chap. 8-13) PLEASE REMEMBER PENCILS
		Web stuff
Wed.	8	No class
Wed.	15	Introduction to Standardized Tests	14
		Standardized Aptitude Measures	15
		DEBATE - Standardized Tests
Wed.	22	Standardized Achievement Tests	16
		Factors Influencing Measurements of Individuals	19

		DEBATE - Cultural Bias
Wed.	29	EXAM #3 (Ch. 11-16,19) PLEASE REMEMBER PENCILS

Grading Procedures

REQUIREMENTS: To receive a final grade of "Satisfactory", a student must exhibit satisfactory performance as measured by:

45% - 3 exams: M/C, T/F, covering text + lectures

10% - Small Group In-class Activities or Quizzes

20% - Class Presentation or Debate:

30% Semester Project:

Attendance - Must attend at least 80% of class sessions (12/14)

In lieu of receiving a grade of Unsatisfactory, an optional comprehensive exam may be taken, on Wed. May 6th, covering all chapters. This exam grade can also replace one of the regular exams, so if you are not satisfied with one of your exam grade, it is suggested that you take the optional final. Students who miss an exam due to illness or family emergency, will be asked to make up that missed exam by taking the relevant portion of the comprehensive exam. If more than one exam is missed, the entire comprehensive must be taken. The comprehensive will only be given at the time indicated.

Debates

Each debate involves 2 sides, 3 people on each side. you are strongly advised to take a position you do NOT agree with in real life. It is also suggested that your team be familiar with both sides of the issue as you will want to anticipate the other teams argument. There will be no formal winners/losers. Each debate will lay the foundation for an in-class activity.

Cultural Bias in Testing - There are at least two schools of thought regarding this issue. One is that standardized tests are culturally biased and therefore institutionalize racism along with some other social ills. Another is the controversial argument made popular in the book, The Bell Curve, which, oversimplified, basically asserts that tests cannot be biased. Therefore, documented differences between races/ethnicities is evidence of differing abilities. This will be an issue debated in class.

Standardized Testing - there are two basic extremes. One is that standardized testing is critical because it is norm-referenced and allows for ability and disability to be understood in the context of a child's peers. The other extreme is that standardized testing is useless because it is a snapshot of performance vulnerable to too many confounds. This is another issue that will be debated.

Tests as Standards -- this has been a big issue, especially in Buffalo, since the students have been raised for graduation. With the elimination of local diplomas, students can either get a Regents Diploma or an IEP (Individualized Education Plan) diploma (which is riddled with issues). Some believe that this change will accelerate learning (a "reach for the moon" mentality). Other believe that the change will doom some students to failure who otherwise would have earned local diplomas or the change will necessitate a scaling-down of the regents exams. This will be the third issue debated.

Presentations

Although the above mentioned debates are to be done amicably, in a non-competitive atmosphere, some people are averse to the confrontational nature of debates. With this in mind, there are two presentation scheduled. Each presentation will involve 3-4 people and is standard in format. One presentation will be on Technology related to Educational Assessment and Evaluation (deliberately vague), and the other will be on Alternative Assessments.

MEASUREMENT PROJECT (CEP 502)

[DUE Wednesday, April. 22]

FINAL OBJECTIVE:

To construct, administer, and fully evaluate the technical merits and weaknesses of a short achievement test.

SPECIFIC REQUIREMENTS: Each individual project must contain the following four sections:

SECTION 1 - The Actual Test

The test must contain 2 separate parts:

Part A - containing 10 multiple choice questions
Part B - containing 10 True-False questions

All 20 questions must cover one of the following subjects:

1. Some aspect of grammar -- e.g. punctuation, capitalization, spelling, syntax, etc. -- at high school graduate level.
2. History
3. Word Processing
4. Biology or Earth Science

The questions may be written either by you or obtained from another source, but all sources must be fully and properly documented. Including questions which appear to mimic what others have included in their tests is not a good idea.

Included within Section I must be copy of the Answer Key for all 20 questions.

SECTION II - The User's Manual

This section must include some discussion of each of the following:

Content of the test (both general and specific)
Knowledge level (taxonomy)
Possible uses with regard to norm and criterion referenced
Appropriate age/grade levels (must include adults)
Directions for administering and scoring (using correction for guessing)
Table of specifications
Percentile Norm Conversion Table and other data of interest
Item statistics (difficulty and discrimination indices as well as recommendations to
revise) for each test item. (Assume norm-referenced)
Reliability estimate (KR20 and/or KR21 for norm referenced)
Reliability estimate (using Spearman-Brown prophesy formula)
Item statistics (difficulty and discrimination indices for each test item
(Assume criterion-referenced use)
Reliability estimate (for criterion-referenced)

SECTION III -- A Comparison of Item Types

Section III should represent a comparison between the two types of test items that have been used -- Multiple Choice and true-false. Comparison must be made on the basis of information presented in Section II (e.g. level of knowledge, reliability and item analysis).

SECTION IV - Appendix

The appendix should include a numerical listing of all formulas that were used and the specific reference where those formulas may be located in the text. Actual calculations may be included, if desired.

OTHER CONSIDERATIONS

1. When possible, time will be made available during class so that students may work with one or two other students (Please no more than two) to help iron out difficulties or seek critical advice. However, each individual student must hand in and is fully responsible for his/her own "unique" final report (unique test items too). (No bonuses for creative layout).
2. The project will contribute 30% of the final grade and positively must be handed in before class on April 22nd!! Although no class lectures have been scheduled for April 8th so that student may use this time to put finishing touched on their project, the instructor will be available (in her office) for last minute help or questions on the project.
N.B. Projects handed in late will be accepted conditionally or not at all.

Dr. Necia A. Black
Spring Semester, 1998

TEXTBOOK:	Measurement & Evaluation in Education and Psychology
	(4th edition, 1991) by Mehrens and Lehmann