Richard Skolnik – Author of Global Health 101
The end of the school year is the perfect time to think about how to measure the extent to which students have met your learning objectives.
To get a ‘fix’ on what students know when they enter my courses, I give an anonymous, ten-question pre-test at the beginning of the first class. The questions are based on a sample of questions from my last final exam and meant to cover the range of learning outcomes for the course. Despite the talents, knowledge, and experience of my students, they usually average around 3 out of 10 on the pre-test. These scores are a good indication of how much students have to learn from an introductory course on global health.
If I were a serious scientist, I would ensure that I ask exactly those same ten questions on the final exam and then compare the scores on the final with the pre-test. I admit - I don’t do this rigorously. However, I do include in the final exam a number of questions from the pre-test and other questions that get at the same issues.
In my undergrad introductory course on Global Health, the final course grade is made up of 15% for each of three policy briefs, 15% for a 30 question mid-term, and 40% for a 40 question final examination. Grading these IS a lot of work for the teacher. However, after trying a number of approaches, I have decided that this is the best mix of papers and tests to motivate my students and assess their performance.
My classes, as I have noted before, are completely interactive. I call on each student each class. Thus, I do NOT include a grade for participation in evaluating my students. In addition, I always worry that it is difficult to assess ‘participation’ in an objective manner.
One specific aim of my Global Health courses is to help the students learn to write Policy Briefs in a clear, concise, precise, and evidence-based manner. I assign three, five-page briefs to my undergrads and three, six-page briefs to my grad students, as discussed in an earlier blog. I grade the papers (now with the help of a TA who also reads them), on the basis of clarity, precision, the linearity of their argument, and the quality of their evidence. We read each paper and track changes and comments on each brief. The best indicator of the grade of a paper, as I tell the students, is how long it takes us to read it. ‘Good’ briefs are easy and fun to read. ‘Bad’ briefs are hard to read and require extensive editorial and substantive comments. In June, we will put on the Global Health 101 website some additional ‘model policy briefs’ written by both undergraduate and graduate students.
For the mid-term and final examinations, I give the students a ‘case’. This is a description of a low-or middle-income country. The ‘case’ is always called ‘Skolnikland’ and it is usually based on a country in East or West Africa or in South Asia. The students are asked to read the case and then answer a set of questions ‘based on the case.’
I try to write questions that test student mastery of key concepts and frameworks that we have covered in the class. I also write some questions that test knowledge that I would expect every Global Health student to have. The typical test could include questions like:
- What is the distribution of the burden of disease in Skolnikland today, how would it have looked 20 years ago, and how might it look in 20 years from now, if present economic trends continue?
- Besides perinatal conditions and congenital anomalies, what are the four leading causes of under-five child death in Skolnikland?
- What would be the two leading causes of child death if Skolnikland were Bolivia?
- What are four of the main pillars of the WHO recommended approach to addressing malaria in Skolnikland?
- What are the ‘three delays’ that relate to maternal mortality? In Skolnikland, what is one measure you would take to address each delay and why would you take that action?
In my first years of teaching, I gave short answer questions on my exams. I felt that these exams were an excellent reflection of student knowledge and thinking. However, they took an age to grade. Encouraged by my colleagues at The George Washington University (GW), I switched to a multiple-choice test. I was very worried that these tests would be too easy. However, I pre-tested my first examination of this type on one of my best former students and was pleased when she said how difficult it was. I do feel like multiple-choice tests give me less of a feel for student mastery of subject matter than short answer tests. Nonetheless, it may be worth this loss, since multiple choice tests get graded electronically and quickly, save me days of work, and allow me to get grades done on time. In addition, multiple-choice tests are more ‘objective’ than short answer tests and I lose less sleep about how ‘fairly’ I am grading when I give multiple-choice tests.
Despite this, I switched back to short answer tests when I moved from GW to Yale. Some of the challenges of these types of test have been just what you would expect:
- Students sometimes see things in your questions that you certainly never intended – even when you pre-tested the questions
- It is sometimes hard to draw the line between an answer that is wrong and right, unless all of your questions allow only for very narrow responses
- It is sometimes almost impossible to read the student responses!
- Some students don’t understand the meaning of ‘short answers’.
Over the summer, I plan to discuss these challenges further with former students and former TAs, with the goal of writing better tests. I am also considering distributing future tests by email, having the students answer in WORD, and telling the students the answers must fit in the space allocated or they will get no credit for them.
I hope at the end of the fall semester of 2013 that I can report back to you on any new lessons I have learned about the challenges of testing.
In the meantime, faculty can request access from Jones and Bartlett to model tests which are on the teacher’s portion of the Global Health 101 website.
Thanks to Rachel Skolnik Light for her comments on the draft of this blog.
Richard Skolnik is a Lecturer at the Yale School of Public Health, where he teaches global health courses at the undergraduate and graduate levels. Richard was previously an Instructor in Global Health at The George Washington University, the Vice President for International Programs at the Population Reference Bureau, and the Executive Director of the Harvard School of Public Health PEPFAR program. Richard worked at the World Bank from 1976 to 2001, last serving as the Director for Health and Education for South Asia. Richard is the author of Global Health 101, a comprehensive, introductory text on global health.