Creating Application Questions for Assessments

By Vincent Wong

I am a faculty in the School of Advancement and I teach introductory biology and physiology and anatomy in the Pre-Health program. We have a very diverse group of students.  Some students just graduated from high school while others are mature students returning to school to upgrade their skills and education. 

One of the challenges I had for converting my courses to online delivery was about how to create suitable test questions for the online environment.  The nature of my subjects involves a lot of knowledge questions that require students to recall facts and information.  These questions become highly ineffectively in assessing student’s learning when we switched to online learning because students can now easily look up answers from the Internet, notes or textbooks.  This required me to come up with a better way for assessing my students. In addition to the knowledge-based questions, I included application questions and multi-select (MS) questions (i.e. questions with more than one correct answer) on the test.   

Here’s an example of a knowledge-based question (1st and 2nd level of Bloom’s taxonomy – Remember and Understand): 

Here’s an example of an application question testing for the same concept (3rd to 4th level of Bloom’s taxonomy – Apply and Analyze): 

The application questions and MS questions require the students to think harder and apply their knowledge rather than relying on recollection of (or Googling) information.  

Lessons Learned

Honestly, it was a disaster the first time I did it. The class average of all the sections were roughly 15% lower than normal.  My inbox was flooded with angry (but mostly polite) emails from frustrated students.  I decided to conduct an anonymous survey to gather more students’ feedback about the application and MS questions: 

Here are some comments that were reflective of all comments: 

*Question 6 – 8 on the survey were related to assessments 

After gathering all the feedback, I started doing case studies with students during class to prepare them for the application questions.   

Here’s an example of a case study I do to prepare students for the application questions above: 

I wish I did the case studies for my first online cohort earlier, that would have avoided unnecessarily anxiety in students.  I was able to do this from the beginning of the course for the subsequent semester.  I conducted another anonymous survey to see how my students felt about the test.  Based on the comments from students (see below), I think my strategy worked for my classes. 

Conclusion

My in-class assessments were not suitable in an online environment.  By increasing the number of questions that are at a higher level of Bloom’s Taxonomy, the assessments became more accurate in assessing students’ learning.   Using case studies as a way to prepare students for application questions on the test also increased students’ interest and engagement.