In #SAcdn, Assessment, Research & Ethics

by Anika Roberts-Stahlbrand, Ontario Institute for Studies in Education, University of Toronto

There’s no such thing as objectivity, there are only lesser degrees of subjectivity
-David Brinkley

This Research, Assessment & Evaluation series is brought to you by the CACUSS Research, Assessment, Evaluation Community of Practice. 

For most, alchemy is the stuff of Harry Potter; for me, alchemy was a class I took in my undergraduate degree. No, I wasn’t one of the lucky few muggles who got my letter to Hogwarts – I studied the history of science and technology at the University of King’s College, a degree focusing on how truth is determined and the way science and technology impact society and vice versa. For knowledge seekers in Medieval Europe who understood the world through Aristotle’s theory of five elements, alchemy was not for greedy buffoons, but rather at the cutting edge of understanding and mastering the world. This goes to show that the ways we know the world and determine truth, validity, and reliability are highly contextual and exist within knowledge paradigms. It is this understanding that informs most qualitative approaches to research.

This is a two-part blog on qualitative research, assessment, and evaluation (RAE) that focuses on building your competency in strategic planning, research, and assessment at the intermediate level (CACUSS, 2017). I also hope it will lead to self-reflexivity on why we aim for a culture of assessment in student affairs, who it is for, and what we want to discover through this culture. My experience is in qualitative research, however, the same principles apply to assessment and evaluation as well. Check out this article for a quick introduction to the difference between research and assessment.

In part one, I will explore the differences between quantitative and qualitative research, and when to use qualitative research. In part two, I will outline the process of selecting and designing qualitative RAE methods.

Qualitative vs quantitative approaches

Qualitative and quantitative approaches do not need to be opposed to each other. In fact, mixed methods research, which uses a combination of qualitative and quantitative approaches, is growing in popularity. However, there are often major ideological differences between qualitative and quantitative researchers (Palys & Atchison, 2008). For the sake of brevity and clarity, I will present them as opposites.

Quantitative approaches to RAE focuses on knowledge or phenomena that can be measured, counted, or turned into a quantifiable abstraction. The only knowledge that can be ‘seen,’ and thus valued, by quantitative approaches is knowledge that can be measured. Many quantitative researchers believe that because they are using numbers, the research and results are objective, and not influenced by their role as a researcher (Palys & Atchison, 2008). A common example of a quantitative approach to RAE in student affairs is an indicator that represents aggregated survey data, such as how many students feel a sense of community, or how many students achieved a learning outcome.

I enjoy koalatative research

Image from __iwastesomuchtime.com

When to use qualitative research

In the tradition of qualitative research, I will try to make my worldview transparent for you, so you know how to evaluate the information in this blog. I am a big fan of qualitative research because I think the most important parts of life – e.g. empathy, love, community – are often the most intangible and impossible to measure. This is the kind of learning that makes me excited to be a student affairs professional. I am currently a graduate student at the University of Toronto, OISE in Adult Education and Community Development designing and performing qualitative research on post-secondary student learning through food.

I am a fan of qualitative approaches to RAE, but they aren’t always appropriate. Creswell and Poth (2018), prominent researchers and the co-authors of multiple books on research methodologies, describe when to use qualitative approaches in RAE (p.45-46):

  • To study a population or variables that cannot be easily measured
  •  To hear silenced voices and empower individuals to share their stories in their
    voices, thus minimizing the power relationship between researcher and participant
  • To gain a complex detailed understanding of the issue
  •  To understand contexts
  •  To amend existing theories or creating new theories that do not adequately
    capture non-dominant populations

Rather than being wishy-washy touchy-feely stuff, qualitative research is rigorous. Rigorous RAE takes a lot of time! For example, it takes time to develop your approach to research, anticipate and attend to ethical considerations, gather data in the field, and analyse your results (Creswell & Poth, 2018). As a current thesis student, I can definitely attest to the fact that this stuff takes a lot of time!

Eek, time. Yes, I know! Time is valuable and often scarce in your job in student affairs. You have pressing crises to deal with, or reports to submit to supervisors. However, as well as being a profession with a culture of assessment, we are also a profession that is student-centred and equity-oriented. With these goals in mind, I believe that as a profession, if qualitative approaches to RAE are the most appropriate to understand a student problem, to answer a given question, to improve a specific program, or to gain insight to enhance the student experience, this should compel us to make the time to do RAE well and in the best service of our students.

For example, do students feel safe on campus? The meaning behind a yes or no answer can get lost in quantitative RAE. For example, on the question of safety, on a survey, a white student may say they feel unsafe in class if the class is one that challenges white supremacy, while a trans student may say they feel unsafe in the gym changing rooms due to body policing and transphobia. In aggregated survey data, the complexity of both of these responses will be collapsed into the same feeling of lack of safety on campus, even though they both have very different causes and very different calls to action. By no means do I think all survey and quantitative data are worthless, I just think they have a time and a place. In this case, qualitative data has the potential to better capture the different kinds of unsafe each student feels.

Conclusion

Hopefully this post has got you thinking about when to use qualitative RAE approaches and why to use them. In the next blog post, I will go over the process of setting up qualitative RAE and various methods so that you can integrate more qualitative approaches into your culture of assessment.

Anika is a Master’s Candidate at OISE in Adult Education and Community Development.  

Works Cited

CACUSS. (2017). Student Affairs and Services Competency Model. Retrieved from
https://www.cacuss.ca/resources/student-affairs-and-services-competency-model/index.html.

Creswell, J., & Poth, C. (2018). Qualitative inquiry and research design: Choosing among five
approaches (4th ed.). Sage.

Palys, Ted & Atchison, Chris. (2008). Research decisions: Quantitative and qualitative research
(4 th ed.). Nelson Education Ltd.

Recommended Posts

Leave a Comment