In #SAcdn, Articles, Assessment

by Anika Roberts-Stahlbrand

This Research, Assessment & Evaluation series is brought to you by the CACUSS Research, Assessment, Evaluation Community of Practice.

I almost got an apple tattoo a few years ago. No, not the company – the fruit. You see, I did my undergraduate thesis on the history of the apple industry in Nova Scotia (Roberts-Stahlbrand, 2016) and became rather engaged in it all. However, it took me a while to figure out how to do research for the first time. At the start of my fourth year, I remember my professors telling me I needed a methodology for my thesis. I didn’t get it. My methodology, I thought at the time, was just reading old agricultural manuals that no one had looked at for decades and then writing about it. 

How wrong I was…And here is an introduction to qualitative methodology so you hopefully don’t need to go on as long a journey as I did.

This is a two-part blog on qualitative research, assessment, and evaluation (RAE). It focuses on building your competency in strategic planning, research, and assessment at the intermediate level (CACUSS, 2017). 

In part one, I explored the differences between quantitative and qualitative research, and when to use qualitative research. In part two, I will outline the process of selecting and designing qualitative RAE. Methodology is often used as a shorthand to include the philosophical assumptions, interpretive frameworks, approaches, data-gathering methods, and data analysis approach. It is what shapes the way you investigate, frame, understand, and interpret what you are studying. 

In this blog you will learn about the importance of unpacking philosophical assumptions, different interpretive frameworks to work within, five common approaches/methodologies, and six common methods in qualitative RAE.

 

Philosophical assumptions

It would hardly be fish who discovered the existence of water.
– Clyde Kluckhohn

So how do we do good qualitative RAE? Often, the hardest part of qualitative RAE is seeing what we take for granted and recognising our own deeply ingrained assumptions, or, in other words, being the fish who discovers water. Qualitative approaches do not seek to find objective truth, but rather require the researcher to be self-reflexive on their own subjectivity and how it shapes the way they approach and do RAE. As researchers, we are multicultural, intersectional subjects who bring our personal perspectives, experiences, and assumptions to our RAE.

I spent much of my undergrad pretending I knew the difference between ontological and epistemological while secretly googling them under the table. Now, I think I’ve got it down, and I also think they are very helpful concepts for informing good qualitative RAE. It is worth taking a moment to look at four key philosophical assumptions that influence our orientation towards knowledge and thus RAE.

Philosophical AssumptionQuestions
Ontological What is the nature of reality?
EpistemologicalWhat counts as knowledge? How are knowledge claims justified? What is the relationship between researcher and that being researched?
AxiologicalWhat is the role of values?
MethodologicalWhat is the process of research?
Excerpt from a chart by Creswell and Poth (2018, p.20)

 

Interpretive frameworks

The CACUSS competencies (2017) encourage us to critique the dominant group perspectives and promote equity, diversity, and inclusion. Critiquing dominant group perspectives recognises that there is no one way to understand student development, or indeed, the world. Instead, the world is often understood and articulated through the lens of the dominant perspective. Without being aware of the systems of knowing that have created inequity, we are in danger of continuing them.

Dr. Michelle Pidgeon from Simon Fraser University discussed this in a previous RAE webinar on decolonising assessment. She said that since assessment is the tool through which things get measured and decisions get made, it often favours the powerful. At an institutional level, she says, if we are assessing to make resource allocation decisions, we need to think critically about whether we are asking the right kind of questions to make the kinds of changes a decolonizing project requires. Dr. Pidgeon calls on those who work in RAE to be transparent about how they assess, and to deconstruct notions of something being ‘the best’ by asking who it is the best for.

Qualitative researchers believe there is no one objective way of knowing the world ‘out there,’ but rather many subjective ways of understanding reality. Indeed, by existing in and studying reality, we ourselves influence it too. Therefore, as qualitative researchers, we must explicitly identify our interpretive frameworks. By identifying and making transparent the interpretive framework and goal of our research, we can be ideologically driven while still practicing rigorous RAE.

Our interpretive framework influences our research strategies, methods of collection, and our practice and politics of data interpretation (Creswell & Poth, 2018). It is important to note that quantitative RAE exists within interpretive frameworks too. Quantitative researchers just tend to be less reflexive about their philosophical assumptions and ideologies and how they influence their research.  Here are the most common qualitative interpretive frameworks, adapted from Creswell and Poth (2018, p.34):

Interpretive FrameworkGoals
Social ConstructivismTo understand the world in which they live
TransformativeTo act for societal improvement
PostmodernTo change ways of thinking
PragmatismTo find solutions to real world problems
Critical theoryTo address issues of power and inequity to empower humans
Critical race, feminist, queer, disabilityReality is based on power and identity struggles. Privilege or oppression based on race, ethnicity, class, gender, mental abilities, physical abilities, and sexuality

 

Five approaches and six methods

It is only now, after all this work, that we get into what most people think of as RAE. Just like student affairs practitioners aim for constructive alignment between learning outcomes, learning experiences, and learning assessment (HEQCO, 2015), in qualitative RAE we aim for methodological congruence (Creswell & Poth, 2018). This means it is important for the assumptions, purpose, questions, and methods of our research to be coherent and cohesive.

Five common qualitative research approaches are narrative research, phenomenology, grounded theory, ethnography, and case studies. Here is a chart to help you figure out which qualitative approach is the best fit for your research needs (Creswell & Poth, 2018, p.67):

 Research focusResearch needs
Narrative researchExplore the life of an individualTell stories of individual experiences
Phenomenological researchUnderstand the essence of the experienceDescribe the essence of a lived phenomenon
Grounded theory researchDevelop a theory grounded in data from the fieldGround a theory in the views of participants
Ethnographic researchDescribe and interpret a culture-sharing groupDescribe and interpret the shared patterns of culture of a group
Case study researchDevelop an in-depth description and analysis of a case or multiple casesProvide an in-depth understanding of a case or cases

Six common qualitative research methods are interviews, focus groups, observation, document gathering, artifact gathering (e.g. audio recordings), and emergent sources (e.g. social network interactions). Within qualitative RAE it is important to use multiple methods of data collection.

For example, in my current research on learning in the meal hall I am using a case study approach. My methods are a combination of interviews, observation of behaviour, observation of physical layout, and artefact gathering (signs and menus). 

 

Conclusion

This was a cursory introduction of different qualitative RAE approaches and methods. Hopefully, it has provided you with enough of a base for you to start digging deeper into qualitative approaches to RAE. Plus, stay tuned for RAE webinars on various specific qualitative methods! 

Anika is a Master’s Candidate at OISE in Adult Education and Community Development.

 

Works Cited

CACUSS. (2017). Student Affairs and Services Competency Model. Retrieved from
https://www.cacuss.ca/resources/student-affairs-and-services-competency-model/index.html.

Creswell, J., & Poth, C. (2018). Qualitative inquiry and research design: Choosing among five
approaches (4th ed.). Sage.

HEQCO. (2015). Learning outcomes assessment: A practitioner’s handbook. Retrieved from
http://www.heqco.ca/SiteCollectionDocuments/heqco.LOAhandbook_Eng_2015.pdf.

Roberts-Stahlbrand, Anika. (2016). Getting to the core of the matter: The rise and fall of the
Nova Scotia Apple Industry, 1862-1980. Canadian Food Studies, 3(2), 4–22.
https://canadianfoodstudies.uwaterloo.ca/index.php/cfs/article/view/165

Recommended Posts

Leave a Comment