What’s in a methodology?

The difference between method, methodology, and theory…  and how to get the balance right

It’s the time of year when students are gearing up to write their thesis, and whether it’s at the undergraduate or graduate level, for many this means coming to grips with a tricky question: how do I best explain what it is I’m doing in my paper, and how do I make sure my explanations are up to the standards of academic research? In other words: how do I put together and write up my methodology?

Answering this question is by no means straight-forward. I’ve just recently had a discussion with a PhD student about the difference between a method and a methodology, and about how these two things relate to questions of theory. These are not problems that cause frustration only at the undergraduate level, but that accompany many scholars their entire careers. In fact, at a meeting I attended a few weeks ago on how to apply for research funding from the European Research Council, one of the concerns that the Council regularly had with applications was that scholars did not provided a good methodology section. So if you are a student, and you are confused, remember that you share that confusion with many of the professionals.

What makes questions of method and methodology so thorny is that the answers depend on the respective discipline and on the particular research project. Someone in the arts and humanities may interpret the word methodology quite differently than someone in the social sciences or the life sciences, and different supervisors usually have diverging expectations about the “methodology chapters” in their students’ research papers. In this post, I will try to highlight different perspectives on this topic, as well as options for coming to grips with methods and methodologies.

I’ll first give you an overview of what a “research method” is, and how a method differs from a “methodology”. I’ll then take a look at how methodology relates to theory, and will discuss where methodological concerns might best fit in a research paper or thesis. I’ll provide examples throughout, but I’ve also included two hypothetical research projects at the end of this post that each deal with methodological issues in a different way: one project is at home in the humanities, and one takes a social science approach. You’ll find suggestions for further reading in the reference section.

Method vs. methodology

A source of some confusion is that the words “method” and “methodology” are often treated synonymously, even though they do not mean the same thing in academia. In scholarly work, “methods” are practical hands-on steps for doing research. This usually includes defining the scope of the research project, coming up with a research question or hypothesis, selecting and collecting data, processing that data with certain tools to enable analysis, and then going through the data systematically to answer the central question. For example: a method for doing quantitative research on Japan’s economy might be to use the statistical software SPSS to check for correlations between different variables in a data set; a method for doing qualitative research on China might be to use differently coloured highlighters to mark metaphors and similes in speeches by Mao Zedong and then discussing which ones draw from different Chinese intellectual traditions. In other words, methods are the tools you use to do your research.

So what is a methodology? In essence, methodology is the discussion of methods. This includes the theoretical ideas and concerns that inform the use of different methods. A methodology section in a research paper needs to achieve three things, though not necessarily in this order: Firstly, it should consider what the nature of academic work is more generally, and what this might mean for anyone who explores the topic at hand. Secondly, it needs to provide a literature review, discussing what methods researchers have traditionally used to study the kind of topic that the project focuses on. Thirdly, it should explain what methods this particular project uses and why.

The first issue is a question of epistemology, the philosophy of knowledge. Crucial epistemological questions include: how can we know something? Is there such a thing as objective “truth”, or are we subjectively creating “truths” ourselves? What have different intellectual schools said on these issues, and what do our own answers to these questions say about the value of our research project? What do they say about the value of academic work in general? These are debates that have occupied thinkers for millennia, and no-one would expect you to answer them in a term paper or thesis. Nevertheless, the practical methods you use to study your subject come with certain assumptions, so it would be a good idea to demonstrate that you are aware of what these are.

For instance, imagine you are planning to do research on how discussions on Facebook influences people’s political views, and that you are planning to do a large-scale survey to get your data. As part of your methodological considerations, you should spell out how we might know about someone’s “political views”, and what you mean by “influence”. These are by no means trivial questions, and even though they are theoretical, they have very real implications for how you conduct your own research. Next, you might want to review what experts in the field have said about the value and drawbacks of using surveys, about the relation between information and human behaviour, and about the problems of establishing causalities between different variables. A note on positivism as a research tradition would also probably be wise. Finally, you should explain where you got your data and what exactly it is you plan to do with it.

Similarly, if you are studying policy documents to find out what the agenda of a specific government is, you would be well advised to think about epistemological questions like the value that such documents might have as an indication of political preferences, about the nature of political decision-making, or about the various philosophical traditions that have debated whether the language in such sources reflects certain beliefs or conjures them into being (or maybe both?). How you then go on to select and study the actual documents will likely follow from your answers to these questions.

How methodology connects to theory

As these examples already show, methodological discussions are both theoretical and practical in nature. This is also what makes writing a methodology section for an article or a thesis so hard. It can be difficult to draw a line between a typical theory chapter and the epistemological discussion of the methods you used. Let’s say you are studying international relations. You’ll likely want to include a theory chapter that discusses what different schools of thought have to say about theoretical concepts like states, power, anarchy, international society, norms, preferences, and so on.  Do you now need to include a second theoretical chapter that discusses how we can know about the system of states? The answer is not straight forward, and will strongly depend on what you are trying to achieve.

Overall, it can help to see this overlap between theory and methodology not as a problem but as an opportunity. In the example above, a good methodology discussion could pick up on earlier theory-driven considerations of what a state is and could then seamlessly connect these to the question of what different schools of thought count as “data” on state behaviour. From there, it is only a small step to outlining what data your research project uses, and what work-steps you took. In this case, the methodology is the puzzle piece that sits between broader theoretical debates and actual hands-on research work.

Nevertheless, it is quite common to get the balance wrong between the theoretical and the practical aspects of a methodology. Imagine a term paper that sets out to study a particular case of how people use digital media in everyday life. The case study will consist of observing and interviewing teenagers in a particular high school in Seoul to see how they use mobile phones during school hours. Here’s two ways this paper could go wrong. The paper could discuss at great length the nature of human knowledge without ever mentioning why this particular high school was chosen, how the researcher conducted the interviews, how the participants were observed, or how the interviews and research notes were later analysed to arrive at a conclusion. This would be a paper that got its emphasis wrong, remaining almost entirely in the philosophical realm of epistemology. Alternatively, imagine the same paper launching into the minutia of every single work-step, but never justifying why it might be useful to conduct a case study in the first place, why observations allow us to say something about people’s behaviour, how much credence we should give to the statements of interview subjects, or whether results from the project are representative of human beings in general or only of kids living in this neighbourhood of South Korea’s capital at this particular point in time.

How you get this balance between theory and practice right will have to be a question you answer on a case-to-case basis. There are certainly projects that do not require a lot of practical work-steps but instead focus more on epistemology. For instance, if you plan to write a paper about a famous philosopher, you might only need one footnote to explain what texts you used and how you went about interpreting them. The question of what an interpretation is or why these philosophical texts matter will be much more central to your study, so that your methodology section will likely focus primarily on these issues. As another example, imagine you are running statistical tests on the relation between different demographic and economic variables in Taiwan, using a dataset published by the United Nations and studied widely by economists. It may not be necessary to go into long discussions about how something like the Gross Domestic Product gets calculated, and what these numbers tell us about incomes in an economy – a few footnotes to other scholars who have discussed these matters will be enough to show that you are aware of such debates. The more interesting questions for your case might be how you set up your statistical calculations and how you went about visualizing the results for your readers. The methodology section of such a study might therefore be rather light on epistemology but heavy on the nitty-gritty practical issues of using this particular data set.

Where in a thesis does the methodology section go?

As you’ve seen, methodological concerns differ widely, depending on the project and the discipline. The same is true for conventions on how to write up a methodology section. In some disciplines, notably the life sciences and certain social sciences, it is customary to write within a standard framework: introduction, literature review (often including theoretical concerns), research design (methodology), research results, discussion, and conclusion. What needs to go where can be very specific, and concepts like theory, methodology, method, and strategy are kept strictly apart (for an example, see Rudestam & Newton 2007). In other areas, particularly in the arts and humanities or in branches of the social sciences that are less positivistic, the setup can be much looser. Questions of methodology might make up a paragraph in the introduction, or the last section of the theory chapter, or the first section of the case study, or even a number of footnotes throughout the study.

The scope of the methodological section will also depend on the level you are working at: most undergraduate degrees don’t normally require hands-on research with primary sources, and it is quite possible that a term paper or even a BA thesis is essentially a literature review. In such a case, it would be wise to include at least a note on what a literature review is, what it can achieve, and what considerations went into picking this set of secondary sources rather than another (for inspiration, see Hart 1998). However, this probably won’t require more than a short paragraph. At the level of a doctoral thesis, the situation is quite different. Such projects usually have fully-fledged methodology chapters, often with sub-sections to discuss epistemological questions, the selection of research materials, and the exact steps taken to conduct the study. You will have to decide where you belong on this scale.

Two examples of how to deal with methodology

To show you how methodological concerns play out in practice, let me walk you through two hypothetical research projects at the graduate level that each deal with an aspect of politics in East Asia. These are the two projects: Alice studies Chinese at a humanities faculty, and she is writing her MA thesis about the role that pre-modern Confucian sources play in the political programme of China’s former president Hu Jintao. Becky studies East Asian Studies at a social science faculty, and she is writing her MA thesis on the way that Japan’s public broadcaster NHK covered the 2011 melt-down at the Fukushima Daiichi nuclear power plant in its flagship news broadcast News Watch 9. Here are the choices that Alice and Becky are making as they are working on their projects.

Case A – Confucian thought in contemporary Chinese politics: Alice’s project starts with a thorough literature review on Chinese politics, the Hu-Wen administration, and the so-called “Confucian Revival” in contemporary China. This literature will later go into a first chapter, in which Alice plans to outline the main issues and debates, along with a few theoretical ideas about how appeals to tradition are said to legitimate political decisions. After reading the literature, Alice decides that her study will focus on how Hu Jintao has used the word “harmonious society” – a term that loosely draws from the Confucian concept of harmony. Since this word was introduced at the National People’s Congress of 2005, she will look at speeches and news announcements that followed that congress, and she will cover six months. She will also look at Confucian classics to see how the word “harmony” is used there, and will then compare these sources.

As Alice works on her project, she decides that the methodological discussions should go at the start of the thesis, in the introduction. She will write a paragraph about her choice of sources, including a footnote on how she will reference these sources throughout the text. Since much of Alice’s work consists of demonstrating her command of the Chinese language, and of convincing her readers of her arguments using translated quotes from the original sources, she decides to also write a paragraph on what it means to translate political texts from such different time periods into contemporary English, and what considerations went into her own translation work. This means that she’ll explain why she is providing both the Chinese original and her own English translation in the main text of her thesis as she examines different sources, and how her translations will be “annotated”, which means she will comment on her translation choices and will provide important cultural or historical information in footnotes along the way.

She will then include an additional methodological section at the start of her analysis chapter, right after the theoretical discussion of how and why political agents appeal to tradition to justify their policies. This short section will discuss the nature of historical source materials, with a particular focus on the Confucian classics and what is currently known about their origin, their authenticity, and their use in later periods of Chinese history (…the highly epistemic question of how we assess something’s “authenticity” will be part of this section). Following her analysis, Alice will draw all these elements together in her conclusion to discuss how Confucian traditions are creatively reworked in contemporary Chinese politics, and to elaborate what this says about the ruling party’s attempts to justify its work.

Case B – NHK news coverage of the 311 disaster: Since Becky wants her study of Japanese media to include quite a few technical elements, like the way that camera angles and studio design contribute to news reporting, she decides to discuss her methodology in a special chapter. Just like Alice, Becky starts her work with a literature review, and she decides that discussions about Japan’s media, Japan’s nuclear industry (the “nuclear village”), and about the Fukushima disaster will all be part of her introduction to the topic. Her research focus will be on how a national broadcaster contributes to knowledge about nuclear energy. To this end, she plans to include a theory chapter that examines how academics usually make sense of mass media and its role in political processes. This is also where she will discuss the works of Japanese media theorists who have written on politics and culture in Japan.

Following this discussion, she will write a methodology chapter, which she calls “Researching Japan’s national news broadcasts”. This chapter is going to have three sub-sections. The first part will follow up on the issues she raised in her theory chapter (like: what is visual communication? What are TV news?) and will discuss epistemology: Does an image on TV represent the actual situation on the ground, or are such images selected and edited in ways that introduce visual rhetoric and specific tropes, biasing the news reports in the process? What does this mean for a person who now analyses these news materials? To explore this issue, Becky will discuss approaches to visual communication analysis, such as semiotics. In the second section of her methodology chapter, she will explain why she picked NHK as a source of material, and which news broadcasts she picked (for instance: all news broadcasts that dealt with nuclear energy in the three months before and the three months after the disaster). The third part of the chapter will discuss the exact work-steps that Becky followed to prepare the material for analysis and interpret her data. She decides that this will include creating sequence protocols of the news broadcasts, and then providing shot protocols for particularly important segments.

Since her actual analysis will consist of a mixed quantitative and qualitative approach, she will explain what this means in this third section of her methodology chapter: she will look at the amount of time that news broadcasts on different days report on nuclear issues, at shot frequencies in the segments that cover Fukushima, and at the meanings that certain camera angles and visual tropes introduce to the overall news narrative. In this section, she will also explain that she is compiling all of her data in an appendix, and that she will include graphics and statistics in tables throughout the actual analysis chapter. Since Becky’s analysis does not focus on the use of language, she will need to explain why she is bracketing this issue (and where readers might find out more about linguistic analyses of the news). Contrary to Alice, Becky decides to not discuss at great length how she is translating the news, and only includes a footnote that states something like “if not noted otherwise, all translations in this thesis are my own”.

Becky’s analysis will now include a chapter with different visualization strategies that NHK used to report on nuclear energy in Japan. She will compare the reporting before and after the disaster, and discuss the implications in her conclusion – where she will tie her own work back to the theoretical concerns she raised in her theory chapter. She will also have a paragraph in her conclusion that outlines what her approach left out and why. To show that she understands the limitations of her research, she will also suggest what kinds of follow-up studies could now shed light on any remaining questions. Her last paragraph will be a forceful argument about how national news play a powerful role in not simply reporting but actually constructing political crises.

Conclusion

As you can see, there is no single answer to how you should build theory, methodology, and method into your research project. The best advice I can give, is: check what your supervisor or your publisher has in mind. They know your field, and they will be the ones judging your work, so you should always see what their specific requirements are. As with all good writing, keep your audience in mind.

I’ve provided sources for further reading below, in case you want to learn more about this fundamental part of academic work. You may want to also take a look at my own discussions of methodology, for instance my blog post on how to do a discourse analysis (which is about methods) or how to set up such an analysis (which includes epistemological questions).

If you are currently working on your methodology, or you are instructing others on how to do so, feel free to leave a comment below. Conventions differ widely, and I’ve for instance just learned that certain life sciences make a distinction between “methodology” (the discussion of methods), “method” (a general technique in research), and “strategy” (the practical work-steps of how to apply a method to a specific case). As always, I’d love to hear how you are dealing with such distinctions, and where you place the emphasis in your own work.

References

Goodin, Robert E., & Lingemann, Hans-Dieter (Eds.) (1996), ‘Part IX: Political Methodology’. In: A New Handbook of Political Science. Oxford & New York: Oxford University Press (pp.717-799).

Hart, Chris (1998), Doing a Literature Review – Releasing the Social Science Research Imagination. Los Angeles et al.: Sage.

Hine, Christine (Ed.) (2005), Virtual Methods – Issues in Social Research on the Internet. Oxford & New York: Berg.

Marsh, David, & Stoker, Gerry (2010), Theory and Methods in Political Science (3rd ed.). Basingstoke & New York: Palgrave Macmillan.

Rogers, Richard (2013), Digital Methods. Cambridge, MA & London: MIT Press.

Rudestam, Kjell Erik, & Newton, Rae R. (2007): ‘The Method Chapter: Describing Your Research Plan’. In: Surviving Your Dissertation – A Comprehensive Guide to Content and Process (3rd ed.). Thousand Oaks, CA et al.: Sage (pp.87-115).

Trachtenberg, Marc (2006), The Craft of International History – A Guide to Method. Princeton: Princeton University Press.