Research Methodology and Research Design H (6807.6)

Required texts

The text below provides an outline of each session as well as the texts that will be made available on Canvas. There is no need to purchase an outside text.

Module 1: Designing Social Scientific Research

The first module lays out the function of social science research as a distinct practice of activity engaged in building theory. We will cover differences between explanation, description, and prediction. This module will show students how to move from identifying a research question to developing a research design which can address that question. As social science research is often used by government departments around the world to evaluate the effectiveness of programs, the topics presented here will be of interest to scholars and practitioners alike.

Session 1: Introducing Social Science Research

Session questions: What kind of knowledge does social science research seek to create? Does social science need a philosophy of science? If so, what would that look like and how does it matter for research? Comparative methods: is science by nature comparative? What does it mean to have a comparative logic? How can case studies provide generalizable or explanatory knowledge?

In the first part of this session, we have two objectives. The first objective is to introduce students to the main objectives of the unit; explain the unit procedures and teaching strategies; and outline the assessment process. The second objective concerns the nature of social science research. We will address how social science enables one to make qualitatively different kinds of assessments than other forms of knowledge practices. Most importantly it involves the move from everyday description to theoretical abstraction, replacing the particular names of things for their qualities which allow researchers to not only describe what happened but to explain why it happened. This moves scientific investigation away from hyperfactualism – i.e. a focus on facts in their individuality – and towards the development of more generalized knowledge claims that allow us to say things about a class of cases or even the operation of whole systems of behaviour. In addition, we will discuss the nature of theory in the study of social and political life, the relation of the philosophy of science to social science methods, and the relationship between social science and its object of inquiry.

We next turn to fundamental debates in the philosophy of science. Although it has never formed an enduring aspect of training within the natural science, the philosophy of science has remained part of the discourse of social science methods. Questions about whether we have cognitive access to an unmediated reality which underlies our observations, the relationship of that underlying reality to theoretical claims, and its implications for developing research designs are enduring debates in the philosophy of social science. The implications of varieties of realism, positivism, and pragmatism in the study of social science and the development of social science methods will be discussed and debated.

In discussing how do scientific research, we begin with the most basic observation that all scientific investigation involves some form of comparison between states either as a form of cross-sectional comparison or as a matter of comparison within one case over time. There are two prevailing logics of comparative research in the social sciences: a most similar systems and a most different systems approach. The former looks across cases which are very much alike along a number of parameters (e.g. Scandinavian countries or Southern European countries) and tries to identify prevailing or common tendencies in how various parts of these systems interact. A most different systems approach takes cases which are very different, say Spain and the US, and tries to use the system-level differences as explanations for individual level variance within each system or to show that these produce different outcomes. In addition to cross-sectional comparisons (i.e. comparisons holding time constant across cases), one might look at a single case over time. This approach is most often used to identify causal relations or evolutionary changes within systems. Whether one focuses on emphasizing difference between systems or minimizing difference between systems depends on how they conceive of the system as a whole as an explanatory factor. Prominent advocates of each of these views are represented in the readings.

Finally, we discuss the role of case studies. The assumption that all scientific research involves comparison does not mean that case studies have no scientific merit. As no research occurs in a vacuum, a well-designed case study may shed important insights on a topic in relation to what is known from the previous literature. In this regard, there is an implicit comparative aspect of a good case study. We discuss five kinds of case studies: idiographic or configurative case studies; configurative studies; heuristic case studies; plausibility probes; crucial case studies. The strengths and weaknesses of each are evaluated with respect to the kinds of analytical leverage they provide. Additionally, we will discuss issues involved in selecting cases with respect to dependent variables, identifying when that can be useful and when it produces biased findings.

Session 1 Required Readings:

Przeworski, Adam, and Henry Teune. 1970. The Logic of Comparative Social Inquiry. New York: Wiley, pp. 3-30.

Gunnell, John G. 1983. "In Search of the Political Object: Beyond Methodology and Transcendentalism." In What Should Political Theory Be Now? Ed. Nelson. Albany: State University of New York Press, pp. 25-52.

Furlong, Paul and Marsh, David. 2010. "A Skin not a Sweater: Ontology and Epistemology in Political Science" in Theories and Methods of Political Science. Eds. Marsh and Stoker. New York: Palgrave Macmillan, pp. 184-211.

Eckstein, Harry. 1992. Regarding Politics: Essays on Political Theory, Stability, and Change. Berkeley: University of California Press, pp 117-176.

Optional Readings

Ayer, Alfred Jules. 2012. Language, Truth and Logic. Courier Corporation.

Taylor, Charles. 1985. Philosophical Papers: Volume 1, Human Agency and Language. New York: Cambridge University Press.

Session 2: Assembling a Research Design

Session questions: What are concepts? How do you relate theoretical concepts to data? What are the problems we might encounter when trying to identify data exemplifying our concepts? What are variables and what role do they play in research? How do you design research so that it answers your research question?

Concepts are critical to conducting social science research. Though they are often invoked, there is little conceptual clarity in the concept of a concept. Our theories are full of concepts such as participation, democracy, efficiency, and so forth. Many research methods often look for evidence of these things. But before we can look for evidence, we need to know what these items mean. There are several complications. First, words like democracy have been used in many ways, each manner corresponding to a different concept. Additionally, even if we agreed upon a single concept denoted by the word "democracy," it would be impossible to say democracy is present here but not there without reference to the various attributes of democracy. Democracy may include things like free and fair elections, de jure and de facto political rights, and so forth. The conceptualization process involves both the definition of the concept and the various attributes that constitute a logical, coherence within that definition. 

The concepts we use in daily life seem self-evident and intuitive, but our theoretical concepts are not the concepts of everyday life. Social science involves finding a theoretical vocabulary to describe a practice in which people already have a vocabulary to describe their activities. The process of operationalization explains the criteria by which empirical observations can be judged as evidence of theoretical concepts. This session will discuss various issues encountered when operationalizing data such as data reliability and validity; whether or not to operationalize variables multiple ways; and the considerations when using proxy variables to measure variables we are unable to directly observe. 

Putting these elements together, we identify five stages in producing a research design. First, all research begins with a research question. The research question should be identified through the literature review. Second, the question should be grounded in the theory that has been produced which in some manner in unsatisfactory because there it leaves something relevant unanswered or because new observations are hard to rectify with the old theory which gives rise to an analytical puzzle. Third relevant aspects of the theory need to be defined as concepts with attributes. Fourth, the relevant data for those concepts must be obtained and analysed. Finally, one needs to consider alternative explanations that may give rise to the observed results. Dismissing alternative explanations is a critical task of any scientific investigation. 

Session 2 required readings:

Mair, Peter. 2008. "Concepts and Concept Formation" in Approaches and Methodologies in the Social Sciences: A Pluralist Perspective. Eds. Porta, Donatella della, and Michael Keating. New York: Cambridge University Press, p. 177-197.

King, G., R. O Keohane, and S. Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton University Press, p. 3-33.

Putnam, Robert D. 1995. "Tuning In, Tuning Out: The Strange Disappearance of Social Capital in America." PS: Political Science & Politics 28(04): 664–83.

Session 2 optional reading:

Gunnell, John G. 2011. Political Theory and Social Science: Cutting Against the Grain. New York: Palgrave Macmillan, pp. 129-154.

Module 2: Techniques of Social Scientific Investigation and Analysis

Social science involves collecting data from persons or aggregates of persons (organisations, states, human systems, networks, markets, etc). During these sessions, we discuss techniques of data collection that have been used to carry out research on humans as individual persons or aggregates. In addition, we cover the kinds of analyses that may be carried out on these kinds of data.

Session 3: Small N methods

Interviewing: Interviewing is one of the most basic methods for learning about persons. We all have practice with interviewing persons in daily life, but interviewing persons who may have personal or professional reasons to be either reluctant or strategically answer questions raises complications in using interviews as a means to generate knowledge. This session will help students learn how to effectively conduct and utilize interviews as a means of conducting social science research. We will discuss how to select interviewees and how to prepare different kinds of interview strategies. Finally, we will discuss how to identify patterns and insights in interview responses and translate those patterns and insights into research findings.

Focus groups: In many ways, focus groups build upon the tools needed to do a competent interview. Focus groups are an important research method, capable of capturing rich qualitative data that emphasises participants' knowledge and experiences. In this session we discuss the rationale for using focus groups, including their strengths and limitations. We unpack how to design and implement focus groups in consideration of various objectives and contexts and explore a range of approaches to analyse focus group data. Included in the session is a chance to design and facilitate your own focus groups for a provided scenario, with critical analysis on the role of focus group data collection for this scenario and your preferred design and facilitation styles.

Interpretive methods: This session aims to introduce and elaborate interpretive approaches and research methods in social science. An ‘interpretive' approach refers to a way of studying the social world that seeks to understand the meaning underlying an intention, action, object or phenomenon. Interpretive studies are informed by social theory which attends to issues of representation through language, text and symbols in the constitution of social life. This session will focus particularly on the use of interpretive approaches in policy studies, but we will also provide various examples of this approach in other areas of political science and Social Sciences in general. 

Discourse analysis: The social world is largely constructed by language. Terms used in language make sense in relation to the systems of meaning that can be described as discourses. Discourses both enable and constrain communication and action, and that is why it is important to study them. There turn out to be a number of different ways to study discourses, here we will look at the more prominent forms of discourse analysis to understand fields of human activity.

Session 3 Required Readings

Barbour, Rosaline S. and Schostak, John. 2011. Interviewing and focus groups. In: Somekh, Bridget and Lewin, Cathy eds. Theory and Methods in Social Research (Second Edition). London, UK: Sage Publications Ltd, pp. 61–68.

Fisher, F. 2006. "Beyond Empiricism." In Deliberative Policy Analysis. Eds. Hajer & Wagenaar. New York: Cambridge University Press, p. 209-227.

Yanow, D. 2007. "Qualitative-Interpretive Methods in Policy Research." In Handbook of Public Policy Analysis. Ed. Fisher, New York: Taylor and Francis, p. 405-415. 

Session 4: From Small to Big N Analysis

Social scientists have often sorted themselves into categories of qualitative and quantitative research. We challenge those categories by looking at ways to quantify qualitative observations and produce meaningful symbolic representations of otherwise quantitative data.

Survey Design: Surveys can be a powerful research tool but there are important considerations in carrying them out. Some things you really cannot ask on a survey and expect to get reliable data. The question order can influence and contaminate responses to subsequent questions. Question wordings can be tricky as subtle differences in wording can introduce systematic measurement biases. Whether one needs to replicate past findings is also an important consideration. We will go through issues in crafting and executing a survey design.

Probability and statistical inference: We will discuss basic issues involved in statistical inference. These include understanding probability and the likelihood of events. In addition we cover basic statistical relationships including formal calculations of association and how to determine when associations are "significant" in a statistical sense.

Regression analysis: This session builds upon the introduction to statistical inference session earlier. The aim is to provide students with an ability to understand regression analyses in published academic work and to identify the assumptions and grounds for its applicability. It will provide a foundation for students who wish to learn these techniques and incorporate them in their own work.

Big data: Big data has become a buzzword in many sectors of the economy and increasingly in academia. More than the sheer size of data implied in the name, big data represents a qualitatively different kind of data requiring specialized skills for its collection and analysis. Big data objects are typically digital in nature, produced as a by-product of transactions such as paying bills, purchasing histories, posting to social media, etc. As they are by-products of transactions, they can be unobtrusively collected. As they are digital, different formats (text, visual, video, numeric, etc.) can be readily combined and information extracted from them; they can be operationalized, modified, manipulated, and so forth without destroying the original data. These operations typically involve a series of competencies developed within computer sciences rather than fields of statistics, anthropology, or even literary criticism. Nevertheless, the approaches of these fields can be incorporated in big data research as, for example, the criteria for analysing a text may be algorithmically operationalized. We will discuss some techniques and applications of big data for understanding social and political life.

Session 4 Required Readings

Gujarati, Damodar. 1995. Basic Econometrics. 3^rd Edition. New York: McGraw Hill. Ch1-3 & Appendix 1.

Session 5: Data analysis

There are four components to this session. The first is microsimulation. This segment will consist of a 45-minute lecture outlining simulation and modelling for social scientists. Types of simulation covered will include agent-based models, cellular automata, microsimulation modelling, systems modelling, CGE modelling and macro-economic models. After the lecture, there will be a practical, showing an agent-based model; a cellular automata model; a microsimulation model; and a systems model. Students will be shown how to provide parameters to the model and get results from it.

Big data: Big data has become a buzzword in many sectors of the economy and increasingly in academia. More than the sheer size of data implied in the name, big data represents a qualitatively different kind of data requiring specialized skills for its collection and analysis. Big data objects are typically digital in nature, produced as a by-product of transactions such as paying bills, purchasing histories, posting to social media, etc. As they are by-products of transactions, they can be unobtrusively collected. As they are digital, different formats (text, visual, video, numeric, etc.) can be readily combined and information extracted from them; they can be operationalized, modified, manipulated, and so forth without destroying the original data. These operations typically involve a series of competencies developed within computer sciences rather than fields of statistics, anthropology, or even literary criticism. Nevertheless, the approaches of these fields can be incorporated in big data research as, for example, the criteria for analysing a text may be algorithmically operationalized. We will discuss some techniques and applications of big data for understanding social and political life.

Mixed methods: No single study will leave a subject matter settled. Different researchers conceptualize entities and events differently; they operationalize concepts differently; and their research designs vary for multiple reasons. One solution is to use multiple sources of data and approaches as a "triangulation" strategy to see if the results of multiple lines of inquiry point towards the same conclusion. If they do or if the results hold across multiple ways of modelling a phenomenon, then we might conclude that a finding is robust. On the other hand, underlying each operationalization of concepts in data is a distinct theoretical rendering of the issue at hand. In the absence of criteria to define equivalences, it is not obvious how varied streams of data can be used to demonstrate evidence of a single underlying phenomenon or entity. In this session we will discuss these issues and the theoretical claims at stake. In so doing, we will bring together discussions regarding research design with considerations of specific methods.

Session 5 Required Readings:

Gilbert, Nigel., & Klaus Troitzsch. 2005. Simulation for The Social Scientist. Maidenhead: McGraw-Hill Education.

Putnam, Robert D. 1993. Making Democracy Work: Civic Traditions in Modern Italy. Princeton: Princeton University Press, pp 12-14, 187-192.

Tarrow, S. 1996. "Making Social Science Work across Space and Time: A Critical Reflection on Robert Putnam's Making Democracy Work." American political science review 90(2): 389–97.

Ekbia, H., Mattioli, M., Kouper, I., Arave, G., Ghazinejad, A., Bowman, T., … Sugimoto, C. R. (2015). Big data, bigger dilemmas: A critical review. Journal of the Association for Information Science and Technology. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/asi.23294/full

Learner engagement

NA

Inclusion and engagement

NA

Participation requirements

NA

Required IT skills

NA

In-unit costs

N/A

Work placement, internships or practicums

NA

• Semester 1, 2021, Intensive, UC - Canberra, Bruce (202582)
• Semester 1, 2020, Intensive, UC - Canberra, Bruce (198070)