ingorohlfing

[ingorohlfing]

Does uncertainty undermine statistical power analysis?

The post is about a paper titled Uncertainty limits the use of power analysis. If you want to avoid the paywalled version on the APA website, there is also an ungated preprint. I came across the paper through this mainly critical blog post. I think there are good points in both texts and some arguments that are taken too far. On the article: I am skeptical about the severity of the two issues…

[ingorohlfing]

Open Science is neither passé nor is it there yet

Is Open Science passé? is the question asked by Xenia Schmalz in this blogpost. I recommend reading it before I share brief thoughts on some points that are raised. I wish an open science movement was not needed anymore, but I agree this is most likely not the answer to the leading question. Neither has the open science movement failed; progress toward more transparent and credible science is…

[ingorohlfing]

"Resolving empirical controversies with mechanistic evidence" - Some thoughts

Recently, I came across the open access article Resolving empirical controversies with mechanistic evidence. I have some thoughts on the arguments made in the article, but first a disclaimer: This is not meant to be a disciplinary beauty contest. The article raises a valuable point: Two or more quantitative models may produce contradictory results. Depending on the research question at hand, it…

[ingorohlfing]

Citation-infusion of research papers with AI?

I came across the AI company Sourcely these days. It promises to “Finish Your Research in Minutes. Save Your Sleep. Paste your essay to find, summarize, and add credible sources. (That’s something Google Scholar can’t do!)” Saving time for sleep is good and if it would work, it would outperform GS, or any traditional search for citations and references, for that matter. However, I have some…

[ingorohlfing]

The peer review process as a preregistration device

The peer review process as a preregistration device

Preregistration of an analysis requires the prior unobservability of data. One publicly declares what one intends to do before one can access data. One goal of preregistration is to diffuse potential concerns that a publication misrepresents the order in which an empirical analysis was implemented. Reasons for prior unobservability of data could be that the data has yet to be produced (as in…

[ingorohlfing]

Confirmation bias in causal qualitative research

Confirmation bias in causal qualitative research

The bi-annual publication of the APSA Section on Qualitative Methods and Multi-Method Research has a highly interesting and controversial symposium on confirmation bias in process tracing in its current issue. I have written a Twitter thread on the question of what the evidence for confirmation bias in qualitative research is. Thanks to the app Threadreader, the thread can be read conveniently…

[ingorohlfing]

Gap-filling, puzzle-solving and replication in empirical research

Gap-filling, puzzle-solving and replication in empirical research

A common justification of empirical research is that a gap in the literature needs to be filled. I have seen this reasoning in all kinds of papers and presentations by Master students, PhD researchers, Postdocs and professors (probably the less, the more advanced the academic career is). It is not wrong to justify a study with gap-filling, but it is also does not seem to be the most exciting…

[ingorohlfing]

Theoretically non-exclusive hypotheses in Bayesian process tracing

Theoretically non-exclusive hypotheses in Bayesian process tracing

Inspired by an email exchange I had with someone on theoretically non-exclusive hypotheses in Bayesian process tracing, I believed it might be useful to write down some thoughts in a blog post. It ended up as a PDF on Github because it has a minor R element and formal notation.

[ingorohlfing]

"Context" is important, but (almost) useless if used as a causal category

“Context” is important, but (almost) useless if used as a causal category

When making causal (or descriptive) inferences, it is important to think about the context within which the causal relationship is expected to hold because it probably does not hold universally and, possibly, only in a limited setting. Falleti and Lynchhave written an excellent article about “context” (I drop the “” now) in relation with causal mechanisms. Personally, I do not like the use of…

[ingorohlfing]

Correlation vs causation: The case of competitive funding and research quality

Correlation vs causation: The case of competitive funding and research quality

On September 27, the German Science Foundation (DFG) announced its decision to award the status of a research cluster of excellence (Exzellenzcluster) to 57 cluster proposals from all disciplines. This was the first step of its so-called excellence strategy, (Exzellenzstrategie/ExStra, formerly known as the Exzellenzinitiative/ExIni). Each cluster receives about 7 to 12 million Euro per year,…

[ingorohlfing]

The COMPASSS statement and QCA solution types

The COMPASSS statement and QCA solution types

About two weeks ago, COMPASSS issued a Statement on Rejecting Article Submissions because of QCA Solution Type. In short, the reasoning was that methodological work on QCA is developing and that reviewers and editors should not judge empirical work based on whether one particular solution type is interpreted as causal. (Disclosure: I am a member of the COMPASSS advisory board, but was not…

[ingorohlfing]

The relevance of Political Science: Some thoughts on the recent critique

The relevance of Political Science: Some thoughts on the recent critique

The charge that Political Science (or other non-STEM disciplines) is lacking relevance and does not produce interesting research is made then and again, with two new pieces published these days. One is written by a political economist, stating that most research is boring; one is written by a German political scientist echoing this claim and arguing that political science lacks relevance (written…

[ingorohlfing]

Review of 'Multi-Method Social Science' (Seawright, CUP) - Chapter 2: Causation as A Shared Standard

Review of ‘Multi-Method Social Science’ (Seawright, CUP) – Chapter 2: Causation as A Shared Standard

Continuing the chapter-by-chapter review of Seawright’s book on Multi-Method Social Science took me longer than I imagined and it should have, but here we go again. The second chapter discusses the fundamentals of multimethod research (MMR) and identifies “Causation as a Shared Standard”. One might think that the fundamentals can be quickly addressed because of the now established distinction…

[ingorohlfing]

Review of "Multi-Method Social Science" (Seawright, CUP): Chapter 1

Review of “Multi-Method Social Science” (Seawright, CUP): Chapter 1

Over the next weeks, I will give a chapter-by-chapter review of the new book Multi-Method Social Science: Combining Qualitative and Quantitative Tools by Jason Seawright. To my knowledge, it is the first book on nested analysis/multimethod research in the social sciences that is not an edited volume. A debate about mixed methods has been going on in adjacent disciplines for decades, but the…

[ingorohlfing]

People require preregistration, not methods

People require preregistration, not methods

Many APSA 2016 panels and discussions in the Section on Qualitative and Multimethod Research and the Political Methodology Section were centered on the Data Access and Research Transparency (DART) Initiative (probably worth a blog post of its own). Even panels not explicitly dedicated to DART have digressed into this topic, which includes a short exchange between Tasha Fairfield and me in a panel…

[ingorohlfing]

Don’t conflate fuzzy set membership with cases in QCA

Don’t conflate fuzzy set membership with cases in QCA

In 2000, Ragin added the idea of fuzzy sets to Qualitative Comparative Analysis (QCA) and they have been widely used since then. However, one sometimes still finds misunderstandings about what fuzzy sets are, in particular when it comes to the interpretation of consistency and coverage scores. In a criticism of QCA by Stockemer (which is criticized for a variety of reasons and to which Stockemer…

[ingorohlfing]

Curiosities of QCA: Fuzzy-set consistency

Curiosities of QCA: Fuzzy-set consistency

Qualitative Comparative Analysis (QCA) is the method of choice for the analysis of set relations and has changed considerably and improved over the years. The more one delves into the method, however, the more things you (I, at least) stumble upon that seem a little bit curious, sometimes also more than a bit (just as with any method, I guess). One issue I came across relates to the calculation…