Research Methods | People Analytics Lab of the Bayou

AOM 2025 Copenhagen: Building Credibility, Collaboration, and Methodological Rigor in Management Research

Sat, 26 Jul 2025 00:00:00 +0000

Introduction: AOM Goes Global

The Academy of Management’s 85th Annual Meeting (AOM 2025) marked a historic milestone as the first time the conference was held outside North America. From July 25-29, 2025, nearly 13,000 management scholars, practitioners, and students gathered at Copenhagen’s Bella Convention Center to engage with cutting-edge research across the field.

This international expansion underscores the Academy’s commitment to building a truly global, inclusive research community. The program featured approximately 4,000 sessions and symposia, along with 900 poster presentations, addressing contemporary topics ranging from artificial intelligence and entrepreneurship to corporate social responsibility, work-life balance, hybrid work arrangements, and employee wellbeing.

Unlike previous years with overarching themes, AOM 2025 embraced diversity by encouraging innovative, research-focused submissions across all divisions. This approach allowed for rich methodological and topical variety, reflecting the multifaceted nature of modern management research.

Session Highlights: Building Research Credibility

Enhancing Research Credibility & Transparency (PDW #21939)

Saturday, July 26, 2025 | 11:00-13:00 CEST | Bella Center, MR5

Organized by Roman Briker (Maastricht University), Fabiola Heike Gerpott (WHU Otto Beisheim School of Management), and Jost Sieweke (Vrije University Amsterdam), this Professional Development Workshop directly addressed the credibility and integrity crisis in management research.

Alongside Nicolas Bastardoz and Johannes Stark, I presented on practical tools for improving research trustworthiness. The workshop provided hands-on introduction to best practices including:

Pre-registration of research designs and hypotheses
Registered reports for reducing publication bias
Causal identification strategies for stronger inference
Better measurement practices and validation
Data sharing and transparency protocols

The session’s key message was clear: building a “credibility community” requires researchers to advocate for these practices systematically. By improving research robustness and transparency, we can increase both policy relevance and the broader impact of management research.

Collaboration Initiative for Doctoral Students: ARIM Symposium (#20376)

Sunday, July 27, 2025 | 12:00-13:30 CEST | Bella Center, Hall C – C4-m3

This symposium introduced the Advancement of Replications Initiative in Management (ARIM), which I’m proud to be part of alongside Andreas Schwab, Xavier Martin, and William Obenauer. The session featured panelists Yannick Griep, Gerdien Sonneveld, Corentin Hericher, and Caitlin C. Belfiore.

ARIM coordinates joint execution and publication of multi-study replications by teams of doctoral students, faculty mentors, and replication-minded scholars. The initiative addresses a critical gap in management research: the need for systematic replication to build evidence-based management theory.

Key Activities:

Brief presentations outlining ARIM’s operational model
Panel discussion exploring replication opportunities and challenges
Q&A session addressing practical implementation questions

Methodological Innovation: Advanced Latent Variable Modeling

Enhancing Model Precision: Advanced Solutions for Common Challenges (#19189)

Tuesday, July 29, 2025 | 08:30-10:00 CEST | Bella Center, Hall C – C2-m4

This symposium addressed critical methodological challenges in structural equation modeling (SEM) and latent variable modeling, featuring presentations by Andrew A. Hanna, Larry J. Williams, and Jinsoo Choi. I participated alongside Steven Culpepper, Bo Zhang, Susu Zhang, and Yonguk Park.

Key Papers Discussed:

Model Selection Using RMSEA-P Fit Index (Williams, Culpepper & Castille)
- Comparison of Traditional vs. Sequential Model Comparison strategies
- Evidence-based guidance for choosing between approaches
Pseudo-Augmentation Strategy for Underidentification (Choi & colleagues)
- Novel method for addressing underidentification in bifactor predictive models
- Supported by Monte Carlo simulations and empirical examples
Parceling Approaches for Multidimensional Constructs (Hanna)
- Examination of how different parceling strategies affect model fit
- Evidence-based recommendations for practice

Large-Scale Collaboration: The Science of Team Science

Navigating Large-Scale Collaborations: Mega-studies, Crowdsourced Research & Massive Meta-Analyses (#11625)

Sunday, July 27, 2025 | 10:00-12:00 CEST | Bella Center, Hall B – B6-m3

Organized by Bart De Jong and Kevin M. Kniffin, this PDW explored the “science of team science” and strategies for managing large-scale collaborations. Presenters included Kevin M. Kniffin, Katherine Milkman, George Banks, Piers Steel, and Bart De Jong, with Matthew A. Cronin serving as discussant.

Session Focus:

Managing mega-studies and crowdsourced research projects
Coordinating massive meta-analyses across multiple teams
Addressing challenges in authorship, data quality, and resource allocation

Key Activities:

Introductory talks setting the stage for collaboration challenges
Roundtable discussions where attendees shared experiences
Report-back sessions synthesizing insights and best practices

Cross-Cutting Themes: Building a Stronger Research Community

Several interconnected themes emerged across these sessions, reflecting broader trends in management research:

Transparency & Credibility

Both the research credibility PDW and the ARIM replication symposium emphasized that transparent methods and systematic replication are foundational to trustworthy science. These sessions demonstrated how management research is actively responding to calls for greater robustness and reproducibility.

Methodological Rigor

The latent variable modeling symposium complemented the credibility theme by addressing technical challenges that, if unresolved, can undermine the validity of findings. The session provided practical tools for enhancing model precision and resolving common methodological issues.

Collaboration & Scale

The mega-studies PDW and ARIM initiative illustrated how team science and large-scale collaboration can advance knowledge. They also highlighted practical considerations for researchers joining large teams, including coordination strategies and credit allocation protocols.

Community Building

Several sessions aimed to build research communities—the “credibility community” for transparency practices, the ARIM network for replications, and collaborative networks for large-scale projects. This reflects a growing trend toward collective effort and shared norms in management research.

Conclusion

AOM 2025 Copenhagen represented a significant milestone in the Academy of Management’s history as the first conference held outside North America. The event brought together nearly 13,000 management scholars, practitioners, and students to engage with cutting-edge research across diverse methodological and topical areas.

The sessions documented here—focusing on research credibility, replication initiatives, advanced methodological techniques, and large-scale collaboration—demonstrate the field’s commitment to building more robust, transparent, and impactful management research. These initiatives reflect broader trends toward collective effort, shared norms, and enhanced methodological rigor in the management research community.

This blog post reflects on the transformative experience of attending AOM 2025 in Copenhagen, highlighting how the conference is shaping the future of management research through enhanced credibility, collaboration, and methodological innovation.

Cognitively Loaded?

Thu, 19 Dec 2024 00:00:00 +0000

Recently, a colleague asked me to review some experimental stimuli for a study. The issue at hand was how cognitively loaded (i.e., how mentally demanding or effortful) the materials were, and whether the stimuli might unintentionally be a confound in the study by taxing participants’ working memory or attentional resources. In lay terms, complex stuff is much easier to read if you know big fancy words and are not initimidated by trying to understand them. If you don’t understand the big fancy words or find them intimidating, you may make less optimal judgments.

I don’t formally study cognitive load but am familiar with the literature. Still, I fired up Consensus.ai for a quick, systematic review of the literature. The platform identified 1,051 papers, of which 592 were screened, 379 were deemed eligible, and the top 50 most relevant studies were ultimately included. These studies represented a range of designs, including psycholinguistic experiments, dual-task paradigms, EEG and fMRI studies, keystroke logging, recall tasks, and self-report/subjective measures of effort. The full Consensus.ai report is available here.

Across these studies, the evidence was consistent: increased linguistic complexity (longer sentences, abstract vocabulary, complex syntax) elevated cognitive load, as shown in slower response times, higher self-reported effort, and physiological indicators such as pupil dilation and EEG theta power (Cohen et al., 2021; Castro-Meneses et al., 2019; Just et al., 1996; Vogelzang et al., 2020). Importantly, the effects were not uniform. Some studies found that complexity primarily affected speed and effort, while others documented reductions in response validity, particularly in participants with lower working memory, lower language proficiency, or higher anxiety (Révész et al., 2015; Güvendir & Uzun, 2023).

So far, no surprise, but it got me thinking about mission statements. I talk about the importance of mission statements in my HR classes, as well as how designing them may go off the rails. Mission statements are often aspirational, but when written in overly complex or jargon-laden language - often by a well-intended committee - they risk introducing unnecessary cognitive load for the very audiences they are meant to inspire—students, faculty, alumni, and community partners.

Consider Dan Heath’s take on mission statements “How to Write a Mission Statement That Doesn’t Suck” (Fast Company, 2010). Heath provides a perfect example of cognitive load confounds in action. He illustrates how a pizza parlor’s mission statement evolved through committee revisions:

Condition A (Original Mission for a Pizza Parlor):

“Our mission is to serve the tastiest damn pizza in Wake County.”

Condition B (Committee-Revised Mission):

“Our mission is to present with integrity the highest-quality entertainment solutions to families.”

Heath did not talk about cognitive load (why would he as it kinda gets in the way of the point he’s rightly making). That said, I was curious to analyze these conditions using a web tool I created (with the help of Cursor) to measure the cognitive load of text extractions. The app is freely available at: cognitive-load-analysis.shinyapps.io. The tool incorporates multiple validated readability indices (Flesch, Gunning Fog, SMOG, etc.) and cognitive complexity measures based on sentence length, word length, and overall readability scores, using the quanteda and quanteda.textstats R packages for text analysis. Not surprisingly, the cognitive load metrics revealed differences that you would expect: the committee-revised mission statement was more cognitively loaded, containing more abstract terms “entertainment solutions” versus concrete “pizza,” and corporate jargon (“present with integrity,” “highest-quality”) versus plain language (“tastiest damn”). More specific details are below.

Flesch Score Gap: 63.5 points (the committee version is rated “very difficult” to read compared to the original, which is “fairly easy” to read)
Word Count Difference: Only 1 word (14 vs 13), yet vastly different complexity. The words are longer in the revised mission statement.
Cognitive Load Gap: 19.7 points (basically the commitee version requires more cognitive processing than the original mission statement)

In prioritizing sounding professional over being understood, a well-intended committee makes it harder for their employees to get behind the company’s mission. In other words, “entertainment solutions” should be swapped for “tastiest damn pizza” (or something close to it – like ‘darn pizza’)!!!

References

Castro-Meneses, L. J., Kruger, J. L., & Doherty, S. (2019). Validating theta power as an objective measure of cognitive load in educational video. Educational Technology Research and Development, 68(1), 181-202. https://doi.org/10.1007/s11423-019-09681-4

Cohen, M. L., Boulton, A., Lanzi, A., Sutherland, E., & Pompon, R. H. (2021). Psycholinguistic features, design attributes, and respondent-reported cognition predict response time to patient-reported outcome measure items. Quality of Life Research, 30(6), 1693-1704. https://doi.org/10.1007/s11136-021-02778-5

Güvendir, E., & Uzun, K. (2023). L2 writing anxiety, working memory, and task complexity in L2 written performance. Journal of Second Language Writing, 62, Article 101016. https://doi.org/10.1016/j.jslw.2023.101016

Heath, D. (2010, February 24). How to write a mission statement that doesn’t suck [Video]. Fast Company. https://www.fastcompany.com/1404951/how-write-mission-statement-doesnt-suck-video

Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F., & Thulborn, K. R. (1996). Brain activation modulated by sentence comprehension. Science, 274(5284), 114-116. https://doi.org/10.1126/science.274.5284.114

Vogelzang, M., Thiel, C., Rosemann, S., Rieger, J., & Ruigendijk, E. (2020). Neural mechanisms underlying the processing of complex sentences: An fMRI study. Neurobiology of Language, 1, 226-248. https://doi.org/10.1162/nol_a_00011

Benoit, K., Watanabe, K., Wang, H., Nulty, P., Obeng, A., Müller, S., & Matsuo, A. (2018). quanteda: An R package for the quantitative analysis of textual data. Journal of Open Source Software, 3(30), 774. https://doi.org/10.21105/joss.00774

This work represents a collaboration between traditional psycholinguistic research methods and modern AI-assisted analysis, motivated by practical research needs and grounded in established cognitive load theory.