What Gets Measured Gets Done.
I used to assume that if a paper was published, it was high quality.
The more I’ve spent time with academic researchers, the more I’ve realized that assumption doesn’t always hold.
I am learning that Academia is a system designed to celebrate quantity, not necessarily quality. There’s a phrase I hear a lot these days… “Publish or Perish.”
Success today in academia is measures in citation counts, impact factors, journal prestige, and author order. These signals are easy to track and widely understood. But they are not the same as quality.
They measure visibility, not rigor.
They measure time on task, ignoring entirely the time to mentor, teach, or thoughtfully review peers.
They do not leave space for parental leave or disruptive life events, regardless of your gender.
They reward getting the next paper out the door.
And so, we end up in a system with two critical risks.
The first risk is quality.
Something can be highly cited and still quietly questioned. A paper can pass peer review and still raise concerns once it reaches a broader audience. I believe this is a systemic issue, not the fault of any individual.
The 2002 Women’s Health Initiative study on hormone therapy is a well-known example.[1] The way this paper’s findings were interpreted led millions of women to stop treatment, often without nuance around age, timing, or formulation. Much of that nuance existed within the scientific community at the time, and has grown in the years since, but it was not visible in a structured, accessible way.
This is where new approaches are starting to emerge. PaperStars is one example.[2]
PaperStars creates space for ongoing, post-publication evaluation, where researchers can rate papers and leave short, constructive reviews, bringing visibility to the kinds of discussions that already happen informally. It does not replace peer review, but complements it with a living layer of community judgment.
It moves us from a binary question of “Was this publishable?” to a more useful one: “How strong is this work, really?”
In the WHI example, a platform like PaperStars would not have changed the study itself, but could have drawn attention to the way risk in the study was communicated. Relative increases were often highlighted without equal emphasis on absolute risk, making the findings feel more dramatic than the underlying absolute risk. And open space to capture expert perspectives in real time, and to distinguish between methodological strength and applicability, might have impacted how media ran with the data back in 2002. Perhaps…
PaperStars is still new, and like any system, its value will depend on how it is used.
The second risk is the focus on quantity.
We have built a system that celebrates throughput. More citations signal success. For many researchers, especially those navigating complex careers and competing demands, this misalignment is not abstract. It shapes what work gets prioritized. It influences how time is spent. It can quietly reward speed and visibility over depth and rigor.
This misalignment is especially visible for caregivers. While there are growing efforts to support parents in academia, including organizations like Mothers in Science, much of the focus remains on helping individuals navigate the system.[3] Fewer efforts address the underlying incentive structures that continue to reward output above all else.
It has been three years of learning from and engaging with academics. I see both the challenges and the potential for meaningful change.
Next Steps
What is one small step we can take to better align how we measure success with the quality of the work itself?
[1] Beverly, Rachel, and Judith Volkar. “Risks and Benefits of Estrogen Plus Progestin in Healthy Postmenopausal Women: The Women’s Health Initiative.” doi.org/10.1093/med/9780190947088.003.0048. (PaperStars, Not Yet Reviewed/0)
[2] Paperstars.org
[3] Mothersinscience.com
