I love getting email from prospective students and postdocs. Recruiting to the lab is one of the most consequential things I do as a professor, which is why I try to make it as easy as possible: our lab website has a dedicated page with instructions for anyone who is interested in joining. There are specific instructions for each career stage, but it all boils down to this: just send me an informal email, in which you tell me about yourself and why you want to join the lab. Very simple, and this worked well for years; it is how I recruited essentially every postdoc and most of the graduate students and staff who have come through the lab.

However, sometime in the past year, things changed. The volume of emails from prospective lab members increased substantially, and I now regularly receive 20 such emails a day. The problem is that almost all of them are AI-generated. I am not alone in this: many colleagues report very similar patterns, where it is now hard to manage the deluge of AI-generated emails showing interest. There are also reports from other professional communities - for example, book authors are reporting a huge increase in various AI-generated emails (although in their case, many of these are scams). Globally, more than 50% of spam email is now AI-generated.

I’m not saying that AI-generated emails showing interest in the lab are spam. But I realized that, for some reason, I cannot bring myself to read them. This happens every time:

And it’s not just me:

This kind of instinctive reaction surprised me, because I am generally pro-AI across most use cases. So I have been trying to understand why this particular thing triggers this particular reaction. Writing this down turned out to be a useful thought exercise for me. This is also a good explanation to anyone who emailed me and haven’t hear back, as well as a way to explain to future potential emailers why they should drop the AI and use their own voice. Here are the reasons I am not reading your AI-generated email:

1. You did not spend time writing it, so I am not going to spend time reading it.

This is the core issue. Writing an email takes effort. That effort itself is a signal that the writer thought about what they wanted to say and decided it was relevant. They thought - even for a few minutes - about why they are a good fit for the lab, how their career would flourish here, what they can contribute. An email that was generated with AI completely eliminates this signal. It tells me the sender spent approximately zero time on this communication. I do not think it is unreasonable to respond in kind.

2. There is nothing original in it, and it hides your real voice.

Every one of these emails looks exactly the same. Exact same structure, register, length, etc. This is not surprising; an LLM, by definition, will generate an average of every cold email ever written, and the result is that all these emails are indistinguishable from each other. This is not just an aesthetic issue: A recent study examining the rollout of an AI cover-letter tool found that after its introduction, the correlation between cover letters and hiring callbacks fell by 51%, because AI eroded the cover letter as a meaningful signal. This is too bad, because the cover letter (or cold email) is one of the only ways for a candidate to distinguish themselves by using their own writing and actual voice. When you use AI, I have no idea who you are - you sound exactly like everyone else.

3. There is zero thinking involved.

I have written before about how writing is the only way to do structured thinking. When you sit down to compose an email showing an interest in joining a lab, you are forced to ask yourself real questions: why this lab, why this work, why now, what do I actually want. A fully AI-generated email tells me this process was likely skipped, and also raises an unsettling possibility: maybe an agent generated a list of labs, sent emails to all of them, and the human candidate is not even aware my lab is on the list. I have no way of knowing.

Ran’s Substack

Just start writing

If you ask a working scientist in the life sciences what they actually do for a living, writing is rarely the first thing they mention, and often not even in the top five. And it’s clear why: in most labs, the unofficial hierarchy puts the experiment and the analysis well above the manuscript that eventually comes out of all of it. Writing is relegated …

Read more

5 months ago · 15 likes · Ran Blekhman

4. I can’t really trust it.

Related to the last point, but if the email was generated by an LLM, I have no way of knowing if any of the information in it was reviewed for accuracy by a human. If the email tells me that you’ve read our latest paper and found it relevant, I have to ask myself, “have you really”? If the emails says that your previous work was relevant to our lab’s research, do I need to go through your CV and verify this is true? This is a strange starting position to be in when I need to think about investing years in someone’s training.

5. It’s the only thing I know about you.

This cold email is usually the first (and often only) contact with a potential candidate. So the only data point I have about this candidate is this: “they had an important task that could determine the rest of their career, but it required a little effort and personal reflection, so they used AI to complete it.” Is that really the one thing you want a potential advisor to know about you?

An important note is that this is not an argument against AI tools. The line I am drawing is between using AI to improve and polish writing you have done, and using AI to avoid writing and thinking altogether.

So I have updated the lab’s recruiting page to say this directly: emails that are clearly AI-generated are unlikely to receive a response, and that note links to this blog post to explain the rationale:

A note on AI: I read every email myself, and I am much more likely to respond to one that you wrote in your own voice and your own words. Emails that are clearly AI-generated are unlikely to receive a reply (here is why).

As a final thought, Henry Shevlin, an AI ethics professor, recently reported receiving an unsolicited email from an AI agent seeking paid freelance work:

So we are moving towards a future where an AI agent is not only writing the emails on behalf of a person, but rather an AI agent writing on behalf of itself, asking to join a lab. Honestly, I don’t know what to make of it, and am not sure if I’m really looking forward to this lab of the future with AI-agent staff. All I know is that right now, I prefer to interact with you, as a real human, with your human writing, and your human thinking.

On Wednesday I shared a paper on Bluesky. Nothing out of the ordinary; sharing and learning about new papers is one of my favorite aspects of social media, so when I saw a paper I thought was interesting published in Nature that day I shared it with a short note. The paper, Miao et al., 2026, describes a framework that converts a scientific paper into an MCP server that an AI agent can query. I think this is a promising approach: it may force authors to package their code, data, and metadata into a runnable pipeline, rather than the static PDF in addition to a half-working GitHub repo, which could improve reproducibility. It might also give readers a way to interrogate the methods interactively, which can be especially useful in fields like genomics, where methods sections can be many dozens of dense technical pages and are often hard to reproduce from the text alone. So I posted about the paper without thinking too much.

The response was jarring. Hundreds of vitriolic replies and quotes, with a level of collective incandescence I completely did not anticipate.

But considering the state of AI discourse in academia, maybe I should have?

To be fair, my post was over-enthusiastic. The realistic case for tools like this is more measured than what I wrote, and if I could edit the post I would definitely soften the “future of scientific papers” framing. But this is not what drove the response volume: the most-endorsed replies, with hundreds of likes, framed my post as an endorsement of a system that teaches students not to read; this interpretation is not what the paper (or my post) proposed at all, but once it took hold everything downstream was a response to the strawman rather than to the actual paper. A minority of the responses were substantive; some were just dismissive. But many were personal attacks, including curses, bullying, xenophobia, calls to take away my funding (LOL), and telling me that I should quit my job (don’t tempt me).

I am not writing this to complain about the nasty personal responses to this post (I have been called worse by peer reviewers). I am writing because this response fits a broader pattern in how academia now reacts to anything AI-adjacent: a toxic, knee-jerk hostility to AI in which the specific tool, application, or question all disappear into a general presumption that the whole category is terrible. This is a common occurrence; for example, the political scientist Alexander Kustov wrote a measured AI-positive piece last year and in return received “professional threats, ad hominem, and coordinated pile-ons,” including calls to his employer to fire him. This is also not a Bluesky-specific problem1 or a social media-specific problem. The anti-AI issue in academia is much broader. You can see it in grant reviews, hallway conversations, paper reviews, Q&A sessions. Mentions of AI are often met with suspicion, with little care about the specific methodological question or model.

And I can understand why. AI has real limitations. The environmental cost may be substantial, the over-claiming from industry leaders is embarrassing, and there are legitimate problems with the actual technology: hallucinations, brittleness in unfamiliar domains, uneven distribution of benefits, unresolved questions about the consent of training-data use. I have also argued before that AI on its own will not speed up research in biology. At the same time, AlphaFold is a remarkable scientific advance that changed the entire field of structural biology; language assistance has lowered the barrier for scientists whose first language is not English; code generation has compressed a lot of routine analysis work. Importantly, “AI” is not one thing: an LLM drafting a paragraph is not the same technology as a variational autoencoder learning a latent representation of single-cell data, which is not the same as AlphaFold. Applications, users, and purposes vary just as widely. Having a single opinion about “AI” is like having a single opinion about “software.” The only serious position on AI in science is nuanced and case by case; any answer to “is AI good for science” that neatly fits into a single yes or no is unserious.

But, unfortunately, somehow AI has been sorted into the same tribal buckets as every other contested topic in public life. It functions less as a technical subject and more as an identity marker; which side you are on tells people who you are, before they hear what you think about any specific system. This is clearly, at least in part, due to social media, which rewards affect over analysis and surfaces the highest-arousal takes. People whose brains have been fried by spending hours a day marinating in the most extreme framings end up reacting to “AI” the way people react to other loaded political words: with a visceral response that precedes any specific claim. A software tool for pattern recognition has been recoded into a morality test.

This is bad for science.

Scientific communities used to be places, at least on their better days, where a specific technical claim could be evaluated on the specific technical merits. We can go back to this. We can treat AI as a normal technology for doing science, and evaluate specific AI methods on their merit, which is what science does for every other tool. For example, an MCP wrapper of genomics papers can improve accessibility and reproducibility of data analysis; it does not mean people are not going to be allowed to read papers. An AI-based preprint ranking system I reviewed recently reproduces the prestige hierarchies it claims to escape; but it does not mean AI is going to replace scientific journals. It’s possible to review specific AI systems without a claim about the global societal impact of AI.

Importantly, when the response to a mildly AI-positive share is a toxic pileup, people learn: they stop posting about AI, or never join the platform to begin with. The thread that started this is a good example: there were no actual AI scientists anywhere in the discussion, and the loudest amplified responses came from scholars in humanities with no knowledge in genomics, or from political accounts with no scientific affiliation at all. These hostile reactions have essentially eliminated any scientific AI content from the platform.

You might think, “OK, stopping the AI discussion is actually good for science!” But your toxic reaction to anything AI does not stop the conversation from happening. It only takes you out of it. The conversation continues without you elsewhere. Science is still moving forward, building AI tools, using AI, discussing AI. The only thing changed by your reaction is whether your voice is included.

So here is an invitation to join the real discussion: you are allowed to have a nuanced opinion about AI.

MIT seems to agree. This week MIT president Sally Kornbluth sent a letter to the MIT community announcing the final report of the Ad Hoc Committee on AI Use in Teaching, Learning, and Research Training. The report itself is overall very insightful, and I found this part refreshingly blunt:

This is not an optional exercise. We must demonstrate how to integrate AI thoughtfully and deliberately into the classroom, the research enterprise, and the experience of residential education in ways that ensure learning, advance discovery, and prepare students for the future.

There is a lot to do.

The report is worth reading. It is direct about the fact that AI is fundamentally changing MIT’s educational experience, that assessments built around a take-home essay or problem set no longer measure what they used to, and that trust between students and instructors is eroding as each side worries about how the other is using AI (the committee calls it an “underground river of mutual suspicion,”). On the research side, the report takes seriously the risk that faculty will start replacing undergraduate researchers and RAs with AI agents, cutting the next generation of scientists out of the research training and mentorship pipeline. It also calls for institutional norms around AI disclosure in theses and other research work.

Importantly, the report is unambiguous about the response: individual instructors and scientists cannot be left to invent their own AI policies from scratch. The committee calls for the institution itself to provide coordinated guidelines, sustained resources, and permanent internal structures to support instructors and researchers as the technology continues to change. The report also states that many universities are working on similar questions, but “no one seems to have it all figured out.” Understatement of the year?

As an example, at Harvard Business School, a new entrepreneurship bootcamp is now built around AI avatars of its own faculty (which is definitely creepy, but that’s beside the point). At the same time, Harvard is also closing its Writing Center and laying off its director, a decision that faculty and alumni have pushed back on, given that writing instruction is being upended by AI. A similar contradiction is visible at my home institution, the University of Chicago, where free Claude Enterprise access was rolled out to all faculty, staff, and students; while the Social Sciences Collegiate Division declared their core courses as “analog” for the coming academic year, banning laptops, phones, and AI use in class and on coursework. The Law School announced a comparable ban for its first-year core courses earlier this summer. At California State University, the same incoherence shows up at a larger scale: CSU renewed its contract with OpenAI for $39 million to provide ChatGPT Edu to their almost half a million (!) students, as well as to faculty and staff. At the same time, according to the CSU AI Survey Report, 40% of its faculty either forbid or discourage AI use in the classroom.

The scale of the mismatch is visible not just among faculty, but also in students’ own reports. In the recent Higher Education Policy Institute survey of UK undergraduates, 95 percent of respondents report using AI in some form. When asked whether their own institution encourages that use, they split almost evenly: 37% agree or strongly agree that their institution encourages the use of generative AI, while 36% percent disagree or strongly disagree. This is what institutional incoherence looks like from the receiving end.

So universities are giving students free access to generative AI tools, which naturally reads as encouragement to use them broadly. Then, where they issue any concrete rules at all, those rules are usually outright bans. What is missing is anything in between: coherent guidance on when and how to use these tools, disclosure norms, and support for the researchers, instructors, and students trying to figure this out themselves.

I’ve been thinking a lot about why this is, and I think much of it comes down to speed and agility. The default university response to a novel challenge is well-known to anyone who has been on a faculty committee: form a task force, commission a report, create a second committee to implement, pilot the recommendations in one unit, fix issues, and revisit in a year. It takes years to achieve an institution-wide change. AI, in contrast, changes on the timescale of weeks, with new model releases, behaviors, capabilities, and cost structures that can shift substantially within a single academic semester. The MIT committee report took 8 months to write; I am not sure how long it will take for the recommendations to be implemented, but by then, the AI landscape will have changed so much that the recommendations may no longer be relevant.

Given this lack of guidelines, it is not surprising that many PIs and instructors are already stepping in by necessity, drafting local policies, buying local tools, and setting local norms. This is also why I recommend every lab should create its own AI use guidelines. But the issue with pushing the response down to labs and instructors is that the risks do not stay local. An inappropriate use of AI in a graduate admissions review, an undisclosed use in a grant proposal, or a data leak through a private account becomes an institutional liability regardless of where it originated. The benefits also are minimized without shared infrastructure, training, and a policy that students can rely on across courses and departments.

Bill Gates makes a related argument in his recent essay on the AI transition, which is also a highly recommended read. His diagnosis of our current state is somewhat bleak:

I don’t see evidence that leaders, experts, and communities are confronting the challenges adequately. There is no plan to ease the entry into the AI era.

Although Gates is writing about the response to AI at the national and international level, the same failure is visible at the university level. His prescription is also worth noting:

The highest priority is a monumental task: creating a domestic and international framework for dealing with AI. None of our current institutions were designed to handle a technology that spreads so fast and touches so many parts of our lives. So we’ll need to make new ones.

This also perfectly applies to universities, who were not designed for a technology that changes on this timescale, and the standard institutional responses cannot handle the challenge. Universities need the same thing Gates is asking for at the national level: new institutional structures that can actually move at the pace AI is moving.

You might say, right, this is just the pace universities are moving - they’re incapable of moving faster. This is not true: in March 2020, when COVID hit, universities moved entire course catalogs online within weeks. Academic integrity policies, exam procedures, grading systems, admissions timelines, housing, health protocols, and the fundamental structure of how classes were taught were all rewritten on very short timescales. The changes were definitely messy and sometimes badly designed, but they happened. What this tells me is that the multi-year cycle I described above is not a hard constraint, it’s just what universities do when they do not perceive an emergency. Universities are choosing to see AI as a slow-moving policy question rather than a fast-moving structural emergency. Instead of treating AI like a pandemic, they treat AI like a curriculum revision.

Universities are slow by design, and that design has served the durability of ideas well for hundreds of years. However, for the current moment, this glacial pace of change is working against them. For AI, universities should adopt a more agile approach to create structural change that can actually keep up with the technology. Or, they can form another committee that will produce another report a year from now to sit on a shelf while the world moves past them.

A few weeks ago, Nature published two papers that are being described as landmarks in AI-driven biomedical science. Google DeepMind introduced Co-Scientist, a multi-agent AI system that generates research hypotheses, proposes experiments, and analyzes data. FutureHouse, a nonprofit AI research lab, introduced Robin, a similar system that identified a candidate treatment for dry age-related macular degeneration, a leading cause of blindness. Both systems accomplished in hours what would typically take a research team weeks or months of literature review, hypothesis refinement, and data interpretation.

The framing from the developers matches the ambition of the work. FutureHouse CEO Sam Rodriques said that “AI is going to massively accelerate the pace of science” and “increase productivity”, allowing scientists make more discoveries much faster. The company’s more recent system, Kosmos, is claimed to perform the equivalent of six months of research in a single day, on the basis of estimates from early users. These two papers are the latest entries in a genre that the leaders of the field have been developing for several years. Demis Hassabis, who shared the 2024 Nobel Prize in Chemistry for AlphaFold, describes the achievement as the equivalent of “a billion years of PhD time done in one year”. Dario Amodei, the CEO of Anthropic, predicts that AI-enabled biology will “compress the progress that human biologists would have achieved over the next 50 to 100 years into 5 to 10 years”. There are other examples, but the common thread we’ve been hearing from AI leaders is that (1) AI will dramatically accelerate productivity in science, and (2) it will do so soon.

I want to push back on this. I think we’re seeing a pattern we’ve seen before, and it did not play out the way the technology’s champions predicted.

2. The productivity paradox returns

A good framing of this pattern actually comes from economics, and is described in this recent piece in Fortune. Briefly, in 1987, the economist Robert Solow made an observation that became well-known in economics: “You can see the computer age everywhere but in the productivity statistics.” Although firms had been investing heavily in information technology, the measured productivity has not grown. This gap between obvious technological change and absent economic gain became known as the productivity paradox. Economists are now reviving this idea to describe the AI boom. In a study of business executives, about 90% of firms reported that AI had made no measurable difference to productivity over the past three years. As in the 80s, a transformative technology arrives, everyone agrees it will change everything, but the productivity doesn’t actually improve.

The original paradox did eventually resolve, and IT-driven gains arrived in the late 1990s. But the lag was over a decade long, and the benefits were unevenly distributed. Importantly, the gains ultimately required broad structural changes in how organizations worked.

I believe science and academia will follow the same trajectory, and possibly a worse one. The reasons are structural, and they have little to do with how good the AI is.

3. AI does not solve the real bottlenecks

What AI automates is useful: searching the literature, generating hypotheses from published findings, analyzing data, generating models to explain complex systems, editing text. These are all important tasks that consume real effort; however, these tasks are not, for most research programs, the rate-limiting step.

The actual bottlenecks in biomedical science are physical, institutional, and human. Getting a grant funded takes 12 to 18 months from submission to award, and that assumes it is funded at all; most proposals require years of revision and resubmission. Recruiting human participants for a clinical study takes years, and enrolling a cohort with rarer characteristics can take longer still. The physical constants of biology are indifferent to how quickly a model can read: cell lines must be cultured, samples collected and shipped, sequencing runs queued and completed, and mouse colonies bred to the correct genotype over many months, with some disease models yielding a first meaningful readout only after the animals have aged. Experiments fail and have to be repeated, protocols need optimization, reagents and specialized instruments have lead times, and shared core facilities have waiting lists. Layered on top of all this is an institutional apparatus that runs on its own clock: IRB approval for human subjects and IACUC approval for animal work, biosafety approvals for pathogens, data use agreements and material transfer agreements with legal back-and-forth between universities, and controlled-access approvals for protected genomic data all take many months. Another bottleneck is human: training a scientist to independence takes years, recruitment and hiring is slow, and visa and immigration delays for international trainees add a layer that can take months to years of bureaucracy. Peer review adds still more delay: revisions can take years, and even finding scientists willing to review a paper now takes months. All these processes, physical and institutional and human, set the real pace of research. Currently, AI improves almost none of them.

This is a well-known principle in engineering and operations research, sometimes called Amdahl’s law: the speedup achievable by optimizing one component of a system is limited by the fraction of total time that component represents. So if literature review and hypothesis generation account for 5% of a research project’s timeline, then even a 100X speedup in those tasks reduces the total project duration by less than 5%. The bottleneck just moves downstream, to the parts of the process that were already slow.

The most instructive illustration of this comes not from a skeptic but from a company built on the opposite premise. A couple of weeks ago, a San Francisco startup called Capable announced that it has built a laboratory capable of going from AI-led drug design to experimental data in 24 hours:

If you scroll through the X thread, what is striking is the description of how they achieved this. Their platform consists of six modules, only one of which is AI-related (drug design); the others are machine learning infrastructure, rapid synthesis, quality control, cell testing, and mouse testing. To compress the timeline they built an in-house in vivo facility, obtained pre-approved protocols, and put veterinarians and in vivo researchers on standby. They are now hiring automation engineers to integrate liquid handlers and robotic arms in order to, in their own words, unlock bottlenecks.

I do not raise this as a criticism of Capable, which appears to understand the problem clearly and is spending money and effort on the actual bottlenecks. I raise it because a company whose entire thesis is that AI will accelerate drug discovery has concluded that the way to accelerate drug discovery is to restructure the wet lab. The speedup did not come from the AI, but from a vivarium, pre-approved protocols, veterinarians on standby, and wet-lab pipelines re-engineered to run faster.

4. To accelerate science with AI, institutions must change

I am not arguing against using AI in research. The tools are powerful and I do believe will eventually transform science. My issue is with the expectations around AI.

When AI companies and prominent commentators claim that AI will dramatically accelerate science, university administrators hear that they can expect more output1 from fewer people. Funders hear that AI might offset the cuts to NIH budgets that are currently suffocating the research enterprise. Trainees hear that reading deeply and analyzing carefully matter less than learning to prompt a model. All of these conclusions are wrong.

To anyone inclined to answer all of this by noting that the original paradox did eventually resolve: it did, but not on its own, and not merely with time. It resolved when organizations restructured how work was done, retrained the people doing the work, and redesigned processes around what computers were actually good at. Organizations did not just wait for gains from technology to arrive; they changed the entire system to remove bottlenecks and make the gains possible.

For science, this points to a clear and somewhat unglamorous conclusion. If the real bottlenecks are the ones described above, then the way to capture gains from AI is to work on those bottlenecks directly. That is largely the job of institutions rather than individual scientists or better models. Funding agencies can shorten the grant cycle and reduce the administrative burden that consumes so much of a researcher’s time. Universities can speedup IRB and IACUC review, streamline the contracting and data use agreements, and invest in the shared infrastructure, core facilities, and technical staff. Journals and funders can rethink a peer review system that is already overloaded and is only coming under more and more strain. The companies most invested in AI for science understand this; it is time for academic institutions to reach the same conclusion.

This is the problem that no model will solve for us. AI that analyzes a dataset 10X faster moves a project forward only marginally, but a proposal funded within 4 weeks of submission cuts years of wasted time. The same is true for a manuscript reviewed and published within 2 weeks of submission. What would transform science are institutions that are willing to do the work and solve the real bottlenecks. The gains from AI tools will eventually be substantial, but they will only be unlocked by deliberately fixing the surrounding system.

So when QED Science launched “The 1%,” a ranked list of the top life science preprints of the past year scored entirely by AI, I was genuinely interested. The goal, evaluating science on its merits rather than on journal prestige or institutional reputation, is one I share. I also think that AI can be an incredibly powerful tool to extract useful information from large amounts of noisy data (like preprints), and appreciate every attempt to disrupt the current publishing system.

But the more I looked at it, thought about it, and explored the data, the more uncomfortable it made me. Here is my full peer review of The 1%.

1. The methods are proprietary, self-validated, and poorly benchmarked

QED Score’s algorithm is a black box. The company describes a “multi-agent AI architecture” that decomposes papers into claims and evaluates evidence, but the actual implementation is not available for inspection, replication, or critique. For a tool that positions itself as a replacement for peer review, the absence of methodological transparency is a fundamental problem.

In addition, the validation study was conducted and reported by QED themselves, on a corpus of their own choosing, with an expert panel they assembled. None of this has been independently peer-reviewed. It is difficult to trust a commercial system validated by the people selling it, which at first glance seems like a conflict of interest.

The benchmark they chose to beat is journal rank, a metric that essentially everyone already agrees is a poor proxy for scientific quality. Outperforming a flawed baseline is not the same as being good. The relevant comparison would be against actual domain expert peer review, conducted independently.

2. The “prestige-free” framing does not hold up

QED’s central claim is that the system evaluates science independent of institution, identity, and reputation. To look more into this, I downloaded the information for the full list of 1% preprints and counted institutional affiliations across all authors. The institution breakdown is telling, with the familiar high-prestige institutions showing up at the top: HHMI, Harvard, Stanford, MIT, Columbia, the Broad Institute, etc. The top of the list is nearly indistinguishable from a conventional “prestige” ranking or U.S. News & World Report. Also worth noting: nine of the top 10 are from the US, and 18 of the top 20 are from either the US or Europe. So the 1% list reproduces the same prestige hierarchy with the same biases it claims to replace.

This is not surprising once you consider how the model was built. The system was trained on expert feedback, from experts embedded in exactly these institutional networks. The biases do not disappear; they get encoded into the model and then laundered into outputs that feel objective because they are numerical. Replacing one prestige filter with another that is harder to see or challenge is not progress toward equity in science; it’s the opposite.

3. Your preprint is being scored without your consent

This is where the uncomfortable feeling comes in. I have posted many preprints to bioRxiv, but had not really considered that this implies consenting to commercial AI scoring and ranking. There is no opt-out mechanism. The work is evaluated, ranked, and potentially circulated in ways we cannot control and may not be aware of. Honestly, this makes me a bit less enthusiastic about posting preprints.

For researchers who upload manuscripts directly to QED’s platform, the terms of service grant QED a license to use that work to train and evaluate their own models. The product being built on your unpublished science is a commercial one. Scientists should read those terms carefully before uploading anything.

4. A single score flattens what science actually is

A scientific paper is not one thing. Methodological rigor, conceptual originality, experimental scope, reproducibility, and relevance to a specific field are distinct dimensions that do not collapse cleanly onto a single numerical axis. Reducing a paper to one score is a modeling choice, and like all modeling choices, it involves assumptions that deserve scrutiny.

Ranking all life science preprints on a single scale simultaneously, across neuroscience, structural biology, bioinformatics, and plant biology, assumes a commensurability that does not exist.

5. The downstream risks are the real concern

The specific danger here is not what QED intends, but how a ranked list that exists in the world will be used. Funders, hiring committees, grant reviewers, and tenure committees are under constant pressure to find signals of quality, and a QED score may be used as a filter regardless of the caveats in the methodology documentation.

Recommendation: major revision

Academic evaluation should be slow, plural, and contested, conducted by the scientific community over years, through mechanisms that are at least partially accountable to it. It takes a decade (and often more) of assessing, citing, probing, replicating, teaching, challenging, and building on a finding to get a real sense of how important a paper is. This is not something that can be determined at the moment of publication.

The core question is not whether AI can help us navigate the scientific literature; I believe it can, and the preprint ecosystem is a reasonable place to try. The question is whether a for-profit company should be the entity that decides, using a proprietary model that perpetuates known biases, which life science preprints are important. I do not think the answer is yes, and I think we should be clear-eyed about the potential risks.

If you ask a working scientist in the life sciences what they actually do for a living, writing is rarely the first thing they mention, and often not even in the top five. And it’s clear why: in most labs, the unofficial hierarchy puts the experiment and the analysis well above the manuscript that eventually comes out of all of it. Writing is relegated to the part you do once the “real” work is done, the bureaucratic chore at the end of the project. I held something like this view myself for years. To be clear, this was an implicit view; I never consciously decided that writing was less important. But that was the implicit hierarchy I operated under nonetheless.

And the broader trend is moving in the same direction. Indeed, in a 2017 eLife study analyzing more than 700,000 abstracts spanning 130 years, Plavén-Sigray and colleagues showed that the readability of scientific writing has been declining steadily for decades. Last week, in an editorial that prompted me to write this post, Nature announced that the future of science communication is no longer written articles, and that it is now joining TikTok.

I think this consensus is exactly backwards. Writing is not just one part of the job. On the contrary: it is the part of the job that matters most. It took me longer than I would like to admit to reach this realization, but when you think about it, a finding that never gets written up has, in any practical sense, not entered the field; it cannot be evaluated, built upon, or used by anyone else. The value of an experiment is only ever realized when its results and interpretation are committed to writing, where they can be read critically by others and folded into the broader scientific record. In other words, unwritten science is not just incomplete; it simply does not exist for the purposes of the field. When you look at it this way, the job of a scientist - having ideas, testing them, and persuading others they are right - turns out to be, fundamentally, an act of writing.

The clearest articulation of this view I have come across is a recent Career Feature in Nature Biotechnology by Dennis Hazelett, titled “An open letter to graduate students and other procrastinators: it’s time to write”. The article is excellent and should be required reading; it is only 4 pages and well worth your time. Hazelett puts the case more clearly than I ever could:

The most important trait for you to have lasting impact on humanity, and indeed to sustain your career, is your ability to have ideas, to gather evidence in support of them, and to publish them. Your job is to have ideas and spread them so that they impact other people’s work. Writing is your job. Nothing else. Therefore, everything else that you do is subservient to the activity of writing. Let this revolution take place in your mind.

The reframe is uncomfortable for a reason that I think is structural rather than personal. Most of us were trained to treat writing as the activity that happens once the science is done; the documentation step at the end of the project, rather than as the activity through which the science actually accumulates and becomes useful. Work that is not written, in any practical sense, has not entered the field, because the field cannot use a result that cannot be read. In other words, if it is not written, it has no value. Hazelett’s sharper version of this point is that everything else, including the experiments and the analyses we are quietly proud of, is properly in service of the writing rather than the other way around.

The deeper claim, and I think the more important one, is that writing is itself how thinking gets done. Ideas only reveal whether they actually hold up once you have forced them into complete sentences. Anyone who has tried to write down a half-formed argument knows the particular feeling of watching it dissolve under the light of its own prose. This happened to me so many times: “this grant proposal writes itself” I think, only to start writing it and realize that some of the ideas that were so clear in my head do not make sense when I try to put them to paper. Hazelett puts it better than I can:

The only kind of thinking that matters in science is structured thinking. The only way to give structure and substance to your thoughts is to write them down. Writing. Is. Thinking.

This has direct implications for how we think about AI writing tools. Do not outsource your writing to AI. If writing is thinking, then asking an AI to produce your first draft, structure your argument, or put your ideas into sentences for the first time is asking it to do your thinking for you. The value of writing a specific aims page, a methods section, or an interpretation of your results is not necessarily the document it produces; it is the process of forcing your ideas into coherent prose and discovering, often uncomfortably, which of them actually hold up. 1

This is also why writing should not be something you do at the end of the project, but something that runs in parallel with the science itself. Write the study design before you run the experiment. Write up your results as you generate them. Write your interpretation as it develops. This may seem like busywork, but it is how you make sense of what you are doing while you are still doing it, and how you catch the ideas that do not hold up before you have spent another six months chasing them.

The same principle extends well beyond the manuscript. Write down an idea when you have it, even a half-formed one, before it dissolves back into the noise. Write a specific aims page for the project you have been thinking about but have not committed to yet. Write the plan for your analysis before you run it. Write up your results and their interpretation as you generate them, not six months later when the context has faded. Write a response to a paper you disagreed with. Start a blog and share your thoughts with a general audience. Just start writing.

It was a full and packed day (six talks, a panel, a reception, and a dinner) and rather than try to recap each talk individually, I wanted to use the occasion to lay out what I think are the most interesting points about where AI in genomics stands right now. Some of these came directly from the talks; others are observations the day brought into focus.

1. DNA models have grown enormously - but haven’t yielded much biological insight yet

The progression of DNA models since 2021 has been dominated by scale. DNABERT (Ji et al., 2021) established the BERT-style self-supervised pretraining paradigm for DNA, which DNABERT-2 (Zhou et al., 2023) refined with byte-pair encoding and multi-species training; the Nucleotide Transformer family (Dalla-Torre et al., 2024) extended it across hundreds of species; Enformer (Avsec et al., 2021) and HyenaDNA (Nguyen et al., 2023) reached for longer genomic context. Then, earlier this year, Evo 2 (Brixi et al., 2026) jumped to a different scale entirely: 40 billion parameters, 9.3 trillion nucleotides spanning every domain of life, and a 1-million-token context window at single-nucleotide resolution.

The more uncomfortable observation is that this enormous scale-up has not translated into the kind of insight one might expect. And that’s because - according to Alex Lu - DNA is inherently different from natural language in several important ways: low signal-to-noise, vast tracts of repetitive sequence, no obvious analog to “words” or “sentences”, and functional elements that are sparse, distant, and combinatorial.

A clear illustration of this disconnect comes from Vishniakov et al. (2025). The authors evaluated seven genomic foundation models across 52 tasks and compared each pretrained model to randomly-initialized baselines of matched architecture. Their finding was striking: randomly initialized models often matched or even surpassed pretrained models in fine-tuning and feature extraction, and the pretrained models systematically failed to capture clinically relevant variants. Tang et al. (2025), evaluating pretrained DNA language model representations against raw one-hot encoded sequences across six regulatory genomics tasks, reached compatible conclusions: the learned representations offer little to no advantage over the raw sequence itself.

The lesson is that scale alone does not deliver biological understanding. Which leads to my next point:

2. Informed models are key

If brute-force pretraining does not reliably produce useful representations, what does? A recurring theme across the symposium (explicit in some talks, implicit in others) was that ignoring theory and history is a mistake.

Genomics is not a young field. As Andy Kern pointed out, population and quantitative genetics have 100+ years of theory to draw on, beginning with the foundational work of Fisher, Wright, and Haldane on the modern synthesis. Similarly, bioinformatics, evolutionary genomics, and comparative genomics have decades of accumulated approaches and intuitions, including the empirical demonstration that functionally important non-coding regions are evolutionarily conserved across distant species (Hardison, 2000; Siepel et al., 2005). AI in genomics that ignores this body of knowledge is essentially trying to rediscover, from data alone, things we already know.

Alex Lu’s work is a clean example of doing this right. In his “reverse homology” approach (Lu et al., 2022), he used evolutionary conservation as the contrastive signal for a deep learning model, leveraging the fundamental principle from comparative genomics that functionally important sequences are conserved over evolution. Rather than trying to learn what matters from scratch, the model is told what sequences are likely important. The result was a small, interpretable model that successfully identified meaningful features of intrinsically disordered regions, something the larger, uninformed approaches had struggled with. Recent follow-up work building on this conservation-as-pretraining-signal idea (Karollus et al., 2024) supports the broader principle.

The general lesson: small, informed models can outperform large, uninformed ones. Leveraging 100 years of theory and prior biological knowledge, rather than asking gradient descent to recover it from scratch, is a much more efficient research strategy.

3. Predictions are fast; validation is slow

A point Katie Pollard made forcefully in her talk, and one that I think the broader AI-in-genomics community underappreciates, is the asymmetry between how fast we can generate predictions and how slowly we can validate them.

Modern models can score every variant in the human genome for predicted effect in a matter of hours. Generating a ranked list of candidate causal variants, predicted regulatory elements, or candidate disease genes is now cheap. But validating any single one of those predictions (e.g. establishing that a specific non-coding variant actually drives a specific phenotype through a specific mechanism) typically requires generating a mouse model, running cell-based functional assays, or executing CRISPR perturbation screens, each of which takes months to years and substantial cost. Pollard’s recent work illustrates exactly this dynamic. Her group developed CardioAkita, a machine-learning model that predicts how structural variants alter 3D chromatin organization, and used it to score de novo structural variants from individuals with congenital heart disease. The computational predictions identified disruptive variants that were then validated experimentally: induced pluripotent stem cells engineered to carry the predicted variants showed the predicted 3D chromatin changes and aberrant expression of cardiac developmental genes (Lee et al., 2026). The computational step was fast; the experimental validation took years.

This asymmetry has a clear implication: the highest-leverage methodological investments right now may not be in better models, but in high-throughput, fast validation platforms that can keep up with computational output. Approaches like massively parallel reporter assays (MPRAs), Perturb-seq, base-editor screens, saturation mutagenesis, and iPSC-engineered variant panels are critical, because they turn AI-prioritized hypotheses into testable, validated biology at scale. This is not a new framing; this has been explicit in protein engineering and synthetic biology for years, where the Design-Build-Test-Learn cycle treats closing the experimental loop as a research frontier in its own right (Carbonell et al., 2018). Genomics is now converging at the same realization.

4. Vast genomic data exists — the bottleneck is metadata

Arjun Krishnan made the extraordinary estimate in his talk that there are roughly 2.5 million human transcriptomes publicly available across resources like SRA and GEO. That is an incredible resource that every other field would die for, and the potential AI applications are staggering. But it is hugely underutilized.

The catch is that the metadata are non-uniform, often incomplete, frequently wrong, and usually unstructured. Tissue annotations vary across studies, disease labels are inconsistent, sample preparation and sequencing details are missing. So the data are there, but without high quality metadata, it is challenging to extract biological signal at scale. This problem has been documented across multiple efforts to harmonize public expression data (Bernstein et al., 2017). This is not unique to RNA-seq, and consistently found across any type of public data, including microbiome.

This means that some of the highest-leverage methodological work in the field right now is not in modeling, but in building tools that improve, harmonize, and validate metadata. It is unglamorous, but it is where a huge fraction of the value will come from.

5. Be skeptical of all benchmarking

Every new model is the best on its own benchmarks. This is not a controversial statement; it is an observation about the literature, and one that has been documented systematically in adjacent fields. Whoever publishes a model also typically chooses the benchmark, and the benchmark almost invariably flatters the model. Even where benchmarks are shared, evaluation protocols, hyperparameter budgets, and dataset splits often differ in ways that advantage the proposing method.

Arjun Krishnan shared a rule of thumb that I think is genuinely useful: the best model is usually the one that is consistently number 2 in benchmarks across the literature. This rule has straightforward reasoning: a model that is consistently competitive but rarely the absolute winner is more likely to be genuinely strong than one that is ranked #1 only on the benchmark its authors designed. I think this is a practical heuristic that reflects how publication incentives shape benchmark design.

6. AI tools are disruptive for trainees, but a real boon when used well

A question that came up repeatedly, both in the panel and in side conversations during breaks, was how graduate students should think about AI tools. There is a clear tension: on one hand, code-generation assistants and large language models can dramatically increase productivity. On the other hand, leaning on these tools too heavily can hollow out the deep understanding a PhD is supposed to build.

A useful rule of thumb is this: if you can validate what the AI produces – namely, if you can do the task yourself comfortably and check whether the AI did it correctly – then you can probably use AI to do the task. Otherwise, you should probably do it yourself, even if it feels hard. The friction of doing something the slow way is often the friction of actually learning it, and an AI tool that produces output you cannot evaluate is a black box you are forced to trust.

This is also a place where PIs need to take the lead. Writing down explicit AI use guidelines for your lab is more straightforward than it might seem, and I have written about how to do it here.

7. There is a backlash against AI in academia, and we should be careful not to lump everything together

A recurring undercurrent at the symposium, and one I think is worth naming, is that academic culture has begun to develop a real skepticism toward AI, sometimes even a knee-jerk hostility. Some of this is understandable: there are real concerns about training data provenance, environmental costs (Strubell et al., 2019), the displacement of human expertise, the use of AI tools by students to bypass coursework, and the flood of low-quality AI-generated content in the literature (Liang et al., 2024).

But a lot of this hostility conflates very different things. When most people say “AI,” they are thinking about ChatGPT, a general-purpose chatbot whose flaws and risks are well-documented. The work the symposium speakers presented is something else entirely: domain-specific models, built and evaluated by careful scientists, used as one tool among many to make progress on hard biological problems. Treating these as the same thing, and rejecting both wholesale, is not productive.

The position I support is the one our speakers occupy, and it is more interesting than wholesale enthusiasm or wholesale rejection of AI: as scientists, we should engage seriously with specific claims, on specific tasks, with specific evidence. The general lesson (which matters beyond AI) is that a defensible opinion on a complex topic is rarely a single sentence. It has to be built case by case, grounded in actual understanding, and updated as the evidence evolves. Almost nothing is one-sided, despite how it often appears from social media or polarized public discourse. The work of building that kind of grounded, case-by-case judgment is not easy, but it is one of the most valuable habits academic training can develop.

Closing

The symposium reinforced something I have come to believe more strongly over the past year: AI in genomics is moving past its first hype cycle. The questions are becoming sharper: when AI helps, which methods actually generate biological insight, how to integrate these tools with the field’s existing theoretical foundations and experimental platforms. And the answers will require deeper engagement with biology, not bigger models.

For the genomics community, and especially our trainees, this is a remarkably exciting moment.

Can you blame them? They’re under pressure to be productive, like everyone else. They see peers at other labs using AI freely. And frankly, these tools are genuinely powerful; I use them regularly in my own work. I am not anti-AI; quite the opposite. I think AI can accelerate literature searches, catch errors in your code, help you learn new syntax, improve your writing, and let you iterate faster on analyses you already understand. I’m enthusiastic about what these tools make possible.

However, there is a real problem with unrestricted AI use during the training period, and it’s one that most PIs haven’t grappled with yet. A Nature editorial last year made the case that doctoral training needs a fundamental rethink in an AI world (Sen Gupta 2025). Among other things, the editorial notes that only 5% of European universities surveyed felt their existing AI guidelines for doctoral education were sufficient. I agree that a rethink is needed. And since universities are not providing clear guidelines, I think it can start at the lab level.

The purpose of a PhD is to develop independent scientific expertise: learning to think critically, analyze data, write clearly, and form your own judgment. When AI automates those cognitive tasks, trainees can produce polished outputs while bypassing the learning process that those tasks are designed to provide. This doesn't mean trainees are doing something wrong. But they can't be expected to set these norms themselves - they don't yet have the experience to know where the line should be. Setting norms around AI use is a mentorship responsibility. I recently went through the exercise of writing AI guidelines for my lab, and I think every PI should do the same. Here’s why, and how.

The core issue

The best framing I’ve found comes from Arjun Krishnan, who articulates a principle he calls “expertise before augmentation” in a recent pair of papers (one that lays out the conceptual framework (Krishnan 2026a), and another that provides practical guidelines (Krishnan 2026b)). The argument is simple. Previous tools (like calculators, statistical software, or search engines) automated mechanical execution. You chose the statistical test; R executed it. You decided what to search; Google found it. Generative AI is different. It automates the cognitive work itself: reasoning, synthesis, judgment. Krishnan illustrates this with a simple framework, shown in Figure 1 from the paper below.

The impact of automation depends on both what is automated (rows) and who is using it (columns). Automating mechanical processes (like running a statistical test or formatting citations) preserves learning regardless of expertise level. But automating cognitive processes (like choosing the analytical approach, interpreting results, or synthesizing arguments) has opposite effects depending on the user: it augments experts who can verify the output, and undermines trainees who can’t.

For an experienced researcher, AI is great. You know enough to guide the tool, catch its errors, and work independently when it fails. But for a trainee still developing those capabilities, the same tool can bypass the learning process entirely, while producing outputs that look polished enough to mask the gap.

AI also changes the nature of the task itself. Instead of doing the work, you're now reviewing it. That sounds easier, but it's not: verifying that an analysis or a text is correct often requires deeper understanding than producing it in the first place. Krishnan calls this the “verification paradox”: you can’t verify AI outputs in areas where you haven’t yet developed competence, because verification requires the expertise you’re still building. Consider a concrete example. A trainee asks AI to write a differential abundance analysis for a microbiome dataset. The code runs without errors, the figures look professional, the statistical tests seem reasonable, and the interpretation sounds confident. But is the normalization appropriate for compositional data? Are the statistical assumptions met? Are there batch effects that the analysis ignores? These are the kinds of questions that require domain expertise to even think to ask, and that expertise is what the trainee is supposed to be developing. The result is a circular trap: the AI produces work that looks right, but the trainee can’t tell whether it actually is.

There’s a deeper issue here, too. The struggle of figuring out the right analysis approach, wrestling with an unexpected result, or trying to turn a messy set of findings into a coherent story is not an obstacle to learning. It is the learning. That’s how you develop judgment, build intuition, and find your own voice as a scientist.

Writing AI use guidelines for your lab

Here’s a process that worked for my lab. Every group has its own culture, emphasis, and style, so adapt as needed, but the basic framework is transferable.

Start with a journal club. Before writing anything down, have a lab discussion. The journal club format worked well for this, with the two Krishnan papers (Krishnan 2026a; Krishnan 2026b) as the assigned reading. You can structure it however works for your group: a set of prepared questions, a free-flowing conversation, or something in between. The format matters less than creating a space for open conversation about how AI fits into scientific training.

Discuss the substance. Go through the paper’s tables and examples. Discuss specific scenarios from the papers and how they map onto your lab’s work. Talk about where the line falls between productive use and dependency. Consider the types of AI errors Krishnan catalogs (hallucinated citations, inappropriate statistical tests, confirmation bias) and discuss which ones are most relevant to your field. You can also discuss when the temptation to reach for AI is strongest: when a task is boring, or when it’s hard and uncomfortable? Kaplan, Palitsky & Raison (2025) call this latter pattern “machinal bypass”; using AI not to support thinking but to sidestep it, especially when we feel stressed or doubtful of our capabilities.

Don’t underestimate the conversation itself: even without producing a written document, our lab discussion was extremely valuable. It surfaced assumptions, clarified where people saw genuine gray areas, and made clear that the a guideline would be a useful resource. If you do nothing else, have the conversation.

Draft guidelines. After the discussion, draft a short document. This doesn’t have to be a rigid policy, but more a set of principles with concrete examples relevant for your lab’s work. Every lab’s document will look different, but for computational labs, I’d suggest covering at least these three areas:

1. Accountability. If you use AI, you own the output. This is the non-negotiable foundation. AI-generated work that contains errors (wrong statistical tests, hallucinated citations, bugs in code, etc) is your responsibility. This means every AI output needs to be verified. Verification means you understand what every step does, why it’s appropriate for your data and experimental design, and what the result should approximately look like. If you can’t perform that level of verification for a given task, that’s a signal that AI use for that task is premature.

2. Writing. This is where I think the stakes are highest. As Hazelett (2025) argues, and I completely agree, writing is the single most important skill in science (Hazelett 2025). The struggle to organize your argument, decide what matters, and articulate what your results mean is the intellectual work itself. When AI writes your first draft, the task gets done, but the thinking doesn’t happen. In our guidelines, first drafts of Results, Discussion, and Introduction sections should be the trainee’s own work. AI can help with grammar, with tightening prose after revision, or with critiquing the logic of an argument you’ve already made. But the hard part of forming the argument needs to be yours.

3. Analysis. For analysis, the question isn’t “Can AI do this?” but “Can I verify what AI produces?” For example, if you understand mixed-effects models and want to learn the syntax of a new R package, AI is a great tool. If you’re asking AI to design a statistical model for a differential abundance analysis, you’re outsourcing the scientific judgment that your training is supposed to build. The same applies to interpreting results: asking AI what your differentially abundant taxa mean in a certain context requires domain knowledge, understanding of your experimental design, and awareness of the data’s limitations.

I’d recommend keeping the tone advisory rather than punitive. The threshold throughout should be functional: can you do the task, explain your reasoning, and catch your own errors? If yes, AI could be a good option. If not, that’s useful information about where you are in your development, not a judgment.

These guidelines won’t be perfect. Ours certainly aren’t, and they’ll evolve as the tools change and as we learn more about how AI affects training. But a flawed document that starts a conversation is better than no document and no conversation. The norms on AI use are being set right now, whether you write them down for your lab or not. You might as well be the one setting them.

References

Hazelett DJ (2025). An open letter to graduate students and other procrastinators: it’s time to write. Nature Biotechnology 43:447–450. https://www.nature.com/articles/s41587-025-02584-1

Kaplan DM, Palitsky R, Raison CL (2025). The ‘machinal bypass’ and how we’re using AI to avoid ourselves. PNAS 122(51):e2518999122. https://doi.org/10.1073/pnas.2518999122

Krishnan A (2026). Build expertise first: why PhD training must sequence AI use after foundational skill development. https://doi.org/10.5281/zenodo.18649847

Krishnan A (2026). Expertise before augmentation: a practical guide to using generative AI during research training. https://doi.org/10.5281/zenodo.18452319

Nordling L (2026). AI and the PhD student: friend or foe? Nature 651:24–27. https://www.nature.com/articles/d41586-026-00843-y

Sen Gupta A (2025). PhD training needs a reboot in an AI world. Nature 647:27–28. https://www.nature.com/articles/d41586-025-03572-w

Look at the last row (“No-DR”). This is the plain random forest trained directly on raw ASV abundances, with zero fancy representation learning – and it is superior on virtually every task and metric.

This should not surprise you

If you’ve been paying attention to the microbiome machine learning literature, this pattern, whereby random forest outperforms more complex approaches, is not new. It keeps showing up, across datasets, diseases, and years.

The most rigorous test of this to date is in a recent paper from Li et al. (Gut Microbes 2025, DOI: 10.1080/19490976.2025.2489074). This is a large benchmarking study: 156 tool-parameter-algorithm combinations, tested across 83 gut microbiome cohorts covering 20 diseases, with both internal and external validation. They compared nine algorithms: LASSO, Ridge, Elastic Net, Random Forest, Linear SVM, Radial SVM, XGBoost, LightGBM, and Neural Networks. The result: ridge regression and random forest ranked best, consistently outperforming neural networks and the gradient boosting methods. Figure 4D from that paper tells the story clearly:

This result held up across many disease contexts and cohorts, in both internal cross-validation and external generalization. Their practical recommendation: when building microbiome-based disease classifiers, use ridge regression or random forest as your default algorithm.

This finding echoes what earlier benchmarks already showed. Pasolli et al. in their MetAML meta-analysis (PLoS Computational Biology 2016), found random forest consistently outperforming SVM across eight large metagenomic datasets, achieving AUCs as high as 0.945 for liver cirrhosis and 0.890 for IBD. Topçuoğlu et al. (mBio 2020) similarly found random forest performing best or tied-for-best across seven different modeling approaches on colorectal screening data, and made the observation that assuming more complex models are better is an assumption, not a finding.

More recently, two 2025 papers drive the point further home. Mu, Tang & Chen (Research Square preprint 2025) applied the same 83-cohort benchmark from Li et al. to test a new wave of microbiome foundation models, including MGM (a transformer pre-trained on 260,000+ samples), TabPFN, and GPT-derived embeddings. Their conclusion: “current foundation-model approaches offer, at best, modest gains over strong classical baselines for microbiome-based disease prediction.” MGM’s performance was disease-dependent and generally lagged behind the best tabular baselines; GPT-derived embeddings consistently underperformed standard numerical features. And Karwowska et al. (Microbiome 2025), analyzing over 8,500 samples from 24 shotgun metagenomic datasets, found that classification performance showed minimal dependence on algorithm choice; random forest and XGBoost were essentially interchangeable, and even a simple presence-absence transformation worked nearly as well as sophisticated abundance-based approaches.

Why does random forest work so well on microbiome data?

There are good structural reasons for this, and they all follow from the properties of microbiome data itself. First of all, sample size. A recent survey of microbiome ML studies found that the median dataset is 161.5 samples, with a third of the studies had <100 samples (Dudek et al., Machine Learning with Applications 2024). Deep learning models with millions of parameters simply cannot learn reliably from a few hundred samples. Random forest’s bagging and random feature subsampling provide powerful built-in regularization that works even when the number of features far exceeds the number of samples. Then consider the feature space: a typical ASV table might have hundreds or thousands of features, but only a small fraction carry any signal for a given phenotype. Tree-based models are natively robust to irrelevant features in a way that neural networks are not; each tree in the forest ignores most features by construction. Related to this is the sparsity problem, with most microbiome abundance tables being >50% zeros. A simple threshold (e.g., is this taxon present) often captures the dominant signal, which is exactly what a decision tree split does naturally. Neural networks must learn this threshold behavior through gradient descent on continuous activations, which is a harder problem. Finally, there may be a ceiling on what abundance data alone can predict. For many diseases, neither random forest nor deep learning achieves particularly high AUCs (often below 0.80, sometimes much lower) suggesting the bottleneck is biological, not algorithmic. Throwing a more complex model at the same abundance table doesn’t help if the signal simply isn’t there.

Where deep learning actually wins

That said, there are specific scenarios where deep learning has real advantages.

The clearest case from Pope et al. is cross-study generalization. When they took their transformer model trained on AGP data and applied it to entirely independent IBD datasets (Halfvarson and HMP2), the transformer outperformed the random forest baseline on the Halfvarson dataset, which had only 34% ASV overlap with the training data. The random forest’s performance dropped substantially on this cross-study transfer, while the pre-trained transformer maintained robust performance. This makes intuitive sense: the transformer learned transferable representations of microbial community structure that generalize beyond the specific ASVs seen during training, while random forest is tightly coupled to the feature space it was trained on.

Deep learning also shows genuine advantages for multi-modal integration. The real gains in microbiome deep learning often come not from applying fancier algorithms to the same abundance table, but from integrating multiple data types altogether. Our own work combining paired gut microbiome and host transcriptomic data across disease cohorts showed that meaningful biological signal – disease-specific microbe-host gene associations – emerges from this kind of multi-modal design (Priya et al., Nature Microbiology 2022). Deep learning architectures that fuse microbiome abundance with metabolomics, host gene expression, or clinical data are the natural extension of that logic. For a more in-depth review on microbiome multi-omic integration I recommend a recent review from my lab (Chetty & Blekhman, Gut Microbes 2023)

Practical recommendations and the road ahead

Given all of this, here is my practical take: start with random forest or ridge regression, and have a good reason before using anything more complex. For the typical microbiome classification problem (a few hundred to a few thousand cross-sectional samples, binary or multi-class disease prediction) the evidence is consistent enough that there’s no reason to default to deep learning simply because it feels more modern. You’ll spend more time tuning hyperparameters, you’ll need a GPU, and you’ll likely get the same or worse performance. What matters more than algorithm choice is proper data processing and cleaning, rigorous cross-validation, and (when possible) external validation on independent cohorts. These decisions routinely have larger effects on real-world performance than switching between RF, XGBoost, and neural networks. More broadly, this is good practice regardless of data type: start with the simplest model that could plausibly work. A random forest takes minutes to train and may solve your problem immediately. Reach for deep learning only when you have clear evidence (more data, a specific architectural advantage, a generalization task) that the added complexity is justified.

Where it really gets exciting is the emerging niche for foundation models. The foundation model era in microbiome research will require the same resource that made it work everywhere else: massive, uniformly processed training data. Resources like the Human Microbiome Compendium (168,000 gut microbiome samples from 68 countries, processed through a single pipeline) are the necessary first step (Abdill, Graham et al., Cell 2025). Building and benchmarking foundation models against datasets of this scale is the next frontier.

You submitted your manuscript for peer review, waited months, and when the reviews finally arrived, one was clearly written by AI. You are not alone: A recent survey of 1,600 academics found that more than half have used AI tools while peer reviewing manuscripts. In addition, data from the American Association for Cancer Research found AI-generated text in 7% of reviewer reports, even as early as 2024. Other estimates suggest up to 17% of peer reviews at major conferences may be AI-written.

So what can you do when that happens?

1. Document evidence

Before raising concerns, it is good to document specific evidence. AI-generated text has telltale markers that have been well-characterized: for example, this recent paper looked at 15 million biomedical abstracts and found specific words (like “delve”) that abruptly increased in frequency in 2024, likely due to the authors using LLMs. Also, a common AI signature are generic, broad, vague comments that do not really engage with the actual methodology or findings of the paper. The biggest red flag is hallucinated citations - references to work that looks very plausible at first glance, but does not exist.

In my experience, there is a general feeling with AI reviews that they sound plausible on the surface (and may look great to a non-expert), but when you actually read the review, it is essentially useless: it doesn’t comment on actual issues, it’s vague, broad, and non-specific, it uses weird formulaic AI language and words, and just doesn’t make sense.

So you can go through the suspected AI review and list all the evidence you have that the paper was written using AI. I advice to wait a day or two, and then look at the report with fresh eyes, perhaps sharing it with colleagues to get their input. Sometimes what seems like clear evidence to the author may not be so conclusive to others.

2. Check the journal policies

Publishers are beginning to take stances against AI reviews. Elsevier explicitly bans reviewers from using AI to assist in scientific evaluation, stating that “the critical thinking and original assessment needed for peer review is outside of the scope of this technology”.

The Committee on Publication Ethics (COPE) has issued guidance specifically on this situation. Their position is that journals should have clear policies, and authors should be able to contact the journal office if they have concerns about a review.

Springer Nature and Wiley permit limited use for language editing but prohibit uploading manuscripts to AI tools due to confidentiality concerns. Reviewers now see this new checkbox when submitting their review:

After checking your journal’s AI policies and confirming If the policy explicitly prohibits AI-generated reviews and you have evidence of violation, reference this in your communication with the editor.

3. Contact the editor

If AI evidence is clear, and the journal doesn’t allow it, consider contacting the editor. You can write a note expressing your concerns, including all the evidence. Focus on the specific issues: you received feedback that appears to have been generated by AI without genuine engagement with your work, and you’re requesting either clarification or re-review. I suggest framing your concern around the quality of the review, not the character of the reviewer. E.g., don’t write “The reviewer used AI”, but maybe something like: “This review contains several factual errors about our methodology and generic comments that don’t engage with the specific content of our manuscript. I’m concerned the review may not reflect careful reading of our work.”

I found it helpful to read a recent case study published in Research Integrity and Peer Review that detailed one author’s experience doing exactly this. The author provided line-by-line annotations of suspicious text and urged the editors to confront the reviewers. The journal’s response was frustratingly neutral: they stated their journal was “not the proper place to conduct a dispute on this issue”, but the author’s documentation created accountability for any appeal process or future submissions.

At the end of the day, the response will depend on the journal, editor, and specific case. Some editors might take it seriously, seek an additional review, and address the matter with the reviewer. Others may dismiss your concerns or simply pass along the review unchanged.

4. Or - and this may be your best option - do nothing

And by “do nothing”, I mean just treat this as a regular review.

AI-generated reviews are, paradoxically, among the easiest to respond to. Because they tend to be vague and generic, the concerns they raise are either trivially addressed (”we have now clarified this point”) or easily refuted (”the reviewer suggests X, but our manuscript already addresses this in section…”). You don’t need to wrestle with substantive scientific disagreements or rethink your experimental design. You just need to demonstrate you’ve engaged with the feedback, which usually can take far less effort than responding to a rigorous human review.

Writing to the editor with a complaint takes time and energy, with uncertain outcomes. Responding to the review and moving forward keeps your paper in motion. Unless the AI review led to a rejection, or the comments are so detached from reality that they’re impossible to address, your path of least resistance may simply be to write your response, address what you can, politely dismiss what doesn’t apply, and let the editor decide.

The bottom line: If you receive an AI-generated review, you have options. Document the evidence, check journal policies, and contact the editor if warranted. But often the most practical path is simply to treat it like any weak review and keep your science moving forward. Don’t let a lazy reviewer derail your work or your morale.

So when Google announced Scholar Labs in November 2025, I approached it with skepticism. Another AI promising to revolutionize literature discovery? Sure. But after weeks of testing, I’ve found something different: an AI tool that cannot hallucinate papers because it only searches real literature.

The key insight is what Scholar Labs doesn’t do. It doesn’t synthesize findings across papers. It doesn’t generate new scientific text. It doesn’t create summaries that integrate information from multiple sources. Instead, it takes your natural language query, runs searches through Google Scholar’s actual database, evaluates real papers, and returns a few dozen results with AI-generated explanations of how each individual paper relates to your question. Every result links to a real paper that you can read and evaluate for relevance. Access requires a Google account and joining the waitlist at scholar.google.com/scholar_labs/search.

Testing Scholar Labs with microbiome queries

I’ve used Scholar Labs quite a bit in the last few weeks; here are some examples of recent searches related to microbiome research, which is an area I can evaluate.

First example: finding comprehensive reviews on host genetics and microbiome interactions. The query “reviews on host genetic variation associated with microbiome composition in humans” returned exactly what it should have, with the most relevant reviews on this topic.

Second example: identifying computational methods for multi-omics integration. When I searched for “computational methods to integrate multi-omic host and microbiome data from humans,” it found a recent review from my lab (Chetty and Blekhman 2024), in addition to other relevant papers.

Third example: uncovering gaps in the literature. This is a bit more tricky: I searched for single-cell RNA-seq applications to microbiome samples—a genuinely rare approach. Scholar Labs found a few papers, mostly reviews describing emerging techniques and challenges.

Fourth example: ultra-specific technical comparisons. The query “comparison of DADA2 vs QIIME2 for rare taxa detection in low-biomass samples” returned detailed pipeline comparisons, including specific performance metrics on rare taxa detection.

Limitations and strengths

The limitations are important. Scholar Labs doesn’t synthesize information across papers, it just shows individual papers relevant to your query. It also doesn’t provide data extraction tables like Elicit where you can compare sample sizes, methods, and outcomes across studies at a glance. There’s no big-picture literature summary that identifies trends, gaps, or emerging areas. The number of papers per search is limited, with no ability to compare specific papers or find similar articles to a reference.

At the same time, the strengths address the fundamental problem with AI research tools: reliability. The papers are all real, and every paper Scholar Labs returns actually exists in Google Scholar’s index. In my experience, the results have been consistently relevant to the queries, without the padding of tangentially related papers common in keyword searches. Importantly, you can click through to each paper, read the text, and evaluate the evidence yourself. There’s no need to trust the AI’s interpretation because you have direct access to the source material.

Scholar Labs appears to be a useful addition to the literature discovery toolkit, especially for exploratory searches and finding papers you might miss with traditional keywords. It won’t replace systematic database searches, won’t do synthesis work, and of course won’t eliminate the need to actually read papers. But for a field drowning in publications and wary of AI hallucinations, a tool that reliably finds real, relevant publications using natural language queries could prove valuable. Time will tell if this becomes essential or remains an interesting experiment. For now, it's the first AI research tool I've used where the biggest feature is what it refuses to do: make things up.

I was honestly shocked by the numbers: Despite 100+ papers per year and $20-25M in annual NIH funding since 2018, the most cited studies have tiny sample sizes (n=20 for some), uncorrected multiple testing, and contradictory findings. Some report higher microbial diversity in autism, others lower, many find no difference. The taxa they report are inconsistent. When properly analyzed with sibling controls and adequate statistical power, associations largely disappear.

The mouse model data is equally troubling. For example, a 2019 Cell paper claimed fecal transplants from autistic donors induced “autism-like behaviors” in mice. But when reanalyzed with proper statistics, the effects vanished. And it is really not clear how mouse behavior can model human autism.

And these are not obscure papers - they are published in the most high profile journals and have thousands of citations each.

Maybe it is not surprising that randomized controlled trials of probiotics show no consistent benefit for autism.

As a result, we’re watching well-intentioned science fuel a billion dollar “wellness” industry selling unproven interventions to vulnerable families.

As a microbiome scientist, this really troubles me. Excellent research IS being done, and the microbiome holds real promise as a diagnostic and therapeutic tool. But these overhyped, methodologically weak studies cast a shadow over our entire field.

The solution, in my opinion – and completely I agree with the authors on this – is to adopt the rigor that transformed human genomics: pre-specified hypotheses, adequate power, standardized protocols and pipelines, large sample sizes, statistical robustness, replication, and above all: resisting the hype machine.

Read the full paper: Conceptual and methodological flaws undermine claims of a link between the gut microbiome and autism, Neuron 2025