It’s no secret that it’s been a rough year for scientific institutions in the U.S. The National Science Foundation and the National Institutes of Health were vandalized by DOGE and they are still struggling to get back on their feet. Those of us awaiting funding decisions are hearing that a layer of political review seems to have been added on top of the normal review process. Last year, several major universities fell into the crosshairs of the Trump administration, which attempted to coerce their behavior by using the nuclear option of freezing all of these institution’s federal funding–primarily funds for biomedical research. Some universities were also invited by the administration to sign a “compact” that would allow an unprecedented intrusion of the federal government into those institutions’ hiring and curricular decisions.

I don’t want to underestimate the damage that’s been done to research universities in the past year and or minimize the suffering of those whose careers have been affected, but I am optimistic about the future of America’s scientific institutions. In the budget bills passed this week, Congress completely rejected the Trump administration’s proposed draconian cuts to the NIH and NSF. Universities are proving to be resilient, and America’s strong scientific enterprise isn’t something that can be destroyed overnight or even over a year.

Subscribe now

But in addition to the more immediate challenges, the scientific community needs to get to work on the longer term problem of rebuilding trust with the broader public. Here I want to suggest that we can rebuild that trust by committing to a successful vision of how the government and universities work together to serve the public.

The vision that we should commit to is not a new one: it is the vision that the U.S. implemented after WWII, leading to our spectacular success in science for over three generations. Most readers have likely heard of Vannevar Bush’s famous report, Science: The Endless Frontier, written for the FDR Administration in early 1945. I imagine that fewer readers have actually read it or know in any detail what its arguments are. Well, I recently reread it and can say that it holds up.

We’re at a moment when the relationship between universities and the federal government seems to be in crisis, and alterante arrangements are being proposed and considered. I’m going to make the case that we don’t need alternatives; we just need to recommit to the principles that Bush laid out 80 years ago.

Before we get to Bush’s specific principles for government support of science, there is one essential prerequisite that Bush, writing as the US was winning a major war, could take for granted, but which we cannot: Government support of scientific institutions must rest on a foundation of broad public legitimacy. Scientific institutions need to maintain the respect and trust of the public that supports them, otherwise they will be seen by many as simply another elite interest group looking for taxpayer dollars. When that happens, Vannevar Bush’s vision for science collapses. I’ll say more on this below, but let’s first dig into the specifics of Bush’s report.

Why should the government fund scientific research?

Science: The Endless Frontier is an answer to the question, why should the government provide stable, long-term funding for basic science? Bush gave four important answers..

1. Basic science is the “seed corn” of technological progress

Whatever you might think of state of the economy today, one of the most important facts about human life in the 21st century is that we enjoy a prosperity that would be envied by nearly everyone alive 100 years ago. The reason for this is long term economic growth, driven by technological progress. Bush recognized that basic research, whose purpose is to generate new scientific knowledge, is the source of scientific capital from which technological progress draws. He called basic research the “seed corn” and “pacemaker” of technological progress. He wrote that

Advances in science, when put to practical use, mean more jobs, higher wages, shorter hours, more abundant crops, more leisure for recreation, for study, for learning how to live without the deadening drudgery that has been the burden of the common man for ages past. Advances in science will also bring higher standards of living, lead to the prevention or cure of diseases, promote conservation of our limited national resources, and assure means of defense against aggression. But to achieve these objectives—to secure a high level of employment, to maintain a position of world leadership—the flow of new scientific knowledge must be both continuous and substantial.

While scientific knowledge and technological progress can benefit the entire world, Bush correctly noted that nations which rely on other societies for scientific knowledge are at a disadvantage. By having scientific institutions here at home, we gain advantages in intellectual property, in human capital, and in the ability for our businesses to act first on new discoveries.

Importantly, it is the role of government to support scientific progress: “Since health, well-being, and security are proper concerns of the government, scientific progress is, and must be, of vital interest to the government.”

2. Scientific progress is critical for the nation’s health, its economic prosperity, and its national security.

Bush noted that major gains in lifespan were due to growth in biomedical knowledge and its application. Not only is a healthier populace a good thing on its own, but a healthier society is also a more economically prosperous one.

The success of our economy also depends in other ways on scientific knowledge:

How will we find ways to make better products at lower cost? The answer is clear. There must be a stream of new scientific knowledge to turn the wheels of private and public enterprise. There must be plenty of men and women trained in science and technology, for upon them depend both the creation of new knowledge and its application to practical purposes.

And finally, Bush noted that one of the most important public goods provided by the government is national defense. Writing in the waning moths of WWII, he said that:

This war emphasizes three facts of supreme importance to national security: (1) Powerful new tactics of defense and offense are developed around new weapons created by scientific and engineering research; (2) the competitive time element in developing those weapons and tactics may be decisive; (3) war is increasingly total war, in which the armed services must be supplemented by the active participation of every element of civilian population.

Today, war is not always total war, but as the war in Ukraine shows, technological innovation is still essential for defending your nation.

3. Progress requires freedom of inquiry

You can’t reliably predict in advance where breakthroughs will happen or what knowledge will prove most valuable:

Many of the most important discoveries have come as a result of experiments undertaken with very different purposes in mind. Statistically it is certain that important and highly useful discoveries will result from some fraction of the undertakings in basic science, but the results of any one particular investigation cannot be predicted with accuracy.

Thus, Bush argued, institutions of scientific research must be committed to freedom of inquiry. Neither politicians nor university administrators should dictate the course of research. Bush thought that universities and research institutes would be optimal sites for basic research because they were committed to open inquiry:

It is chiefly in these institutions that scientists may work in an atmosphere that is relatively free from the adverse pressures of convention, prejudice, or commercial necessity. At their best they provide the scientific worker with a strong sense of solidarity and security, as well as a substantial degree of personal intellectual freedom. All of these factors are of great importance in the development of new knowledge, since much of this new knowledge is certain to arouse opposition because of its tendency to challenge current beliefs or practices.

If universities fail to be places where researchers are free to challenge current beliefs, then they are failing to live up to their part of the bargain that Bush is proposing.

4. Need for a scientific workforce

A trained, talented scientific workforce is the single most important ingredient for scientific progress, and for converting scientific knowledge into technologies that benefit society:

We shall have rapid or slow advance on any scientific frontier depending on the number of highly qualified and trained scientists exploring it.

Critically, Bush argued that the nation needs to draw on talent from anywhere it is found. It is in the government’s interest to ensure that social or economic barriers don’t stand in the way of bringing talented people into science:

Here is a tremendous waste of the greatest resource of a nation—the intelligence of its citizens. If ability, and not the circumstance of family fortune, is made to determine who shall receive higher education in science, then we shall be assured of constantly improving quality at every level of scientific activity.

How to recommit to Bush’s Vision

Bush’s arguments for why the federal government should provide substantial, long-term funding for science are still sound today. To promote the national welfare, the government should support the production of scientific knowledge, at institutions that are committed to free inquiry and to drawing in talent from wherever it is found.

But the obligation doesn’t just go one way, from the government to universities. What Bush is proposing is that universities hold an important public trust. In exchange for public support, universities will work in the broad public interest, and they should be accountable to the public for holding up their end of the bargain. We need to be proactive about communicating to the public that we value this trust, and that we take our obligations seriously. Here are some principles scientists should follow, especially university leaders and those who choose to engage with the public:

1. Don’t link science with partisan identities. Don’t act like science and universities are the province of political liberals. Science is for everyone, and we should welcome colleagues and trainees regardless of their political affiliation. One way that universities have failed at this is to rely too heavily on language and ideas that come across as partisan. For example, DEI efforts, in principle, align with Bush’s charge to ensure that social and economic barriers don’t keep talented people out of science. In practice, these efforts often rely on tendentious left-wing arguments that have the counterproductive effect of convincing a big chunk of the population that the scientific community doesn’t care about them or share their values.

2. Hold high standards and aggressively police fraud. The public needs to know that when we spend taxpayer funds, we are doing do with care, and that we take misconduct seriously.

3. Defend academic freedom from all threats. Elected officials right now represent the most serious threat to academic freedom, but the last decade has seen unacceptable levels of censorship in academia. Too often the problem has been dismissed as minor, affecting only a few people. I do agree that the overwhelming majority of academics have not been affected by censorship, but the problem is real and a dismissive attitude sends the wrong signal to the public. The public needs to know that universities are living up to Bush’s claim that they are committed to free inquiry against all threats.

4. Be willing to listen to public concerns. Amander Clark, director of the UCLA Center for Reproductive Science, recently wrote in a Nature commentary that

Those of us working in the sector should reflect on the purpose, mission and vision of universities. It’s crucial that educators and researchers listen to the public on whether university visions align with public needs, values and the jobs that will be needed in the future workforce.

National Academy of Science President Marcia McNutt wrote in Science that

The scientific community must also better recognize that it may not be helpful to emphasize consensus in policy reports’ recommendations when the underlying values are not universally shared.

Good doctors learn to communicate effectively with patients by cultivating a bedside manner, which more or less means letting the patient know that you are hearing their concerns. We need to cultivate a similar approach in how we present ourselves to the public–and these days, basically anything we say or do online is public.

Americans understand the importance of science

The good news is that Americans across the political spectrum do genuinely understand the importance of science, as recent results from Pew Research show (see below). They don’t need to be convinced of that. Instead, they need to be convinced that scientists and scientific institutions are committed to working in the interests of everyone.

Today (April 25) is DNA Day, in recognition of Watson and Crick’s transformative 1953 paper describing the double helical structure of DNA. DNA Day also recognizes the completion of the first Human Genome Project fifty years later. And yesterday the Genetics Department at Washington University in St. Louis celebrated its 50th anniversary. One of the keynote speakers was Bob Waterston, our former department head who played a critical role in making the Human Genome Project happen. In his talk, Waterston recounted a chilling anecdote that illustrates what might have happened if the Human Genome Project turned out differently. Here’s a version of the anecdote from his recent memoir:

At one meeting in December 1999, called to explore possible collaboration [between the publicly-funded Human Genome Project and the private genomics company Celera], Tony White, the CEO of Perkin-Elmer, insisted that any combined genome-sequence product would only be available through the Celera website for years to come, and Celera would be the only source for any analyses, including annotations, that might come from joint efforts.

Many readers will remember that Craig Venter’s Celera was competing with the publicly-funded, international project to sequence the human genome. If you don’t work in biology, you may not appreciate just how disastrous it would have been if the CEO of Perkin-Elmer had gotten his way. Just about every aspect of biology has been transformed by genome sequencing–an analogy might be the impact that quantum mechanics has had on physics. Virtually no part of the broader field is untouched by the transformation. A key part of the genomic transformation of biology has to do with new tools that depend on access to reference genomes. Gene editing to cure disease, mRNA vaccines, ancient DNA studies, gut microbiome analysis, brain changes in autism, ecological studies–all of these depend on technologies that would either not exist or have been delayed for years if a private company had owned the human genome sequence.

Why? Because the technologies and computational methods that are so central to medical and biological research today depend heavily on aligning new DNA sequencing data to a reference genome sequence. These technologies aren’t only about ‘reading’ a DNA sequence–many of them use sequencing to measure critical molecular events, quantify the state of a cell, and to design new sequences that can be used in lab research and therapies. The Human Genome Project generated one of the first and most critical reference sequences. If every lab around the world had to pay Celera for access to that reference genome, much of the innovation that drives biology today would never have gotten off the ground.

Subscribe now

This isn’t just speculation. We can make the direct comparison. Parts of the genome that were sequenced first by Celera locked up as its IP until the public project resequenced a freely available version. Comparing the Celera-first versus public-first parts of the genome, 2010 analysis found that “Celera's IP led to reductions in subsequent scientific research and product development on the order of 20 to 30 percent.”

Yesterday at our department’s 50th anniversary symposium, Bob Waterston told of how he and others pushed hard for open genome access at a now-famous 1996 meeting held at the Princess Hotel in Bermuda. At the time, the genome sequence of the nematode C. elegans was nearing completion and large-scale sequencing of the human genome was about to begin. What emerged are known as the Bermuda Principles, a central part of which was that new sequencing data would be shared immediately. Sequencing centers would upload their new sequencing data to an open database every day, thereby putting this data in the public domain. As one history of the Bermuda Principles puts it,

the Bermuda Principles addressed concerns about gene patents impeding scientific advancement, and were aspirational and flexible in implementation and justification. They endured as an archetype for how rapid data sharing could be realized and rationalized, and permitted adaptation to the needs of various scientific communities.

Biomedical science and the biotechnology industry would not be what they are today if those principles had not prevailed. Because they did prevail, patents on human genes and gene variants were already on their way to a natural death by the time the U.S. Supreme Court killed them for good in 2013.

Share

Scientific knowledge is one of the most critical of what economists call public goods. Public goods are “non-rivalrous” and “non-excludable”. Non-rivalrous means one person’s use of a public good doesn’t diminish anyone’s ability to use it, and while non-excludable means it’s difficult to keep others from using it. Because of those features, private incentives to produce scientific knowledge are low, and economies then under-produce them, to everyone’s detriment. The Human Genome Project is a clear example of scientific knowledge as a public good. Public domain reference genomes can be freely used and reused by academic and industry labs to advance knowledge and invent new technologies and therapies. A 2013 report found that every $1 invested by the US government in the Human Genome Project led to a $141 addition to the US economy. Had Celera succeeded in paywalling the reference genome, it would have increased research costs and stifled innovation. But it also would have been difficult and time-consuming for Celera to enforce its monopoly, since bootleg sequence data could easily be shared and labs could (inefficiently and redundantly) resequence parts of the genome they cared about.

Publicly-funded work like this is the foundation on which nearly all of today’s technological innovation is built. It’s why we can now cure some fatal genetic diseases, treat cancer more effectively with better diagnoses, have a successful drug against obesity, shop at well-stocked grocery stores, text friends, stream movies, and fly to visit family on holidays. Had the human reference genome been locked up behind a paywall, our society would have been sicker and poorer for it.

For those interested in the early history of the technologies that made the Human Genome Project possible, I’ve transcribed a talk given by another former Genetics Department Head, Mark Johnston, who had a front row seat and made some critical contributions of his own. Click the link for the video and annotated transcript.

For the last couple of months, I have had this strange experience: Person after person — from artificial intelligence labs, from government — has been coming to me saying: It’s really about to happen. We’re about to get to artificial general intelligence.

Klein defines AGI as AI that is “capable of doing basically anything a human being could do behind a computer — but better.” Achieving AGI is clearly a major goal of the big players in AI. Google, clearly concerned about being perceived as responsible in its quest for AGI, recently released a paper on AGI safety. The underlying premise is that work pursued by Google and others is definitely on the path to AGI.

George Mason University economist Tyler Cowen also thinks AGI is imminent, and he also defines it in terms of human intelligence:

Others are skeptical that there is any clear threshold that would count as achieving AGI. Arvind Narayanan and Sayash Kapoor, authors of AI Snake Oil, argue that we’ll see AI models climb a “ladder of generality,” with new models capable of new tasks; however, they are skeptical that we’ll get to anything one might call AGI simply by building bigger LLMs (large language models):

An unknown number of steps lie ahead before we can reach a level of generality where AI can perform any economically valuable job as effectively as any human (which is one definition of AGI).

Why all the fuss about AGI? A big part of the answer is that AGI has been the holy grail of AI throughout its entire history because the field is specifically motivated by human intelligence. There are many ways that you can use probability and statistics to build powerful models of the world. But the AI field, starting at least with the very first artificial neurons has proceeded on the premise that we can learn how human intelligence works by building computer models with similar capabilities. In other words, one purpose of AI is to simply understand how intelligence works–and thus AGI would represent an important milestone.

But AI clearly has value when it’s applied to many tasks that humans don’t do especially well. And so it’s worth asking, is the focus on human-like capabilities holding back AI progress in applications that don’t depend on recreating features of human reasoning? To be more specific, in the context of genomics, are AI architectures cribbed from language models and image recognition tools really the right way to go?

Borrowing a Jetsons analogy from Max Bennet, are we wasting effort trying to build a humanoid robot that can hold a conversation and clean the house (left) when what you really need is a one-armed robot (right) that can do complex but specialized tasks in a factory?

I don’t pretend to have a serious answer to this question, but it seems useful to raise it. In the rest of this post I want to explore the question by thinking about thoughtful benchmarks. To measure progress and spur innovation, we need to think carefully about tests of AI that are not merely based on whatever evaluations are convenient to make, but rather on tasks that are specifically designed to reflect our big goals for AI. The punchline is that progress in biological applications of AI is going to depend on devising much better ways to test our models.

Benchmarking AI on fluid intelligence

While Tyler Cowen thinks AGI might be here in a few days, AI scientist François Chollet does not. The Atlantic recently ran a story profiling Chollet, called “The Man Out to Prove How Dumb AI Still Is”. The piece is largely about a famous set of benchmarks that Chollet created with his collaborators, ARC-AGI. (ARC stands for Abstraction and Reasoning Corpus.) The ARC benchmarks are based on an interesting 2019 paper by Chollet called “On the Measure of Intelligence.” He argued that most tests of AI’s capabilities overestimate how good it is, because they they are too heavily based on prior knowledge:

[S]olely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power.

Showing that GPT can score in the 90th percentile on the LSAT is not really a significant test of intelligence, because the GPT was trained on the entire corpus of the internet. What we should be measuring, Chollet argued, is the capacity to solve problems that don’t depend on lots of prior skill and knowledge, something called fluid intelligence. The ARC benchmarks were designed to test fluid intelligence and, as the Atlantic piece points out, AI models completely bombed when they were first tested.

The ARC benchmarks have spurred innovation and AI now does much better. OpenAI’s o3 showed impressive gains, but it still fails to solve puzzles that most elementary school students could solve. And when it does solve these problems it takes on average 14 minutes of compute time to do something that takes most humans just a few moments of inspection. Here’s an example of a puzzle o3 couldn’t solve:

You can try these puzzles yourself. Spend a few minutes with them and you’ll quickly realize that they test features of intelligence that are much more universal among humans and more profound than what’s needed to get a high score on the LSAT.

Thinking about the ARC-AGI benchmarks is useful for a few reasons. First, it should deflate any expectations that AGI is just around the corner. My ninth-grader solved the above puzzle in a few seconds. Second, ARC-AGI shows that thoughtful benchmarks, as opposed to benchmarks of convenience like the LSAT, can spur innovation. And third, they highlight the heavy emphasis of AI development on achieving features of human intelligence. ARC-AGI is easy for humans but hard for AI. What about applications of AI that are not easy for humans? Does it make sense to rely almost exclusively on AI methods that are developed to achieved human-like capabilities?

Subscribe now

What’s the equivalent of fluid intelligence for molecular reasoning?

One could reply to the above question by pointing to the Nobel Prize won by the Demis Hassabis and John Jumper for AlphaFold. AlphaFold’s ability to accurate predict 3D protein structures is incredibly useful, and its is based on decades of human-intelligence-inspired AI development. However, AlphaFold does include some critical innovations in its architecture, so it’s not just an off-the-shelf LLM. That’s not always true of the increasingly common genomic LLMs that seem to appear in the literature every week. Not all, but much of the AI in biology is taken almost off-the-shelf from applications in computer vision or human language.

I’m not an AI expert and definitely not qualified to discuss innovations in AI architecture. But as a field, we should ask ourselves, how can we spur innovation that’s relevant for biology? Humans are good at solving the ARC-AGI puzzles, but we’re terrible at finding subtle patterns in megabases of genomic sequence. What’s the genomic equivalent of fluid reasoning that we want genomic AI to achieve?

Right now, the field relies on the equivalent of benchmarking GPT on the LSAT. We use convenient large-scale datasets to evaluate our models. Sometimes this makes sense, because we want AI models that can accurately predict the outcomes measured in those datasets. But, to push the LSAT analogy, an AI model that gets a perfect score on the LSAT likely won’t be able to engage in real-life legal reasoning if it lacks the kind of fluid intelligence that ARC-AGI tests for. Similarly, it’s one thing for a genomic model to predict gene expression levels in a handful of cell types; it’s something else to be able to predict gene expression in any cell type, in any genetic background or disease state. One definition of genomic AGI might be AI that can replace our need to do an experiment, because the model predicts what happens in a new context with as much accuracy as our experimental measurements would have had.

Again, I don’t have the answer. But to highlight the problem, I’ll summarize some important genomic benchmarking papers that are relevant to considering where we go from here.

Protein-ligand binding AI models are still just memorizing training data

A group out of the University of Basel has introduced the “Runs N’ Poses” benchmark dataset, a set of 2,600 high-resolution protein-ligand structures not included in the training data of the latest deep learning 3D structure predictors, including AlphaFold3. They binned the structures in their benchmark dataset according to how similar the structures were to other structures in the training data. They then asked, how accurate are the models on structures that are very different from the training data, compared with structures that are similar. The results show that these models don’t generalize well:

There is a big decrease in accuracy on structures that don’t look like what the models have previously seen. This is evidence that success in this area still largely comes from memorizing the training data, rather than learning the biophysical rules that govern how proteins bind ligands.

Predicting regulatory genetic variant effects: Big AI doesn’t beat simpler models

A group at UC Berkeley and Genentech has released TraitGym, a benchmark dataset of non-coding regulatory DNA variants associated with a set of Mendelian diseases or a set of complex traits. For readers not steeped in genetics, Mendelian diseases are usually caused by a variant in one gene with a large effect on the phenotype, while complex traits are typically affected by variants in many different genes with more subtle effects.

The key result is that relatively simple models based on evolutionary alignments of DNA sequences. In the plot below, performance (AUPRC) is represented on the x-axis.

The best model, CADD, as the TraitGym authors describe it, is a “is a logistic regression model trained to distinguish proxy-deleterious from proxy-neutral variants,” built on a large set of feature annotations, some of which were derived from protein language models. Enformer (ranked #2 on complex traits) is a a supervised, transformer-based AI model. Notably, the big self-supervised AI models (HyenaDNA, Evo2 with 40 billion parameters, etc.) did much worse.

Large language models don’t outperform simpler models, again

Anshul Kundaje and his lab at Stanford found similar results in their benchmark analysis, DART-Eval. The benchmarks were designed to test “sequence motif discovery, cell-type specific regulatory activity prediction, and counterfactual prediction of regulatory genetic variants.”

There are multiple interesting results in this analysis. One suggests that language models have not learned to identify DNA sequence features that define cell type-specific regulatory DNA. The UMAP plots below show the results of clustering the model embeddings of cell type specific regulatory sequences. The models were presented with a collection of cell type-specific regulatory sequences (derived from scATAC-seq data) and tried to learn which sequences are active in the same cell type.

There is much more in the paper, but I’ll just cite a key conclusion:

Our systematic evaluations reveal that current annotation-agnostic DNALMs [DNA Language Models] exhibit inconsistent performance and do not offer compelling gains over alternative baseline models for most tasks, despite requiring significantly more computational resources.

Language model embeddings underperform

The last study I want to highlight is from Peter Koo’s lab at Cold Spring Harbor. The authors asked whether the more complex representations learned by language models were better at predicting the activity of regulatory DNA compared to simpler representations based on one-hot encoding. The question is whether the language models learned a good (and generalizable) representation of the underlying DNA sequence grammar. The answer is that simpler representations do better:

Their conclusion:

To assess the transferability of knowledge acquired during pre-training for current genome language models, we evaluated the predictive power of pre-trained representations from four gLMs on whole genomes (without fine-tuning) across six functional genomics prediction tasks with appropriate baselines for comparison. We found that within cis-regulatory elements, representations from pre-trained gLM provide little to no advantage compared to standard one-hot sequences.

There is much more to discuss, but this post is already getting too long. What the results above illustrate is that very large models–the GPTs of genomics–don’t seem to be learning general representations of protein-ligand interactions or DNA sequence grammars. They’re not learning the biology, and in many cases, simpler machine learning tools outperform the big models. We’re far from AGI for biology, whatever that might mean. To spur innovation, we need more benchmarking analyses like those above.

Perhaps this should include new experimental datasets that are specifically designed, like ARC-AGI, to test whatever the equivalent of fluid intelligence should be for genomic reasoning. Our goal should be to achieve AI that learns meaningful representations of fundamental biology that generalize across contexts. Rather than aiming for AI that’s can do anything that humans can do, but better, in genomics we need AI that can accurately predict anything that humans can measure in an experiment, but haven’t.

1. Bad AI will be bad for science

In this week’s issue of Nature, computer scientists Arvind Narayanan and Sayash Kapoor warn about the potential for bad AI to set science back. Narayanan and Kapoor are the authors of AI Snake Oil, a book warning about AI hype, and one that I included in my recent post of recommendations for learning about AI. The authors suggest that an overreliance on AI in science is bad for two reasons.

The first is incompetence: like conventional statistics, AI can be used badly by people who lack the training or commitment to use it properly. “Machine-learning modelling is the chainsaw to the hand axe of statistics — powerful and more automated, but dangerous if used incorrectly.” One of science’s dirty secrets is that many, if not most scientists have inadequate formal training in statistics. That includes me–I never took a stats course or even a course with a serious statistical component. Bad statistical practices like p-hacking, due in part to poor training, has caused pervasive reproducibility problems in some fields. Widespread, inexpert use of AI has the potential to be much worse. Narayanan and Sayash write that

[E]rrors are becoming increasingly common, especially when off-the-shelf tools are used by researchers who have limited expertise in computer science. It is easy for researchers to overestimate the predictive capabilities of an AI model, thereby creating the illusion of progress while stalling real advancements.

One of the critical problems is “leakage”, which means that the test dataset is not fully independent of the training data. If information is shared between test and training data, AI models will score well by just memorizing the training data, and then fail to generalize to other datasets when someone tries to use the model for another analysis. The authors created a list of recent studies across disciplines that identify common problems with data leakage in published research. Everyone should read a few of those papers to learn about problems in their field.

It is common in genomic AI modeling to evaluate models on held-out data from the same experiment, such as RNA reads from a subset of chromosomes in a single RNA-sequence experiment. While this may not formally count as data leakage, I don’t think it’s great for the field to almost exclusively evaluate AI models on test data taken from the same experiments as the training data. We should hold ourselves to higher standards and evaluate our models either on independent benchmark datasets, or, if feasible, just do another experiment to generate your test data.

The other problem that Narayanan and Kapoor worry about this that we’ll confuse AI predictive success for real conceptual progress:

AI excels at producing the equivalent of epicycles. All else being equal, being able to squeeze more predictive juice out of flawed theories and inadequate paradigms will help them to stick around for longer, impeding true scientific progress

In other words, we shouldn’t outsource our thinking to AI models. This gets to the ultimate goal of scientific theories, which, aside from their practical uses, is to describe nature in a way that is comprehensible by humans. Science is for human minds. Centuries of scientific progress show that a program of ruthless experimental testing of big ideas about the world leads to new discoveries. In genomics, AI can be a powerful discovery tool that will help us learn how genomes function, but in the end what we want is an understanding that explains how DNA sequence, interacting with other molecular players, leads to cellular and organismal phenotypes.

Aside from a commitment to the empirical foundations of science, one of the most important solutions to the problems raised by Narayanan and Kapoor is education. Everyone, especially everyone who does work or aspires to work in science should learn about AI.

Subscribe now

2. Different diseases, same genes: mutations in cancer and autism

My WashU Genetics Department colleague Tychele Turner and her collaborator Rachel Karchin of Johns Hopkins have a paper out in Cell Genomics looking at genetic variation genes that are associated with both cancer and neurodevelopmental disorders (NDDs) like autism. They sought to learn why mutations or genetic variants in the same genes can lead to different outcomes (cancer or NDDs). They looked at missense variants, whose effects are often difficult to figure out because these variants cause single amino acid changes in a protein, as opposed to larger, more obviously damaging mutations.

The challenge with any genetic variant analysis is that there are many variants that exist in cancer or in individuals with NDD that have no effect whatsoever on disease. Thus geneticists try to come up with clever ways to sift the wheat from the chaff. In this case, Turner and Karchin hypothesized that causal variants for either NDD or cancer would cluster in similar parts of the protein. That is, if we take a protein that plays a role in both cancer development and NDD, we would see that mutations that genuinely lead to NDD would disrupt the protein in similar ways. Mutations that cause cancer would affect the same protein but in different a way, and thus would cluster in a different part of the protein. That is indeed what they found:

In the figure above, showing two views of the same protein, blue represents sites of mutations found in cancers, while red represents genetic variants found in NDD patients. You can see that the sites of the clustered mutations are different for different diseases, suggesting that causal mutations for each disease affect different parts of the protein.

The study illustrates the power of AI for discovery in genomics–the scientists used AlphaFold structure predictions to perform their 3D clustering analysis. Looking at the 3D structure is important because mutations that may be distant in the linear sequence of the protein may be close together in folded, 3D structure. The big takeaway is that while both NDD and some cancers are associated with missense mutations in the same gene, the patterns of missense mutations, at least for some genes, differ between the two diseases. The same gene can be damaged in different ways and lead to dramatically different outcomes.

Share

3. Data problems for AI models in drug development

In 2023, a group of news organization sued OpenAI and Microsoft for copyright infringement, claiming that GPT was trained on millions of copyrighted works without permission from the copyright holders. The suit highlights one of the challenges of cutting edge progress in AI: you need enormous amounts of data, on a scale that couldn’t possibly be generated by one organization. At the same time, it’s probably not feasible in most cases to pay for access to all of the data you need, when that data is scattered among many different owners. If the data you need is a bunch of online content, then, like OpenAI and others, you just scrape everything you can get from the internet and figure you can settle the lawsuits later.

But this strategy doesn’t work if your AI needs to be trained on big datasets obtained from experiments, which aren’t always so east scrape from the web. A comment piece in the latest issue of Science highlights the lack of large, high quality open datasets that could be used to advance AI methods for drug discovery. Not even the largest pharmaceutical companies will, at least any time soon, have their own datasets at the scale needed to implement truly cutting-edge AI. The whole drug discovery enterprise would be better off if the players involved collectively supported the development of high quality, open datasets that would allow the field, including industry AI teams, to innovate:

To help AI mature, developers need nonproprietary, open, large, high-quality datasets to train and validate models, managed by independent organizations.

The authors draw an analogy to the bad old days of gene patents, when a lot of useful genomic data was locked up in proprietary databases. Open genomic data has spurred an enormous amount of innovation, and the same thing could happen for AI-based drug discovery if academic and industry players worked together to contribute open data. There is an opportunity here to avoid the problems at the center of the news versus OpenAI lawsuit, and to make deliberate decisions about how to support AI innovation in this field. While I’m on board with the open datasets, I do think the authors of the opinion piece underestimate the impact of task-specific AI models that can be trained on smaller datasets. Not everything needs to be GPT or AlphaFold.

4. Can statistical models really just regress out all confounding factors?

Some time ago I read a piece by Brown University health economist Emily Oster that has stuck with me. I can’t remember the original piece, but there is a 2024 version on her blog that confronts a critical question about epidemiological studies: is it really possible to control for all of the confounding factors in a statistical model? Oster suggests that confounding is much worse than we typically think.

There are many common examples epidemiological claims that are a longstanding source of controversy, such as the question of whether moderate drinking is better or worse than no drinking, whether there is a risk between certain foods and cancer, etc. For example, are people who drink more at a higher risk for cancer because of the alcohol, or because those who drink more also tend to smoke? To answer that, epidemiologists try to control for smoking (and age, sex, exercise, etc.) in their analyses of what alcohol’s effects are.

Oster gives another example of this kind of reasoning:

[T]he reason we know that the processed food groups differ a lot is that the authors can see the characteristics of individuals. But because they see these characteristics, they can adjust for them (using statistical tools). While it’s true that education levels are higher among those who eat less processed food, by adjusting for education we can come closer to comparing people with the same education level who eat different kinds of food.

But what about controlling for the variables that you don’t see? This is of course something that is well known to statisticians, called residual confounding. Oster says:

We all agree that this is a concern. Where we differ is in how much of a limitation we believe it to be. In my view, in these contexts (and in many others), residual confounding is so significant a factor that it is hopeless to try to learn causality from this type of observational data.

She argues that many of the differences among human study subjects that statisticians treat as random (after controlling for confounders) are actually not random. They are much more highly correlated with confounders than scientists think and, importantly, subject to social feedbacks in a way that is not sufficiently appreciated.

Oster gives an example of studies looking at a link between breastfeeding and intelligence:

The link between breastfeeding and IQ is a good example. This is a research space where you can find many, many papers showing a positive correlation. The concern, of course, is that moms who breastfeed tend to be more educated, have higher income, and have access to more resources. These variables are also known to be linked to IQ, so it’s difficult to isolate the impacts of breastfeeding.

So what happens to this link when you control for confounding variables? She points to a 2006 paper that controlled for parent IQ and demographics, as most papers do. But then they kept adjusting for additional effects, such as within family effects (comparing siblings, only one of whom was breastfed). After these adjustments, the correlation between breastfeeding and IQ disappeared entirely.

How bad is this problem across all observational studies in economics, health, sociology, etc? Oster thinks it’s very bad, “that a huge share of the correlations we see in observational data are not close to causal.” These poor studies lead to bullshit headlines and bad media coverage, and populace that is confused by a series of flip-flopping scientific claims about moderate drinking, processed foods, breastfeeding, etc. I’m inclined to agree–we should be very skeptical about the ability of scientists to statistically control for all of the relevant confounding variables in observational studies. While in some cases observational studies are the best we can do, being ‘the best we can do’ should not be an excuse to simply accept flawed evidence.

5. The 100-year old science fiction of the Radium Age.

To round off the newsletter, here’s something for fans of science fiction. Most people who are into science fiction are aware of the so-called Golden Age, when pulp magazines dominated the genre. This ran largely from the 1930’s (John Campbell became editor of Astounding in 1937) through first decade or so after WWII when mass market paperbacks took over. But before the Golden Age was the fascinating Radium Age, which was both more innovative and less male-dominated than what came later. Science fiction of the first quarter of the 20th century, called the Radium Age by publisher Joshua Glenn and characterized by him as filled with a “dizzying, visionary blend of acerbic social commentary and shock tactics.” Most of the classic plots and character types that we ascribe to the Golden Age were already present in the Radium Age. There is a long list of really interesting works by women of this era. Consider the plot of a book written in 1909:

When Mary Hatherley, an intrepid British explorer, is kicked in the head by the camel she was riding through the Arabian desert, she finds herself transported to what seems to be an alternate version of Earth. Arriving in Armeria, she discovers a society in which the very concept of gender is unknown.

How about a 1935 Bengali cult classic describing lost race of Bengali supermen in Uganda?

The era also includes a post-apocalyptic plague novel by Jack London, a satirical cyborg novel, H.G. Wells’ 1913 novel about nuclear war, a 1926 eugenic dystopia, and much more. MIT Press is publishing many of these works, so if this is sounds interesting to you, go check out the series.

Drug development became much more effective in the 20th century as medical scientists began to rationally design potential therapies grounded in a molecular-level understanding of disease. One of the great successes of 20th century medicine is the treatment of childhood acute lymphoblastic leukemia, which was almost 100% fatal in 1900. Millennia of human efforts to treat disease had been completely ineffective at finding a successful treatment for what we now call childhood leukemia. In the mid-20th century, Sidney Farber and his colleagues transformed the treatment of childhood leukema when they introduced folic acid antagonists as chemotherapy. This treatment was based on their growing understanding of the role of folic acid metabolism in leukemia. After decades of further improvement, nearly 90% of children diagnosed with childhood leukemia now survive.

As our understanding of the molecular basis of disease has grown, so has our ability to rationally design drugs. While we still rarely know all of the relevant molecular variables involved in any disease, we do have a significant capacity to rationally design potential therapies based on our knowledge of the proteins and other molecules that are active in pathogenic pathways. We get a lot of mileage out of a fairly simple framework: Disease can often be treated by inhibiting or activating specific macromolecules (proteins and RNA). Therefore, we can devise new treatments by designing therapeutics that physically bind to a target molecule, in order to inhibit or activate it.

It’s hard to state how profoundly different this framework is from what came before. In 1949, Linus Pauling, Harvey Itano, and Ibert Wells published a classic paper in Science, titled “Sickle Cell Anemia, a Molecular Disease.” In it they compared the chemical and physical properties of hemoglobin from patients with sickle cell disease to hemoglobin from unaffected people. They proposed, for the first time in history, an evidence-based molecular mechanism of disease, which, they said, “if it is correct, it supplies a direct link between the existence of ‘defective’ hemoglobin molecules and the pathological consequences of sickle cell disease.”

Of course, the route from an understood molecular disease mechanism to a successful cure can be a long one. Just last year, a CRISPR gene therapy for sickle cell disease became the first CRISPR-based therapy approved by the FDA, 75 years after the paper by Pauling and his colleagues. But our improving capacity for rational drug design allows scientists come up with potential therapies much faster, and enables us to treat diseases that no amount of human ingenuity could have solved under the old paradigm of drug discovery.

Share

Rational drug design with nucleic acids

Today, rational drug design most often refers to using 3D models of protein structure to find potential surfaces to which a designed chemical or peptide inhibitor could bind. One of the most famous examples is Gleevec, a molecule that was specifically designed to inhibit the ATP binding pocket of the Abl protein kinase that drives chronic myelogenous leukemia. This kind of rational drug design is still difficult, because most proteins do not have an obvious druggable surface or binding pocket. The development of therapeutic antibodies and, more recently, AI models of protein structure do help, but the fact remains that it is very difficult to design specific inhibitors or activators of particular proteins.

But one class of therapeutics avoids the complex problem of 3D modeling altogether–DNA and RNA. Because nucleic acids interact (usually) through standard Watson-Crick base pairing, you don’t need to solve a complex 3D puzzle to design a DNA or RNA molecule that binds a specific target; all you need is the sequence of A’s, C’s, G’s, and T’s from a genome database. And because every protein that you might want to inhibit is derived translation from RNA, there is an RNA target for everything. It is completely trivial to design an RNA or DNA molecule that will bind to an RNA target. Perhaps the most striking example of the ease of RNA design is the COVID mRNA vaccine, which took just two days after the virus sequence was posted.

I don’t want to get too carried away–mRNA vaccines and oligonucleotide therapies rest on decades of work. It may be simple to design a DNA or RNA sequence, but raw DNA and RNA delivered into human tissues are not effective. They need to be chemically modified to increase their stability, reduce their toxicity, etc. It was this work that led to the 2023 Nobel Prize awarded to Katalin Karikó and Drew Weissman for their work on mRNA vaccines.

Rationally treating disease with antisense oligonucleotides

In 2017, the New England Journal of Medicine published an astonishing example of rational drug design. The study reported results of a new treatment for a neurodegenerative disease called spinal muscular atrophy (SMA). In its severe form, symptoms of motor neuron degeneration appear in infants, who are never able to sit up and eventually can’t breathe unassisted. SMA was first recognized as a distinct disease in the 1890’s, and it was invariably fatal before a child turned one or two. (There are “milder” forms of the disease with symptoms that appear in adults and become progressively worse.) In one tragic example, described in 1902, four out of eight children in the same family died of SMA:

In 1902, Beevor described a male patient with a more severe form of SMA. The patient was the eighth child in a family with eight children, and he was the fourth affected child. The first affected sibling (sister) was noticed to be paralyzed all over at the end of the first month of life and died before 4.5 months of age. The second affected sibling (sister) developed symptoms at 6 months of age, and died at 8 months of age. The third affected sibling (gender unknown) developed symptoms at 6 weeks of age and died at 7 months of age. The patient was noticed to be paralyzed all over, excepting the diaphragm, before 5 weeks of age, and died after living eight weeks.

In the 2017 study, led by Richard Finkel of the Nemours Children’s Hospital in Orlando, Florida, 51% of infants treated with a new therapeutic reached a developmental milestone of motor function, compared to 0% of infants who received the placebo. The effect was so impressive that the study was terminated early, so that other children could be treated. A related study, which began treating infants with the disease before the onset of symptoms found that all 25 children in the study were alive several years later, at a median age of 4.9. The new therapeutic that successfully treated this fatal disease was nusinersen, an antisene oligonucleotide (ASO). Nusinersen was not the first ASO treatment, but it has become the most successful. The nusinersen story illustrates the power of rational nucleic acid drug design

The concept of an ASO to treat disease is simple, though development is still challenging. Thinking back on our framework for rational drug design, the goal is typically to design a drug that either inhibits or activates a molecular target. In the case of ASOs, the molecular target is the RNA transcript of a gene. The ASO is designed to have a sequence that is the reverse complement (‘antisense’) of a short region of the RNA molecule. When the ASO binds to the RNA, the RNA can be affected in several ways. The most straightforward effect is shown below: ASO-bound RNA is degraded by RNase H1, which breaks down RNA/DNA double helices.

Other ways that an ASO destabilized an RNA target is prevent translation of the RNA into protein by physically blocking the binding of the ribosome, or by preventing the RNA from being properly spliced, leading to nonsense-mediated decay. You can also stabilize an RNA in various ways, including by physically obstructing a binding site for a negatively-regulating microRNA (miRNA), thereby preventing RNA degradation. This recent review has a nice discussion of different approaches. The key point is that, while a big fraction of proteins in their 3D forms might be undruggable, every messenger RNA that codes for those proteins can likely be inhibited or stabilized by an ASO.

Subscribe now

In the case of nusinersen, a rational design was based on careful knowledge of the underlying cause of SMA. In the 1990’s scientists mapped the genetic basis of SMA to a gene called SMN1, which is necessary for the survival of motor neurons. When an infant is born with two loss-of-function mutations in SMN1, they develop the disease. However, there is a gene that can compensate for the loss of SMN1, called, sensibly enough, SMN2. The problem is that SMN2 rarely produces a functional protein, because it is spliced improperly, and one of the critical exons is skipped. In 2006, a team led by Ravindra Singh at U Mass Worcester described an ASO that that blocked the site on the RNA that caused exon-skipping. The ASO thereby led to the production of functional SMN2 protein. In patients whose SMN1 genes are non-functional, nusinersen increases the expression of functional SMN2, which compensates for the missing SMN1.

It took 10 years to go from this original ASO design to an FDA-approved drug. For a previously untreatable, fatal genetic disease, that is a fairly fast drug development timeline. Since the approval of nusinersen, gene therapy to cure the disease and a small molecule drug that has the same effect as nusinersen have been approved. That’s great news for SMA patients, and nusinersen is now being considered a bridge therapy to keep patients healthy before receiving gene therapy.

Future ASO Therapeutics

I don’t want to sound too pollyannish; identifying individuals with ASO-correctable diseases and finding RNA sites that can be effectively targeted with ASOs requires careful development. And delivery of ASOs can also be a challenge. To treat neurological diseases, ASOs, which can’t get through the blood-brain barrier, need to be injected intrathecally (into the spinal cord). SMA patients have a miracle drug available, but it requires such injections every four months.

But the tremendous advantages of rational drug design with ASOs, which now include a massive collection of biotechnological and computational tools that didn’t exist in 2006 when nusinersen was designed, has created a growing opportunity. One of these includes the ability to create organoids derived from a patient's cells, which allows scientists to rapidly screen many different ASOs for effectiveness in the lab. These new tools have created broader opportunities, and ASOs are in development to treat various cancers and other neurological diseases like Huntington’s.

One of the most interesting opportunities is the potential to treat very rare genetic diseases, even diseases for which the size of the patient population might only be 1. In a review article published last year, a group called the N=1 Collaborative, led by Timothy Yu at Boston Children’s Hospital, argued that the potential to create rationally designed, individualized therapies with of ASOs and other nucleic acids requires the medical community to think about how to evaluate treatments designed for a single person. There are thousands of extremely rare genetic diseases, which cumulatively affect a lot of people. Many of these diseases could be treated with ASOs, but there are too few individuals in each case to assemble a statistically well-powered clinical trial to evaluate the therapy. Yu and his colleagues layout a framework for developing rigorous N=1 trials.

In a sense, such efforts at individualized drug testing are a throwback to the era before rational drug design, when potential therapies were tried on small numbers of patients, and physicians would carefully follow the patients to see whether they got better or worse. Without a foundation of knowledge of the biological basis of disease, this approach wasn’t very effective. But with today’s molecular knowledge and capacity for rational drug design, we have the chance to reclaim part of that older paradigm and design afforable, individualized treatments for thousands of rare diseases.

One of the most important abilities you need for a successful career is knowing how to update your skills. As I tell my kids, this isn’t just important advice for scientists, who work in a profession that is supposed to generate new knowledge and new technology, and thus is always changing. The New York Times recently covered the “Gen X career meltdown”, describing the challenges of media professionals who entered journalism, advertising, film, etc. in the 1990’s and early 2000’s. The media business looks nothing like it did 20-30 years ago.

The same is true of much of biology. When I started grad school, a little over 20 years ago, I had no sense whatsoever of the pace of change in science. While some of the skills I learned in grad school (e.g., yeast molecular genetics) are still useful, most of them, in my graduate field of functional and structural genomics, haven’t withstood the transformative impact of next-generation sequencing, CRISPR, cryoEM, deep learning, long-read sequencing, and high-throughput imaging. What did withstand those transformations was solid training in scientific reasoning and in the ability to teach myself what I needed to learn. Back then, when the draft human genome sequence was still hot off the press, some of my fellow grad students and I decided to teach ourselves Perl and R so that we could be better participate in this trendy field called genomics.

In that spirit, I want to encourage everyone learn more about AI, including scientists whose training and work may be mostly non-computational. Especially if you’re still early in your career, you have time to build your skills and bring deep learning into your work. To be clear, if you’re serious about doing computational biology, you’ll need to do more than just read some books; you’ll need to spend some time training with real computational biologists. But it is possible to, in the words of Harvard computational biologist Sean Eddy, “ go where a question takes you, not where your training left you.” Eddy described his own background, in this great 2005 piece, “Antedisciplinary Science”:

I've been a computational biologist for about 15 years now. We're still not quite sure what “computational biology” means, but we seem to agree that it's an interdisciplinary field, requiring skills in computer science, molecular biology, statistics, mathematics, and more. I'm not qualified in any of these fields. I'm certainly not a card-carrying software developer, computer scientist, or mathematician, though I spend most of my time writing software, developing algorithms, and deriving equations. I do have formal training in molecular biology, but that was 15 years ago, and I'm sure my union card has expired. For one thing, they all seem to be using these clever, expensive kits now in my wet lab, whereas I made most of my own buffers (after walking to the lab six miles in the snow, barefoot).

For the antedsciplinary scientists (and non-scientists) out there, here are my recommendations for books to learn AI. They range from gentle and popular introductions to fantastic textbooks that rigorously cover the math. (Eddy’s piece, by the way, helped tip the scales in favor of my decision to come to Washington University in St. Louis, where Eddy was at the time. I wanted to be around people who thought like that.)

Subscribe now

1. The Hundred-Page Machine Learning Book, Andriy Burkov

If you want to spend one weekend jump starting your AI education, this is the book. Anyone with decent high school math skills can pick this up and learn about probability, linear and logistic regression, decision trees, SVMs, basic learning algorithms, and your standard neural networks. There are some small code examples in Python. It’s well-written, well-illustrated and covers all of the basics. After reading this book you’ll know the basic vocabulary of machine learning, have a general idea of what the basic algorithms are, and know how they are applied to some standard problems. I recommend this book as your first foray into AI because it includes accessible math, rather than only vague verbal descriptions of what AI does. You can download chapters for free to try before you buy.

2. AI Snake Oil, Arvind Narayanan and Sayash Kapoor

A colleague serving on a university AI committee with me recommended this book. Read this book, by two computer scientists, to teach yourself not to be credulous about AI. The authors write a popular Substack bearing the same name of the book (see the previous link), and their big argument is this: predictive AI is wildly overrated, performing worse than advertised, while generative AI is much more useful. They are very skeptical that AGI (artificial general intelligence) is imminent, and find AGI itself to not be a very coherent concept. It’s a good book, another one you can read in a weekend.

The most important message, not emphasized enough in contemporary discussions of AI, is that we should be very skeptical about AI performance claims, because so often models that perform well on benchmark datasets used during development will fail when applied in another context. Domain shift is when a model is asked to make predictions in a setting that is systematically different from its training data, and it is one of the most difficult problems to solve in AI. But that fact doesn’t stop snake oil salesmen from overselling their models to predict everything from insurance risks to recidivism to product recommendations. As a culture, we should be much more skeptical of predictive AI.

Share

3. Why Machines Learn, Anil Anathaswamy

AI pioneer Geoffrey Hinton blurbed this book as “a masterpiece,” and he’s right. If you want to read a popular-level book that covers the math and history of AI, in a non-technical way, this is the book to read. It’s pitched at about the level of Cornell mathematician Steven Strogatz’s book Infinite Powers or Johns Hopkins physicist Sean Carroll’s Biggest Ideas in the Universe books. Anathaswamy takes the story of AI from the perceptron in the 1950’s up to today’s deep learning models, while very gently presenting the math along the way. He’s interviewed many of the pioneers in the field, who describe what it felt like to work at various stages in machine learning history. Read this book to build your intuition for the field, learn some history, and to get a lucid layperson’s explanation of Bayes’ theorem, matrices and vectors, maximum likelihood estimation, and the other important math that makes AI work.

4. The Science of Deep Learning, Iddo Drori

Now we get to real textbooks. Iddo Drori wrote this concise textbook based on courses he taught. If you have a decent background in calculus and probability, you can dive right in to deep learning with this book. It starts with forward and backpropagation, and covers optimization, various deep learning architectures, including transformers, generative models, and reinforcement learning. If you’re busy with your day job but want to learn the math behind the critical backpropagation algorithm, or to understand how transformers work, this textbook has what you want.

Along with the clear explanations and rigorous math, there are very helpful illustrations. When it comes to math, I’m a visual learner. I can parse the math in a condensed description of backpropagation, but I can’t get an intuition about how it works without pictures. Drori’s illustrations are very helpful for building intuition. Some of the chapters are free online - see the link above.

5. Probabilistic Machine Learning, Kevin Murphy

This is a recent update of a classic textbook. In contrast to Drori’s concise book, which starts in media res with backpropagation, this book starts at the beginning with a rigorous presentation of basic probability and statistics. While in principle anyone could pick up this book and learn probability from scratch, the reality is that the notation and concepts, while clearly presented, will be hard to follow if you don’t have some college-level background in this field. However, the good news is that, if you have had some introductory, college-level calculus, probability, and statistics, this book has everything you need to learn the math of AI.

The book is free online via Github and has tons of figures and code examples, all of which can be downloaded as jupyter notebooks. For advanced, up-to-date topics, there is the 1300-page volume 2. I have a whole shelf full of math and statistics textbooks, few of which are written as clearly as this one by Kevin Murphy. If you are really serious about learning the mathematical foundations of AI, and ready to invest a substantial chunk of time, this is the book to use.

The one thing missing from these recommendations is programming for AI. There are plenty of good books out there; I like this one from the publisher Packt.

If you like recommendations for learning scientific topics like this, I suggest checking out John Baez’s How to Learn Math and Physics.

Our new dean of the school of public health at Washington University in St. Louis, Sandro Galea, lays out the real stakes in the current head-on assault of our health institutions in an editorial he co-authored with UC San Francisco’s Kirsten Bibbins-Domingo. They start out by correctly noting that the Trump administration’s crudely implemented policies, such as the proposed cut of indirect costs to 15%, would be “a near-existential threat to academic health research as we know it.” They also note that it is not clear how much, if any, of the proposed changes will stick. I’m still inclined to think that most of them won’t, but it’s a low-confidence opinion.

One way to think about the value of health research is through a cost-benefit lens. The authors point out that the scale of NIH funding, which is greater than any other funding body in the world enables large-scale projects and research infrastructure that wouldn’t be possible otherwise. Being a country that leads in such large-scale projects brings enormous benefits to the U.S. Publicly funded advances in vaccine technology and public health efforts for routine childhood vaccination prevent tens of thousands of deaths and tens of millions of illness over the lifetime of each year’s birth cohort, saving billions of dollars in societal costs.

But there is more to say in defense of academic health research than cost-benefit analyses. Galea and Bibbins-Domingo ask “How should we consider the value of academic health research, and perhaps, implicitly, of health itself?” Their answer is that the purpose of fostering health among all of our citizens is to give people the opportunity to have more agency over their lives:

The point of this work is to allow people to be healthy so they can live, however they choose to. Hence, in a world of ideological and partisan division, health should be the ultimate nonpartisan good. We do the research we do so that more people can live long, healthy lives, and live them as they wish to live them.

Those of us in academic health research should make it clear to our fellow citizens that we take this purpose seriously. We value the trust we’ve been given, largely through tax-payer dollars, to do work that is ultimately meant to make everyone’s lives better. As health researchers, regardless of what motivated us to go into science, the ultimate purpose of our work is to allow people and democratic society more broadly to thrive by reducing the burden of disease:

If a country sees itself as a robust, vibrant, thriving, and growing country—as certainly the US of myth and nationalistic narrative suggests it to be—it requires a strong academic health research enterprise to allow it to inhabit that vision of itself. That should make academic health research as core to the national identity as our vision of a democratic country that permits and encourages self-determination.

2. The mystery of long distance enhancers deepens.

Most of the functional genome consists of regulatory DNA, but exactly how regulatory DNA works is still one of the biggest mysteries in biology. Distal enhancers, which, in linear genomic sequence, are often located very far away from the genes they regulate, are among the most puzzling functional elements of the genome. We know a few things about them: they are bound by DNA-binding proteins, they are transcribed by RNA polymerase to produce enhancer RNAs (whose function, if any, is unclear), and their chromatin epigenetic state is often distinctly different from the DNA around it.

We also know that distal enhancers are typically very cell type-specific. Their specificity is why the same genome can specify the gene expression programs of the great diversity of cell types in the human body, which include motor neurons that transmit an electrical signal across the ~1 meter distance between the human foot and the base of the spine, and a ~100 µm cardiomyocyte that keeps up an unrelenting rhythmic contraction every moment of our life.

But what exactly happens, biochemically, when an enhancer activates its target genes? This is still mysterious. What does an enhancer deliver to the target gene promoter? How close in 3D space, and for how long, does it need to be to deliver whatever it is that it delivers? Why do enhancers often skip by the nearest gene to target another, more distant one?

One source of clues about enhancers are cases when one long distance enhancer regulates more than one gene. In these cases, two different target genes seem to compete for the enhancer, suggesting that the enhancer can only do whatever it does at one gene at a time.

A new paper in Genes & Development casts doubt on that idea. The paper describes a single enhancer that regulates two functionally unrelated genes, and it can do so without any competition between the target gene promoters.

The study, led by Wouter de Laat of the University Medical Center Utrecht in the Netherlands, describes an impressive series of deletion experiments that tease out the role of this particular enhancer in shaping the local 3D genome structure, thereby allowing its two target genes to come into close proximity.

Those who care about the details should check out the paper, but I’ll just make two comments about this investigation into one of the most critical molecular processes at the foundation of life: First, the enhancer itself, and not just the normal CTCF/Cohesin machinery seems to be important for shaping the compact 3D genomic domain that brings the enhancer and its target genes close together. Not all enhancers do this, as far as we can tell. And second, there seems to be no competition between the enhancer’s target genes, meaning that whatever it is that this enhancer brings to a target gene promoter, there is enough of it to go around. When the investigators inhibited one target gene, there was no effect on the other, which is not what we expect if the gene promoters were competing for the enhancer’s attention.

This is further evidence that not all enhancers work in the same way, and that is what makes these regulatory DNA elements so mysterious. Enzymes aren’t like this - alcohol dehydrogenases all more or less work the same way (to the best of my knowledge). But ‘enhancers’ may not be a natural biophysical category–different regulatory DNA sequences may have evolved to activate their target genes in different ways.

Share

3. Will GMO regulations slow down new therapeutic biotechnologies?

The March 2025 issue of Trends in Biotechnology has several pieces reflecting on the 50-year legacy (or notoriety) of the Asilomar meeting on the then-emerging technology of recombinant DNA. For some scientists, Asilomar is an example of the community showing the ability to regulate itself: the rapidly developing ability to move genes around among species had potentially catastrophic consequences (in the view of some), and so scientists agreed on a moratorium until the issues could be more carefully considered. Others consider Asilomar the epitome of scientific arrogance, because scientists assumed that they, and not legislators, should be the ones to decide how to handle the risks of recombinant DNA. (This is for another day, but I think neither of those positions is correct.)

A piece by Hans-Georg Dederer, a law professor at the University of Passau, Germany, discusses how GMO regulations might inhibit medical advances based on new genetic technologies. He begins with a premise that I agree with (and, again, will discuss another day):

The 1975 Asilomar conference contributed to the misperception that recombinant DNA (rDNA) technology is inherently risky to human health and the environment.

Dederer argues that Asilomar confused the risks of a process–methods and technologies that involve recombinant DNA–with risks of specific products that should be evaluated on their own merits, and not by virtue of the fact that recombinant DNA was used to create them. He writes about how this confusion, a legacy of Asilomar, could obstruct the approval of new therapies based on CRISPR or other gene editing technologies, or pharmaceuticals that are produced by what Europe categorizes GMOs. While Europe has made some modifications to their laws to address concerns like this, Dederer argues that the law has not kept up with the rapid technological progress, and thus may work to hinder the arrival of say, new cancer therapies based on genetically engineered cells.

It’s a thought-provoking article that puts current debates within the larger historical context of post-Asilomar views on genetic engineering.

4. Early origins of the human genome project

Bob Waterston, former chair of the Department of Genetics at Washington University in St. Louis, was one of the early movers and shakers of the Human Genome Project. His insights were crucial to developing many of the important technologies that got the project off the ground, and his role is why we here at Washington University have a Genome Institute today. Waterston later moved from Washington University in St. Louis to the University of Washington in Seattle, where he became chair of the Department of Genome Sciences. (The fact that he was chair at both places does not help the common confusion of the University of Washington with Washington University.)

Waterston has a memoir out in the Annual Review of Genomics and Human Genetics. It’s an insightful and well-told account of how one of the most consequential efforts in biology came to be. If, like me, you’re inspired by frank accounts of how stuff really gets done in science, don’t miss this. Also, for a second perspective, I posted a transcript of a talk by another of our former chairs, Mark Johnston, who was deeply involved in these early days of genomics.

5. A Brief History of Intelligence

I have been a New York Times subscriber since the 90’s. For most of that time, one of my favorite sections was the book review. But something there seems to have changed–these days I rarely find reviews of anything that interests me. Part of the problem is that they’ve turned half of the book review into a listicle section, and the other problem is that they now rarely (not never, just rarely) review serious books on science or intellectual history that connects to science. They also review basically no fiction that interests me. The whole book review section has become boring.

I now get my book review fix from The New York Review of Books, a list of interesting literature posters on X that I follow, and from recommendations on Substack. I found something interesting this week that I’ll pass along here–Max Bennett’s A Brief History of Intelligence. It came out in 2023, but if it was reviewed in the NYT I missed it. (And a search didn’t turn up anything.) I found the book due to a recommendation by Stetson, author of the interesting Substack newsletter Holodoxa, and the economist Jason Furman:

Here is the book’s description:

Artificial intelligence entrepreneur Max Bennett chronicles the five “breakthroughs” in the evolution of human intelligence and reveals what brains of the past can tell us about the AI of tomorrow. In the last decade, capabilities of artificial intelligence that had long been the realm of science fiction have, for the first time, become our reality. AI is now able to produce original art, identify tumors in pictures, and even steer our cars. And yet, large gaps remain in what modern AI systems can achieve—indeed, human brains still easily perform intellectual feats that we can’t replicate in AI systems. How is it possible that AI can beat a grandmaster at chess but can’t effectively load a dishwasher?

Max Bennett, the book’s author, is a tech entrepreneur and not a historian or scientist. I don’t usually buy big think books by people who aren’t scholars in the field, but I’m making an exception for AI. The field is moving really fast, and so much of the innovation is taking place outside of academia. Anyway, I’m on an AI reading binge. While I haven’t read this one yet, I thought the recommendation was worth passing on.

Today there were nationwide “Stand Up for Science” rallies. The organization website suggests that we explain why we stand up for science. Here is my take.

I studied for my PhD at the University of Rochester, whose motto is meliora, a Latin adjective that the University translates to mean “ever better”. When I first heard the motto, I associated it with the more familiar ameliorate–to make better. If I were to sum up what it means to stand up for science, it would be that motto: meliora. When we stand up for science, we should be standing up for a better life for all. More specifically, we should stand up for the following:

1. Science as the long-term engine economic progress.

200 years ago, most people lived on less than the modern equivalent of two dollars a day. The reason why more than 90% of us today are not that poor is because of science-driven technological progress that thoroughly transformed the material conditions of human life. No amount of education, hard work, free markets, or government reform would have achieved this transformation without scientific advances that increased crop yields, reduced the burden of disease, and are the basis of the materials and technologies involved in modern transportation and communication. We should stand up for the institutions and approach to knowledge that make up this engine of progress.

Subscribe now

2. Public trust that makes modern science possible.

Those of us who work in science are able to do so thanks to decades of public trust underlying the state-funded work that is the seedbed of the scientific enterprise. That public trust is founded on the expectation of meliora–that what we do as scientists can, over the long run, make all of our fellow citizens’ lives better. We can’t control the wise or unwise decisions of politicians that have the biggest impact on how broadly prosperity is shared. But as scientists we can stand up for the idea that our work is for everyone, regardless of partisan affiliation, for minorities and majorities. Standing up for that principle means acknowledging that people are different, and that, to take the example of health research, we do need to study differences between sexes, genders, races, etc. Everyone deserves evidence-based medicine.

3. Honesty, rigor, and a commitment to evidence.

The reason science has been the engine of progress is because, when its institutions are healthy, it is reliable. Those of us who work in science need to be absolutely clear about our commitment to the values that make science work. It means speaking out against fraud, arguing with but not canceling dissenting views, not shelving negative results, especially not with the excuse that we did so to avoid “weaponization” of the results, and not trying to pretend that low quality studies supporting our prior views are in fact strong evidence. It means not our pulling punches against cranks, like the one who now runs HHS, but it also means standing up against attempts to foist on science unreliable epistemic values that come from more radical corners of the humanities–such as risible claims that “indigenous” knowledge should be taught alongside science.

Share

4. A willingness to be accountable for our work.

We should be willing to answer honest questions. Yes, the world is filled with bad-faith trolls who are trying to discredit science, and they should be treated as such. But the attitude we shouldn’t take is something I saw on a sign in a photo of a protest– “In science we trust.” “Trust the science” is too often used as a way to brush off criticism and avoid answering tough questions. While politicians, journalists, and the general public do not have our disciplinary expertise, we owe it to them to explain what we’re doing and take critical questions seriously. One of the worst recent examples is the effort to shut down debate over the origins of Covid. Philosopher Peter Godfrey-Smith describes what happened as “a degree of insistence and refusal to doubt that I think is very discouraging… an attempt to pre-label future dissent as incompetence or deception.” Every one of us with a Ph.D. in science was trained to answer tough questions. We owe the public more than just “trust the science.”

5. Stand up for our students and colleagues.

I am as much an unbeliever as Richard Dawkins, but I greatly appreciate my wife’s church. Each week, when the congregation is invited to come up for communion, the pastor finishes his lines by says “all are welcome.” And he means it. I’m sure many churches do this, but I know for a fact that a great many don’t. All should be welcome in science too, and we should stand up for them. I know we’re not going to fix misogyny and racism with platitudes, but at least as individuals we can make it clear that our student, technician, postdoc, and faculty colleagues belong here, that they are evaluated on how they do science and not on who they are. Almost everyone here could be making more money doing something else. From a purely mercenary perspective, we get a better scientific workforce when we draw talent from all across society. From a humane perspective, we should defend our community from malicious outside efforts to label our non-white colleagues as DEI hires, or to deny respect to our gay and trans colleagues.

If you scanned the headlines of some leading science outlets the other week, you could be forgiven for thinking that the generative AI can now design a working genome. Here is a story in Nature describing the release the Arc Institute and NVIDIA’s’s new foundation model, Evo 2:

This story reached beyond the bounds of the scientific community and was even mentioned on The Ezra Klein Show this week. But can Evo 2 (or any generative AI) design completely new genomes that actually function? The answer is that we have no idea. Nobody has shown it yet.

I’m not writing this post to pick on Evo 2. Brian Hie, Patrick Hsu, Hani Goodzari and the Arc Institute team (working with NVIDIA) are pushing the envelope of genomic foundation models. Their innovations in model design are described in a companion paper to Evo 2. I am enthusiastic about this work. But we need to be clear: while ever larger generative AI models (40 billion parameters for Evo 2) can natural-looking sequences at scale, there is very little evidence–and in some cases none at all–showing that these sequences function as expected. The rate-limiting step in genome design is still the experiment.

Subscribe now

So why are we hearing claims that generative models are writing genes and genomes? At this point, the field largely relies on comparisons to existing datasets, rather than a thorough program of experimental testing. In the case of Evo 2, the team prompted the model with various sequences,such as a bacterial genome, and then examined the generated sequence to see how well it resembled natural sequences. In the case of the generated bacterial genome (prompted with the very small M. genitalium genome), the authors show that a model-generated 580 kb sequence has a gene density resembling the natural genome, and that the synthetic gene sequences show structural matches to natural enzyme genes.

But could this genome function coherently, with properly regulated, fully-functioning genes? We won’t know until someone synthesizes the entire thing and puts it into a bacterial cell. The experiment is feasible with today’s technology (actually with 2008 technology), but it is resource intensive. My intuition is that no generated genome will actually function without a lot more tinkering to make it work, or without training models in a different way (more on that below). I could be wrong! Be we don’t know because the experiment hasn’t been done.

Here is a more specific example of one of the problems. The Arc team had Evo 2 generate a mitochondrial genome. For those not familiar with mitochondrial genomes, they are independent, mini genomes present in the eukaryotic cells’ mitochondria. They encode a handful of genes, many of which work in complexes.

Evo 2 generated a set of mitochondrial genomes that seemed to have the right complement of genes. But there are three issues. First, we don’t know which of the generated genes are actually functional. They resemble the natural mitochondrial genes, but they may have variation in them that leave them inactive. Second, regulation of genes in the very compact mitochondrial genome is complex, and we have no idea whether Evo 2 produced functional regulatory sequences. And third, even if we assume that all these genes are indeed functional, each one most closely resembles a natural gene from a different species, as seen in Table S6 of the paper:

Those species are the muntjac (a deer native to China), a type of mouse-eared bat, a species of labeonins (freshwater fish), a harbor porpoise, a moose, a yak, the Balkhash marinka (freshwater fish), the Sambar deer, the domestic sheep, a species of antelope, the zebu (a type of cow), a species of eelpouts (marine fish), and a species of carp found in Thailand. While these are all vertebrates, and mitochondrially encoded genes are generally well-conserved, I am very skeptical that you can put together a functioning electron transport chain with a mix of fish and mammalian genes.

The problem is that Evo 2, trained on over 8 trillion bases of sequences from many species, may not have learned how different members of a complex within a single species genome work together. Yes, Evo 2 learned which genes go together in the mitochondrially encoded genome. But the 45 carp genes for mitochondria Complex I have evolved to work with each other in carp cells, and the 45 harbor porpoise genes evolved to work each other in porpoise cells. The carp genes did not evolve to work with the porpoise genes-and Evo 2 probably doesn’t know that.

Share

Again, I could be wrong, but until the experiment is done, we do not know how realistic the genomes generated by Evo 2 are. They could be genuinely functional, or they could be like the non-sensical AI-generated engine blueprint up at the top of this post. We just don’t know until we’ve done the experiments.

To be fair, the Evo 2 team is working on a set of experiments, and in their Evo 1 paper last fall, they tested some designed CRISPR-Cas complexes and a new transposable system. But AI-designed sequences at the scale of even a small genome have not yet been tested.

One of most successful gene designers is the recent Nobel laureate David Baker, who has spent decades investing in tools for both careful design and experimental synthesis and testing. In their most recent preprint, the Baker lab tested their AI-designed peptide inhibitors of the proto-oncogene Ras. They characterized the designed proteins in vitro, and showed that these inhibitors bound their targets in live cells.

Thorough experimental testing needs to become the standard in this field. I was surprised to see a paper last fall in Genome Research, which claimed to design realistic sequences but did not show that any of them function. Again, this paper was from a great team (at Genentech) whose work I admire, and I understand the value of papers that report innovation in model architecture and training. There are non-trivial computational problems to solve. But without experiments, we do not know whether these models work at all.

I suspect that large models trained on large sequence datasets will not be enough. There are real efforts to design and physically build synthetic genomes, the most ambitious of which is the yeast Sc2.0 project. It has taken years of debugging and re-synthesis to build a synthetic yeast genome, which is, by eukaryotic standards, pretty small. AI design will require a lot more human intervention to be successful, at least in the near future. Successful design will also require skilled experimentalists who can show that the designed sequences work.

Understanding how genotype relates to phenotype has been one of the fundamental challenges in biology for more than a century. More explicitly, the challenge is to understand how the linear DNA sequence of the genome encodes the necessary information to robustly generate an adult organism, and how variation in DNA sequences leads to differences between members of the same species.

Kevin Mitchell (at Trinity College Dublin, and author of multiple books on neurobiology, including most recently Free Agents) and Nick Cheney (at the University of Vermont) have a thought-provoking piece in Trends in Genetics that addresses these questions by working out an extended analogy between the genome and generative neural networks, specifically variational autoencoders (VAEs). I like this comparison because it is a very specific way of seeing the genome as a compressed representation of an organism that is decoded during development.

To motivate their argument, Mitchell and Cheney point out the flaws in the more common metaphors of the genome–as a program, a recipe, or a blueprint. Here they are on the genome as a blueprint:

First, an architectural or engineering blueprint is isomorphic with the desired product, that is, distinct parts of the blueprint correspond directly and specifically to distinct parts of the product. In this way, the blueprint concept is almost preformationist, with the genome containing a direct mapping of the final product. Second, a blueprint does not usually contain instructions on how to build the object in question, it only has information on what it should look like when completed. This clearly leaves a major question unanswered: how are the processes of development specified so as to yield the desired outcome? And finally, a blueprint typically specifies an object in such detail as to be effectively deterministic, leaving little room for the kind of variability in developmental trajectories and outcomes that is typically observed even in genetically identical organisms raised in highly controlled, effectively identical environments.

A vivid example of the genome as a compressed representation is the single-celled zygote, the state in which we all began.

What does the genome encode directly? Sequences of proteins and functional RNAs, as well as regulatory elements (whose encoding in DNA sequences is still not well understood). In all of these sequences, Mitchell and Cheney say that the key feature that is encoded is “differential affinities.” Proteins and RNAs function by forming selective interactions with other proteins, RNAs, DNA, and small molecules. Back in a 1958 paper, a few years before the genetic code was cracked, D.L. Nanney made a similar argument, that the genome is a “library of specificities.”

Mitchell and Cheney argue that, like a VAE which reconstructs new images or other outputs by running the compressed information through a series of higher dimensional neural network layers, genomic information is realized through a succession of cell types as development and differentiation occur. I haven’t considered this enough, but I’m not sure how induced pluripotent stems cells and direct reprogramming of one type of cell into another fit into the analogy of a compressed representation being decoded through a series of developmental states. With direct reprogramming, you skip directly from one state to another.

Mitchell and Cheney do make it clear that they’re not arguing for a perfect analogy between any kind of neural network and development. The real strength of their conception is that it lets them set up a more specific, formalized model.

There is a lot more in the piece, including a section on evolution as the encoder in the VAE analogy. It’s worth a read for anyone interested in ways to conceptualize the relationship between genotypes and phenotypes.

Subscribe now

2. Why different mutations in the same transcription factor genes have such different clinical outcomes

Speaking of cell biology being largely about differential affinities and specificities, a new paper in the American Journal of Human Genetics shows that the phenotypic impact of mutations in the gene for the transcription factor BCL11B corresponds to whether those mutations alter the affinity or specificity of this protein’s binding to DNA.

Mutations in BCL11B cause a variety of developmental syndromes that feature cognitive, morphological, and immunological problems. The challenge is that, since all of these mutations occur in the same gene, it is very hard to predict which mutations will have severe versus mild outcomes.

The scientists analyzed pathogenic or likely pathogenic BCL11B mutations from 92 patients, and found that they could group patients into three clinical subtypes based on where in the gene the mutation occurred. Interestingly, mutations at positions in BCL11B that contact DNA, and which are responsible for its DNA binding affinity or its specificity, resulted in the more severe and variable clinical outcomes. Mutations elsewhere in the protein were relatively mild.

The results indicate that the most damaging mutations are those that change the ability of BCL11B to target the right set of genes, rather than simply attenuating its ability to activate genes. This alone isn’t so surprising: if you fail to turn on the right genes, or turn on the wrong ones in a particular context, bad things happen. What this paper shows, and what I think is less appreciated, is that different mutations in the same transcription factor will not affect all of its target genes in the same way. The different clinical outcomes probably reflect the fact that each mutation affects different sets of targets, with different biological consequences. (We made a similar argument in a paper last year.)

3. Regulating AI in medicine

Former FDA Director Scott Gottlieb has a written a thoughtful piece about how AI in medicine should be regulated. It’s obvious to everyone that the incorporation of AI into health care is inevitable, because there are clearly enormous potential benefits, but there are also enormous risks. The issue facing everyone, but especially the FDA as a regulatory agency, is how to balance the risks.

Gottlieb is worried that AI tools “with advanced analytical capabilities” will be automatically classed as medical devices by the FDA, under recent change in its guidelines. Why is that a problem? Because it would prevent new AI capabilities from being incorporated as add-ons to existing software tools, like electronic medical record (EMR) systems or other software intended to support clinical decision making. If an AI tools is classified as a medical device, that means EMR developers would be incentivized to exclude them, because it brings a higher level of regulation to bear on the entire software suite:

If these tools are classified as medical devices merely because they draw from multiple data sources or possess analytical capabilities that are so comprehensive and intelligent that clinicians are likely to accept their analyses in full, then nearly any AI tool embedded in an EMR could fall under regulation. The risk is that EMR developers may attempt to circumvent regulatory uncertainty by omitting these features from their software. This could deny health care clinicians access to AI tools that have the potential to transform the productivity and safety of medical care.

Gottlieb notes that the 2016 21st Century Cures Act was written to avoid exactly this kind of outcome, in which digital health tools are regulated not based on their clinical use but rather their analytical sophistication. AI tools that don’t make autonomous diagnoses or treatment decisions shouldn’t be treated as medical devices.

The rapid growth of EMRs in clinical practice has opened up a lot of potential to improve health care, by letting different doctors for the same patient easily share information, and by facilitating medical research with very large cohorts and high statistical power. The ability of AI tools to make recommendations based on the synthesis of different data sources is another way EMRs could lead to better care. That will only happen if the incentives are there to develop these tools.

Share

4. What’s the return on investment for the NIH budget?

The NIH has been a target for the new administration’s firing and spending freezes. Despite a court ruling that the NIH needs to spend the money appropriated by Congress, the works are still gummed up and study sections have by and large not been able to review new grant submissions. This is a frustrating and completely unnecessary self-own; the U.S. benefits by being the place where much of the world’s biotechnology is invented. (Though there are ways we could benefit more–choices made by our government have left us with higher drug prices than elsewhere.)

Any time the NIH or NSF or NASA budgets come up, the perennial question returns to the discourse: what do we get for our research dollars? One of the best places to find an answer to that question is at Matt Clancy’s New Things Under the Sun Substack page, where he has a continuously updated piece on the ROI of government spending on R&D. His answer:

This is very challenging to estimate, but a variety of research points to an additional dollar of government sponsored R&D generating $2-$5 in benefits via economic growth.

If you want the details of the calculation, with references to the literature on this subject, check it out.

As someone who has received multiple NIH grants, there are a few points about these grants I think are worth highlighting:

1. I, and most colleagues that I am aware of, value the trust placed in us by the taxpayers whose money funds our labs. The NIH funds our work in order to benefit everyone through advances in knowledge and technology, and I take that seriously. We all should–the knowledge generated by science should benefit everyone, regardless of partisan affiliation, identity, etc.

2. Grant budgets are tight. There is not a lot of slack in a standard NIH R01. You need at least two funded R01s to run a modestly-sized lab these days. It does not cover all of costs of research, some of which are in fact borne by the universities and hospitals we work for. This is not intended as a complaint. The point is that government-funded scientists have very strong incentives to spend their grant money efficiently, because if you waste it you won’t have enough to accomplish your scientific aims.

3. Over the decades, the U.S. has developed world-leading scientific research institutions, and, even though there are some aspects many of us would reform, we are still the place where scientists from around the world want to work. We have our edge in biomedical research thanks to a longstanding, bipartisan commitment to government-funded science. This is an enormous national resource that we shouldn’t recklessly discard, and we shouldn’t assume it can easily be rebuilt if we break it through ignorance and carelessness.

On the related issue of indirect costs paid by the NIH to universities, Harvard geneticist Sasha Gusev, who write The Infinitesimal, has a detailed discussion of indirect costs and much more. Indirect costs have been a big source of misunderstanding:

A research contract is work: when the government hires General Dynamics to build a jet, they pay for the hangar; and when the government hires a university to study cancer, they pay for the lab space.

In his conclusion, which I agree with, he doesn’t mince words:

What happens while everything is shut down? Excellent funding applications that took months to develop are summarily rejected because their mechanism was cancelled; high-scoring grants are passed over for funding because the review council cannot meet; effective drugs are delayed for approval because the FDA has a staff shortage; longitudinal cohorts that have been collected for decades collapse and their data becomes polluted; and — most importantly — the next generation of talented people who planned to pursue research for the public good are cruelly let go or not hired and drift away. In most cases we may not know for years that a study or a dataset was important and should not have been abandoned. And because the whole thing is being conducted without rigor, even when cuts increase efficiency they will be confounded by all the other functions grinding to a halt.

5. What is convolution and why is it important in deep learning?

Convolutional neural networks are everywhere these days, and they often have impressive predictive power compared to more standard statistical models, like linear regression. So why is a convolutional neural network so special? If you’ve wondered just what convolution is, Christopher Olah has a superb series of blog posts on the concepts behind neural networks, including convolution. It was written in 2014, but it’s still a great explanation of the idea.

If you’ve had any statistics, you are probably familiar with some of the ideas behind convolution. If you have two random variables, each with their own probability density function, what is the probability density function of the product of these two variables? Let’s take a discrete case: if I roll two dice, a six-sided die and twenty-sided die, what is the probability that the product of the two rolls is 18? Well, if you roll a 1 with the first die, then you only get 18 if you roll 18 on the second die. But you could also get 18 by rolling a 2 and a 9, etc. You get the idea. To see why this matters, check out Olah’s post.

OK, there was one other piece of big news this week in the world of AI and biology. The Arc Institute and Nvidia announced a new foundation model, Evo-2. I’ll have some things to say about it next week, along with some commentary about what standards we should have for judging the performance of generative DNA models. Have a great weekend.

Why was there an opioid addiction epidemic in America? One cause was Purdue Pharma’s aggressive efforts to convince physicians to freely prescribe a drug that the company knew was addictive. But another cause is that treating pain effectively is an unsolved problem. Millions of Americans suffer from not only acute pain, after surgery or an injury, but also debilitating chronic pain. For awhile, I was one of them. Terrible ergonomic habits at work caused me to develop severe neck and shoulder pain that lasted for years. If you had asked me, I would have rated my pain as at least a six out of ten on most days, and it was rarely better than that. Chronic pain is not just uncomfortable but distracting, making it difficult to focus on anything. I suffered, but my productivity also suffered, as it does for many people with chronic pain. Across the U.S., according to a 2012 study, the total annual cost of untreated pain, including both health care costs and lost productivity, is more than half a trillion dollars.

For a long time, the only drugs on the market to treat severe pain were opioids. Because they can be addictive, and because they lose their efficacy over time as people develop tolerance, opioids are not a solution for chronic pain, but people suffering from chronic pain take them anyway. I was once prescribed opioids after an outpatient procedure. I didn’t need them to recover from the procedure, but I took them anyway to get some relief from my incessant neck and shoulder pain. I stopped taking opioids when my prescription ran out, but others aren’t so lucky. In one study of Medicare patients, 10% continued to persistently use opioids after receiving a short-term prescription with a surgery. When you combine an unmet need for pain treatment with lax prescribing practices for an addictive drug, you get the origins of an opioid epidemic.

America’s opioid epidemic, now fueled by cartel-produced fentanyl, has developed beyond its origins in the abuse of prescription pain medicine. But the need for better drugs to treat severe pain is still unmet. That is why the FDA’s first approval of a non-opioid drug for severe pain is an impotant milestone. Approved at the end of January, suzetrigine (commercial name is Journavx, don’t ask me how to pronounce it) is a first-in-class, non-opioid analgesic from Vertex Pharmaceuticals, the company known for its breakthrough cystic fibrosis drugs. Suzetrigine was found to be effective for treating pain in two randomized, double-blind trials of patients who had undergone a surgical procedure. These trials looked at efficacy for acute, post-surgical pain, but ongoing trials will determine whether the drug is safe and effective for other indications, such as diabetic neuropathy and other forms of chronic pain. If it is, then this drug, and likely others in the pipeline, will transform how we treat pain.

Reaching this milestone is something that could only be accomplished on a deep foundation of neurobiology built with decades of publicly-funded research. The story behind Suzetrigine shows how enterprising and forward-looking industry research programs, working in tandem with academic scientists, can turn scientific knowledge into therapeutics that make people’s lives better.

Subscribe now

Blocking pain before it reaches the brain

Suzetrigine is a small molecule that works by inhibiting a voltage-gated sodium channel called Na_V1.8. Sodium channels mediate the electrical signaling of neurons by allowing sodium ions to pass through the cell membrane. (Wikipedia has a nice primer if you need it.) By blocking the action of Na_V1.8 sodium channels, Suzetrigine prevents pain signals from being transmitted to the brain.

This sounds simple, but the contrast with opioids illustrates the challenge of inhibiting the neuronal signaling in very specific ways. Opioids act on µ-opioid receptors, many of which are located in pain-processing regions of the brain. Unfortunately, these receptors are also present in the reward systems of the brain, as well as in the brain stem, which controls our breathing, and in the small intestine. Hence, opioids not only affect our sensation of pain, but they cause euphoria, produce constipation, and depress breathing. In other words, opioids are dangerous because they aren’t selective enough for those parts of our nervous system responsible for signaling pain.

The history of the past few decades of pharmaceutical research for analgesics is basically a quest for drugs that are selective inhibitors of pain signaling. Suzetrigine is a medical milestone because it is highly selective for Na_V1.8 sodium channels, which are almost exclusively present in the peripheral nervous system, and not the brain. By inhibiting sodium channels in the peripheral nervous system, primarily at the dorsal root ganglion where the peripheral nervous system connects with the spinal cord, Suzetrigine blocks pain signals before they reach the central nervous system. The drug thus leaves the pain and reward centers of the brain unaffected, and avoids the side effects that make opioids so dangerous.

Before we get into the scientific backstory, it is important to emphasize another important contrast between Suzetrigine and opioids. This is the difference between drugs that are developed to be specific using the molecular knowledge of modern science, and non-specific drugs based on a phenomenological understanding of medicines common in pre-scientific thought. 8000-year-old cuneiform tablets from Sumer describe the use of opium as medicine. For most of human existence, people discovered and used plant products as medical treatments without knowing anything at all about how they worked. While there are undoubtedly useful medicines that were discovered this way, all drug action happens at the molecular level. It is much more effective to develop drugs based on a molecular-level understanding of biology.

This may seem obvious, but we live in a society in which loud so-called health influencers with big platforms want to re-mystify biology and medicine by invoking pre-scientific theories like miasmas. Just like you can’t fix a car that won’t start unless you know the parts of the ignition system, we can’t develop medicines that address challenging medical conditions without detailed knowledge of the specific molecular interactions at the foundation of biology. The knowledge of those molecular interactions is the product of decades of publicly-funded research. In the rest of this post, I will walk through the important discoveries that made Vertex Pharmaceuticals’ success possible.

Discovery 1: Sodium channels

Alan Hodgkin and Andrew Huxley, at the University of Cambridge, discovered the existence of transmembrane sodium currents in neurons in 1952. They received the 1963 Nobel Prize for their fundamental work on the action potential of neurons. Bertile Hille (University of Washington) and Clay Armstrong (University of Rochester) began characterizing the properties of sodium channels in the 1960’s and 70’s, including how these channels were acted on by anesthetics. Daniel Beneski and William Catterall (University of Washington, supported by NIH grant HL-22234 and an NIH postdoctoral fellowship) characterized the protein components of sodium channels.

Over the subsequent decades, researchers discovered different subtypes of sodium channels, eventually identifying nine in humans. The gene for the subtype targeted by Suzetrigine, Na_V1.8, was cloned from rat neural tissues in 1996 by two groups, one at University College London (funded by the Welcome Trust) and the other at Roche. Sodium channels are critical throughout the central and peripheral nervous systems, and different subtypes are expressed in different neural tissues. The discovery of Na_V1.8, not expressed in the brain but rather in the peripheral nervous system, was an important advance.

Discovery 2: Link between Na_V1.8 and pain

In 1999, the University College London group (led by John Wood) showed that mice missing the gene for Na_V1.8 were less sensitive to pain. A University of Arizona group, led by John Hunter, showed a similar effect in rats. Three years later, this same group, in collaboration with Roche, showed that a suppressing the production of the Na_V1.8 with an anti-sense oligonucleotide induced resistance to pain in rats. Further establishing the link between Na_V1.8 and pain, an international group led by Stephen Waxman at Yale discovered that some people with painful neuropathies cary gain-of-function mutations in Na_V1.8 (funded in part by Pfizer and by the U.S. Department of Veterans Affairs). Many other academic groups further characterized the role of Na_V1.8 in pain signals in the peripheral nervous system, such as work led by Richard Carr and Matthias Ringkamp at Johns Hopkins University and the University of Heidelberg (supported by NIH grants R01NS09722 and F32DA036991.)

With the link between pain sensation and Na_V1.8 established, Abbott laboratories, Pfizer, and others, developed molecules that specifically targeted Na_V1.8, in the hope that these could lead to new analgesic drugs.

Discovery 3: Na_V1.8 is expressed only in the peripheral nervous system

The critical piece of the success of Suzetrigine is that it selectively targets a sodium channel that is not expressed at significant levels in the brain or other non-targeted organs like the heart. It is this feature of Na_V1.8 that made it the focus decades of research by both pharma and academic scientists, who knew years ago that a drug that could successfully inhibit only Na_V1.8 would almost certainly avoid the dangerous side effects of opioids.

The specific expression of Na_V1.8 in the peripheral nervous system was established early on. But two important publicly funded human expression atlas projects were cited by Vertex in their clinical trial report as evidence of the expression pattern of Na_V1.8: the Human Protein Atlas and GTEx, two publicly funded projects that set out to comprehensively catalog the distribution of every gene and protein among human tissues. The knowledge of where in the body each gene and protein is expressed is crucial for developing selective drugs that act only where you want them to act.

Share

Converting knowledge into therapy

Fundamental knowledge of sodium channels, and Na_V1.8 specifically, is what enabled pharma scientists to choose Na_V1.8 as an attractive target for drug development. Vertex Pharmaceuticals tested several compounds and discovered one that was much more selective than the rest. Other companies also have Na_V1.8 inhibitors in development. Getting a working drug across the finish line is difficult, even with a deep foundation of molecular knowledge, in part because our knowledge is always imperfect. One reason, though not the only one, that drug development is so expensive is that the failure rate is high. An improvement in success rates for taking a drug from pre-clinical work to FDA approval will help bring down the cost producing new drugs. One of the most effective ways to improve success rates is to base drug discovery on genetic knowledge, most of which is produced by publicly funded academic scientists. A 2019 analysis found that using human genetic evidence to target drug development increased approval rates by more than two-fold. The story of Suzetrigine illustrates why. Extensive evidence in model organisms and in human patients with Na_V1.8 mutations established a strong scientific foundation for a drug development program.

The final important piece of drug development is the regulatory agency. The FDA has an obligation to keep unsafe and ineffective drugs off the market, but equally important is its mission to facilitate the release of drugs that are safe and effective. The agency established important guidelines and processes for drugs that could treat urgent, unmet medical needs, and suzetrigine was approved under the FDA’s Breakthrough Therapy, Fast Track, and Priority review pathways.

The opioid epidemic was driven in part by the criminal activity of one pharmaceutical company, but a major, unmet medical need was another important driver. There are still many unmet medical needs that could be solved with better drugs. Those future drugs will, more often than not, emerge from the fertile soil of our rapidly growing, publicly funded stock of genetic and genomic knowledge.

If you want to learn more about the development of Na_V1.8 sodium channel inhibitors, The New England Journal of Medicine’s podcast Intent to Treat put out an informative episode with Yale neurobiologist Stephen Waxman back in 2023, when the suzetrigine study was published. Waxman has long been one of the leaders of the Na_V1.8 field.

An obvious solution to the data problem is to simulate training examples. With a good simulation, you could generate as much training data as you need. How do you create a good simulation? For LLMs like GPT-4, you could use the trained model to generate much more text, and then train the next on the generated text, etc. That kind of recursive training risks sending your model into a strange place. A paper published last year reported that AI models collapse when trained on data they generated. The authors wrote that “the model becomes poisoned with its own projection of reality.”

An alternative to recursive training is to simulate your data with a different model, one that is not tied into deep learning model that you are attempting to train. My WashU Genetics colleague Willie Buchser and his lab and collaborators at the McDonnell Genome Institute (MGI) have a preprint out that uses an interesting approach: they use the popular 3D computer graphics software Blender to simulate biophysically realistic tissue samples. They then use the simulated data to train a convolutional neural network (CNN) that learns to automatically annotate real tissue samples.

Share

Using all of the data in spatial ‘omics

The context in which Willie and his colleagues are trying to automate tissue annotation is the exploding new field of spatial ‘omics. The ability to measure the expression of all genes in cells and bulk tissues was transformative back when RNA-seq arrived in the scene in 2008. But tissues are composed of complex arrangements of cells of different types, and even cells of the same type may be in different gene expression states at different locations in a tissue. The invention of technologies to perform RNA-seq on single cells was an important step forward because now different cell types in a tissue could be distinguished based on their gene expression profiles. But while single-cell RNA-seq resolves the identity of individual cells, information about the spatial distribution of these cells within a tissue or a tumor is lost.

Barely a decade after the introduction of RNA-seq, RNA sequencing within intact tissues arrived, known as spatial transcriptomics. Science advances in large part by the ability to make measurements at new scales, and the capacity to determine cell types and their expression states on the spatial scale of whole tissues is beginning to transform disease genomics.

A detailed discussion of these technologies will be a post for another day. In the context of the new work by Willie and his colleagues, an important advantage of spatial transcriptomics is that it provides not just gene expression values and spatial coordinates, but actual images of tissue slices whose features can be annotated with labels that mark known anatomical features. But this anatomical information isn’t often used because most spatial ‘omics analyses take what the authors call a bottom-up approach. When working from the bottom up, the molecular sequencing data is used to group individual cells by their gene expression patterns, and then only afterwards are cell clusters mapped back onto spatial coordinates. Instead of directly using prior information about anatomical structure of the tissue, anatomical regions are roughly reconstructed by the clusters of cell types mapped back on to the tissue coordinates. The MGI group argues instead for a top-down approach, in which cells are first labeled by the annotated anatomical region from which they came, after which gene expression patterns can be compared:

Automation is great but annotation is expensive

The top-down approach would have many important applications, such as comparing healthy and pathological brain samples. But the limitation is that annotating tissue sections is time-consuming and requires a pathologist or some other expert to manually mark the boundaries of different regions on each new image. This is just not feasible in most settings, which is why the top-down approach isn’t widely used. But the result is that, as the MGI group points out, information in the dataset is left on the table.

Software that could automatically segment tissue images into different anatomical regions would make the top-down approach feasible. Image analysis is a task that neural networks excel at, so such automatic segmentation of tissue images would be a natural job for a CNN. This, however, takes us back to the data problem I mentioned at the outset: obtaining labeled data to train neural networks is expensive, and in the case of tissue annotation, the data doesn’t exist at the scale you’d need to train a sufficiently good model.

The solution that Willie and his colleagues came up with is the main point of this post: they developed a method to simulate realistic images of tissue sections, and then used the simulated data to train a CNN capable of automatically segmenting anatomical regions in images of tissue slices. Here is the outline of how it works:

They being by simulating individual cells with different sizes, shapes, and intensities, determined by the simulated cell type. This microscale simulation is then used to build a regional simulation, in which populations of cells are built up into larger regions based on simulation parameters for cell density and variation in cell types. Regions are then built up into whole tissue sections, such as the simulated coronal section of a mouse brain shown in panel C above. All of this is done with the CAD software Blender, an open-source suite of tools used for game and movie animations.

Once the 3D tissue simulations are put together, the MGI group then simulates the process of microscopic imaging, including adding debris, tears, optical distortions, etc.:

To do this, we use optics and the rendering engine within Blender. Specifically, we employ a virtual camera with a 15.5 mm focal length and a low f-stop (0.01) to focus on the tissue section from about 1.4 mm away. Then the rendering engine ‘Cycles’ calculates the light being emitted by or bouncing off the samples and uses it to produce an image. After the image layer is produced, we use a compositor layer to add varying amounts of white noise (0-95%), making the image slightly harder to interpret. This process is repeated (per ‘frame’) for the total number of images and produces the simulated ‘microscope’ images for our dataset.

The simulated images are then used to train the segmentation CNN, which they call Si-Do-La (Simulate Don’t Label). Using a few desktop computers, they generated ~4000 simulated images per hour.

How well does Si-Do-La predict the boundaries of cells and anatomical regions? For the detailed performance assessment, check out the paper, but the authors show that Si-Do-La performs well at different scales on samples of mouse brain and spinal cord, and pig sciatic nerve. Here is an example from the paper:

I like this paper because it is a clever example of a realistic simulation that is independent of the machine learning model that will be trained on the output. Unlike recursive training, this approach avoids the risk that the model will fall ever deeper into its own mistaken projection of reality. Realistic simulations probably can’t be done to address every type of data shortage, but it will be worthwhile to put effort into developing good simulations where they are feasible. This might include genome sequence, since the human genome itself is too small and lacks the sequence diversity we’ll need to achieve many of our modeling goals. (Though not all… see AlphaFold.) Simulating data is only one tool in the toolbox when it comes to dealing with limited training data, but there is surely more potential in simulations that could be realized with clever methods like the one described by my MGI colleagues.

Sitting on a bookshelf in my living room are the fifty volumes of my great-grandfather's Harvard Classics. This series, first published in 1909-1910, was selected and edited by Harvard President Charles Eliot, not as an attempt to pick the “best books of all time”, as he put it, but rather “to give, in twenty-three thousand pages or thereabouts, a picture of the progress of the human race within historical times, so far as that progress can be depicted in books.” At the time the Harvard Classics came out, my great-grandfather was a young Latvian political refugee fleeing the violence of the 1905 Russian Revolution. He arrived in America in 1906 at the age of 19, and eventually became an accomplished bacteriologist at Merck Sharp & Dohme.

I never met him, but I suspect that my great-grandfather would have subscribed to Eliot's notion of human progress, progress that is the result of, as Eliot put it, “observing, recording, inventing, and imagining." The name “Harvard Classics” may sound pretentious today, but Dr. Eliot’s "five foot shelf of books" was mass-marketed to an American middle class that was presumed to share Eliot’s optimism about the trajectory of our civilization. The Harvard Classices were billed as a means by which, with just fifteen minutes of reading a day, anyone could gain access to the “prodigious store of recorded discoveries, experiences, and reflections which humanity in its intermittent and irregular progress from barbarism to civilization has acquired and laid up.” Since he owned these books, I presume my great-grandfather agreed with this general idea.

To describe what he was looking for in his selection of books, Eliot invoked the methods of science: “observing, recording, inventing, and imagining.” This is no accident — the Harvard Classics were released at a time when the modern scientific view of the world was arriving at a dominant position in our culture, carrying at least as much cultural heft as the major religions. In a cultural conquest as significant as the rise of major world religions in earlier centuries, science’s description of the world has become a near-universal baseline with which every other belief system, religious or not, is forced to reckon. Most items in the vast and surprising inventory of the scientific Big Picture—electrons, atoms, chromosomes, cells, galaxies, black holes, microwaves, mantle plumes, coronaviruses—are accepted as real by almost everyone, even though much of this inventory was unknown when my great-grandfather was born in 1887. (One striking example: scientists didn’t establish that there was more than one galaxy until the 1920’s.)

There is one volume in the Harvard Classics that most prophetically anticipates the coming cultural influence of scientific thought: Darwin's The Voyage of the Beagle. Too often, the Voyage is left out of Great Books lists in favor of Darwin’s more famous On the Origin of Species. The essayist Adam Kirsch, in an insightful piece on the Harvard Classics, chided Eliot for the excessive enthusiasm that led him to devote two of the fifty volumes to Darwin. Kirsch suggests that a modern edition of the Classics should "doubtless" keep On the Origin of Species and ditch the Voyage for James Watson's Double Helix. Darwin's Origin is often included in collections of great books by default, because it is one of the very few important primary scientific documents that can be read without specialized training. But if I had to pick one book by Darwin to recommend, it would be the Voyage. The Origin is an argument for a specific scientific theory. The Voyage is a much more general foreshadowing of science's cultural force in the modern world, written before science’s overwhelming presence in modern life was fully established.

Share

Observations

The Voyage of the Beagle is Darwin's account of the five formative years abroad (1831-1836) with the crew of the HMS Beagle, an experience that transformed him into the scientist who would discover natural selection. The book was first published in 1839 as the most popular part of a multi-volume record of the Beagle's exploits, and later in 1845 as the definitive edition that we read today. The book is a hybrid, a scientific account fused to the ever-popular travel narrative. Darwin wrote both to summarize his scientific findings and to sell books to a reading public eager for accounts of exotic regions that most would never visit. The result is one of literature's great voyages, much like Moby Dick in its mass of scientific detail and similarly concerned with the meaning of nature and our place in it. What makes Darwin's Voyage stand out in this genre is his authentic, uncompromising scientific vision. Herman Melville in Moby Dick provides a stark contrast. While presenting pages and pages on the science and technology of whales and whaling, Melville agonizes over Providence, fate, purpose, and ultimate meaning in the universe. Darwin also has profound things to say about our place in the universe, but he does so without much fuss over the kinds of unanswerable questions that tormented Melville. Darwin patiently observes, records, and imagines, and in doing so sees a vast, complex planet that nevertheless can ultimately be explained.

Darwin's considerable powers of observation are immediately evident in The Voyage of the Beagle. His aim as a scientist is to infer the story underneath the visible events of nature, which is why he captures the small, but high-information details that allow him to narrow the space of possible explanations. Imagine arriving on the scene of the chaotic aftermath of a city devastated by an earthquake, with misery everywhere. While another writer might have written about human lives being subject to the whims of fate, here is Darwin on the 1835 earthquake that destroyed the Chilean city of Concepción:

The town of Concepción was built in the usual Spanish fashion, with all the streets running at right angles to each other; one set ranging south-west by west, and the other set north-west by north. The walls in the former direction certainly stood better than those in the latter; the greater number of the masses of brickwork were thrown down towards the north-east. Both these circumstances perfectly agree with the general idea of the undulations having come from the south-west; in which quarter subterranean noises were also heard; for it is evident that the walls running south-west and north-east which presented their ends to the point whence the undulations came, would be much less likely to fall than those walls which, running north-west and south-east, must in their whole lengths have been at the same instant thrown out of the perpendicular...

The different resistance offered by the walls, according to their direction, was well exemplified in the case of the Cathedral. The side which fronted the north-east presented a grand pile of ruins... The side walls (running south-west and north-east), though exceedingly fractured, yet remained standing; but the vast buttresses (at right angles to them, and therefore parallel to the walls that fell) were in many cases cut clean off, as if by a chisel, and hurled to the ground. Some square ornaments on the coping of these same walls were moved by the earthquake into a diagonal position.

Instead of chaos, Darwin sees geometry in the patterns in the surviving walls, down to the angles of the ornamental bricks. These patterns allow him to infer the behavior of the earthquake. It is this kind of observation and inference that eventually leads to the capacity to build more earthquake-resistant buildings.

Here is Darwin observing a wasp hunting a spider:

The wasp made a sudden dash at its prey, and then flew away: the spider was evidently wounded, for, trying to escape, it rolled down a little slope, but had still strength sufficient to crawl into a thick tuft of grass. The wasp soon returned, and seemed surprised at not immediately finding its victim. It then commenced as regular a hunt as ever hound did after fox; making short semicircular casts, and all the time rapidly vibrating its wings and antennae. The spider, though well concealed, was soon discovered, and the wasp, evidently still afraid of its adversary's jaws, after much manoeuvring, inflicted two stings on the under side of its thorax. At last, carefully examining with its antennae the now motionless spider, it proceeded to drag away the body.

Darwin notes where and how often the spider has been stung, and discovers the rationale behind the wasp's motion that others might have taken to be a random walk. Driving Darwin's method is a belief that the world has an underlying rationale, one that can be inferred if we look closely enough. The behavior of wasps and spiders is not unfathomable; it is a careful survival strategy. Earthquakes don't randomly knock down walls; they travel in waves and exert forces on buildings at particular angles. Darwin's observations are precise because the world is precise, locked into tight constraints of cause and effect, and therefore we use the tools of reason and observation to understand why the world behaves the way it does.

Darwin showed a disciplined commitment to avoiding unjustified shortcuts in the pursuit of answers. Curiosity may be a common human trait, but rigorously and reliably satisfying that curiosity is not. Darwin often had difficulty convincing people that he was genuinely driven to understand the workings of nature, and that he wasn't secretly prospecting for valuable ores:

I found the most ready way of explaining my employment was to ask them how it was that they themselves were not curious concerning earthquakes and volcanos?--why some springs were hot and others cold?--why there were mountains in Chile, and not a hill in La Plata? These bare questions at once satisfied and silenced the greater number; some, however (like a few in England who are a century behindhand), thought that all such inquiries were useless and impious; and that it was quite sufficient that God had thus made the mountains.

Science depends upon a restless, intellectual dissatisfaction with inadequate, speculative answers. It was this dissatisfaction that drove Darwin to brave thousands of miles while enduring discomforts, seasickness, earthquakes, severe thirst, cold nights on beds of rocks, vile food, and even violent political upheaval, with an energy that is evident on every page of the Voyage.

People

It would be wrong to assume that Darwin's scientific instincts make him a passionless observer of the world. In fact, much of the power of the Voyage comes from Darwin's capacity to find emotional resonance in the targets of his scrutiny. His knack for seeing precisely works with his sense of humanity to make Darwin particularly skilled at rendering people, and describing their poignant and ironic moments as they respond to their natural and social environments.

This is perhaps most vividly seen in Darwin's account of his encounters with the natives at the end of the world in Tierra del Fuego. The crew of the Beagle, on a previous voyage, had basically kidnapped several Fuegians, and the Beagle's captain was bringing them back home. One of these natives, 'Jemmy Button', was purchased as a boy for the price of a pearl button. Now a young man, he is about to return to his stone age culture, after having spent three years within one of the world's most technologically sophisticated civilizations. He hardly remembers his native language and speaks broken English, but, like any teenager, is very impressed with the fact that he can make himself look sophisticated:

Jemmy was short, thick, and fat, but vain of his personal appearance; he used always to wear gloves, his hair was neatly cut, and he was distressed if his well-polished shoes were dirtied. He was fond of admiring himself in a looking glass; and a merry-faced little Indian boy from the Rio Negro, whom we had for some months on board, soon perceived this, and used to mock him: Jemmy, who was always rather jealous of the attention paid to this little boy, did not at all like this, and used to say, with rather a contemptuous twist of his head, “Too much skylark.”

Jemmy is struggling with acute and ambiguous feelings about his background: he is defensive about his people, yet he wants to prove that despite his family roots, he can be at least as civilized as any of his British shipmates. And now he's about to be unceremoniously dropped back among his kinfolk. In the mid-nineteenth century, too many thought nothing of such callous treatment; for them, there was a natural order to things, and Jemmy was best off in his proper place. But Darwin manages, almost inadvertently, to poignantly render the ambiguity of Jemmy's reunion with his family:

We had already perceived that Jemmy had almost forgotten his own language. I should think there was scarcely another human being with so small a stock of language, for his English was very imperfect. It was laughable, but almost pitiable, to hear him speak to his wild brother in English, and then ask him in Spanish ("no sabe?") whether he did not understand him.

Some time later, when the Beagle briefly returns to the region before leaving for good, Jemmy shows up again in a canoe, unclothed, scrawny, and long-haired. His first action is to turn his back on his former shipmates in embarrassment.

Darwin's account of the Fuegians is unforgettable because Darwin was more unsettled by his encounters with these people than with any others during his five year voyage. He is honest about his rattled feelings and about the difficulty he has making sense of what appear to him to be lives of unnecessary misery:

A woman, who was suckling a recently-born child, came one day alongside the vessel, and remained there out of mere curiosity, whilst the sleet fell and thawed on her naked bosom, and on the skin of her naked baby! These poor wretches were stunted in their growth, their hideous faces bedaubed with white paint, their skins filthy and greasy, their hair entangled, their voices discordant, and their gestures violent. Viewing such men, one can hardly make oneself believe that they are fellow-creatures, and inhabitants of the same world. It is a common subject of conjecture what pleasure in life some of the lower animals can enjoy: how much more reasonably the same question may be asked with respect to these barbarians! At night five or six human beings, naked and scarcely protected from the wind and rain of this tempestuous climate, sleep on the wet ground coiled up like animals. Whenever it is low water, winter or summer, night or day, they must rise to pick shellfish from the rocks; and the women either dive to collect sea-eggs, or sit patiently in their canoes, and with a baited hair-line without any hook, jerk out little fish. If a seal is killed, or the floating carcass of a putrid whale is discovered, it is a feast; and such miserable food is assisted by a few tasteless berries and fungi.

Putting ourselves in Darwin’s place, if we were confronted with such people who seemed thoroughly alien, but are nonetheless as human as anyone else in their intelligence, their desires, and their needs, how would we react? In this case, Darwin's method of observing and then inferring the story again serves him well, because he does not try to fit the Fuegians into some morality tale. Instead, recognizing them as fully human, he attempts to understand how it could be that they exist in such an apparently miserable condition.

[O]ne asks, Whence have they come? What could have tempted, or what change compelled, a tribe of men, to leave the fine regions of the north, to travel down the Cordillera or backbone of America, to invent and build canoes, which are not used by the tribes of Chile, Peru, and Brazil, and then to enter on one of the most inhospitable countries within the limits of the globe? Although such reflections must at first seize on the mind, yet we may feel sure that they are partly erroneous. There is no reason to believe that the Fuegians decrease in number; therefore we must suppose that they enjoy a sufficient share of happiness, of whatever kind it may be, to render life worth having. Nature by making habit omnipotent, and its effects hereditary, has fitted the Fuegian to the climate and the productions of his miserable country.

Language

Darwin is a scientific stylist, writing a travel narrative that shows how the practice of science is not an impersonal, algorithmic process, but is instead a human act of imagination. The Voyage shows, at the fine level of sentences and paragraphs, a scientific imagination churning away, observing, sifting, and finally inferring the story of nature's driving forces. As Darwin puts it, “The limit of man's knowledge in any subject possesses a high interest, which is perhaps increased by its close neighbourhood to the realms of imagination.”

In this, Darwin compares favorably with another first-rate travel writer, Herman Melville. Both Darwin and Melville cut their teeth producing best-selling accounts of their voyages to the Southern hemisphere. (In Melville's case the accounts were lightly fictionalized.) Both knew what would sell among the travel-narrative-reading public, but the differences in their styles highlight the power that Darwin's scientific instincts bring to his language.

Melville's imagery is impressionistic, as in this description of the Galapagos from The Encantadas:

In many places the coast is rock-bound, or, more properly, clinker-bound; tumbled masses of blackish or greenish stuff like the dross of an iron-furnace, forming dark clefts and caves here and there, into which a ceaseless sea pours a fury of foam; overhanging them with a swirl of gray, haggard mist, amidst which sail screaming flights of unearthly birds heightening the dismal din. However calm the sea without, there is no rest for these swells and those rocks; they lash and are lashed, even when the outer ocean is most at peace with, itself. On the oppressive, clouded days, such as are peculiar to this part of the watery Equator, the dark, vitrified masses, many of which raise themselves among white whirlpools and breakers in detached and perilous places off the shore, present a most Plutonian sight. In no world but a fallen one could such lands exist.

Darwin, while also drawing a comparison with iron-foundries, doesn’t make vague observations about "blackish and greenish stuff." His imagery depends on observational precision and tight organization of his thoughts. His words, while not always technical, are usually specific, and descriptions are often given in geometrical terms of symmetries, regularities, lines, and circles. Here is Darwin on Chatham Island:

The entire surface of this part of the island seems to have been permeated, like a sieve, by the subterranean vapours: here and there the lava, whilst soft, has been blown into great bubbles; and in other parts, the tops of caverns similarly formed have fallen in, leaving circular pits with steep sides. From the regular form of the many craters, they gave to the country an artificial appearance, which vividly reminded me of those parts of Staffordshire where the great iron-foundries are most numerous. The day was glowing hot, and the scrambling over the rough surface and through the intricate thickets was very fatiguing; but I was well repaid by the strange Cyclopean scene. As I was walking along I met two large tortoises, each of which must have weighed at least two hundred pounds: one was eating a piece of cactus, and as I approached, it stared at me and slowly walked away; the other gave a deep hiss, and drew in its head. These huge reptiles, surrounded by the black lava, the leafless shrubs, and large cacti, seemed to my fancy like some antediluvian animals.

Another strategy that Darwin uses to powerful effect is the shocking contrast between violence and beauty so commonly found in the world. In this case, the murder of a sea captain intrudes on a serene moment in the Galapagos:

The water is only three or four inches deep and rests on a layer of beautifully crystallised, white salt. The lake is quite circular, and is fringed with a border of bright green succulent plants; the almost precipitous walls of the crater are clothed with wood, so that the scene was altogether both picturesque and curious. A few years since the sailors belonging to a sealing-vessel murdered their captain in this quiet spot; and we saw his skull lying among the bushes.

Note that the lake is not fringed with generic green plants, but green succulent plants. The walls of the crater are "almost precipitous"; here precipitous means literally perpendicular, forming a 90 degree angle with the surface. And contrasting with this very specific, orderly scene is a messy reminder of human conflict.

Subscribe now

God

In the early 17th century, Francis Bacon warned that “from [an] unwholsome mixture of things human and divine there arises not only a fantastic[al] philosophy but also a heretical religion.” It’s a warning that scientists have taken seriously ever since. Darwin, in all of his arguments, inferences, hypotheses, and narratives of natural history, quietly refuses to ever invoke God as an explanation. The geographical distribution of animals, the causes of extinctions, the composition of mountain ranges, the layout of the plains of the South American Pampas, are all explained exclusively in terms of natural processes. These processes operate over vast scales of space and time, and are thus often not directly observable. They are inferred from observable evidence: raised beds of fossilized sea shells, thousands of miles from any ocean; folded layers of various types of rock exposed in the great mountains of the Cordillera; the resemblance of the skeletons of long-extinct, gigantic quadrupeds to those of living species. It is by inferences made from observations like these that Darwin and many others developed the grand explanations of the origin of the world around us, explanations that have now thoroughly infiltrated and in many cases completely supplanted those provided by other systems of belief.

Here is Darwin inferring a thoroughly materialistic creation story of the great mountain ranges of the South American Cordillera:

As the beds of the conglomerate have been thrown off at an angle of 45 degrees by the red Portillo granite (with the underlying sandstone baked by it), we may feel sure that the greater part of the injection and upheaval of the already partially formed Portillo line took place after the accumulation of the conglomerate, and long after the elevation of the Peuquenes ridge. So that the Portillo, the loftiest line in this part of the Cordillera, is not so old as the less lofty line of the Peuquenes. Evidence derived from an inclined stream of lava at the eastern base of the Portillo might be adduced to show that it owes part of its great height to elevations of a still later date. Looking to its earliest origin, the red granite seems to have been injected on an ancient pre-existing line of white granite and mica-slate.

The Voyage makes it clear that scientists' refusal to invoke divine agency as a causal explanation does not arise from an innate or personal opposition to religion. At issue is what suffices as an explanation. Darwin refuses to settle for vagueness, and rightly considers statements like 'it was born in nature', or 'God made it' to be no better (or perhaps worse) than saying 'I don't know.' This refusal to accept the unfathomable as an explanation is the first step in the scientific process, and it clears the ground for observation, hypothesis, inference, and experiment to operate. It is based on a belief that the world, unlike God, is not inscrutable, and that our minds can comprehend the detailed links of cause and effect that happen behind the scenes of the everyday, observable world. What you read in the Voyage is an almost fanatical demand for precision and rigor, a curmudgeonly insistence that we don't fill in our stories with what scientists call 'hand waving.' Again and again Darwin rejects such hand-waving, content-free explanations.

Of petrified trees:

How surprising it is that every atom of the woody matter in this great cylinder should have been removed and replaced by silex so perfectly that each vessel and pore is preserved! These trees flourished at about the period of our lower chalk; they all belonged to the fir-tribe. It was amusing to hear the inhabitants discussing the nature of the fossil shells which I collected, almost in the same terms as were used a century ago in Europe,--namely, whether or not they had been thus "born by nature."

Boiling water:

At the place where we slept water necessarily boiled, from the diminished pressure of the atmosphere, at a lower temperature than it does in a less lofty country; the case being the converse of that of a Papin's digester. Hence the potatoes, after remaining for some hours in the boiling water, were nearly as hard as ever. The pot was left on the fire all night, and next morning it was boiled again, but yet the potatoes were not cooked. I found out this by overhearing my two companions discussing the cause, they had come to the simple conclusion "that the cursed pot (which was a new one) did not choose to boil potatoes."

Today, scientific explanations of the world are overwhelmingly what we teach and what we research, and it is these exaplanations on which we base our technologies, our medical practice, and much of our social and and economic organization. This was not yet true when Darwin was writing his travel narrative, but the eventual cultural success of science was probably inevitable. Galileo, Newton, and their 17th century colleagues invented a method, rooted in extremely precise observation and careful inference, to reliably learn about the world. Three centuries of steady growth in reliable knowledge brought about the most astonishing material transformation of human life in the history of our species. By the time of the Voyage, science was making serious inroads into matter and earth's history, and it was about crack open biology in a spectacular way.

My great-grandfather, during his career as a bacteriologist, witnessed the first great taming of infectious disease by the development of antibiotics and new vaccines. Within the past few years, scientists have made the first direct observation of a black hole, invented new therapies to treat disease by editing a patient’s DNA, and built AI that can write code and design proteins. One hundred eighty years after the publication of the Voyage, science is a relentless current in our society whose power can be compared with the streams that Darwin observed in the Chilean Cordillera:

As often as I have seen beds of mud, sand, and shingle, accumulated to the thickness of many thousand feet, I have felt inclined to exclaim that causes, such as the present rivers and the present beaches, could never have ground down and produced such masses. But, on the other hand, when listening to the rattling noise of these torrents, and calling to mind that whole races of animals have passed away from the face of the earth, and that during this whole period, night and day, these stones have gone rattling onwards in their course, I have thought to myself, can any mountains, any continent, withstand such waste?

Pick up a free, electronic copy of The Voyage of the Beagle over at Project Gutenberg. (http://www.gutenberg.org/ebooks/3704)

This essay is adapted from a much earlier piece published at my old blog, The Finch & Pea.

1. Replication initiatives conflate replication with reproducibility

I’m a little late to this, but last December Science reported on a new NIH initiative to fund contract research organizations (CROs) to replicate studies. The NIH invited researchers to apply for $50,000 to be used to help a CRO re-run their experiments. Apparently there were few takers. I think anyone who has spend time trying to get a difficult assay to work in the lab might understand the reluctance to participate. The Science article quotes one researcher who sums up the problem well:

Sean Morrison of the University of Texas Southwestern Medical Center, who edited some of the papers resulting from that project, notes the contract labs didn’t have the resources to do pilot experiments or repeat studies to work out kinks. And such labs often lack “the expertise of academic laboratories, especially when it comes to advanced or specialized techniques,” he says. In some cases, “this led to uninterpretable results.” The murky outcome of some replication attempts may have left scientists behind the original studies feeling their reputation was unfairly tarnished, he adds.

While there are some specific types of studies that should be replicated (replication cohorts for GWAS are good), I and others have been arguing for a long time that it is foolish to conflate replication with reproducibility in science. Chris Drummond of the Canadian National Research Council summed it up well in a piece written way back in 2009 (PDF):

A critical point of reproducing an experimental result is that irrelevant things are intentionally not replicated. One might say, one should replicate the result not the experiment…The sharing of all the artifacts from people’s experiments is not a trivial activity.

Cell Reports in 2014 published an example of just how hard it is to exactly replicate someone’s experiments:

Our two laboratories, one on the East and the other on the West Coast of the United States, decided to collaborate on a problem of mutual interest—namely, the heterogeneity of the human breast. Despite using seemingly identical methods, reagents, and specimens, our two laboratories quite reproducibly were unable to replicate each other’s fluorescence-activated cell sorting (FACS) profiles of primary breast cells. Frustration mounted, given that we had not found the correct answer(s), even after a year. Rather than giving up or each publishing our data without the other laboratory, we decided to work together to solve these differences, even traveling from one laboratory to the other in order to perform experiments side by side on the same human breast tissue sample. This exercise confirmed our suspicions and resolved our problem. Here, we summarize our cautionary tale and provide advice to our colleagues.

Reproducibility, I contend, is usually achieved when multiple labs that use different methods show that a phenomenon is robust. Reproducibility can be poor when journals publish papers with poor controls, p-hacking, and other bad statistical practices. The solution to that is not replication initiatives but rather holding papers to higher standards for publication.

To reiterate, there are specialized cases in biomedical science where something closer to exact replication is indeed important, but those cases generally involve studies with methods that can and should be standardized: GWAS, clinical trials, pre-clinical testing of therapeutics, etc. But exactly replicating a basic science study like this 2010 Cell paper (one included in the Cancer Reproducibility Project) is a waste of time and resources.

Subscribe now

2. Yet another DNA language model

Yun Song and colleagues at UC Berkeley have a paper in Nature Biotechnology that describes a DNA language model trained on the multiz multi-species alignment of 100 vertebrates. What makes their model stand out, the authors claim, is the computational efficiency of the approach compared the big foundation models out there. (I covered one of them here.) Their method is efficient because they didn’t actually train their model on 100 vertebrate genomes, but rather only on the top 5% conserved regions, plus some randomly selected regions.

They tested their model by asking it to classify pathogenic variants from various databases (ClinVar, COSMIC, OMIM, etc.). Their method sometimes does about as well as, and occasionally better than some other standard variant predictors, like CADD and ESM-1b. None of the methods do particularly well on deep mutational scanning data. They scored all 9 billion possible SNVs in the human genome, and have thus created another database of predicted variant effects.

There is nothing special about the performance here, but as this field develops, it is useful to watch the performance of two different approaches to language models: multi-species models, like this one, and functional genomics models, like Enformer. It’s not clear which approach is more effective.

3. Accelerating in vivo enhancer assays

Evgeny Kvon, of the snakefied ZRS enhancer fame, has a cool new paper out describing an efficient, two-color assay to test enhancer variants in vivo:

Here we introduce dual-enSERT, a robust Cas9-based two-color fluorescent reporter system which enables rapid, quantitative comparison of enhancer allele activities in live mice in less than two weeks.

They make one-color reporter mice, each carrying either the wild-type or alternate allele of an enhancer. The mice are crossed to generate two-color offspring, which allows you to measure both versions of the enhancer in the same mouse. This promises to make in vivo enhancer assays much more efficient and reproducible.

Share

4. Training data for deep learning beyond the genome

I don’t intend to use too much of this space to discuss my own work, but we have a paper out in Cell Systems this week that, imho, works on an angle in genomics and machine learning that I think is a under-appreciated. The paper is the thesis work of Ryan Friedman, a fantastic former graduate student who is now a postdoc in Cole Trapnell’s lab. Ryan used a technique called active machine learning to iteratively train models of regulatory DNA on successive rounds of functional assays performed in the mouse retina:

An important point of this approach is that, with functional assays and affordable DNA synthesis, we aren’t limited to training data from the genome. In fact, we can design our experiments to produce training data that is better optimized for deep learning. That the genome may not offer the best training data is something that is increasingly recognized in the field — see this commentary (“Hold out the genome”) by De Boer and Taipale (bioRxiv here), and this very good review by La Fleur, Shi, and Seelig. There isn’t enough sequence diversity in the genome to learn some of the things we want to learn, and a big chunk of the genome is inactive and probably not especially informative. Obviously not all (or even most) genomic assays use synthesized DNA, but we have a number of powerful functional assays that do, and we should take advantage of the opportunity to design experiments to better achieve our deep learning goals.

5. Is it harder to make scientific progress?

Now and again people write big think pieces asking whether scientific progress is slowing. I don’t think you should generalize. AI and biology seem to be firing on all cylinders right now. But perhaps things are different in physics. Physicist Chad Orzel, who write Counting Atoms made the point in a way I found interesting. If you consider that huge progress in physics has been made by finding anomalies, like the Michealson-Morley experiment that failed to detect the movement of the earth, or precession of Mercury that could not be accounted for by classical physics, then this point seems important:

All the anomalies in the sixth decimal place were measured a century ago— modern physicists are looking for changes in the 14th decimal place. You can still come up with new ideas about physics, sure, but they’re incredibly tightly constrained by prior experiments, and testing those theories is vastly more complicated an expensive than it was in the golden era— you used to be able to discover new particles by leaving some photographic plates out on top of a mountain, but now you need millions to billions of dollars of sophisticated electronic detectors.

“All the anomalies in the sixth decimal place were measured a century ago.” Science is cumulative in an important sense, and the scale at which we need to search to find new phenomena in some fields becomes increasingly harder to achieve, because all of the relevant measurements at the easier scales have been made. Biology is not quite the same, but there are some parallels. Samples sizes of GWAS need to be very large to detect the associations that we’re going after these days. Bulk tissue measurements of gene expression aren’t enough; we now need single-cell resolution, and maybe some spatial information too. There are still big questions to answer, but smart people have been at this business for awhile, always pushing at the technological frontier.

Last week Nature published a provocative think-piece by the eminent human geneticist Peter Visscher and his colleagues that works through the potential impact of human embryo editing on disease. They assume for the sake of argument that we can reliably edit individual embryo genomes in tens or hundreds places. Reliable here means that you edit all and only the target positions, so that there is little risk of off-target or incomplete editing. Assuming that capacity, Visscher and colleagues ran a model showing that you could substantially reduce the burden of common diseases among a hypothetical genome-edited population. Disease with polygenic risks, like type 2 diabetes, coronary artery disease, Alzheimer's disease, schizophrenia, and major depressive disorder could be prevented by editing in protective alleles.

They make the bold claim that “editing 40 variants could greatly reduce an individual’s lifetime risk of AD, SCZ, T2D and CAD to less than 0.2%”. The degree of risk reduction obviously depends on the base rate of disease, but the predicted reduction is impressive. For example, editing ~10 variants to protect against type 2 diabetes could reduce its prevalence among genome-edited people by 50 fold! The big point is that if such edits ever become feasible, we need to consider the potentially massive benefit to human health, at least among those whose genomes are edited.

What about the risks?

Anyone reading this surely is immediately thinking about a bunch of caveats, many of which Visscher and colleagues acknowledge and discuss. The model involves some important assumptions, and you could ask, how would this hold up across different environments? How much does genomic “background” matter — would these numbers hold for different populations across the world? What about pleiotropy — would editing a dozen variants to reduce your risk for major depressive disorder also increase your risk for some other condition in as yet unknown ways? And won’t routine genome editing basically bring on a dystopian Gattaca scenario in which those with the means to access editing for their kids dominate the world?

Share

These are all good questions, and Visscher and colleagues discuss them. So do Shai Carmi, Henry Greely, and Kevin Mitchell, in a responding piece in the same journal issue. They are much less sanguine about the risks of genome editing than Vissher, et al. Their most compelling point is that even if you assume that embryo editing technology gets much better than it now is, the method would still impose unknown risks on the non-consenting future child. Mistakes and incomplete knowledge about the impact of particular alleles on development could have catastrophic effects on some children who had no say in the decision to submit their genomes to editing. We need to keep in mind that while idealizations may work for thought experiments, in practice no technology is ever mistake-free. A counterpoint would be that back in the 1960’s and 70’s, similar concerns were raised about in vitro fertilization (IVF). But IVF turned out to be an enormous benefit to millions of families, including mine, and it didn’t have a detrimental impact on the health of children born by the procedure. I recognize that the parallel to embryo editing is very imperfect, but we should keep that example in mind when we talk about unknown risks.

I have followed the individual work of Carmi, Greely, and Mitchell, and I have tremendous admiration for all three of them. Greely, for example, is the author of CRISPR People: The Science and Ethics of Editing Humans, and has clearly thought very carefully about the issues surrounding embryo editing. But I found this Nature perspective a bit frustrating, even a little hysterical too combative in its tone. They repeatedly accuse Vissher, et al of ignoring issues that are in fact discussed in their piece, though clearly not to the satisfaction of Carmi, Greely, and Mitchell. (Caveat: I don’t know what the editing process was, and perhaps an earlier draft of the Vissher piece, shared with those who wrote the response, didn’t address these topics.) Carmi, Greely, and Mitchell ask whether it’s responsible to even have this discussion, concluding their piece with the following sentence: “Is it wise to distract stakeholders, including the public, with a technology that is still a long way off at best, and might never actually be safe?”

The technology will get there but will we need it?

While I’m glad these three wrote an important response, I’m not so sure it’s unwise to discuss the possibilities of polygenic embryo editing. We should be wary of underestimating the technological advances in genome editing that will occur over the next generation. Vissher and his colleagues suggest a time frame of 30 years. 30 years ago, the technology to complete the human reference genome was still under development, and none of today’s major sequencing and gene editing technologies existed.

There are critical applications of gene editing technology other than modifying embryos, especially in basic research and gene therapy for people suffering from catastrophic disease. These applications, not embryo editing, are motivating hundreds of innovative projects to develop more reliable gene editing. This field is moving very fast, and an enormous number of potential technologies are quickly being explored. I think reliable, polygenic gene editing will get here sooner than we might guess. In parallel, the functional genomics field is currently putting in a big effort to expand our catalog of actionable genetic variants, and in 30 years I have no doubt at all that we will have enough causal variants on hand to implement the strategy proposed by Vissher, et al.

Subscribe now

So I do think that, with the technologies moving as quickly as they are, it is important to start thinking about this issue now. But for me the biggest reasons to not go the route of polygenic embryo editing are ones that were hardly touched on in these two pieces:

First, while Vissher and colleagues showed that you can greatly reduce the incidence of disease among genome-edited people, there are easier ways to reduce many of these diseases now or in the near-term future. Drugs, in general, are the cheapest and easiest therapeutics. I recognize that’s not true for everything, but I propose that in 30 years, we will likely see substantial progress in drug treatments for all of the diseases discussed, including Alzheimer’s. Alongside that, gene therapy for already-born individuals is also likely to achieve major progress. The first FDA-approved gene therapy came out in 2018, and many, many more are in the pipeline. Not only will technology to treat disease advance, but we’ll also likely see major progress in preventative treatments. As some of my Washington University colleagues wrote back in 2012, “more than half of the cancer occurring today is preventable by applying knowledge that we already have.” By 2054, editing embryos to reduce disease may seem expensive and difficult compared to other the treatments and environmental interventions available.

And finally, the second barrier to a Gattaca society is the fact that if you are going to edit your embryo, you need to do IVF. Maybe someday we’ll edit embryos in utero, but that is a much more distant technology. It seems implausible that there will ever be enough people who voluntary conceive their children by IVF for embryo editing to have a big impact on human health or societal inequality. Sex is cheaper, easier, more spontaneous, and much more enjoyable, while at the same time, greater forces are driving inequality in society. Embryo editing will likely never be more than a niche factor.

One of the most profound questions that biology tries to answer is, how does the organized behavior of a living cell arise from the physical interactions of non-living components? The physicist Erwin Schrödinger posed this question in a particularly compelling way in a set of public lectures given in 1943, which were published the next year as the book What is Life? Writing about a decade before molecular biology emerged as a distinct field, Schrödinger basically said that physicists can’t explain the picture of life developed by geneticists:

Today, thanks to the ingenious work of biologists, mainly of geneticists, during the last thirty or forty years, enough is known about the actual material structure of organisms and about their functioning to state that, and to tell precisely why, present-day physics and chemistry could not possibly account for what happens in space and time within a living organism. What is Life, p. 4, emphasis added.

Life shouldn’t work but it does

Why couldn’t “present-day” (1943) physics and chemistry account for what happens within a living organism? Because the motion of individual atoms is dominated by random thermal motion, and this “does not allow the events that happen between a small number of atoms to enrol [sic] themselves according to any recognizable laws (p. 10).” The orderly behavior of matter, argues Schrödinger, is described by the physics of statistical mechanics and only holds for large collections of atoms. Thermodynamics, for example, gives a very accurate description of the pressure, volume, diffusion, and heat capacity of a gas at large n, but not for ~1000 atoms. And yet, “incredibly small groups of atoms, much too small to display exact statistical laws, do play a dominating role in the very orderly and lawful events within a living organism (p. 20).”

Subscribe now

Statistical physics as come a long way since 1943, but the primary challenge to reconciling physics with the workings of a cell is still this paradox that Schrödinger identified: cells exhibit remarkably organized behavior that arises from the interactions of an astonishingly small number of molecules. The promoters that regulate the expression of critical genes exist in just two copies in each typical cell, and those promoters are bound by a handful of transcription factor proteins that activate gene transcription. The results for the organism are quite reliable. For example, 99.9% of all babies are born with five fingers on each hand, and this depends in part on the proper functioning of the ZRS enhancer, a regulatory DNA element that is present in only two copies of each cell and bound by a relatively small number of transcription factors. And not only do we need to consider that each cell operates with only two copies of each gene and its associated regulatory elements, but also that all of us start out life as only a single cell. Life works amazingly well at low copy number.

I won’t belabor the point, but I don’t think we have answered Schrödinger’s paradox (the biological one, not the cat one) in a satisfactory way. To be absolutely clear, I am not claiming that there is something happening in the cell that violates the laws of physics. The fact that we can assemble, in a test tube, operational cellular subsystems from synthetic components shows that organized behavior is an intrinsic feature of the interactions between these components. But we don’t have the quantitative models that would, I think, have satisfied Schrödinger. Sure, molecular biologists have identified millions of different types of functional molecules at work in the cell, and we are steadily mapping how they interact with each other. But from a physical perspective, I don’t think we have a great account of how a heterogenous collection of proteins, nucleic acids, and other molecules organize themselves into a coherent operation. Part of the answer, not recognized by Schrödinger but clearly appreciated by the 1950’s, is specificity. Specific, non-covalent interactions among biological molecules are a big part of the answer to Schrödinger’s question. But we don’t have accurate, generally applicable physical models that apply to the kinetics and thermodynamics of a cell. The growing interest in physics of biomolecular condensates is one area of the current frontier. (Here is another classic on this general topic, “Life at low Reynolds number” from 1976 (PDF)).

Share

Life at low copy number

This is a long preamble to some recent work describing a set of exciting new technologies that enable us to more directly tackle the big question that Schrödinger posed: single-fiber assays. If a critical feature of life is regulating genes, which are present at two copies per cell, then we need to understand the binding events that are happening at those individual DNA molecules. So much of cell biological data comes from bulk assays, assays that measure averages over enormous numbers of cells. Single-fiber assays let biologists zoom in and see how many factors are bound on individual copies of regulatory DNA, but they do this at scale using sequencing technology, rather than imaging individual cells.

One of the papers I’m most excited about came out in November, work of the labs of Lacra Bintu and Will Greenleaf at Stanford. (Free bioRxiv version here.) They used one version of this technology to measure the binding states of individual regulatory DNA elements. In a nutshell, the method works by treating DNA with a methylating enzyme that marks sites that are accessible to the enzyme. If a transcription factor is bound to that site, the methylating enzyme can’t get there, and thus the site remains unmarked. By looking at the distribution of marked and unmarked sites on individual DNA molecules, you get binding statistics. For example, you can ask, if I have four DNA binding sites for a regulatory factor, how often are all four sites bound, versus fewer or no sites? How about if I increase the number of binding sites to six or eight?

What the experiment gives you is the frequencies of the different occupancy states of a DNA molecule. There is a direct analogy here to statistical thermodynamics: just like molecules in a gas occupy different energy states as described by the Boltzmann distribution, DNA molecules exist in different protein-bound or -unbound states with probabilities that can be described by a biophysical model. But unlike a gas, in this case we’re observing the microstates of the system directly.

With data like this, the Bintu and Greenleaf labs were able to parameterize biophysical models that describe how protein factors compete and cooperate for binding to regulatory DNA. Using this approach, they drew some conclusions about the relationship between transcription factor occupancy and gene expression and how binding of individual factors to individual sites works (binding seems to be independent). I won’t walk through the full paper here, but I do want to highlight some other interesting papers using related technologies. Andrew Stergachis, at the University of Washington, pioneered the technique called Fiber-seq with Stirling Churchman (at Harvard) and John Stamatoyannopoulos (University of Washington). Stergachis, who came and spoke to our department late last year, has a new preprint out describing DAF-seq, a more versatile form of Fiber-seq that lets you focus in on specific DNA elements of interest.

The DAF-seq preprint by the Stergachis lab has a number of very cool technical improvements over prior versions of this assay, part of which involved developing a very efficient new enzyme to mark the DNA. They also perform a binding analysis to show that the E-box of the NAPA promoter (element 1 in the figure below) nucleates the binding for elements 2 and 3. In other words, protein binding to these sites is not only cooperative, but directional, with one site driving the occupancy of the other sites.

The idea of using DNA-modifying enzymes to mark unbound sites of DNA has been around for awhile (see here, here, here, and here for a few of the important papers). As the technology becomes democratized, we’re going to see more work directly addressing the mechanisms of gene regulation, which get to the heart of Schrödinger’s biological paradox: how a handful of binding events at a single DNA molecule give rise to reproducible, orderly behavior in the face of disorderly, random thermal motion.

One of the most profound findings of genomics is that a majority of the functional portion of our genome is devoted to regulatory sequence rather than genes that code for proteins. (This result was anticipated before the Human Genome Project.) Thus one of the major efforts in the field has been to catalog all of these regulatory DNA elements, which is difficult because they a) don’t follow a clear code like the genetic code for protein-coding genes and b) are often very cell type-specific and are hard to discover unless you make your measurements in the correct cell type. And the easiest, most scalable measurements we can make to detect regulatory DNA elements are only indirect measures of function, such as chromatin accessibility or epigenetic state. Thus making an inventory of all regulatory elements in the human genome is a tall order.

A new preprint reports an updated registry of candidate regulatory DNA elements for the human and mouse genomes. Using new analyses of ENCODE data, there are now 2.35 million human sequences that qualify as candidate regulatory elements. This is one of the most thorough and systematic analyses of regulatory elements across tissues.

There are a few notable features of the updated registry: More than half of these elements are not close to the transcription start sites of genes, meaning that act at a distance to control gene expression. 90% of the candidate elements (identified by indirect measures of function) have been tested by some sort of direct functional assay (which helps one decide whether to move an element out of the “candidate” category). This to me is a surprisingly high number and a nice sign of progress. Also, this update includes a big set of silencers — repressive regulatory elements that have been under-studied and which are not well understood.

The updated registry is a useful resource, but it’s important to keep some important limitations in mind: regulatory elements from many cell types are still not represented because the measurements haven’t been done, and the assays that directly measure regulatory function are still heavily weighted towards manipulable cell lines, rather than primary cells. In other words, there is still substantial work to do before we achieve a compete inventory of the most abundant type of functional element in the human genome.

2. On the meaning of “post-genomic”

One of my goals for 2025 is to work out a better science media diet, now that science Twitter has fragmented into too many sub-communities to keep track of. My old ‘must-read’ book mark folder has links to a bunch of dead blogs and publications that no longer publish anything that captures my interest. Trying to keep track of posts from all of the people I follow on Xitter, Bluesky, Substack notes, and LinkedIn usually leaves me disoriented.

Undark, an independent publication supported by MIT’s Knight Science Journalism Fellowship Program is an outlet that has flown under my radar, but which looks fantastic. It’s going on my daily must-read list for 2025. A recent piece by Yale biologist C. Brandon Ogbunu asks “What does it mean to be in the ‘post-genomic’ age?”Ogbunu discusses the kinds of problems biologists should be thinking about in this era in which sequence data no longer seems to be rate-limiting.

In today’s world, post-genomic is built on two important ideas: that, as previously mentioned, doing genomics is easier than ever, and that genetic information is not enough. Post-genomic embodies a world where we can and should focus on the next big (or small) revolutionary ideas in the study of the biological world.

“Genetic information is not enough” — while it may seem obvious, it’s also a good prompt for creative ideas. If genetic information is not enough to understand your disease/system of interest, and if “doing genomics is easier than ever”, how should we frame our questions? Will the solutions require more large consortia, or are ‘omic technologies sufficiently democratized so that important advances will be most likely made in individual labs?

Subscribe now

3. How would you explain heritability to a fifth grader?

Harvard statistical geneticist Sasha Gusev explains how to explain heritability, and much else, in an interesting interview with Awais Aftab, who write the Substack Psychiatry at the Margins. Those of us working in genomics these days come from many different disciplinary backgrounds (I trained in biochemistry), and the grasp of key statistical genetic concepts in the broader community is uneven, to put it charitably. And get those statistical concepts are critical in the work that the genomics community does. What does a GWAS association really mean and what mechanisms could produce one? What does it mean to say that there is ‘missing heritability’? I don’t think our current biomedical PhD programs do a great job teaching non-statistical geneticists these concepts.

Gusev also discusses some fascinating ongoing challenges in the field, so even if you are a statistical geneticist, this interview is worth checking out. And if you want more, Gusev writes the excellent Substack The Infinitesimal.

Psychiatry at the Margins

A Critical Introduction to Behavioral Genetics: Q&A with Sasha Gusev

Sasha (Alexander) Gusev is a statistical geneticist and an Associate Professor of Medicine at Harvard Medical School and the Dana-Farber Cancer Institute. His work involves the development of statistical methods for making sense of disease mechanisms and heritability from Genome-Wide Association Studies. He blogs about the genetics of complex traits at…

Read more

a year ago · 55 likes · 17 comments · Awais Aftab and Sasha Gusev

4. Great reading ideas for 2025

Aside from getting in shape, a reading plan seems to be one of the most common New Year’s resolutions. I enjoyed this list of 9 Reading Ideas for 2025 from Jared Henderson who writes Commonplace Philosophy. Many of these ideas can be applied to scientific papers as well. For example: join a book (or journal) club, pick an author (or lab) and read the complete works, pick a topic and really master it. I like to read in stacks, such a bunch of books on the scientific revolution or general relativity. I’m trying to do the same thing with papers now, rather than only haphazardly reading whatever just came out in Cell or bioRxiv.

How do you master a topic? There is more to it than just reading (take notes, organize the information in your mind, write about it, even if just for yourself). But to narrow the question to just reading, how many papers should you read to master a topic? The advice from my director of graduate studies to those of us prepping for our qualifying exams was good: Identify 20 critical papers and know them in-depth. Along with that, read more quickly another ~100 other papers in the field. The process of identifying those 120 papers will already be a great start towards deeply understanding the field.

The final piece of advice from Henderson is to actually read the books you buy. (I admit I have a problem.) The same could be said for dozens or hundreds of the browser tabs many of us have open: just read the paper — or close the tab if you decide it’s not worth the time.

Share

5. August Weissmann is underrated in the history of biology

On that last piece of reading advice, sitting on my shelf is a door-stop biography of the pioneering 19th century biologist August Weissmann. I’m finally going to read it this year. When people think about 19th century biology, Darwin gets an overwhelmingly disproportionate share of the attention, followed by Mendel as a distant second. Weissmann deserved much more, especially given the major influence of genetics and developmental biology on today’s biological sciences. Weissman developed important ideas about heredity and development well before the 20th century rediscovery of Mendelian genetics. He was also a superb experimentalist.

Here is the book summary from Harvard University Press:

The evolutionist Ernst Mayr considered August Weismann “one of the great biologists of all time.” Yet the man who formulated the germ plasm theory—that inheritance is transmitted solely through the nuclei of the egg and sperm cells—has not received an in-depth historical examination. August Weismannreintroduces readers to a towering figure in the life sciences. In this first full-length biography, Frederick Churchill situates Weismann in the swirling intellectual currents of his era and demonstrates how his work paved the way for the modern synthesis of genetics and evolution in the twentieth century.

In 1859 Darwin’s tantalizing new idea stirred up a great deal of activity and turmoil in the scientific world, to a large extent because the underlying biological mechanisms of evolution through natural selection had not yet been worked out. Weismann’s achievement was to unite natural history, embryology, and cell biology under the capacious dome of evolutionary theory. In his major work on the germ plasm (1892), which established the material basis of heredity in the “germ cells,” Weismann delivered a crushing blow to Lamarck’s concept of the inheritance of acquired traits.

In this deeply researched biography, Churchill explains the development of Weismann’s pioneering work based on cytology and embryology and opens up an expanded history of biology from 1859 to 1914. August Weismann is sure to become the definitive account of an extraordinary life and career.

Last spring, my colleagues and I organized a symposium in honor of our colleague Gary Stormo, a pioneer of computational biology who (nominally) retired last year. (He still has some papers in the works.) The outstanding closing talk was given by Mark Johnston, former chair of the Washington University Genetics Department and now emeritus chair of the Department of Biochemistry and Genetics at the University of Colorado School of Medicine. In his talk, Mark made a bold claim: that our Genetics Department here in St. Louis was the “founder” of genomics in its modern sense. Mark argues that it was here — specifically on the eighth floor of the McDonnell Sciences building where the Genetics Department was housed— that genomics was first instantiated.

Those from other institutions who are less partisan than I am may dispute this claim, but no matter where you come down on it, Mark Johnston’s talk is a fascinating oral history of the technological advances that were needed to make genome sequencing possible. (Read Mark’s talk and you’ll learn about the early St. Louis careers of several important figures who made Seattle’s University of Washington a genomics powerhouse. Our department also trained the current director of the National Human Genome Research Institute.) Imagine a scientist in the early 1980’s considering the daunting prospect of sequencing the entire human genome. Sequencing very short segments of DNA was a tedious process that involved manually examining bands on a polyacrylamide gel. Sequencing hundreds of base pairs was daunting, and the technology to sequence billions of base pairs did not exist.

Today, I can submit my sample to our sequencing core and get back a few hundred billion bases of sequencing in a couple of days. This capacity is, and will continue to be transformative, but how did it originate? Mark Johnston covers some of the critical early developments that anyone interested in genomics should learn about. Below I’ve highlighted the key landmark papers/technologies that Mark presents in his talk. If you read these dozen or so papers, you will have an excellent understanding of the key technologies that made the Human Genome Project possible. You can watch the video and read over the annotated and lightly edited transcript below. Mark was the person who drew me to Washington University few years after the draft human genome sequence was published. In the talk below you can hear his wonderfully dry sense of humor, his intelligence, and his very clear sense of the history of this field. It’s an inspiring story

Landmark 1: Restriction fragment clone mapping

Genomes aren’t sequenced in order, from the start to end of individual chromosomes. Your sequencing data comes in shorter fragments. To sequence a genome, you need a map. Work by Maynard Olson at Washington University in St. Louis on S. cerevisiae, and complementary work at the MRC in Cambridge mapping the genome of C. elegans showed to create these maps.

Random-clone strategy for genomic restriction mapping in yeast, Olson, et al., PNAS, 1986 Oct; 3(20):7826-30.

Toward a physical map of the genome of the nematode Caenorhabditis elegans, Coulson, et al., PNAS 1986 Oct; 83(20): 7821–7825.

Landmark 2: Yeast Artificial Chromosomes

Restriction fragment clones are good for relatively short range sequencing, but they aren’t capable of mapping sequences on the scale of millions of base pairs. Larger clones for mapping were made possible by the development of yeast artificial chromosomes (YACs) in Maynard Olson’s lab.

Cloning of large segments of exogenous DNA into yeast by means of artificial chromosome vectors, Burke, Carle, and Olson, Science May 15;236(4803):806-12.

Landmark 3: Sequence-tagged Sites

Eric Green, the current director of the National Human Genome Institute at the NIH, did critical work with Maynard Olson to develop a clever mapping method called sequence-tagged sites. The technique involves using PCR primers to identify landmarks in cloned fragments.

A common language for physical mapping of the human genome, Olson, Hood, Cantor, and Botstein, Science 1989 Sep 29;245(4925):1434-5.

Chromosomal region of the cystic fibrosis gene in yeast artificial chromosomes: a model for human genome mapping, Green and Olson, Science 1990 Oct 5;250(4977):94-8.

Landmark 4: Automated 4-color sequencing

Automating DNA sequencing was a critical step to scale up capacity. With automated methods to read sequencing output, you also needed computational methods to assess the quality of that output. Phil Green developed a prototype of these methods (notably PHRED) here at Washington University, before he moved to the University of Washington in Seattle, along with a number of other former member of the WashU Genetics Department.

Fluorescence detection in automated DNA sequence analysis, Smith, et al., Nature 1986 Jun;321(6071):674-9.

Base-calling of automated sequencer traces using phred. II. Error probabilities, Ewing and Green, Genome Res. 1998 Mar;8(3):186-94.

An improved method for photofootprinting yeast genes in vivo using Taq polymerase, Axelrod and Majors, Nucleic Acids Res. 1989 Jan 11;17(1):171-83.

Yeast genome sequenced

As Mark Johnston explains, it was necessary to sequence the yeast genome because, if you were going to use YACs in yeast as your physical vector for C. elegans or human genome sequencing, you needed to identify the yeast sequences so that you could throw then away. The yeast genome was the first sequenced eukaryotic genome.

Complete nucleotide sequence of Saccharomyces cerevisiae chromosome VIII, Johnston, et al., Science 1994 Sep 30;265(5181):2077-82.

Life with 6000 genes, Goffeau, et al., Science 1996 Oct 25;274(5287):546, 563-7.

Subscribe now

Watch Mark’s talk below:

Transcript of “Genomics Instantiated: A View from the Front Row”, Mark Johnston, May 21, 2023, Washington University in St. Louis Gary D. Stormo Symposium

My talk today really has nothing to do with Gary. That's because I don't know any computational biology and all the other speakers before me have amply phrased Gary's accomplishments and his contributions to our field. So I won't do that. I'll just preface my remarks by saying that I know that Gary was well known from the very beginning of the field. I was a graduate student in 1978. We were doing some sequencing and we had a little bit of sequence you know 100 bases or something and we didn't know how to analyze it. And I remember somebody telling me, I can't remember who, but I clearly remember them saying, “Oh you should talk to this guy Stormo in Colorado. He knows how to analyze sequence.” I never found his email address so I wasn't able to get in touch with him.

What I want to do today is to give you a history of why Washington University and in particular the Genetics Department here was really the founder of the field of Genomics. So my talk is “Genomics Instantiated.” Now that word “instantiated”, academics love that word. I've seen it all over the place. I've never used it and I'm an academic and I've always felt bad that I've never been able to never had an opportunity to use the word instantiated. So I want to thank the organizers for inviting me to give me the chance to actually use this word for the very first time now. Of course I had to look it up because I don't really know what it means. I had to look it up. And I looked up in the dictionary: “Instantiate -  to represent an abstraction by a concrete instance.”

What I'm going to present to you today is my contention that the first instantiation of genomics happened right here up on the eighth floor of McDonnell Sciences building. I'm going to tell you the story of how that came to be and how WashU came to be known as a DNA sequencing Power, and my view from the front row. I had the privilege to observe all of this. I saw the whole story from the very front row, so I'll tell you my view of what of how it all happened.

Share

So back then, this is the ‘80s we're talking about, the goal was to map genomes. And so you have your genome here and the goal is to get a single-nucleotide resolution map of the genome. Well, back in the 80s of course you couldn't do that directly. You really can't quite even do it now directly. What you need is to map the genome first before you get it down to single-nucleotide resolution. You have to cut the genome up into smaller pieces, figure out where they lie in the genome, and then you can get your single-nucleotide resolution map. And it was those methods that you need to map the genome.

And how to do that was really instantiated here in the Department of Genetics by this guy Maynard Olson, a faculty member who arrived here I think in 1978. And Maynard was really the first one to do genomics, the first one to study an organism on the whole genome scale. And what Maynard did was to to map clones of the yeast genome. He was the first one, along with John Sulston, who you'll hear about in just a minute here, to lay out the need, plus the mechanism of the approach for how to map clones to genomes.

What Maynard did was simply do a restriction site digest of a large number of clones and measure the fragment sizes. So here's a bunch of clones of the yeast genome cut with EcoRI and HindIII, run out on a gel to size the fragments. And you just count the sizes of the fragments. If you do that enough times to enough clones, you can identify clones that overlap. They share fragments of the same lengths, and so those clones must overlap. So if you're clever enough to be able to run gels that can accurately size DNA fragments, and then you're clever enough to write algorithms that can handle the data and put it all together, you end up with something that looks like this a clone map of the genome, where the horizontal lines are individual clones, the hash marks on the clones are EcoRI and HindIII sites. And so you can generate a series of clones that map physically map the genome, and Maynard was the first guy to do this.

And he was doing it at a time when nobody could understand why you'd want to do that. I remember a very well-known scientist, a very smart guy who said to me who the hell cares where all the EcoRI and HindIII sites are in the yeast genome, I don't really care. Well, the reason you care is because you need a map of a genome to get a sequence of a genome, and all that follows from that.

Okay, so Maynard was the first one to map genomes. Maynard was working at this level, in the range of clones of about tens of kb in phage vectors and plasmid vectors and later on in cosmid vectors. He wasn't working at the larger scale of thousands and hundreds to thousands of kilobases of DNA. So now comes the key event that really led to WashU having a Genome Sequencing Center. You all know that it's much easier to put together a puzzle with a few number of pieces than a puzzle with lots of pieces, right? A thousand-piece puzzle is much harder to put together than a hundred-piece puzzle, and Maynard was working in this range, and then comes the key event.

Bob Waterston, Maynard's colleague and my colleague up on the eighth floor of McDonnell Sciences Building, goes off in the mid 80s, ‘85 I think, ‘86, to do a sabbatical with John Sulston. And Bob's over there working on his muscle genes in John's lab, watching John map the C. elegans genome. John was doing the same thing Maynard was doing, trying to get a map of the C. elegans genome, eventually for the purpose of sequencing it. And John was also mired in this area of small fragments. He had 750 contigs and there were 750 gaps in the clone map, when he should have had seven. And he just couldn't get past those 750 gaps in the DNA sequence. Bob is watching John be frustrated with that.

Bob comes back to St Louis and he talks to these two guys, David Burke and George Carl. They were graduate students in Maynard's lab, and they told Bob about work they were doing to generate yeast artificial chromosomes. These were vectors that would take large pieces of DNA, and Bob learned about this before it was published. Bob goes back to John Sulston’s Lab in Cambridge England and begins to work with John on closing those 750 gaps in the genome sequence. And that's really what got Bob into DNA sequencing, which is what brought a genome sequencing center with hundreds of millions of dollars in grant income to Washington University.

Okay so there's another landmark [to talk about]. We have clone mapping from Maynard and John Sulston, we have yeast artificial chromosomes from David Burke and George Carl, again working in Maynard’s lab, and there's a third major landmark in genome mapping that also came out of Maynard’s lab called sequence-tagged sites. Maynard, in 1989, publishes this paper, with a few other people to give it credence I guess, “A common language for physical mapping of the human genome.”

Sequence-tagged sites are simply PCR priming sites that will yield a PCR product. So you simply get random sequence from a bunch of clones, design primers, and if those primers give a product, amplify a sequence, then that means the clone that was the template has those sequences in it. So it simply sequenced landmarks of about 100 to a thousand base pairs. Sequence-tagged sites, really a brilliant idea, quite simple, which as most brilliant ideas are, they're brilliant after you know what the idea is.

The guy who put sequence-tagged sites into practice, who instantiated sequence-tagged sites, was this guy Eric Green, a postdoc with Maynard. Eric was a student here, an M.D./P.h.D. student, got his degree in immunology, and decided he wanted to go into genomics and join Maynard's lab. And he really put into practice Maynard's idea of sequence tagged-sites and ended up generating, by 1990, a map of a region of the human genome. It was a model for how you map the human genome with sequence-tagged sites. Here's all the clones [image below], the horizontal lines, and now those dots on the clones are sequence-tagged sites, PCR priming sites that are present in those clones. You generate a map of the genome that way. Eric went on to become the second director of The National Human Genome Research Institute where he has been the director now for 15 years.

So genomics was born in the [Washington University] Genetics Department, was extended by the Sequencing Center in the Genetics Department, and then we exported it to the National Human Genome Research Institute. Mr. McDonnell [James S, McDonnell] would have been proud that his money invested in the Genetics Department really came to fruition.

Okay, so now we got a map, and now we're ready to do the sequencing. You all know how this works, right, di-deoxynucleotides get incorporated, cause chain termination, you generate a family of molecules that have each stopped at a nucleotide, you separate those by their size on a gel and you read the sequence off. Now this was how it started for C. elegans sequencing. I remember them developing films and reading off the sequence. I remember you know fishing gels out of the fixer and reading the sequence before they were even dry, because you couldn't wait to to see your exciting sequence come off. Now of course you're not going to be able to sequence a genome that way. You’ve got to have some automated way to do it.

So a great leap forward came with the development of fluorescently-tagged DNA for sequencing where you put on your di-deoxys some dyes that report a color, and now you can load all of those reactions into one lane and simply read off the sequence from the color of each of those bands. So that was a major advance that allowed automation and sort of portended that the sequencing the human genome was going to be possible. You get these kinds of kinds of gels where you read off the sequence and here's really what the readout was. These colored peaks are the peaks of those bands coming off the gel, and you just simply read the sequence off from the color.

But the problem is, where do you start your sequence read. Down here it's not going to be a very good sequence, it gets pretty good in here, but then the sequence starts degrading, so where do you stop reading your sequence. And here's where a major advance came that really enabled the human genome sequencing and it also comes from the Department of Genetics on the eight floor of McDonnell Sciences building. That is the development of quality statistics by Phil Green who was a faculty member in Genetics.

Here's Phil, and what Phil did is develop algorithms that would read those peaks, those colored peaks, and put a quality score on them. So his algorithm could read the peaks and tell you how sure of each base call you are. So a score of 10 means that there was a one in 10 probability the base was called wrong, so you're 90% certain that base is called right. A score of 50 said that there was a one in 100,000 chance that you were wrong so you're most certain of that of that base call. And so that enabled you to turn these peaks into scores which enabled automation of the sequence reading, and that was a major advance in fluorescent sequencing of the human genome. [Note: Phil Green moved from Washington University in St. Louis to the University of Washington in 1994, bringing with him the protoypes of PHRED and PHRAP, initially developed at WashU, and which he later published at the University of Washington.]

There were other advances this is the slide I got from Maynard. These are Maynard's landmarks in the evolution of four-color sequencing. There were advances in polyacrylamide gel technology, better dyes that gave you more sensitive detection , DNA polymerases that would accept better the dye-labeled nucleotides. And there's one more advance that was critical that, again also came from Washington University. This is one not from the McDonnell Sciences building, but from the first floor of the North building in the biochemistry department, and that was the development of cycle sequencing: using PCR technology to amplify your sequencing reaction so you can sequence small amounts of DNA. And that was done by our late colleague John Major and his student Jeff Axelrod.

So here's John, and what Jeff and John did was they use multiple rounds of anealing and extension with Taq polymerase to detect sequences in the genome. You do a one-sided PCR reaction, essentially a linear amplification of the reaction, and John pointed out that their primary extension assay could be used to determine DNA sequence directly from the yeast genome. That was the first instantiation of cycle sequencing, which was another major advance that enabled the sequencing of the of the human genome.

So mapping a genome happened here, many other advances happened here at Washington University. So why am I up here telling you the story? It's all about Maynard and Bob and Phil. Right, why am I here? Well, I had a front row seat because I was peripherally involved in the project. So when Bob set out to sequence the worm genome, he and John decided they needed to get the sequence of the yeast genome also. Why? Well, they thought that they were going to be sequencing C. elegans DNA cloned in YACs. And if you isolate YAC DNA from yeast, you're going to necessarily be contaminated with yeast DNA. So you got to know the yeast sequence so you can throw it out.

So they wanted to sequence the yeast genome and they recruited me to be in charge of that of that project. So this was our first paper on this in 1994, we sequenced one of the yeast chromosomes. This was the second whole chromosome sequenced and that's why it was published in a relatively high, in a sensibly high-profile Journal. We did it in quite quick order because we had Rick Wilson’s and Bob Waterston’s Sequencing Center that enabled us to do that.

Now what does it mean to manage a project like that? So my role to manage this project was to rely on Linda Riles. Linda was Maynard's technician who played a big role in his mapping of the yeast genome. So Linda was the caretaker of all the clones of the yeast genome. Here's Linda in front of the maps of the yeast genome.

So my role was to go tell Linda what chromosome we were going to sequence next Linda would go in and figure out what clones span the chromosome. She'd give them to me, I'd walk them the three blocks over to the 4444 building, hand them to Lucinda [Fulton], and they'd get sequenced. The sequence would come in and I had the privilege of sitting in front of a computer and annotating it, which was really fantastic and I got all the credit. It's great work if you can get it, highly recommended.

Share

Then just a little while later we collaborated with a worldwide Consortium of sequencers — back then sequencing was hard and so lots of people were involved — to produce the sequence of the first eukaryotic genome, yeast, about 12 megabases in size. This was an interesting sociological and political science experiment, which we can talk about later, maybe.

Then soon thereafter, Bob and John's Sequencing Center sequenced a real genome, you know, a large genome [C. elegans]. About 100 megabases is about roughly 10 times the size of of yeast. The first multi-cellular organism whose genome was sequenced. And then of course you know that they sequenced the human genome. Here they are in 2000 celebrating the completion of the yeast genome. Here's Bob Waterston and Rick Wilson at the White House, with a few other minor players in the process, celebrating what they called the completion of the human genome sequence.

So really this guy [Bob Waterston], was I think one of the moral compasses of the project, for data release, for appropriate allocation of the resources of the project. He really, I think, does not get all of the credit that he deserves for his leadership role, his quiet leadership role in that project. Here is Bob at a press conference with Craig talking about talking about the sequence.

That's how genomics was instantiated in the field, how genomics was invented, I claim here, at Washington University on the eighth floor of the McDonnell Sciences building by Maynard, implemented extraordinarily well by Bob, and advanced by people like Phil Green. So we should all be proud to have been part of Washington University, and I had the great privilege of having a front row seat for all of that which I've just told you about.

1. Genomic foundation models aren’t there yet

One question that comes up in recent discussions about AI is what happens when the big large language models run out of training data? There are concerns that progress is starting to slow since companies like Google and OpenAI have already used up just about all of the available text on the internet.

There is a parallel concern about large DNA language models, which haven’t made as much progress as one might have expected. Two recent papers benchmark these models and find that, despite the fact that they are the biggest AI models in genomics, they don’t compare favorably to humble CNNs trained on smaller datasets for specific tasks.

In one paper, out in Nature Methods, the team at InstaDeep trained transformer models on different datasets and then benchmarked their performance on a variety of tasks. In the paper, they give a glass-half-full interpretation of results that show models with 2.5 billion parameters, trained on hundreds of thousands of genomes, often don’t do worse on specific tasks than smaller specialized models. They generally don’t do better either. For example, a 2.5 billion parameter model trained and then fine-tuned to predict activities of regulatory sequences in a massively parallel reporter gene assay didn’t do any better than a small CNN:

Despite these disappointing results, this a well-done study that presents a clear, systematic evaluation of these models.

A second DNA language model benchmarking paper was just posted on arXiv by Anshul Kundaje and his lab, at Stanford. This is also a nicely-executed study that includes a worthwile discussion of the state of the field. Their top-line conclusion is that “current annotation-agnostic DNALMs exhibit inconsistent performance and do not offer compelling gains over alternative baseline models for most tasks, despite requiring significantly more computational resources.” So yeah, we’re not there yet.

I’ll highlight a critical point from their discussion that, at least in the circles I move in, is one that is increasingly recognized as a challenge: the genome itself isn’t necessarily the best training data for the models we want:

Although DNALMs successfully discriminate regulatory DNA from background sequences, they appear to learn incomplete repertoires of regulatory sequence features. This limitation likely stems from the sparsity and the uneven distribution of regulatory features; regulatory elements constitute only 10-20% of the human genome, and certain classes of regulatory features occur at substantially different frequencies.

Subscribe now

2. A tool to measure the activity of a cell’s regulatory factors

Transcription factors are the key regulators of a cell’s identity, since they direct the expression of the right repertoire of cell type-specific genes. To know which transcription factors are active in a cell type people typically look at RNA-seq data, but this only tells you something about the amount of transcription factor present in a cell and not its level of functional activity.

Bas van Steensel’s lab, at the Netherlands Cancer Institute, has a paper out in Cell Systems that describes how to use multiplexed reporter gene libraries as a tool to read out transcription factor activity in a cell. They came up with a scheme to design optimal sensors of transcription factor activity, and because the libraries are multiplexed, you can read out the activities of dozens of factors.

3. Everything you ever wanted to know about commercial single cell technologies.

One of my plans over the holiday break is to get up to speed on some areas I should know better than I do, including the latest single-cell omics technologies. Fortunately, the latest issue of Cell Genomics has a comprehensive, open access review of single-cell genomic and epigenomic assay technologies. Even better, this is merely the introduction to what they call a “living review”: a Substack page at sctrends.org discussing all things related to single-cell and spatial omics technologies. (Noted: find a way to promote your Substack by writing a paper about it…)

4. The laws underlying the physics of everyday life are completely understood

In 2010, the physicist Sean Carroll wrote what is probably one of my favorite blog posts of all time, in which he claimed that “the laws underlying the physics of everyday life are completely understood.” He said:

A hundred years ago it would have been easy to ask a basic question to which physics couldn’t provide a satisfying answer. “What keeps this table from collapsing?” “Why are there different elements?” “What kind of signal travels from the brain to your muscles?” But now we understand all that stuff. (Again, not the detailed way in which everything plays out, but the underlying principles.) Fifty years ago we more or less had it figured out, depending on how picky you want to be about the nuclear forces. But there’s no question that the human goal of figuring out the basic rules by which the easily observable world works was one that was achieved once and for all in the twentieth century.

This claim caused a little controversy, but I like it because it pushes back against a view of science that people may have picked up from Thomas Kuhn (or Larry Laudan for a deeper cut), namely that scientific paradigms are always going to change and that today’s theories are almost certainly going to be discarded by scientists of the future. I don’t think that is uniformly true. In biology we’ve figured out the basics of mammalian sexual reproduction (two haploid gametes fuse to form a diploid zygote). We have a good understanding of the structure of DNA and why that structure enables DNA to act as a template for transcription and its own replication. Not everything in science is going to be obsolete.

I can’t speak to the physics, but Carroll wrote an interesting paper laying out the details of his claim. (It was posted in 2021 but I didn’t see it until this week.) If you’re interested in this discussion, it’s worth at least skim, even if you don’t have a background in quantum mechanics.

5. Are we living in a simulation?

Speaking of philosophy and science, here’s one more interesting paper, this one by the philosopher Peter Godfrey-Smith, who is one of the most interesting philosophers of biology working today. He takes on David Chalmer’s best-selling book Reality+ and gets into questions like the substrate independence of minds.

6. Three history of science books that directly address philosophy

It’s getting a little late to make holiday book recommendations, but here are three I enjoyed, one well known, and two less familiar. I enjoy books that present a careful history of some aspect of science, and then use that history to directly address big questions in philosophy of science.

David Wotton, The Invention of Science. This book got quite a bit of attention when it came out in 2015. I’ve read more than a dozen books on the Scientific Revolution and this is easily my favorite. Wootton argues that science was not a gradual evolution from earlier natural philosophy, but was instead a specific invention of the long 17th century. He makes his case by analyzing the emergence of language of science. For example, when did people start using the words discovery, experiment, and fact, and in what context were they using them? This proves to be an extremely effective way at understanding what the scientists of the 17th century thought they were up to. (They didn’t use the word ‘scientist’, but they used other words that captured the same idea.)

Stephen Brush, Making 20th Century Science. Historian Stephen Brush asks, what is more convincing to scientists, a successful new prediction or an explanation of some long-standing anomaly? To answer this question, Brush examines the history of how scientific consensus was achieved for major theories developed in the 20th century. He traverses quantum mechanics, organic chemistry, general relativity, the Big Bang, the hereditary role of chromosomes, and the Modern Synthesis of evolution and genetics. You may have thought that Eddington’s famous 1919 eclipse expedition to test Einstein’s theory of general relativity is a classic case of a successful new prediction persuading scientists, but Brush explains that it took nearly five more years of careful measurements before most of the physics community came around. In general Brush’s answer is that both prediction and explanation can be persuasive in different contexts, but there is more to it. To get the full, nuanced answer, read the book.

Elof Axel Carlson, How Scientific Progress Occurs. This concise book reviews the history of cell and molecular biology, with the aim of contrasting scientific progress in biology with Thomas Kuhn’s model of paradigm shifts. Carlson argues that biology is much more incremental and, as far as he can see, basically never involves paradigm shifts. It’s a provocative book with some great history - history that really anyone working in the biomedical sciences should know.

Unfortunately Robert F. Kennedy Jr. is now a regular presence in the national news, and it means that we are going to suffer through yet another cycle of public debate over the false claims that autism has been linked to vaccines. Responsible journalists, medical institutions, and public figures will yet again need to remind everyone that a there is no link between the MMR vaccine and autism, that the issue has been thoroughly investigated in large studies and meta-analyses, whose results are publicly available. It is good to cite this science, but it can lead people who are inclined to be skeptical of vaccines to feel like this is a complex debate with large datasets and difficult statistics that once could parse in different ways.

I want to try something different here. In this post, we’ll go back to the original source of the claim that the MMR vaccine might cause autism, the notorious, now-retracted 1998 paper published by Wakefield, et al. in The Lancet. What people typically know about this paper, if they are aware of it, is that it was retracted and Wakefield and his colleagues were accused of fraud and ethics violations. (They had undisclosed financial interests, they didn’t get the necessary consent for conducting invasive examinations of their child subjects, and they lied about how they assembled their study population. See this summary.)

But you don’t need to know any of that to see that the paper, taken at face value, is obviously terrible. It’s study design (and I’m being generous using the word “design”) has problems can be easily understood by anyone who has even a vague recollection of high school science. When you read this paper, you can understand why a whole movement for “evidence-based medicine” had to be a thing, because the 1990’s Lancet editors clearly didn’t understand how medical evidence works.

Subscribe now

Problem 1: There is no statistical power in 12 kids who happen to show up at your clinic

The 1998 Wakefield paper is an “early report” that describes an assessment of twelve children who visited a UK gastroenterology clinic. Their ages ranged from 3-11, and, according to the paper, they had “a history of normal development followed by loss of acquired skills, including language, together with diarrhoea and abdominal pain.” You don’t need any more information than this to understand that a study like this has zero power to find any association between autism and the MMR vaccine. Twelve kids who show up at your clinic, among a population of tens of millions of children who had received the MMR vaccine, will tell you nothing about MMR and autism.

Problem 2: The association of autism with vaccination was a parent report and not a statistical finding

Nine of the twelve children in the study were diagnosed with autism, though the paper does not describe how the diagnosis was made. Three others were ambiguous cases. The main evidence used to claim a link with vaccination, I kid you not, is this:

In eight children, the onset of behavioural problems had been linked, either by the parents or by the child's physician, with measles, mumps, and rubella vaccination.

That is literally the entire basis for suggesting a link between vaccines and autism in this paper - the parents or the child’s physician thought the vaccine caused their child’s condition. There is no statistical analysis, no clear histological or other unambiguous evidence of vaccine effects. The children were all given colonoscopies, and the study reports a miscellany of findings, each of which only occurred in some of the children. Wakefield had earlier explored the idea that measles and the MMR vaccine might be associated with the development of irritable bowel disease (IBD). This idea morphed into his belief that vaccines predispose children to autism via gut pathology. Hence the colonoscopies for his patients.

There is a table in the paper titled “Neuropsychiatric diagnosis” whose columns show the behavioral diagnosis for each subject, the “exposure identified by parents or doctor”, the interval from exposure to first symptoms (usually a few days to a week), etc. This table does not present evidence for anything. It merely lists a small number children who happened to show up at a GI clinic, who have autism and who supposedly received the MMR vaccine within a week of the first symptoms (according to the parents, with no other documentation).

Problem 3: No controls

When you want to find an association between some cause and a disease, you need to include disease cases and healthy controls. If, for example, you wanted to claim that green jelly beans are associated with a risk of cancer, your analysis would need to include control subjects who didn’t get cancer, so that you could check whether the rate of green jelly bean consumption differed among the groups. Alternatively, you could follow two groups, one that ate green jelly beans and one that did not, and ask whether there was a difference in cancer incidence between the groups. The Wakefield study has no real controls. There is no work-up of healthy children presented or an analysis of children who were not vaccinated.

Figure 1 does show a plot of urinary methylmalonic-acid in 8 of the 12 patients versus 14 “age-matched controls.” There is no description of where these control patients came from or anything else about them. There is a p value (p = 0.003), but how it was calculated is not described. And methylmalonic-acid says nothing about autism or vaccines; it is a potential indicator of vitamin B-12 deficiency.

How could anyone consider this science?

There are many other problems in this short paper we could discuss, but by now it should be clear: this work, taken at face value, ignoring the later evidence of fraud, couldn’t possibly say anything about a link between vaccines and autism. It’s a description of twelve children who were vaccinated (like millions of other children the same age), who had autism, and whose parents thought the vaccine caused their autism. This is not science.

It’s true that the authors write that “We did not prove an association between measles, mumps, and rubella vaccine and the syndrome described.” But this leaves out the fact that they didn’t even investigate an association between the MMR vaccine and autism. And yet, in the very next paragraph, the authors write that “If there is a causal link between measles, mumps, and rubella vaccine and this syndrome, a rising incidence might be anticipated after the introduction of this vaccine in the UK in 1988.” It was this idea that caught fire in the media and, nearly three decades later, has left us us with a nominee to lead the U.S. Department of Health and Human Services who goes on television to tell people that vaccines cause autism.

The most perplexing question is how any serious person at The Lancet thought a manuscript like this met the standards for publication in their journal. Medical science, like all science, is an astonishingly effective generator of reliable knowledge when it is based on reliable methods. This paper flagrantly ignores those methods. It is worth noting that in the year after the Wakefield paper The Lancet published a real investigation with nearly 500 subjects and found no association between the MMR vaccine and autism, no evidence of a distinct increase in incidence after the MMR vaccine was introduced, and no relationship between the timing of vaccination and the age of autism diagnosis. But the damage had been done.

A quick read of the Wakefield paper is enough for almost anyone to understand that false claims about vaccines and autism did not arise from some suggestive or worrying early study that was later clarified. The tragedy is that there was never any basis whatsoever for believing the claim.