The Technium

Recent Innovations in the Method

[Translations: Japanese]

What would you say are the innovations in the scientific method of the last 50 years? What has changed the nature of science in practice in your lifetime? I am primarily interested in innovations in the process of science itself, rather than the discoveries made by that process.

I asked this question to some prominent scientists and science observers in astrophysics, biology, evolutionary theory, computer science, psychology, and science fiction. Those who responded include the following:

(JB) John Barrow, (GB) Gordon Bell, (GtB) Gerrit Breoekstra, (RD) Richard Dawkins, (NE) Niles Eldredge, (TE) Terry Erwin, (FD) Freeman Dyson, (GD) George Dyson, (JG) Jim Gray, (DH) David Hillis, (NH) Nick Humphrey, (SK) Stuart Kaufman, (CL) Chris Langton, (SP) Steven Pinker, (LS) Lee Smolin, (BS) Bruce Sterling.

I’ve combined and edited their ideas. Here are what some thinkers believe are the major innovations in the process of science in the last 50 years.

Personal Computers — I’d say that putting personal computers on the desktop of every scientist has had a bigger influence on science than the first computer simulation in 1946. This may not seem like a “new” development, but statistical analysis, simulation, etc. has become an integral part of science because of this. (DH) The development of the PC should be distinguished from computers in general. Before the PC, computers were big and expensive and directed at block busting problems. The arrival of inexpensive small computers with good interactivity and graphics enabled the study of chaos and complexity to proceed experimentally, by simulation. (JB)

simulationComputer Simulations — Simulation is a big one, but even that is an evolution of modeling.  A whole collection of models have evolved to explain nature at various levels of detail with resulting savings in cost and time. Ken Wilson, a Nobel laureate, and others have claimed that simulation and modeling is the 3rd paradigm of science … after observation, and theory. Visualization has helped make simulation really “work” and is a strong component of simulation (GB)

Exhaustive Combinatorics — You can explore the world scientifically by building a model and then sweeping the entire possibility space of that model to see what it produces. There is the image of sweeping a space, or sampling broadly from it, to search for what you want. For Boolean nets, I call it the “ensemble method” in that one invents different classes of models, and for each establishes their typical properties by random samples from the ensemble. Later, physicists used the same approach for spin glasses. (SK)

Statistical analysis — Statistical significance is worth mentioning. (JB) So is analysis of variance. (RD) And Bayesian approaches to statistical analysis. One might date this to the Reverend Bayes’ fundamental work in the 1700s, but that isn’t realistic. In fact, implementation of Bayesian analysis depended on the development of Markov chain Monte Carlo computational approaches, which only became practical in the 1990s. For many fields of science, this has provided a new way to approach and tackle very complicated analyses, and provides a new framework for interpretation. In many areas of science, I think it has become the dominant paradigm for statistical analysis of data. (DH) One example, which may be more than fifty years old (depending on how one demarcates the origin as opposed to full flowering), is statistical inference, without which psychology, social science, epidemiology, and so on would be impossible. Clearly this began more than a century ago with Galton, then was further developed by Pearson, Neyman, Fisher, Spearman, and others in the first three decades of the twentieth century. But it really only became practicable with computers, and the past fifty years have seen the first systematic use of the main techniques used to establish cause and effect in this fields: Multiple regression (and its special case Analysis of Variance), path analysis, time-series analysis, and so on. (SP)

Analysis-by-synthesis — This is a variant of a technique I worked on in 1959 at MIT where the phrase was coined to describe how one does analysis (i.e. science). For speech the goal was to understand the speech production process. It worked by taking in a speech signal, and comparing it with the computer’s generated signal to infer what the vocal track was doing, and ultimately what sound was being created. Note Google says there are 11K hits for the technique that is used in many different situations. (GB)

databaseLarge scale databases — Another innovation is the rise of data-processing as the central component of experimental science. This is not the same thing as computer-simulation. It is the handling of massive amounts of real data in such a way as to make it intelligible. It requires experts who know how to organize data-bases and make them accessible to users. It has the effect of turning experimental scientists into software engineers. (FD) Modern databases permit discovery by enabling one to query, try something, and get reinforcement in a feedback loop that has the computer in the loop. (GB) Bioinformatics, including genomics and proteomics is the doing of biology via the analysis of large-scale databases. This is seen by many (e.g., Eric Lander) as a revolution in biomedicine. (SP)

Pattern Mining — Data mining (automatic inference of trends, detection of patterns,..) is becoming a large part of the scientific method. (JG) -Data mining is an artificial intelligence application that searches databases or even text articles looking for hidden patterns. (SP)

Digital Repositories — Public electronic databases (scientific literature, databases of raw scientific information such as GenBank) that are searchable over the internet are an innovation. GenBank changed the nature of molecular biology from a local (single lab) to a global enterprise, and other databases have had similar effects on other fields of science. (DH) In the observational sciences, the scientific method is changed by the emergence of digital libraries, and now online digital libraries that integrate all the data and literature (Genbank+Pubmed+BLAST are a good Bio example of this). (JB)

Raw Data Depositories — It’s easy to gather terabytes of expensive data (voxels of brain activation values) very quickly, too many for the original researcher to analyze. Recently neuroimaging (PET and fMRI) researchers agreed to place all their data in a public archive, so others can analyze their data for new findings. (SP)

Web – IR (Info retrieval), the web, hypertext, and search have probably done as much for than science as simulation. (GB) The internet and the web have turned science into a highly collective activity in which many minds work together on problems that used to be considered in isolation or a long time. (JB)

google logoSearch Engines — One (maybe two) obvious things are the recent emergence of online pre-print archives, resources like Wikis, and sites like MathWorld, all of which show up from searches with Google or other search engines. This capacity has so reduced the turnaround time for finding the answer to a question that may emerge in one’s research or thinking, that you can get an answer while the context that generated the question is still foremost in your mind. This has, I think, fundamentally changed our ability to think about complex problems, especially those which lead us out of our area of expertise – enough to qualify as a true phase-transition in our thinking and problem solving capabilities. The potential for a new idea to spread far and wide in a very short time, combined with the filtered-search capacity that allows us to quickly find a specific idea-needle in that enormous data-haystack, qualifies, in my mind, as a genuine advance in the scientific method. After all, it is simply an enhancement of the basic algorithm of the scientific method (evolution really): the generation and distribution of variant ideas combined with a strong selective filtering process. (CL)

E-print — Electronic publications and dissemination by PDF files is a major innovation. (TE) This really speeds the process up. LANL’s x-server and archive of not-yet-published work was a truly revolutionary innovation. (GD) Downstream, we might hope of getting rid of proprietary, expensive journals that limit the flow of knowledge. Varmas’s technical journal for the web funded by the Gordon Moore foundation could be a biggie in this regard. (GB) A more recent and more benign change is the publication of research papers on the web. This practice is rapidly making printed journals obsolete. It has the great advantage of making research results more promptly and more widely accessible. It has the disadvantage of depriving the learned societies that publish the printed journals of their main source of income. (FD) The biggest change I experienced is the enormous increase in accessibility and speed of scientific information through the Internet (papers’ immediate availability on for example, which thereafter may still be published in regular journals. (GB) Electronic publication. (BS)

Massive multi-authorship — The greatest innovation in the last fifty years was the organization of research into big projects with tens or hundreds of people to do a single experiment. The change from small to big teams was most spectacular in experimental particle physics, but has also happened to a lesser extent in other fields such as biology and astronomy. The change is driven by the increasing size and complexity of instruments. It has the undesirable effect of turning students into slave-laborers without giving them much chance to be creative. (FD) I had a personal experience in jointly writing a paper through the Internet. I completed a paper (in the Netherlands), with a Frenchman and his postdoc, a Chinese fellow, working at UCLA in California, never having met these guys nor spoken them. Purely by means of email exchanges over a period of almost half a year the paper was accomplished. This was an exhilarating experience, also given the time differences (they worked while I slept, and vice versa). (GtB) Massively multi-authored, multinational papers is a major innovation. (BS)

telescope arrayDistributed instrumentation — Things like arrays of small radio telescopes, that can see things collectively which would not be visible to any individual instrument. These tools have fostered a new level of collaborative work, making projects like whole-genome sequencing possible etc. Whether this is an evolution of the scientific method I’m not sure, but it’s certainly an evolution in methodology. (GD)

Institutionalization of science – One thing is the corporatization of scientific communities, so that there are large communities spread over the planet who identify themselves rigidly with one research program, for example, string theory. A second is the completion of the institutionalization of science within universities and research institutes, based on a rigid hierarchical model of governance in which powerful senior people near retirement control the careers of young people. (LS) National science funding; industrial, for-profit R&D; Gigantic, billion-dollar instruments are big time. (BS)

Patents – Don’t forget lucrative commercial freakouts over intellectual property (BS)

Gedankenexperiment or thought experiment is something to include. (GD)

Hierarchy theory—Hierarchy theory is a major innovation in the scientific method. I would put this more broadly: the major revolution is how the reductionist method (going from higher to lower levels) was complemented by the systemic method (going from lower to higher levels), starting with von Bertalanffy’s Genreal Systems Theory (in the 1930s) , through cybernetics (1950’s), up to chaos and complexity sciences, self-organization and circular causality (1970s). (GtB) Hierarchy theory is an epistemological tool for tackling the structure and inner workings of complex systems. It is ontological (usually–some disagree) in its claims about the structure of systems– like information and economic systems in the biological world–and I would, say, even more so in the material cultural realm–though this work is just beginning. (NE)

Awareness of Paradigm Shifts — The coming of “scientific self-consciousness” may in itself have been a watershed in the history of science. C.H. Waddington wrote a book called “The Scientific Attitude” in 1941. I don’t know how much earlier the concept goes. (NH)

Global science education — The other thing that science has taken advantage of is the growth in education and average prosperity around the world. As a result there are far more scientists alive today than there have ever been. Science has just become bigger. (JB) Add state-supported universities and mass education. (BS)

Prizes — The Nobel and other lucrative, prestigious prizes; Celebrity status — more gold and glory. (BS)

  • Hi Kevin,

    Evolutionary Programming, its variants and antecedents (Holland, Koza, Simon’s GPS) as an invention technique should be on your list, both because it is an innovation in the scientific method and because it is (I suggest) the next major evolutionary advance, and meta-evolutionary advance, in the evolution of the scientific method.

    I hope you’ll unpack these ideas. For the second time, you seem to be writing and researching a book I’ve long envisioned and fantasized about writing. So glad you’re actually able to do it!

  • Kevin Kelly


    Yes, I think the “organizational” aspect of science is in the end the most important part. I believe the structure of science changes over time, and the cultural aspects reflect/drive this change.


    I waver on how widespread Evo Programming is. I tend to lump it with “combinatorial” as a method.

  • Gary Cehelsky

    Hello Kevin,

    Regarding Patents- I think we mistake them as an innovation of method or process of science. I think patent law and patents more of a beurocracy to scope intellectual ownership (i.e. my brain thought this way and it’s my thought – not yours). A patent owner shares knowledge with us but we cannot utilize the the idea unless we pay the patent holders “royalty”.

    So, patents are not about scientific method, but distribution of wealth.

    And this is not an original thought, but once an idea gets out on the internet, who owns it? Who can claim the 1’s and 0’s that make up the information out there in the ether?

    The patent system holds together now, but China and subscribers of the internet in general, are putting enough pressure on the system to make the whole thing blow apart in a decade. This is my thought, and I cannot back it by any particular research I have accomplished. But I offer it to you royalty free.

  • jp

    Firstly great idea for a book, i think it takes courage to take on such an important question that is inherently linked to your professionnal and personal life. Admitting that its complex is always a first step.

    I have an economist background so my comment has that bias. In essence, i rate all the mentioned inventions and I think that the path dependant nature of the world means that weighing up an invention with another is very difficult.

    I do think that the organisational aspect of the “production of science” is very important. I think that organisations help people learn better and more. The communities all over the world who have developed, side-by-side, their know-how is the general context of science today.

    Lots of steps have helped this happen. The pdf, global web, databases, all created a fruitful terrain for people to practice their research area with other people.

  • martin king

    I really worry about this idea of 21st century science being so analysis driven.

    I think that great science comes from inspiration, creativity – analysis is used to confirm the “blue sky” thinking

  • Kevin Kelly


    I agree the patent system is broken, and not optimized for our current economy. But they are the only meager measure we have for ideas, and useful as a historical metric.

  • e

    I am curious why “MAN” was not placed first. MAN has changed, adapted and progressed from all the movement and change that has accured around him. The study of… science is the vessel that MAN uses to flow thru life at the speed that MAN moves. From the birth of each of us, we all have progressed with in science.
    Have a great moment…