The Technium

Maps of Knowledge


[Translations: Japanese]

Previous maps of the relationship between branches of modern science were done by mapping the citations among journal articles. These citations are footnotes referencing previous articles on the same or related subjects. Citations are the equivalent links in a posting. They take you to the source. Citation indexes tally these links by subject. Mapping software can display the patterns of links. One example is below:

Structure Of Science

A map of linkage.

A new way of mapping these intra-science relationships has just been published. Instead of mapping links, this new method maps clicks. The program reads the logs of the servers offering online journals (the most popular way to get articles today) and records the clickstream of a researcher as they hop from one article to the next.  Then these clickstreams (1 billion interactions in this case) are mapped to sort out the relationships generated by users.  Here is their recent portrait of the Map of Scientific Knowledge:

Mapofscience

A map of clickage.

According to the authors of the the paper the advantages of the clickstream versus citation method is that clickstreams give you a real time picture and are broader in scope. They note that “the number of logged interactions now greatly surpasses the volume of all existing citations.” 

I’ve been wondering about the future of Google and search engines in general. It is obvious that on the web the number of clicks outnumber the number of links. That is, people click more often than they create a link. Yet, to my knowledge, PageRank and other search index ranking algorithms primarily work by weighting links. Wouldn’t be smart to also incorporate the wisdom of crowds of people clicking on sites as well. Mining the clickstream as well as the link graph? I wondered if Google was already doing this? Since so many sites now run Google Adsense (and Analytics), Google knows how often and from where people click on a page, do clicks play any part in PageRank ranking now?

I got a reply from  a Google VP in response to my question: “Search quality is based on a combination of page rank and ‘information retrieval score’.  Clicks (and other indicators of user interest) play a role in the IR score.”

In other words, Yes. Google does count clickstreams in creating its own maps of knowledge.

The number of clicks will continue to outpace the number of links, so I expect that in the future more and more of the web’s structure will be determined by clickage rather than linkage.




Comments
  • Tom Buckner

    I see a comparison to neurons here. Not an earthshaking observation; clicks are to links as synapse firings are to synapses. In the brain, of course, frequently fired synapses are strengthened and rarely fired synapses weaken. The same is not automatically true of web sites, which can sometimes go from popularity to nonexistence almost overnight. A poignant example for me was Omni magazine, which was my favorite for about 15 years before becoming one of the first high-profile websites, and then ceased dead-tree publication. Editor Kathy Keeton died about that time and then the web site spent several years as a sort of abandoned, yet still visible, relic, before vanishing altogether.

    Are there any other methods that strengthen links upon clicking? Well, Twine is a little like that, as are such sites as Digg or Metafilter.

    • http://www.kk.org Kevin Kelly

      @Tom Buckner: Yes, synaspe to nueron is a nice analogy for click to link.

  • dave tribbett

    Here is a post that represents some very cool maps of the internet, science and complexity. These are more than maps, they are works of art.

  • Brandon Byers

    Clickstreams are important indeed. After all, links suggest somebody thinks something is important … or wants the world to think so, and after a few years people realized they could game the system. This is where spam-blogs came from. But clicks on links mean that not only did the content creator think things were important & related, but intelligent (compared to bots) visitors agreed.

    I’d wager that Google absolutely uses Analytics for this, and consider if you combine Analytics, AdSense, and DoubleClick (now a Google-owned outfit), Google has its tendrils in 95% of the web … at least, the area of the web the most people visit. Deeper sources of knowledge (academia, government, other sites that aren’t likely to run ads) may have to be based more on links than on clicks.

  • Ehsanul

    This is very interesting to me, because I have an idea for better information retrieval on the web using primarily this concept. I wasn’t thinking in terms of clicks, but more of a wikipedia-type approach to users voluntarily generating and rating an artificial network of links to content on the web. The overall concept is the same though, that users and not website owners, decide what links to what else and how important these links are as well. As I attempt to build this artificial-link-network system, I’ll try to keep an eye as to how one might implement automatic link generation based on clicks and perhaps overall web navigation history even.

    Right now, I use user-generated tags on delicious.com bookmarks, in an attempt to find and automatically generate links between web pages based on the tags that people assign to the pages. This is also pretty much the same concept as following clicks, but somewhat more pointed.

    I think you make too much of a distinction between “links” and “clicks”, or rather have a narrow definition of links. With clicks, what changes is that the links are created based on clicking, but the clicks still do create “links”, just in a new way. Rather than web authors creating links, users are the ones creating them, and I’m convinced that this will be a big change in how the web functions at a global level.

  • mini forex trading

    I have been searching for this information and finally found it. Thanks!

  • Matthew

    Many browsers now provide a feature that lets you select text on a page, and then right-click to search for the selected item in Google or Wikipedia. In effect creating your own link. Because the user creates this link rather than the author, I suspect their value to knowledge mapping is greater than clicking or linking, if they can be harvested.

  • fostiak

    John Battelle expressed a similar intuition in 2004:

    “That’s when I remembered As We May Think, Vannevar Bush’s famous essay in The Atlantic. I had read it earlier in my research, and was struck not by the idea of the Memex, which is well understood, but by Bush’s explication of the problem – that knowledge and learning has become so complicated, so layered, so inefficient, that it is near impossible for anyone to be a generalist, in the sense Aristotle was. Bush’s answer to this problem was the Memex, of course, but what I find interesting is the mechanism by which the Memex is made potent – the mechanism for capturing the traces of a researcher’s discovery through the Memex’s corpus, and storing those traces as intelligence so the next researcher can learn from them and build upon them.

    Searchstreams, I realized, are the DNA which will build the Memex from the flat soil of search as it’s currently understood. Engines that leverage searchstreams will make link analysis-based search (ie, nearly all of commercial search today) look like something out of the pre-Cambrian era. The first fish with feet are all around us – A9, Furl, del.icio.us. We have yet to build the critical mass of searchstreams by which this next generation engine might be built (nor will it necessarily be built with our tacit consent). But I can sense it coming.”

    http://battellemedia.com/archives/2004/08/the_memex_the_s.php_test.php

    • http://www.kk.org Kevin Kelly

      @fostiak: I asked Battelle whether the move from linkage to clickage was new and he said “This approach is incorporated as a signal into most search engines now. It was brought to market early by Mike Cassidy and his company Direct Hit, which was acquired by Ask if I recall.”

  • John

    Kevin,

    This is off subject here, but I would love if you could draw the connections (if there is one) between the rise of the technium and those with Asperger’s. I think there’s a direct connection between the two and as those with Asperger’s have become connected with each other via the internet over the last decade, the rate of expansion of the technium has accelerated accordingly.

    Your ‘technium’ might be nothing more than the result of the labor and thoughts of those with Asperger’s over the centuries.

    All the best,
    John

    Note: Here’s a good article about the IT industry and Asperger’s:

    http://www.computerworld.com/action/article.do?command=viewArticleBasic&taxonomyName=Management&articleId=9072119&taxonomyId=14&pageNumber=2

    • http://www.kk.org Kevin Kelly

      @John: If you mean to ask, is tech culture, now main stream culture, becoming more autistic? The answer might be yes.

  • Tarun

    Matthew,
    The methodology you mention is equivalent to considering keywords used for internet search while ranking pages.

    A link is given weightage because it is created by the content creator AFTER the content creator knows the value of the link.

    On the other hand, a search page per your suggestion, is created by the content consumer BEFORE he or she has evaluated the content that will come up through the search engine. Not sure if this should be given weightage.

    Finally, clicks on the actual content have weightage because multiple clicks imply stickiness that implies value of the content.

  • Ilkka Hamalainen

    Is there any public linkage to create a map of clickage for the management science discipline (relations to economics, psychology, linguistics, philosophy, cognitive science, neuroscience, anthropology etc)?

  • energyequation

    Can anyone recommend software that does this: allows you to create and manipulate relationship-maps like the ones shown above?

    A 3D parametric CAD system would work, but that’s overkill.