The Technium

Lifelogging, An Inevitability


The goal of lifelogging: to record and archive all information in one’s life. This includes all text, all visual information, all audio, all media activity, as well as all biological data from sensors on one’s body. The information would be archived for the benefit of the lifelogger, and shared with others in various degrees as controlled by him/her.

Some form of this total recall is inevitable, at least for some people. Partial recall from partial recording seems inevitable for the rest. We are all likely to record more and more of our lives.

The extreme form of lifelogging is very radical at the moment. If you are seriously lifelogging you have microscopic cameras that record all you see, all the time, your computer archives every action you take, all your email, phone calls and conversations are kept and annotated, your location is tracked, and dozens of nano-sensors in your body record your vital functions. However you may be the only one who sees all this information. It is sort of like radical self-surveillance.

The technical specs of recording every moment in your life are entirely plausible right now. In ten years, it will be cheap, too. But the social, legal, ethical, and cultural consequences are gnarly, weird, unpredictable, over-the-top, controversial, and thrilling.

Gordon Bell1

A lifelog would offer these benefits;

* A 24/7/365 monitoring of vital measurements such as body temperature, heart rate, blood pressure, and presence or absence of bio-chemicals. This data could serve as a warning system and also as a personal base upon which to diagnosis illness and to prescribe medicines.

* A digital memory of people you met, conversations you had, places you visited, and events you participated in. This memory would be searchable, retrievable, and shareable.

* A complete archive of your work and play, and your work habits. Deep comparative analysis of your activities could assist your productivity, creativity, and consumptivity.

* A way of organizing, shaping, and “reading” your own life.

To the degree this lifelog is shared, this archive of information can be leveraged to help others work, amplify social interactions, and in the biological realm, shared medical logs could rapidly advance medicine discoveries.

By 2007 several dozen people have experimented with lifelogs. The first person I met doing this was Ted Nelson in the mid-1980s who recorded every conversation he had, no matter where or what importance. To my knowledge his archives have never been revisted, even by him. In the 1990s Steve Mann at MIT outfitted himself with a head-mounted camera and recorded his daily life on video tape. Because there was a camera half-obscuring his face, it was hard to be natural around him. For the past 6 years, beginning in 2000 Gordon Bell at Microsoft Research has been documenting every aspect of his work life. His experiment is called MyLifeBits. Bell wears a special camera around his neck, the SenseCam, which will notice a person’s body heat if they are near and photograph them, or snap a picture of a new place if it detected a change in light. Bell records and archives every keystroke on his computer, every email, every phone conversation transcribed, every face to face conversation, and of course every photo he takes, every movie he watches, every web site visits, every window on his computer and how long it remained after – any and all data he can record he does.

For the past two years Daniel P.W. Ellis, an associate professor of electrical engineering at Columbia University, has audio recorded – or audio lifelogged – his life. (One DVD hold 2 week’s  worth of life’s sound.)

Alex Pentland at the MIT Media Lab built a “socioscope” which continuously tracked the location of 81 volunteers for 9 months, as well as track the pattern of their conversations.(To oversimplify their results they found that if you looked the patterns of people living and working in a city they are indistinguishable from tribal herding and foraging behavior.)

While a student at MIT, Bradley Rhodes invented a “Remembrance Agent,” which “pays attention to everything you type; every few seconds it checks inside your hard drive to see if it can find anything relevant. If it does, it alerts you in the corner of your screen by showing a line or two of the related document.”

So how well does lifelogging work? Journalist Clive Thompson visited Gordon Bells MyLifeBits lab and hung out a while. His report for Fast Company is the best description yet of what lifelogging feels like.

For all of its machine muscle, Bell’s virtual memory wasn’t quite what I imagined. When I first heard about his work, I expected someone who would dazzle me mentally, pulling off feats of recollection like some cyborgian savant….MyLifeBits is now so big that it faces a classic problem of information management: It’s hellishly difficult to search, and Bell often finds himself lost in the forest. He hunts for an email but can’t lay his hands on it. He gropes for a document, but it eludes him. While eating lunch in San Francisco, he tells me about a Paul Krugman column he liked, so I ask him to show it to me. But it’s like pulling teeth: A MyLifeBits search for “Paul Krugman” produces scores of columns, and Bell can’t quite filter out the right one. When I ask him to locate a phone call from one of his colleagues, he hits a bug: He can locate the name of the file, but when he clicks on it the data are AWOL. “Where the hell is this friggin’ phone call?” he mutters to himself, pecking at the keyboard. “I either get nothing or I get too much!”

In the winter of 2006 Scott Carlson, writing in the Chronicle of Higher Education, reported his experiment with simply recording his life on digital audio tape. “My wife, who is also a journalist, banned recording at home for the first week because she said I acted like I was “on stage.” I had noticed that, too. I never really forgot that the recorder was on, and now and then I sensed I was talking differently, as if to a crowd. I consciously avoided saying things that might be deemed politically incorrect or downright gross, although some of that slipped out and into my memex.”

Clive Thompson again:

In spring 2004, [Bell’s co-worker] Jim Gemmell lost a chunk of his memory. The Microsoft senior researcher had built his own personal MyLifeBits database, filling it, like Bell, with oodles of his email, Web surfing, and pictures. But one day, Gemmell’s hard drive crashed, and he hadn’t backed up in four months. When he got his MyLifeBits back up and running, the hole that had been punched in his memories was palpable, even painful. He’d be working at a project and vaguely remember some Web site or document that was important, then begin drilling down to find it–only to discover that it was part of the missing period. “It was like having my memories stolen,” he says. He was amazed to realize his backup brain was no longer some novelty but a regular part of his psychological landscape. “I realized I count on this now. It’s like I expect to drive cars and have flush toilets.”

Persondemo

Lifelogging is not yet ready for prime time. The tools for lifelogging today are scarce. You can’t buy Microsoft’s SenseCam yet. There’s a lame Web 2.0 Nokia site called Lifelog, which is just a fancy blog site. And a curious app. called TimeSnapper which takes a screen shot of your computer every few minutes and allows you to view a sped-up week’s worth. You can throw web sites, emails, memos, pdfs, and other digital documents into DEVONthink, a kind of electronic associative memory, but the input is pretty labor intensive. Google’s free Gmail with its promise of eternal email is the closest consumer actual lifelogging tool available.

Writing in the March 2007 issue of Scientific American, Gordon Bell says,

Another technical challenge will be ensuring that users are able to open their digital files decades after storing them. We have already run into cases where we could not access documents because their formats were obsolete. Digital archivists will have to constantly convert their files to the latest formats, and in some cases they may need to run emulators of older machines to retrieve the data. A small industry will probably emerge just to keep people from losing information because of format evolution.

Still the overriding concern is how to read, retrieve, and use this huge – and I mean huge – ocean of data that your life will generate. There’s one solution not normally offered in discussions of lifelogging that seems reasonable to me. Ten years ago computer scientist David Gelertner envisioned a new organizing metaphor for one’s growing cache of personal data. Instead of boxes and “windows” a lifelog should be centered on well, a life log, a timeline, chronological log of one’s life. “We’re going from an artificial information storage scheme to a far more natural one. The idea of a timeline, a chronology, a diary, a daily journal, or a scrapbook is so much older and so much more organic and ingrained in human culture and history than the idea of a file hierarchy.”

Gelertner calls his new perspective Lifestreams. For a while a startup company headed by him called Mirror Technologies offered a downloadable demo of the beta version of the software, but it no longer appears to be available. However, his vision is still valid and in my opinion the  mostly likely model for lifelogging. In an interview with a Sun computer rep he spells it out:

I can imagine all the electronic information in my life collected into one beam, or (equivalently) one flowing stream. Every electronic document: every email, photo, draft, URL, audio, video, calendar or address note, and so on. Life is a series of events in time — a timeline with a past, present and future. The events of your life and the memories in which they’re recorded aren’t parceled out into directories, or typed shoeboxes. An information beam incorporates documents of all types into one (focussable) beam. The question “where did I put that piece of information?” always has exactly one answer: it’s in my beam.

The stream has a past, present and future. The future flows into the present into the past. If I’ve posted an appointment or reminder in the “future” part of the stream, eventually it flows automatically into the present where I’ll notice it and be reminded, and then into the past where it’s part of the permanent, searchable, browsable archive. When I acquire a new piece of “real-life” (versus electronic) information — a new memory of (let’s say) talking to Melissa on a sunny afternoon outside the Red Parrot — I don’t have to give this memory a name, or stuff it in a directory. I can use anything in the memory as a retrieval key. (I might recall this event when I think about Melissa, or sunny afternoons outside the Red Parrot.) I shouldn’t have to name electronic documents either, or put them in directories. And I ought to be able to use anything in them as a retrieval key.

In our view of the future, users will no longer care about operating systems or computers; they’ll care about their own streams, and other people’s. I can tune in my stream wherever I am. I can shuffle other streams into mine — to the extent I have permission to use other people’s streams. My own personal stream, my electronic life story, can have other streams shuffled into it — streams belonging to groups or organizations I’m part of. And eventually I’ll have, for example, newspaper and magazine streams shuffled into my stream also. I follow my own life, and the lives of the organizations I’m part of, and the news, etc., by watching the stream flow.

Once lifelogging becomes prevalent, the really interesting times begin. Dozens of legal and cultural puzzles immediately surface.

5323Lifelogging-As-1

* What part of your life is someone else’s privacy?

* Is remembering a scene with your brain different from remembering with a camera?

* Can the government subpoena your lifelog?

* Is total recall fair?

* Can I take back a conversation I had with you?

* Is it a lie if a single word is different from the record?

* How accurate do our biological memories have to be?

* Can you lifelog children without their “permission”?

For many skeptics the social challenges of lifelogging will doom it to a small minority, or else earn it full prohibition. They don’t want ubiquitous lifelogging, and find it implausible than anyone will once they see it in action. In an interview with Scott Carlson, Daniel P.W. Ellis, the guy audio lifelogging, admits, “You know, the reactions I get are mostly pretty strong and mostly pretty negative. Maybe about 50 percent of people I discuss this with have an immediate negative reaction, which they often have difficulty really explaining. There’s clearly a strong intuitive resistance to having a more detailed record of what they’re saying than memory already provides.”

I believe we’ll invent social norms to navigate the times when lifelogging recording is appropriate or not, but for the most part total recording will become as pervasive as text is to us now. It will be everywhere and we won’t even notice it – except when it is gone.




Comments
  • Ronald Snijder

    I have mixed feelings about this, probably because I’m older then 30 (36) :-)

    It feels ‘wrong’ to record every aspect of your lif, and make it available. But on the other hand – isn’t it great to show it all? Most people – and I sure am one of them – find themselves extremely interesting, wonderful, unique etc. It would be the ultimate ego booster to show it all to the world, beyond blogging or sharing your musical taste with tools like last.fm.

    Lifelogging may be one of the true revelotions of the information age. I wonder what my daughter will make of it…

  • dylan fries

    I already bookmark more web-sites for future viewing then i actually return to, people write huge blogs where nothing really interesting happens at all. Do we really want to inspect our lives and the lives of others in such extreme detail? We generate content almost faster then we can read it. I think the most telling sentence is about Ted Nelson, and how nobody had ever bothered to visit the archives. Who are we recording this all for? We already take more home movies and pictures then we know what to do with, imagine going to someones house and not even getting to see vacation pictures, but an all expansive feed of them brushing their teeth, watching tv and eating cheezies? brilliant. We should be careful what we’re getting into. :)

  • http://thetopiarybunny.blogspot.com/ The Topiary Bunny

    Lifelogging seems to afford people the limitless potential to become a photo album in their own life, real time.

    Just like tourists who are too busy taking pictures to actually see and enjoy the experience.

    Also, it appears to, more importantly, give oneself and others the opportunity to bore yourself and your friends to death.

    This Bunny has noticed most blogs which discuss the daily, or even weekly doings of the author, are sedately pedestrian. Lacking thought and interpretation, who the heck cares that you went to Starbucks and met friends and had a latte?

    It’s through analysis, interpretation, discussion, learning that events become interesting.

    Hippity-hop!

  • DH

    Hi Kevin,

    Thank you for your wonderful post. While reading it, lots of conflicting ideas popped up and out in my head. Nevetheless, I got to learn a lot about the topic. Again, thank you for your hard work. Mind if I can post this into my blog? Of course, I will refer to your blog.

  • http://www.yuri.typepad.com Yuri van Geest

    Hi Kevin,

    Thanks again for a great post.

    This post triggered a lot of associations and related topics in my mind. Here goes:

    - Lifelogging will be embraced by the digirati/innovators, bloggers and young(er) generations. Why ? It is an extension of current social networking sites, blogging, Twitter and Jaiku (co-presence), RSS feeds etc.
    - It complements with the Augmented Reality trend (Steve Mann/MIT)
    - It reminds me of the movie Being John Malkovich (living the life of someone else)
    - There are text-based and video-based bottom up, automated tagging/indexing solutions like Autonomy and Blinkx respectively to make indexing our lifestreams easier
    - Lifelogging is the next step in the Transformation/Self Awareness/Exploration trend (Joseph Pine II in Experience Economy). If we record everything we experience, we will be increasingly aware of what we like, share, read, see, recommend, do, hear etc. This will bring the concept of Identity to higher levels, behaviorally based. Lifelogging is a tool for the ultimate need in Maslows’ hierarchy of human needs.
    - It extends the current trend of live webcasting our lives -> http://www.ustream.tv and http://www.justin.tv. Webcasting from mobile devices will be on the rise due to newer mobile technologies like 4G, WiMax etc.
    - Lifelogging is an enabler to boost our creativity and innovation, in my view a necessary skill in our future due to the increasing rate of innovation around us. You mentioned the social (permission-based, opt-in) network integration in your post. In solutions like Twitter and Jaiku we already see the value of this ‘meme sharing’ playing out in this respect.
    - Lifelogging is merging with the trend of Authenticity as to objectify experiences
    - Lifelogging is related to the Web 3.0 movement called Semantic Web with companies like Metaweb, Radar Networks, RealTravel, Joost etc. Example: a more structured web (web as database) will be more useful to provide more targeted news feeds in your lifelogging experience. Our experienced themselves might be structured in RDF and related technologies using tools like Blinx and Autonomy.
    - This week a report will be published on the future of the Metaverse (www.metaverseroadmap.org) looking into the future of lifelogging, Augmented Reality and Virtual Worlds.
    - Lifelogging is related to the current collective intelligence trend and your beautiful post in Wired on We Are The Web.

    In case you would like to see more information on my views on this topic:
    http://yuri.typepad.com/yuri_blog/2007/03/on_authenticity.html

  • http://trevor.smith.name/ Trevor F. Smith

    We can see shadows of these questions being addressed in virtual worlds, as the service providers often maintain server logs which, if given to the user, could be a detailed lifelog. You could also see some of the same social friction around the in-world lifelogging slstats.com as occurs when someone wears a SenseCam.

  • Jaqian

    I semi lifelog. I started recording events with my sons birth, my marriage etc. Then started back-filling it by datamining my emails, text messages for any interesting (to me) material. I’ve also scanned in old concert tickets, wage slips, letters offering me jobs (that I acccepted) etc. I track what I read, films watched, weight, what I eat and how much exercise I get. I’ve also started recording voice notes as sometimes do not get time to jot down my thoughts. Everything is tagged with date YYYY-MM-DD, the subject, person and/or place. I try to keep everything in basic OS independent formats like pdf, jpg, png and txt. I will not use .doc as have experienced trouble opening documents after ten years. Its been a great aid in my memory (which is shite) and I search it at least once a month sometimes more.

    Jaqian (Ireland)

  • zygote5

    In reality, everything IS recorded as electromagnetic ripples in the universal background. While we as a speicies cannot freely and reliably access this information YET, it doesn’t preclude it’s eventual replay.
    Our Vanity and sense of embarasment are stumbling blocks in our way to total freedom and awarness.

  • Peter Debik

    Lifelogging may become a reality by autobiography services on the Internet like http://www.annoknips.com . They are more than simple weblogs. They are somewhat “life logs”, in which people write a journal and illustrate it with pictures and other media of their choice. On annoknips.com e.g. every event can be geocoded, so that events that happened at the same place (the same address) even if they happened to different people are automatically grouped together.

  • Shiri

    Much more than “Being John Malkovitch”, it’s worth watching “The Final Cut” (http://www.imdb.com/title/tt0364343/) for a serious look at the implications of life logging at its fullest.

    Thanks for the informative article. Obviously technology will have to be significantly more advanced before life logging can have a serious effect on (most of) our daily lives. To me it seems that search/IR is the most clearly lacking feature in such a repository, with the necessary additions of photo/sound/video recognition.

  • http://satoshi.blogs.com/uie/2008/07/life-logging-wi.html Satoshi Nakajima

    I think this type of “automatic life logging” is still years away and has some privacy issues (not only to oneself, but to the other people around).

    I, however, believe that the camera-phone, especially iPhone represents a beginning of life-logging (please take a look at my blog entry below).

    http://satoshi.blogs.com/uie/2008/07/life-logging-wi.html

  • ernest williams

    Good post. It gives a brief view of attempts, there are other companies such as Everclip doing similar stuff. But certainly is also worth mentioning the impact of networked spaces and sensory networks as information is distributed in storing/retrieving. More social and interesting stuff is the scanmemories project and william mitchel at MIT, otherwise those devices like sensecam and so on look almost kitsch!

  • Norman Zyland

    I have some concerns about my whole life being recorded: I live in an environment that is increasingly tolerant of diversity but what if it is a Weimar republic about to topple from internal and external pressures? However the issue of ‘can I even be bothered to spend all that time recording and collating?’ is currently a bigger factor.

  • http://www.iambooks.blogspot.com vieome

    Very interesting article, I guess in part we could say that each of us pluged into the web has a life log, or a web life log. The problem lies in the people who attempt to life log, will we ever be able to ignore the observer and be ourselves. My assumption is that the life log will be like our web log, a portrait of what one wants others to think of them rather then the true nature of the individual.

    We see this alot with reality TV, where things get stage managed

  • Jawn

    To lifelog, it must automatically happen (set it and forget it), and it must be search-able.

    Without that it’s too much of a hassle to lifelog and “normal” people will not do it.

    I too am trying to start up an audio log of my life. So far it’s paid off by giving me records of a work injury I had (who, what, where, when) which I used to file a claim a month and a half later. Plus I do the logs on time that I would have wasted anyway, like commuting.