future of ui

Minority Report science adviser and inventor John Underkoffler demos g-speak — the real-life version of the film’s eye-popping, tai chi-meets-cyberspace computer interface. Is this how tomorrow’s computers will be controlled?

Below are a few passages from Natural-Born Cyborgs, Andy Clark relating to that of the evolution of technology and their influence on human society;

…lets look briefly at a familiar item, one that long-ago passed from the realm of opaque technology into that of transparent symbiotic partner–the humble wristwatch. We humans didn’t always keep precise, objectively measured time. Before the dawn of the city, the factory, and the organized religious order, human beings used natural cycles to prompt daily activities. The sun rises and farming begins, interrupted only by a brief break when the sun is high in the sky. Darkness signals food and sleep. Today, a great many humans are not like this. We work all hours. We plan to meet friends for coffee at 11:45 a.m. We make a date for supper at 10:00 p.m. and a film at midnight, and so on. The transition from a natural-time society to our present arrangements for work and play was mediated by a long thread of technological evolution: a thread that leads from heavy, fixed, unreliable sundials and water clocks, through the development of early oscillating -element based timekeeping, right up to cheap, accurate, personal quartz crystal wristwatches. But the technological story, though fascinating, pales beside the human-centered story. In a mere five hundred years, the opaque, unreliable, fixed-location tower clocks of the Middle Ages gave way to the reliable, cheap, personal timekeepers that we now take so much for granted. Along the way our relationship to time itself was irrevocably changed and transformed.

Once the average city worker was awakened by the call of the night watch, a living person whose task was to patrol the streets shouting the time. A little later the tolling of a bell, either owned by the town or perhaps by a specific employer, woke the townspeople. These measures instilled a degree of what David Landes nicely calls “time obedience.” But with the availability of personal timepieces, int he form of chamber clocks or wristwatches, came the possibility of something new and different–“time discipline.” The presence of easily accessible, fairly accurate, and consistently available time-telling resources enabled the individual to factor time constantly and accurately into the very heart of the endeavors and aspirations. This made possible ways of though, and cultural practices and institutions, which were otherwise precluded by our basic biological nature. Landes makes the point well:

The public clock could be used to open markets and close them, to signal the start of work and its end, to move people around, but it was a limited guide to self-imposed programs. Its dial was not always in view; its bells not always within hearing. Even when heard, hourly bells are at best intermittent reminders. They signal moments. A chamber clock or watch is something very different: an ever-visible, time spent, time wasted, time lost. As such it was prod and key to personal achievement and productivity.

Notice that what counts here is not always consciously knowing the time. Noe of us, I suppose, looks constantly at his or her watch! Rather, the crucial factor is the constant and easy availability of the time, should we desire to know it. Therefore, a prime characteristic of transparent technologies is their poise for easy use and deployment as and when required. Daily, unreflected usage bears this out. As you walk down the street, you are accosted by the familiar cry of the temporarily watchless. “Excuse me, sir, do you happen to know the time?” Asked this question on a busy street, most of us will unhesitatingly reply, even before consulting our wristwatches, that yes, we surely do. Grasping the request hidden in the formulaic question, many of u will also, and without further request, share our knowledge with the time-challenged supplicant. As we do so, we may find ourselves producing one of the characteristic body motions of the modern world. In the suited male or female, this takes the form of a controlled, punch-like extension of the arm, a clockwise half-rotation of the emerging wrist, and a slight lowering of the gaze. This knowledge-retrieval tropism serves, of course, a single practice function–it permits you to focus your gaze briefly upon the face, dial, or display of your watch, that humble example of cyborg technology.

The idea of “mind as spirit-stuff” is no longer scientifically respectable. Instead, mind is seen as the working of a purely physical device. In identifying that physical device solely with the biological brain, we again make a leap of faith, depicting the biological brain itself as the sole and essentially insulated engine of mind and reason. This conception is the old idea of special spirit-stuff in modern dress. A thoroughgoing physicalism should allow mind to determine-by its characteristic actions, capacities, and effects-its own place an location in the natural order. We should not, at any rate, simply assume that it is correct to identify and locate the individual thinking system by reference to the merely metabolic frontiers of skin and skull.

We can, in any event, take away two somewhat less contentious lessons from our discussion of modern timekeeping. The first is that transparent technology is by no means a new invention. It is with us already in a wide variety of old technologies, including; pen, paper, books, watches, written words, numerical notations, and the multitude of almost-invisible props and aids that scaffold and empower our daily though and action. The second is that the a passage to transparency often involves a delicate and temporally extended process of co-evolution. Certainly, the technology must change in order to become increasingly easy to use, access, and purchase; but this is only half of the story because at the same time, elements of culture, education, and society must change also. In the cast at hand, people had to learn to value time discipline as opposed to mere time obedience, and this transition itself, Landes tells us, took over a hundred years to fully accomplish.

Our sense of bodily presence is always constructed on the basis of the brain’s ongoing registration of correlations. If the correlations are reliable, persistent, and supported by a robust, reliable casual chain, then the body image that is constructed on that basis is well grounded. It is well grounded regardless of whether the intervening circuitry is wholly biological or includes non-biological components.

What happened with timekeeping is now happening with the flow of information itself. Mark Weiser’s vision of ubiquitous computing is finding concrete expression in attempts to design and market what Normal calls “information appliances.” Information appliances are characterized by three central features:

1)An information appliance is geared to support a specific activity, and to do so via the storage, reception, processing, and transmission of information.

2)Information appliances form an intercommunicating web. they can “talk” to each other

3)Information appliances are transparent technologies, design to be easy to use, and to fade into the background. they are poised to be taken for granted.

The term “augmented reality” was first used by a group of Boeing engineers and scientists in the early 1990s. Their idea was to use such systems to help workers install complex wiring harnesses in aircraft. The workers would see the desired positioning superimposed upon the actual physical structure of the plane. In a similar, vein, engineers seeking to repair broken equipment might soon see the innards of the machine along side specific repair instructions highlighting the elements to be removed and replaced. Surgeons seeking to repair human brains or bodies could benefit int he same way, seeing ultrasound scans or brain imaging information projected on the appropriate areas. Researchers at the University of Central Florida have overlaid a model of a knee-joint on a woman’s leg. Using infrared LED’s to inform the system about current leg position, the Augmented Reality interface allows onlookers to see just how the bones move while the woman walks and bends. The use of overlaid digital resources to enhance our ordinary daily experience of the world and to provide new means of physical-virtual interaction is likely to play a major role in the next decade. Very soon we may expect to see various kinds of electronically overlaid information, from advertising to information about incoming cell phone calls or even about ourselves apparently suspended in the air as we roam about. Such information might appear attached to the space around an individual, or a shop, or to allow the physical informational realms to seamlessly merge and mingle, in ways that unobtrusively support daily activity and that make maximum use of our normal means of embodied, socially embedded activity. It is worth repeating that such Augmented Reality, though it uses some of the same technologies Virtual Reality, is really quite different at root.

This kind of blurring has education importance too. If we are indeed becoming complex biotechnological hybrids, a major challenge for the future will be to train young minds to think well about a world in which the physical and the informational/digital are densely and continuously interwoven.

Nurtured by such experiences, and living and moving in a world populated with ubiquitous computing devices, augmented reality displays, and various kinds of tangible computing, next-generation human minds will not invest very heavily int he virtual/physical divide. Instead these minds will focus on activity and engagement, seeing both the virtual and the physical as interpenetrating areas for motion, perception and action. Mixed reality play intends to block the stale opposition between the real and the virtual, or the bodily and the information, revealing each for what it is; just one more aspect of a larger world in which hybrid selves lives, move, work, and play.