Ambient Telepresence: Colleague Awareness in Smart Environments
Hans-W. Gellersen and
Michael Beigl{
michael,hwg}@teco.eduAbstract. Ambient Telepresence is a method to support informal awareness in distributed collaborative work, and to promote a sense of presence of people who are in fact not co-located. In this approach, everyday things that people use are augmented with awareness technology, creating a smart environment in which information on background activity can be collected and interpreted. This information is transmitted to remote sites, where it is rendered for peripheral awareness, using ambient media. The approach is demonstrated with the MediaCup environment, in which coffee cups are augmented with sensor, processing, and communication to obtain some basic cues on what people do.
Ambient Telepresence is a method using smart environments for providing informal awareness in distributed collaborative work. ‘Smartness’ of the environment is used in two ways: to obtain information on background activity in the local work place, and to present this information in a non-distracting way at a remote location. The information communicated between locations is interpreted by the smart environment; this is in contrast to conventional methods for informal awareness which generally use video-based techniques to provide remote people with a sense of what’s going on. In [4] we recently introduced the notion of white box context vs. black box context to distinguish interpreted awareness information from information simply channeled to a remote site.
In the field of computer-supported collaborative work (CSCW), awareness support systems are designed to provide distributed people with similar kinds of cues as available in face-to-face settings. Cues such as whether a colleague appears to be very busy help to assess availability for interaction and guide the social coordination of collaborative work. The common approach to provide such cues over distance is to use video communication to let people follow up on what’s going on at a remote site. Awareness systems based on this approach mediate video material but doe not extract cues or any kind of higher-level context: this is up to the human recipient, distracting them from other tasks, and potentially overloading them (and their desktop displays) with information. Besides video-based awareness, some systems use technology infrastructure to obtain very specific cues for collaboration. The ActiveBadge system for example provides a smart environment to track the whereabouts of people, a fundamental cue for initiation of contact and collaboration [13]. In this kind of system, the people’s location is a white box context, understood by the support system which can use this knowledge to generate suitable target representations, such as graphical location maps and audio event notifications.
The ambient telepresence approach that we propose in this paper is based on a smart infrastructure to obtain a multiplicity of specific cues useful for informal awareness. The key ideas are:
- Tracking manipulation of the everyday things people use rather than tracking the people themselves
- Building smartness into the things that surround us rather than introducing new smart devices
- Collecting white box context rather than black box context to support informal awareness
- Mapping awareness information to a target representation that is not monopolizing attention and not competing for display resources
The paper is now organized as follows: we will first discuss related work on awareness support systems, on the use of physical devices for mediating awareness, and on ambient display of awareness information. In Section 3 we will introduce ambient telepresence, and in Section 4 we will describe the implementation of a system demonstrator. The demonstrator is based on MediaCups, ordinary coffee cups augmented with sensor technology, processing and communication to track their use.
In co-located teams, spontaneous meetings and casual interaction are used to coordinate the flow of work. Spontaneous meetings are facilitated by informal awareness of who is around and what they do. For distributed teams, real-time groupware for is readily available to provide communication channels for spontaneous casual interaction but as Cockburn and Greenberg point out, people still have trouble making contact for lack of mutual awareness [5]. In the CSCW community, this issue has been addressed with video-based methods for awareness support.
2. 1 Video-based Informal Awareness
Video-based methods for providing informal awareness include media spaces, video glances and snapshots. In media spaces, people can view remote offices and spaces through continuous video [1]. The use of continuous video raises issues of giving rather too much information, waisting bandwidth, and compromising privacy. In video glances the continuous video is replaced by a user-initiated brief two-way video connection to a remote person’s location, which is like peeping into a colleague’s office [12]. Video snapshots, on the other hand, provide for continuous awareness but replace the continuous video stream with snapshots of low resolution that are updated only every few minutes [10].
There are a number of problems associated with video-mediated informal awareness. Video provides only somewhat static and restricted views of a remote site, capturing awareness information only within limitations. Mediated video images are usually of low resolution or even deliberately blurred for the sake of privacy; this means the awareness information carried in the images is rather coarse. Further, people have to attend to their computer screens to obtain awareness information, and awareness displays compete with other applications for screen real estate. This latter issue was addressed in Buxton’s Ubiquitous Media Spaces, using multiple videoconferencing units separate from the computer and placed in the architectural work space, each as surrogate for a remote person [3].
2. 2 Sensor-based Context-Awareness
An alternative to capturing awareness information with video is to use sensors. In contrast to video, sensors capture rather specific information from which can be interpreted and processed by computers. A simple example are infrared or radio frequency sensors that keep track of electronic badges worn by people, as in the ActiveBadge system [13]. Special-purpose sensors can be used to obtain other context relevant for informal awareness, for example to monitor whether the telephone is engaged, or whether the door is open or shut and [7].
Individual sensors capture only limited awareness information but it can be assumed that with the use of many sensors and the combination of different kinds of sensors a large degree of awareness can be achieved. Smart environments with ubiquitous sensors are in contrast to perceptual intelligence, shifting the cost for obtaining awareness from processing to infrastructure. Given the current advances in sensor technology with respect to size, cost and accuracy, their ubiquitous deployment in smart environments is becoming viable.
2.3 Things That Mediate
As alternative to the desktop-bound use of video, a number of systems demonstrate the use of physical devices separate from the computer to mediate awareness. In most of these systems, people interact rather explicitly through physical devices. Examples are Shaker [11] and inTouch [2] which facilitate interaction through pairs of haptic devices. In contrast there are only few examples in which physical devices are used for informal awareness of remote activity. Kuzuoka and Greenberg have designed a number of Digital but Physical Surrogates which are tangible representations of remote people [9]. These surrogates are used to indicate activity and availability of the people they represent. For example the peek-a-boo surrogate is a figurine that rotates to face away if the represented person becomes unavailable, as measured with simple sensors.
The use of physical devices in our approach, ambient telepresence, differs in two ways from Kuzuoka and Greenberg’s work: ambient telepresence is based on computationally augmented everyday devices rather than newly introduced devices, and these devices are used to collect awareness information while Physical but Digital Surrogates are used to display the information. More directly related to ambient telepresence is the Internet Bed which is, however, rather one-of-a-kind installations. In the Internet Bed, the presence of a person in one bed is translated to warmth and heartbeat sounds on a remote bed [6].
2.4 Ambient display of awareness information
Ambient displays as investigated in the MIT MediaLab’s AmbientRoom overcome limitations of conventional computer display and can be used to reflect activity in the information world in our surrounding physical environment [14]. Examples for ambient displays demonstrated in the AmbientRoom are water ripples, light patches and sounds. These displays exemplify calm technology and lend themselves to peripheral awareness, presenting information in the background rather than having it intrude in the foreground of other work activity.
In our ambient telepresence system, ambient displays are used to convey the feeling of a remote person’s presence. This relates to one of the installation in the AmbientRoom, promoting a remote hamster’s presence by representing it’s activity in a hamster wheel in a physical vibrating object [8].
We work with a simple definition for ambient telepresence:
Ambient Telepresence is a method to give someone the feeling that someone else is present while they are actually not co-located
Ambient telepresence connects people at different locations through several steps as shown in figure 1, and as illustrated in figure 2. First, information on the background activity of a person is obtained in their local environment. The technical approach is to have sensors and processing embedded in the everyday things that people use in their environments. The collected information is interpreted to obtain context information at symbolic level, handled as events. The next step is to transfer these events to the remote location. At the remote site the event is processed by a dispatcher, mapping the event (i.e. the activity represented by the event) to an ambient display, available to a connected person for peripheral awareness. We assume a mapping to a representation that is perceived as natural, for example by generating sounds directly associated with the represented background activity.
Fig. 1. Connecting locations for ambient telepresence: background activity sensed in one location is transmitted as event, and given an ambient representation at the other location.
Fig. 2. Ambient Telepresence: a person’s background activity is tracked, transferred as awareness event and transformed to an ambient representation at a remote location
Figure 3 depicts the system architecture. The remote sites are connected via the Internet to exchange events. Locally, different media and computationally augmented devices are connected in a smart environment. The devices in this environment broadcast awareness information, and a dispatcher is used to channel information to remote sites. The dispatcher is also used for dispatching events received from remote sites to local media for their representation.
Fig. 3. System architecture for ambient telepresence: local devices are integrated in a smart environment controlled by a dispatcher, and remote sites are connected via the Internet.
For demonstration of ambient telepresence we have implemented a setup as shown in figure 4. Background activity in the workplace is tracked by monitoring manipulation of computer keyboards and coffee cups. At a connected remote site sounds are generated to represent the remote activity, to give it a virtual presence.
Fig. 4. Setup for demonstration of ambient telepresence
4. 1 A Smart Environment for Ambient Telepresence
In our office environment we have created a smart environment to demonstrate ambient telepresence. The backbone of this environment is an infrared network with a transceiver infrastructure mounted under the ceiling. We have used HP’s HSDL 1001 IrDA Transceiver with 15 ° range, and about 1m˛ footprint. Transceivers are connected via serial line to a computer that connects the environment to the Internet, to connect with remote sites.
As example for augmentation of everyday objects with awareness technology we have developed MediaCups which are coffee cups with built-in sensors, processing and communication. The MediaCup hardware comprises sensors for temperature and acceleration, a PIC 16F84 microcontroller, an infrared diode for communication, and a standard Lithium battery (3V, 120mAh). To track how the cup is handled, we have integrated the two-axis acceleration sensor ADXL202AQC of Analog Devices, which can measure both dynamic and static acceleration. The sensor uses 0,6 mA and is turned off between measurement cycles to save power. For temperature sensing we have integrated the DS1621 Dallas Semiconductor chip measuring from –55 to +125 °C, with 1ľA standby current, and 400ľA communication current. The microcontroller has 1792 Byte Flash RAM for programs, 68 Byte RAM, and 13 I/O ports used for control of temperature chip, accelerometer, and infrared diode. With 4 MHz, power consumption is below 2mA, and in sleep mode below 1 ľA. With the Lithium battery, the MediaCup can be powered for approximately 2-3 weeks.
Figure 5 shows two MediaCup prototypes. As shown on the left, the MediaCup hardware is embedded in a non-obtrusive way at the bottom of a coffee cup. The latest prototype shown on the right now has the hardware mounted in the rubber base of the HUC99 coffee cup, allowing removal so that the cup can be dish-washed. At present we have 8 of these cups operational in our office environment.
4.2 Capturing of Context
In the MediaCup, sensor readings are taken every 50ms for acceleration, and every 3 seconds for temperature. The raw sensor data is processed on the MediaCup, applying heuristics to obtain cues regarding handling and situation of the coffee cup. Acceleration sensor data is mapped to three distinct cues: cup is stationary, drinking out of the cup, and playing with the cup. Temperature data is mapped to the cues: filled up, cooled off, and actual temperature.
Cues are communicated every 15 seconds via a low-powered 3mm infrared sender SFH 409-s, using IrDA physical layer coding. MediaCups are tracked in the infrared transceiver network, so their location becomes available as additional context. The MediaCup can also communicate via transceivers already present in desktop and laptop computers.
In addition to context obtained from the MediaCups we also collect awareness information available from people’s interaction with their computers. To demonstrate this, we have included a monitor that keeps track of keyboard hits and of mouse manipulation.
4.3 Ambient Display of Context Information
In our current demonstrator we use audio only for ambient display of awareness information. The mapping of context to sounds is implemented in the media dispatcher. Table 1 lists a few examples for this mapping. The idea is basically to reproduce the noise associated with a certain activity. However, we have to note that our primary concern was technology demonstration, and that at this stage we abstracted from the available body of research on audio display.
MediaCup rotate |
Hard rubbing sound |
MediaCup put down |
Hard clack noise |
Keyboard hit |
Key click noise |
Table 1. Mapping awareness information to audio representation
Fig. 5. MediaCup prototypes. Sensors, processor, and infrared diode are built into cup base
We have introduced ambient telepresence as a new concept for support of informal awareness. The key ideas are: tracking of the everyday things people use to obtain information on background activity; augmentation of everyday objects with sensors and processing to facilitate their tracking; and use of ambient media to display obtained awareness information. To demonstrate the approach, a smart environment with computerized coffee cups, the MediaCups, was implemented. This environment was demonstrated at an exhibition and is now operational in the authors’ work group.
We have used the MediaCup environment also for another colleague-awareness application [4]. In this application, context information obtained from the cups and from other sensors in the smart environment was used to support analysis of video streams for production a storyboard-like representation of recent activity in a workplace. This application as well as the discussed ambient telepresence demonstrate opportunities that smart environments create for support of collaborative work. However the presented work is only a first step to inform further research which most importantly will have to consider how these technical opportunities effect the people and their collaboration.
References