The Centre for Speech Technology Research, The university of Edinburgh

Instrumented Meeting Room (AMIDA WP3)

Project Summary

Smart room environments for communication scene analysis

Project Details

AMI WP2 (Infrastructure, data collection and data management ) Jan. 2004 - Dec. 2006

AMIDA WP3 (Infrastructure and data management) Oct. 2006 - Sept. 2009.

In order to support research activities within the AMI project, the CSTR instrumented meeting room has been developed - one of three instrumented meeting rooms installed as part of the 190 person month data collection and annotation effort within AMI.

 Since initially it was unclear what information collected within the room would be important in the analysis of meetings, the decision was taken to record as much data from as many complementary modalities as possible. To this end the room contains a large number of microphones (both close talking and far field), video cameras (close-up and wide angle), an EBeam whiteboard capture system, and image capture of projected slides. This data is all synchronised to a common timeline, allowing complete reconstruction of the activities within the room at any time.

During the AMI project, the room was used for the collection of the publically available AMI Corpus . This contains 100 hours of meeting recordings, both 'scenario based meetings' where groups take on roles in a fictitious design task, and 'real meetings' which would have occured anyway. CSTR has co-ordinated the annotation of the data to include high quality, manually produced orthographic transcription for each individual speaker, including word-level timings that have derived by using a speech recognizer in forced alignment mode. The corpus also contains a wide range of other annotations, not just for linguistic phenomena but also detailing behaviours in other modalities. These include dialogue acts; topic segmentation; extractive and abstractive summaries; named entities; the types of head gesture, hand gesture, and gaze direction that are most related to communicative intention; movement around the room; emotional state; and where heads are located on the video frames.

The data recorded in the meeting room has been used for research in a number of areas. The far field microphone recordings, for example, have been used for investigating microphone array beamforming algorithms as shown here: Mic Array Demo ; and the video and close talking microphone data has been used in the development of meeting browsers and automatic summarisation systems as shown here : Summarisation Demo

The focus of the AMIDA project is on meetings involving remote particpation by means of Video Conferencing. The instrumented meeting room has recently been upgraded to include a Visual Nexus Video conference server and endpoint, and a system based on VNC for remote collaboration. A second room has also been equiped with a further endpoint allowing us to record further 'scenario based' meetings including a remote participant, so that the effects of video conferencing on group meetings can be studied. Pilot recordings involving video conferences have recently been made, and full recordings and subsequent data annotation will be made in late 2007 as part of the 90 person month, 800 000 euro ''Infrastructure and data management" AMIDA workpackage.

Personnel

Funding Source

EU IST programme (Framework VI)