One of Zig’s associates to take the stage was Larry Heck, Chief Scientist, Speech at Microsoft. Larry believes that there are three areas of research and development that will combine to make speech a part of everyday interactions with computers. First, the advent of ubiquitous computing and the need for natural user interfaces (NUIs) means that we cannot keep relying on GUIs and keyboards for many of our computing needs. Second, cloud computing makes it possible to gather rich data to train speech systems. Finally, with advances in speech technology we can expect to see search move beyond typing keywords (which is what we do today sitting at our PCs) to conversational queries (which is what people are starting to do on mobile phones).

I attended four other talks with topics relevant to my research. Brigitte Richardson discussed her work on Ford’s Sync. It’s exciting to hear that Ford is coming out with an SDK that will allow integrating devices with Sync. This appears to be a similar approach to ours at Project54 – we also provide an SDK which can be used to write software for the Project54 system [1]. Eduardo Olvera of Nuance discussed the differences and similarities between designing interfaces for speech interaction and those for interaction on a small form factor screen. Karen Kaushansky of TellMe discussed similar issues focusing on customer care. Finally, Kathy Lee, also of TellMe, discussed her work on a diary study exploring when people are willing to talk to their phones. This work reminded me of an experiment in which Ronkainen et al. asked participants to rate the social acceptability of mobile phone usage scenarios they viewed in video clips [2].

I also had a chance to give a talk reviewing some of the results of my collaboration with Tim Paek of Microsoft Research. Specifically, I discussed the effects of speech recognition accuracy and PTT button usage on driving performance [3] and the use of voice-only instructions for personal navigation devices [4]. The talk was very well received by the audience of over 25, with many follow-up questions. Tim also gave this talk earlier this year at Mobile Voice 2010.

For pictures from SpeechTEK 2010 visit my Flickr page.

References

[1] Andrew L. Kun, W. Thomas Miller, III, Albert Pelhe and Richard L. Lynch, “A software architecture supporting in-car speech interaction,” IEEE Intelligent Vehicles Symposium 2004

[2] Sami Ronkainen, Jonna Häkkilä, Saana Kaleva, Ashley Colley, Jukka Linjama, “Tap Input as an Embedded Interaction Method for Mobile Devices,” TEI 2007

[3] Andrew L. Kun, Tim Paek, Zeljko Medenica, “The Effect of Speech Interface Accuracy on Driving Performance,” Interspeech 2007

[4] Andrew L. Kun, Tim Paek, Zeljko Medenica, Nemanja Memarovic, Oskar Palinko, “Glancing at Personal Navigation Devices Can Affect Driving: Experimental Results and Design Implications,” Automotive UI 2009

Peter and his colleagues are investigating automotive navigation aids and are currently preparing for an on-road study. I’m happy to report that this study will utilize one of our eye trackers. My visit provided an opportunity for us to discuss this upcoming study and how the eye tracker may be useful in evaluating the research hypotheses. Part of this discussion was a Telecommunications Forum talk I gave – see the slides below:

I want to thank Peter and his colleagues at FTW for hosting me and I’m looking forward to our upcoming collaboration. I also want to thank FTW for providing funding for my visit.

References

[1] Albrecht Schmidt, Anind L. Dey, Andrew L. Kun, Wolfgang Spiessl, “Automotive User Interfaces: Human Computer Interaction in the Car,” CHI 2010 Extended Abstracts

Enter Project54. According to Matt “[r]esearchers are working to reduce the risk.” He goes on to describe UNH’s Project54 system which allows officers to issue voice commands in order to interact with in-car electronic devices. This means officers can keep their eyes on the road and their hands on the wheel. The article includes praise for the Project54 system by Captain John G. LeLacheur of the New Hampshire State Police. The Project54 system was developed in partnership with the NHSP and almost every NHSP cruiser has the Project54 system installed.

Both the print and the online versions of the article begin with a picture of the Project54 in-car system. This great picture was taken by Sheryl Senter and it shows Sergeant Tom Dronsfield of the Lee, NH Police Department in action.

How should we design a speech interface that will support multi-threaded human-computer dialogues when the human is engaged in a manual-visual task? In order to begin answering this question Alex explored spoken dialogues between two human conversants. The hypothesis is that a successful HCI design can mimic some aspects of human-human interaction.

In Alex’s experiments one of the conversants (the driver) operated a simulated vehicle while the other (an assistant) was only engaged in the spoken dialogue. The conversants were engaged in an ongoing and in an interrupting spoken task. Alex’s dissertation discusses several interesting findings, one of which is that driving performance is worse during and after the interrupting task. Alex proposes that this is due to a shift in the driver’s attention away from driving and to the spoken tasks. The shift in turn is due to the perceived urgency of the spoken tasks – as the perceived urgency increases the driver is more likely to shift her attention away from driving. The lesson for HCI design is to be very careful in managing the driver’s perceived urgency when interacting with devices in the car.

Alex benefited tremendously from the help of my collaborator on this research Peter Heeman. Peter provided excellent guidance throughout Alex’s PhD studies for which I am grateful. Peter and I plan to continue working with Alex’s data. The data includes transcribed dialogues, videos, driving performance as well as eye tracker data. I am especially interested in using the eye tracker’s pupil diameter measurements to estimate cognitive load as we have done in work lead by Oskar Palinko [1].

References

[1] Oskar Palinko, Andrew L. Kun, Alexander Shyrokov, Peter Heeman, “Estimating Cognitive Load Using Remote Eye Tracking in a Driving Simulator,” ETRA 2010

Our SIG deals with human-computer interaction in the car. This is an exciting field of study that was the topic of a CHI 2008 SIG [2] as well as the AutomotiveUI 2009 conference [3], and the AutomotiveUI 2010 CFP will be posted very soon. In the last several years human-computer interaction in the car has increased for two main reasons. One, many cars now come equipped with myriad electronic devices such as displays indicating power usage and advanced driver assistance systems. Second, users (drivers and passengers) bring mobile devices to cars. The list of these brought-in mobile devices is long but personal navigation devices and mp3 players are probably the most common ones.

At the SIG we hope to discuss user interface issues that are the result of having all of these devices in cars. Some of the questions are:

• How can we reduce (or eliminate) driver distraction caused by the in-car devices?
• Can driver interactions with in-car devices actually improve driving performance?
• Can users take advantage of novel technologies, such as streaming videos from other cars?
• How do we build interfaces that users can trust and will thus actually use?
• How can car manufacturers, OEMs, brought-in device manufacturers and academia collaborate in envisioning, creating and implementing automotive user interfaces?

The 2008 CHI SIG [2] attracted over 60 people and we’re hoping for similar (or better!) turnout.

References

[1] Albrecht Schmidt, Anind L. Dey, Andrew L. Kun, Wolfgang Spiessl, “Automotive User Interfaces: Human Computer Interaction in the Car,” CHI 2010 Extended Abstracts (to appear)

[2] D. M. Krum, J. Faenger, B. Lathrop, J. Sison, A. Lien, “All roads lead to CHI: interaction in the automobile,” CHI 2008 Extended Abstracts

[3] Albrecht Schmidt, Anind Dey, Thomas Seder, Oskar Juhlin, “Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2009″