Author Archives: Andrew

Personal and Ubiquitous Computing theme issue: Automotive user interfaces and interactive applications in the car

I’m thrilled to announce that the theme issue of Personal and Ubiquitous Computing entitled “Automotive user interfaces and interactive applications in the car” is now available in PUC’s Online First. I had the pleasure of serving as co-editor of this theme issue with Albrecht Schmidt, Anind Dey, and Susanne Boll.

The theme issue includes our editorial [1], and three papers. The first is by Tuomo Kujala, who explores scrolling on touch screens  while driving [2]. The second is by Florian Schaub, Markus Hipp, Frank Kargl, and Michael Weber, who address the issue of credibility in the context of automotive navigation systems [3]. The third paper is co-authored by me, my former PhD student Alex Shyrokov, and Peter Heeman. We explore multi-threaded spoken dialogues between a driver and a remote conversant [4]. The three papers were selected in a rigorous review process from 17 submissions, by approximately 50 reviewers.

 

References

[1] Andrew L. Kun, Albrecht Schmidt, Anind Dey and Susanne Boll, “Automotive User Interfaces and Interactive Applications in the Car,” PUC Online First

[2] Tuomo Kujala, “Browsing the Information Highway while Driving – Three In-Vehicle Touch Screen Scrolling Methods and Driver Distraction,” PUC Online First

[3] Florian Schaub, Markus Hipp, Frank Kargl, and Michael Weber, “On Credibility Improvements for Automotive Navigation Systems,” PUC Online First

[4] Andrew L. Kun, Alexander Shyrokov, and Peter A. Heeman, “Interactions between Human-Human Multi-Threaded Dialogues and Driving,” PUC Online First

2012-2013 senior project ideas

Here is a list of senior project ideas I would be very interested in working on. I’m looking for teams or individuals from the ECE and CS departments.

  1. Augmented reality (AR) on the cheap. Work in our lab has shown that in-vehicle AR navigation aids can effectively guide drivers, that they do not distract, and that drivers like them [1] – see video below. However, they’re expensive to make. In this senior project students will build a device that will augment the speech output of a personal navigation device/app (e.g. Google Navigation) with LED displays indicating upcoming turns. The device will be tested in driving simulator experiments.

  2. Instrumented steering wheel. Today’s vehicles have myriad buttons, many on the steering wheel [2]. This project will build on our work with a push-to-talk glove [3, 4] to explore how drivers could interact with in-vehicle devices by tapping the steering wheel. Additionally, sensors on the steering wheel will produce feedback about the driver’s state (e.g. a stressed driver might squeeze the steering wheel much harder than a relaxed driver). Multiple driving simulator experiments will validate the design of the instrumented steering wheel.
  3. Video call. Work in our lab has shown that video calling can be a real distraction from driving [5] – see video below. This project will explore how different topics of conversation (e.g. playing word games vs. arguing), different relationships between conversants (e.g. friends vs. strangers), and different driving conditions (e.g. city vs. highway) influence driver’s ability to operate a simulated vehicle while video calling.

  4. Tangible user interfaces that support exploring large, time-sequence data sets. The Environmental Response Management Application (ERMA) is a web-based data visualization application. It visualizes geo-coded time series, without requiring users to know how to access specialized databases, or overlay data from these databases on virtual maps. ERMA was developed at UNH, under the guidance of the Coastal Response Research Center (CRRC). Nancy Kinner is the co-director of the UNH Coastal Response Research Center. Building on Nancy’s experiences with ERMA, she and I are interested in exploring how a tangible user interfaces utilizing a multi-touch table could be used to access and manipulate geo-coded time series. In this project students will develop a user interface on a multi-touch table. The interface will allow a human operator to access remote databases, manipulate the data (e.g. by sending it to Matlab for processing) and display the results on a virtual map or a graph.
  5. Tangible user interfaces for children. How can we entertain and teach kids using technologies such as the Microsoft Surface and tangible interfaces? Students working on this project would seek opportunities to collaborate with other researchers on the UNH campus to further explore this question.
  6. Your ideas related to user interfaces in vehicles and on multi-touch tables. Do you have an idea you’d like to explore? Tell me more about it!
If this sounds interesting send me email and let’s talk.

References

[1] Zeljko Medenica, Andrew L. Kun, Tim Paek, Oskar Palinko, “Augmented Reality vs. Street Views: A Driving Simulator Study Comparing Two Emerging Navigation Aids,” MobileHCI 2011

[2] Dagmar Kern, Albrecht Schmidt, “Design Space for Driver-based Automotive User Interfaces,” AutomotiveUI 2009

[3] Oskar Palinko, Andrew L. Kun, “Prototype Wireless Push-to-Talk Glove,” IET 2008

[4] Oskar Palinko, Andrew L. Kun, “Comparison of the Effects of Two Push-to-Talk Button Implementations on Driver Hand Position and Visual Attention,” Driving Assessment 2009

[5] Andrew L. Kun, Zeljko Medenica, “Video Call, or Not, that is the Question,” to appear in CHI ’12 Extended Abstracts

Introducing ubiquitous computing through videos

Students and technical professionals can quickly gain an introduction to a variety of topics in pervasive/ubiquitous computing by watching online research videos. Of course, watching videos is no replacement for reading papers (yes, starting with Weiser’s) and using an excellent textbook, such as John Krumm‘s ubicomp fundamentals. Yet, videos can quickly convey many basic ubicomp ideas in an engaging manner.

Good places to look for videos are the usual suspects: Vimeo and YouTube. Additionally, many conferences now allow uploading videos as support material for papers, and conferences such as Ubicomp and Pervasive have tracks dedicated to video submissions (see e.g. Ubicomp 2012 and Pervasive 2012).

In my 2010 intro ubicomp course I used the following videos, grouped by general topic area:

Field studies:

  1. Project54 police radio study (YouTube)
  2. Tagging photographs using voice commands (YouTube)
  3. Linking with Flickr using voice commands (YouTube)
  4. AmbiKraf (YouTube)
  5. HealthLine: Information Access for Community Health Workers in Developing Regions (YouTube)
  6. LINC, An Inkable Digital Family Calendar (YouTube)
  7. Rexplorer – an urban interactive game for tourists (YouTube)

Privacy:

  1. Sharing data on public displays: academic transcript example (YouTube)
  2. Sharing data on public displays: family pictures example (YouTube)
  3. Sharing data on public displays: map example (YouTube)
  4. Multi-user interaction using handheld projectors (YouTube)
  5. Empathy mirror (YouTube – removed) (Website)
  6. How to hack RFID-enabled credit cards for $8 (YouTube)
  7. IBM RFID commercial (YouTube)
  8. Minority report – mall scene (YouTube)
  9. RFID parking access control long range system (YouTube)
  10. Sensecam: Cambridge day out (YouTube)
  11. The Ambient Clock (YouTube)
  12. The RFID Ecosystem Project (YouTube)

User interfaces:

  1. Natural User Interfaces: Utilizing physiological data (YouTube)
  2. Skinput: Appropriating the body as an input surface (YouTube)
  3. 10GUI: 10 finger multitouch user interface (YouTube)
  4. BumpTop 3D multi-touch desktop (YouTube)
  5. Ford SYNC and Pandora (YouTube)
  6. Google Maps navigation (Beta) (YouTube)
  7. Microsoft Courier in action (YouTube)
  8. Microsoft Research: Project Gustav (YouTube)
  9. Microsoft future vision: productivity (YouTube)
  10. Microsoft’s vision of the future (Parody) (YouTube)
  11. Microsoft future vision: Windows home concept (YouTube)
  12. Microsoft future vision: manufacturing (YouTube)
  13. Microsoft future vision: personal health (YouTube)
  14. Microsoft future vision: banking (YouTube)
  15. Microsoft future vision: retail (YouTube)
  16. NanoTouch (YouTube)
  17. Reboard (YouTube)
  18. The invisible train (YouTube)
  19. Searchvox (website)

Of course this list is now 2 years old – time to update!

Video calling while driving? Not a good idea.

Do you own a smart phone? If yes, you’re likely to have tried video calling (e.g. with Skype or FaceTime). Video calling is an exciting technology, but as Zeljko Medenica and I show in our CHI 2012 Work-in-Progress paper [1], it’s not a technology you should use while driving.

Zeljko and I conducted a driving simulator experiment in which a driver and another participant were given the verbal task of playing the game of Taboo. The driver and the passenger were in separate rooms and spoke to each other over headsets. In one experimental condition, the driver and the other participant could also see each other as shown in the figure below. We wanted to find out if in this condition drivers would spend a significant amount of time looking at the other participant. This is an important question, as time spent looking at the other participant is time not spent looking at the road ahead!

We found that, when drivers felt that the driving task was demanding, they focused on the road ahead. However, when they perceived the driving task to be less demanding they looked at the other participant significantly more.

What this tells us is that, under certain circumstances, drivers are willing to engage in video calls. This is due, at least in part, to the (western) social norm of looking at the person you’re talking to. These results should serve as a warning to interface designers, lawmakers (yes, there’s concern [2]), transportation officials, and drivers that video calling can be a serious distraction from driving.

Here’s a video that introduces the experiment in more detail:

References

[1] Andrew L. Kun, Zeljko Medenica, “Video Call, or Not, that is the Question,” to appear in CHI ’12 Extended Abstracts

[2] Claude Brodesser-Akner, “State Assemblyman: Ban iPhone4 Video-Calling From the Road,” New York Magazine. Date accessed 03/02/2012

Summer course to introduce tools of research

Click image for source

This summer I’ll be teaching ECE 900 Electrical and Computer Engineering Seminar. The seminar introduces graduate students to the general tools of research. Students gain practical experience with framing research questions, setting goals, and proposing hypotheses. We also discuss ideas on how to read, write and review research publications, and on how to give oral presentations about such documents to different types of audiences. Finally, we explore best practices for success in graduate school.

A key outcome of the seminar is a research proposal. Proposals address the steps required to complete the research requirement of students’ graduate degrees (MS or PhD).

Interested? Please take a look at the facts below and/or send me email with questions and suggestions.

Who is this course for?
While ECE 900 is a requirement for UNH ECE graduate students, the course is open to graduate students from all UNH departments. I certainly hope non-ECE students will join us this summer.

Will the course be online?
Yes. Students resident at UNH can participate in person, remote students can participate online (no in-class meetings required).

How many credits?
During the academic year ECE 900 is offered both in the fall and spring. Material covered in the spring builds on that covered in the fall. Similarly, the summer course will run in two consecutive sections, each four weeks long. Section 1 will cover material covered in ECE 900 in the fall, while section 2 will cover the spring material. Students will earn 2 credits for completing each section, for a total of 4 credits for the summer.

What are the prerequisites?
Graduate standing.

Is there online material from previous years?
Yes, e.g. fall 2010 and spring 2011.

Schedule
ECE 900, Section 1 (fall material): June 11-July 6, Tuesdays and Thursdays 12-2 PM
ECE 900, Section 2 (spring material): July 9 – August 3, Tuesdays and Thursdays 12-2 PM

Further progress towards disambiguating the effects of cognitive load and light on pupil diameter

In driving simulator studies participants complete both visual and aural task. The most obvious visual task is driving itself, but there are others such as viewing an LCD screen that displays a map. Aural tasks include talking to an in-vehicle computer. I am very interested in estimating the cognitive load of these various tasks. One way to estimate this cognitive load is through changes in pupil diameter: in an effect called the Task Evoked Pupillary Response (TEPR) [1], the pupil dilates with increased cognitive load.

However, in driving simulator studies participants scan a non-uniformly illuminated visual scene. If unaccounted for, this non-uniformity in illumination might introduce an error in our estimate of the TEPR. Oskar Palinko and I will have a paper at ETRA 2012 [2] extending our previous work [3], in which we established that it is possible to separate the pupil’s light reflex from the TEPR. While in our previous work TEPR was the result of participants’ engagement in an aural task, in our latest experiment TEPR is due to engagement in a visual task.

The two experiments taken together support our main hypothesis that it is possible to disambiguate (and not just separate) the two effects even in complicated environments, such as a driving simulator. We are currently designing further experiments to test this hypothesis.

References

[1] Jackson Beatty, “Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources,” Psychological Bulletin, 276-292, 91(2)

[2] Oskar Palinko, Andrew L. Kun, “Exploring the Effects of Visual Cognitive Load and Illumination on Pupil Diameter in Driving Simulators,” to appear at ETRA 2012

[3] Oskar Palinko, Andrew L. Kun, “Exploring the Influence of Light and Cognitive Load on Pupil Diameter in Driving Simulator Studies,” Driving Assessment 2011

2011 opportunity for UNH CS students: multi-touch surface manipulation of geo-coded time series

When I think back to the recent BP oil spill in the Gulf of Mexico, the images that come to mind are of wildlife affected on beaches, idle fishing vessels, and a massive response that involved thousands of people across multiple states.

How can such a massive response be managed? There is no single answer. However, one thing that can help is to make data about various aspects of the disaster, as well as the response effort, accessible to those conducting the response activities. This is the role of the Environmental Response Management Application (ERMA). ERMA is a web-based data visualization application. It visualizes geo-coded time series, without requiring users to know how to access specialized databases, or overlay data from these databases on virtual maps. ERMA was developed at UNH, under the guidance of the Coastal Response Research Center (CRRC).

Nancy Kinner is the co-director of the UNH Coastal Response Research Center. Building on Nancy’s experiences with ERMA, she and I are interested in exploring how a multi-touch table could be used to access and manipulate geo-coded time series.

Seeking UNH CS student

To further are effort, we are seeking a UNH CS student interested in developing a user interface on a multi-touch table. The interface would allow a human operator to access remote databases, manipulate the data (e.g. by sending it to Matlab for processing) and display the results on a virtual map or a graph. This work will be part of a team effort with two students working with Nancy on identifying data and manipulations of interest.

What should the user interface do?

The operator should be able to select data, e.g. from a website such as ERMA. Data types of interest include outputs from various sensors (temperature, pressure, accelerometers, etc.). Data manipulation will require some simple processing, such as setting beginning and end points for sensor readings. It will also require more complex processing of data, e.g. filtering.

What platform will be used?

The project will leverage Project54’s Microsoft Surface multi-touch table. Here is a video by UNH ECE graduate student Tim April introducing some of the interactions he has explored with the Surface.

What are the terms of this job?

We are interested in hiring an undergraduate or graduate UNH CS student for the 2011-2012 academic year, with the possibility of extending the appointment for the summer of 2012 and beyond, pending satisfactory performance and the availability of funding. The student will work up to 20 hours/week during the academic year and up to 40 hours a week during the summer break.

What are the required skills? And what new skills will I acquire?

Work on this ream-project will require object-oriented programming that is necessary to control the multi-touch table. You will explore the application of these skills to the design of surface user interfaces as well as experiments with human subjects – after all we will have to systematically test your creation! Finally, you will interact with students and faculty from at least two other disciplines (civil/environmental and electrical/computer engineering), which means you will gain valuable experience working on multi-disciplinary teams.

Interested? Have questions, ideas, suggestions?
Email me.

Budapest information session – fall 2011

This Tuesday I organized an information session for UNH ECE students interested in participating in the Budapest Exchange Program. Under the program UNH CEPS students can spend a semester at the Budapest University of Technology and Economics (BUTE) during their junior year. The program also brings BUTE students to UNH for a semester.

Under the exchange program five UNH ECE students spent the spring 2011 semester in Budapest. All five (picture above) were present at the information session to share their experiences with the eight juniors interested in the program (picture below).

The experiences we heard about were awesome, in fact life-changing. All five alumni of the exchange program agreed that spending a semester in Budapest was an excellent decision, with some calling it their “best decision.” The program was challenging, but that was one of its most important aspects because of the skills and confidence it built in each and every one of them. They all enjoyed their classes at BUTE, with one student describing a BUTE professor as the “best professor” he’s ever had – enthusiastic, knowledgeable, and helpful. For more on studying in Budapest, read the eceblogger posts by Carol Perkins, one of the five alumni of the program.

The juniors received instructions on administrative steps to take in order to participate in the program. These instructions were assembled by Kathy Reynolds and Caitlin Baldwin – thanks Kathy and Caitlin! Also thanks to József Porohnavec, a BUTE student spending a semester at UNH, for participating in the session.

As Kathy said in a follow-up email to the eight juniors: we can’t wait to hear their stories next fall when it is their turn to meet with the next group of students going to Budapest.

World Learning exchange students at UNH ECE

For the second year in a row the UNH ECE department has been selected by World Learning (WL) as a host institution for international exchange students participating in WL programs. WL is a non-profit organization that, according to its website, “provides education, exchange, and development programs that cultivate the global leadership and social innovation needed in a shrinking world.”

During the 2010-2011 academic year we hosted Angelica Sanabria from Honduras. Angelica participated in the WL UGRAD program. The picture to the left shows Angelica with her UGRAD Post image – UGRAD students use this image as a way to report on their travels around the US. Exploring the US, including by travel, is an integral part of the exchange program. During her time at UNH Angelica impressed me greatly as a smart and capable person. You can read more about her experiences at UNH on eceblogger.

During the 2011-2012 academic year, UNH ECE is hosting two students, smiling in the picture to the left. Huy Dong (left) has completed his freshman year at Vietnam National University, Ho Chi Minh City. Ivan Razumenic is a senior at the University of Belgrade, Serbia. Btw, Ivan is a blogger – take a look at his view of the WL FORECAST program and life in general on his blog.

I’m thrilled that WL has selected UNH as a host institution, and I’m really looking forward to working with both of our exchange students in the coming months. I am also greatful to the UNH CEPS Dean’s Office for their support in bringing in the exchange students.

2011 Cognitive Load and In-Vehicle Human-Machine Interaction workshop

I’m thrilled to announce the 2011 Cognitive Load and In-Vehicle Human-Machine Interaction workshop (CLW 2011) to be held at AutomotiveUI 2011 in Salzburg, Austria. I’m co-organizing the workshop with Peter Heeman, Tim Paek, Tom Miller, Paul Green, Ivan Tashev, Peter Froehlich, Bryan Reimer, Shamsi Iqbal and Dagmar Kern. 

Why have this workshop? Interactions with in-vehicle electronic devices can interfere with the primary task of driving. The concept of cognitive load helps us understand the extent to which these interactions interfere with the driving task and how this interference can be mitigated. While research results on in-vehicle cognitive load are frequently presented at automotive research conferences and in related journals, so far no dedicated forum is available for focused discussions on this topic. This workshop aims to fill that void.

Submissions to the workshop are due October 17. Topics of interest include, but are not limited to:

– Cognitive load estimation in the laboratory,
– Cognitive load estimation on the road,
– Sensing technologies for cognitive load estimation,
– Algorithms for cognitive load estimation,
– Performance measures of cognitive load,
– Physiological measures of cognitive load,
– Visual measures of cognitive load,
– Subjective measures of cognitive load,
– Methods for benchmarking cognitive load,
– Cognitive load of driving,
– Cognitive overload and cognitive underload,
– Approaches to cognitive load management inspired by human-human interactions.

For a detailed description of workshop goals take a look at the call for papers.