Josh and Shawn created a tangible user interface for the Microsoft Surface multitouch table.  Their interface allows users to play a game of air hockey on the Surface. Josh, a computer engineering senior, was in charge of creating the Arduino-based game controller. Shawn, a computer science senior, created the game on the Surface.

Here is a video introducing the work of Josh and Shawn:

Congratulations Josh and Shawn!

The Project54 lab was created in 1999 in partnership with the New Hampshire Department of Safety to improve technology for New Hampshire law enforcement. Project54’s in-car system integrates electronic devices in police cruisers into a single voice-activated system. Project54 also integrates cruisers into agency-wide communication networks. The Project54 system has been deployed in over 1000 vehicles in New Hampshire in over 180 state and local law enforcement agencies.

Research focus

Both the PhD and the MS student will focus on the relationship between various in-car user interface characteristics and the cognitive load of interacting with these interfaces, with the goal of designing interfaces that do not significantly increase driver workload. Work will involve developing techniques to estimate cognitive load using performance measures (such as the variance of lane position), physiological measures (such as changes in pupil diameter [1-5]) and subjective measures (such as the NASA-TLX questionnaire).

The PhD student will focus on spoken in-vehicle human-computer interaction, and will explore the use of human-human dialogue behavior [6-11] to guide the design process.

The work will utilize experiments in Project54’s world-class driving simulator laboratory which is equipped with two research driving simulators, three eye trackers and a physiological data logger.

Appointment

The PhD student will be appointed for four years, and the MS student for two years. Initial appointments will be for one year, starting between June and September 2012. Continuation of funding will be dependent on satisfactory performance. Appointments will be a combination of research and teaching assistantships. Compensation will include tuition, fees, health insurance and academic year and summer stipend.

How to apply

For application instructions, and for general information, email Andrew Kun, Project54 Principal Investigator at andrew.kun@unh.edu. Please attach a current CV.

References

[1] Oskar Palinko, Andrew L. Kun, “Exploring the Effects of Visual Cognitive Load and Illumination on Pupil Diameter in Driving Simulators,” ETRA 2012

[2] Andrew L. Kun, Zeljko Medenica, Oskar Palinko, Peter A. Heeman, “Utilizing Pupil Diameter to Estimate Cognitive Load Changes During Human Dialogue: A Preliminary Study,” AutomotiveUI 2011 Adjunct Proceedings

[3] Andrew L. Kun, Peter A. Heeman, Tim Paek, W. Thomas Miller, III, Paul A. Green, Ivan Tashev, Peter Froehlich, Bryan Reimer, Shamsi Iqbal, Dagmar Kern, “Cognitive Load and In-Vehicle Human-Machine Interaction,” AutomotiveUI 2011 Adjunct Proceedings

[4] Oskar Palinko, Andrew L. Kun, “Exploring the Influence of Light and Cognitive Load on Pupil Diameter in Driving Simulator Studies,” Driving Assessment 2011

[5] Oskar Palinko, Andrew L. Kun, Alexander Shyrokov, Peter Heeman, “Estimating Cognitive Load Using Remote Eye Tracking in a Driving Simulator,” ETRA 2010

[6] Andrew L. Kun, Alexander Shyrokov, and Peter A. Heeman, “Interactions between Human-Human Multi-Threaded Dialogues and Driving,” PUC Online First, to appear in PUC

[7] Andrew L. Kun, Zeljko Medenica, “Video Call, or Not, that is the Question,” to appear in CHI ’12 Extended Abstracts

[8] Fan Yang, Peter A. Heeman, Andrew L. Kun, “An Investigation of Interruptions and Resumptions in Multi-Tasking Dialogues,” Computational Linguistics, 37, 1

[9] Andrew L. Kun, Alexander Shyrokov, Peter A. Heeman, “Spoken Tasks for Human-Human Experiments: Towards In-Car Speech User Interfaces for Multi-Threaded Dialogue,” Automotive UI 2010

[10] Fan Yang, Peter A. Heeman, Andrew L. Kun, “Switching to Real-Time Tasks in Multi-Tasking Dialogue,” Coling 2008

[11] Alexander Shyrokov, Andrew L. Kun, Peter Heeman, “Experimental modeling of human-human multi-threaded dialogues in the presence of a manual-visual task,” SigDial 2007

The theme issue includes our editorial [1], and three papers. The first is by Tuomo Kujala, who explores scrolling on touch screens  while driving [2]. The second is by Florian Schaub,
Markus Hipp, Frank Kargl, and Michael Weber, who address the issue of credibility in the context of automotive navigation systems [3]. The third paper is co-authored by me, my former PhD student Alex Shyrokov, and Peter Heeman. We explore multi-threaded spoken dialogues between a driver and a remote conversant [4]. The three papers were selected in a rigorous review process from 17 submissions, by approximately 50 reviewers.

References

[1] Andrew L. Kun, Albrecht Schmidt, Anind Dey and Susanne Boll, “Automotive User Interfaces and Interactive Applications in the Car,” PUC Online First

[2] Tuomo Kujala, “Browsing the Information Highway while Driving – Three In-Vehicle Touch Screen Scrolling Methods and Driver Distraction,” PUC Online First

[3] Florian Schaub, Markus Hipp,
Frank Kargl, and
Michael Weber, “On Credibility Improvements for Automotive Navigation Systems,” PUC Online First

[4] Andrew L. Kun, Alexander Shyrokov, and Peter A. Heeman, “Interactions between Human-Human Multi-Threaded Dialogues and Driving,” PUC Online First

1. Augmented reality (AR) on the cheap. Work in our lab has shown that in-vehicle AR navigation aids can effectively guide drivers, that they do not distract, and that drivers like them [1] – see video below. However, they’re expensive to make. In this senior project students will build a device that will augment the speech output of a personal navigation device/app (e.g. Google Navigation) with LED displays indicating upcoming turns. The device will be tested in driving simulator experiments.

2. Instrumented steering wheel. Today’s vehicles have myriad buttons, many on the steering wheel [2]. This project will build on our work with a push-to-talk glove [3, 4] to explore how drivers could interact with in-vehicle devices by tapping the steering wheel. Additionally, sensors on the steering wheel will produce feedback about the driver’s state (e.g. a stressed driver might squeeze the steering wheel much harder than a relaxed driver). Multiple driving simulator experiments will validate the design of the instrumented steering wheel.
3. Video call. Work in our lab has shown that video calling can be a real distraction from driving [5] – see video below. This project will explore how different topics of conversation (e.g. playing word games vs. arguing), different relationships between conversants (e.g. friends vs. strangers), and different driving conditions (e.g. city vs. highway) influence driver’s ability to operate a simulated vehicle while video calling.
4. Tangible user interfaces that support exploring large, time-sequence data sets. The Environmental Response Management Application (ERMA) is a web-based data visualization application. It visualizes geo-coded time series, without requiring users to know how to access specialized databases, or overlay data from these databases on virtual maps. ERMA was developed at UNH, under the guidance of the Coastal Response Research Center (CRRC). Nancy Kinner is the co-director of the UNH Coastal Response Research Center. Building on Nancy’s experiences with ERMA, she and I are interested in exploring how a tangible user interfaces utilizing a multi-touch table could be used to access and manipulate geo-coded time series. In this project students will develop a user interface on a multi-touch table. The interface will allow a human operator to access remote databases, manipulate the data (e.g. by sending it to Matlab for processing) and display the results on a virtual map or a graph.
5. Tangible user interfaces for children. How can we entertain and teach kids using technologies such as the Microsoft Surface and tangible interfaces? Students working on this project would seek opportunities to collaborate with other researchers on the UNH campus to further explore this question.
6. Your ideas related to user interfaces in vehicles and on multi-touch tables. Do you have an idea you’d like to explore? Tell me more about it!

References

[1] Zeljko Medenica, Andrew L. Kun, Tim Paek, Oskar Palinko, “Augmented Reality vs. Street Views: A Driving Simulator Study Comparing Two Emerging Navigation Aids,” MobileHCI 2011

[2] Dagmar Kern, Albrecht Schmidt, “Design Space for Driver-based Automotive User Interfaces,” AutomotiveUI 2009

[3] Oskar Palinko, Andrew L. Kun, “Prototype Wireless Push-to-Talk Glove,” IET 2008

[4] Oskar Palinko, Andrew L. Kun, “Comparison of the Effects of Two Push-to-Talk Button Implementations on Driver Hand Position and Visual Attention,” Driving Assessment 2009

[5] Andrew L. Kun, Zeljko Medenica, “Video Call, or Not, that is the Question,” to appear in CHI ’12 Extended Abstracts

Students and technical professionals can quickly gain an introduction to a variety of topics in pervasive/ubiquitous computing by watching online research videos. Of course, watching videos is no replacement for reading papers (yes, starting with Weiser’s) and using an excellent textbook, such as John Krumm‘s ubicomp fundamentals. Yet, videos can quickly convey many basic ubicomp ideas in an engaging manner.

Good places to look for videos are the usual suspects: Vimeo and YouTube. Additionally, many conferences now allow uploading videos as support material for papers, and conferences such as Ubicomp and Pervasive have tracks dedicated to video submissions (see e.g. Ubicomp 2012 and Pervasive 2012).

In my 2010 intro ubicomp course I used the following videos, grouped by general topic area:

Field studies:

1. Project54 police radio study (YouTube)
2. Tagging photographs using voice commands (YouTube)
3. Linking with Flickr using voice commands (YouTube)
4. AmbiKraf (YouTube)
5. HealthLine: Information Access for Community Health Workers in Developing Regions (YouTube)
6. LINC, An Inkable Digital Family Calendar (YouTube)
7. Rexplorer – an urban interactive game for tourists (YouTube)

Privacy:

1. Sharing data on public displays: academic transcript example (YouTube)
2. Sharing data on public displays: family pictures example (YouTube)
3. Sharing data on public displays: map example (YouTube)
4. Multi-user interaction using handheld projectors (YouTube)
5. Empathy mirror (YouTube – removed) (Website)
6. How to hack RFID-enabled credit cards for $8 (YouTube)
7. IBM RFID commercial (YouTube)
8. Minority report – mall scene (YouTube)
9. RFID parking access control long range system (YouTube)
10. Sensecam: Cambridge day out (YouTube)
11. The Ambient Clock (YouTube)
12. The RFID Ecosystem Project (YouTube)

User interfaces:

1. Natural User Interfaces: Utilizing physiological data (YouTube)
2. Skinput: Appropriating the body as an input surface (YouTube)
3. 10GUI: 10 finger multitouch user interface (YouTube)
4. BumpTop 3D multi-touch desktop (YouTube)
5. Ford SYNC and Pandora (YouTube)
6. Google Maps navigation (Beta) (YouTube)
7. Microsoft Courier in action (YouTube)
8. Microsoft Research: Project Gustav (YouTube)
9. Microsoft future vision: productivity (YouTube)
10. Microsoft’s vision of the future (Parody) (YouTube)
11. Microsoft future vision: Windows home concept (YouTube)
12. Microsoft future vision: manufacturing (YouTube)
13. Microsoft future vision: personal health (YouTube)
14. Microsoft future vision: banking (YouTube)
15. Microsoft future vision: retail (YouTube)
16. NanoTouch (YouTube)
17. Reboard (YouTube)
18. The invisible train (YouTube)
19. Searchvox (website)

Of course this list is now 2 years old – time to update!

Zeljko and I conducted a driving simulator experiment in which a driver and another participant were given the verbal task of playing the game of Taboo. The driver and the passenger were in separate rooms and spoke to each other over headsets. In one experimental condition, the driver and the other participant could also see each other as shown in the figure below. We wanted to find out if in this condition drivers would spend a significant amount of time looking at the other participant. This is an important question, as time spent looking at the other participant is time not spent looking at the road ahead!

We found that, when drivers felt that the driving task was demanding, they focused on the road ahead. However, when they perceived the driving task to be less demanding they looked at the other participant significantly more.

What this tells us is that, under certain circumstances, drivers are willing to engage in video calls. This is due, at least in part, to the (western) social norm of looking at the person you’re talking to. These results should serve as a warning to interface designers, lawmakers (yes, there’s concern [2]), transportation officials, and drivers that video calling can be a serious distraction from driving.

Here’s a video that introduces the experiment in more detail:

References

[1] Andrew L. Kun, Zeljko Medenica, “Video Call, or Not, that is the Question,” to appear in CHI ’12 Extended Abstracts

[2] Claude Brodesser-Akner, “State Assemblyman: Ban iPhone4 Video-Calling From the Road,” New York Magazine. Date accessed 03/02/2012

Click image for source

This summer I’ll be teaching ECE 900 Electrical and Computer Engineering Seminar. The seminar introduces graduate students to the general tools of research. Students gain practical experience with framing research questions, setting goals, and proposing hypotheses. We also discuss ideas on how to read, write and review research publications, and on how to give oral presentations about such documents to different types of audiences. Finally, we explore best practices for success in graduate school.

A key outcome of the seminar is a research proposal. Proposals address the steps required to complete the research requirement of students’ graduate degrees (MS or PhD).

Interested? Please take a look at the facts below and/or send me email with questions and suggestions.

Who is this course for?
While ECE 900 is a requirement for UNH ECE graduate students, the course is open to graduate students from all UNH departments. I certainly hope non-ECE students will join us this summer.

Will the course be online?
Yes. Students resident at UNH can participate in person, remote students can participate online (no in-class meetings required).

How many credits?
During the academic year ECE 900 is offered both in the fall and spring. Material covered in the spring builds on that covered in the fall. Similarly, the summer course will run in two consecutive sections, each four weeks long. Section 1 will cover material covered in ECE 900 in the fall, while section 2 will cover the spring material. Students will earn 2 credits for completing each section, for a total of 4 credits for the summer.

What are the prerequisites?
Graduate standing.

Is there online material from previous years?
Yes, e.g. fall 2010 and spring 2011.

Schedule
ECE 900, Section 1 (fall material): June 11-July 6, Tuesdays and Thursdays 12-2 PM
ECE 900, Section 2 (spring material): July 9 – August 3, Tuesdays and Thursdays 12-2 PM

However, in driving simulator studies participants scan a non-uniformly illuminated visual scene. If unaccounted for, this non-uniformity in illumination might introduce an error in our estimate of the TEPR. Oskar Palinko and I will have a paper at ETRA 2012 [2] extending our previous work [3], in which we established that it is possible to separate the pupil’s light reflex from the TEPR. While in our previous work TEPR was the result of participants’ engagement in an aural task, in our latest experiment TEPR is due to engagement in a visual task.

The two experiments taken together support our main hypothesis that it is possible to disambiguate (and not just separate) the two effects even in complicated environments, such as a driving simulator. We are currently designing further experiments to test this hypothesis.

References

[1] Jackson Beatty, “Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources,” Psychological Bulletin, 276-292, 91(2)

[2] Oskar Palinko, Andrew L. Kun, “Exploring the Effects of Visual Cognitive Load and Illumination on Pupil Diameter in Driving Simulators,” to appear at ETRA 2012

[3] Oskar Palinko, Andrew L. Kun, “Exploring the Influence of Light and Cognitive Load on Pupil Diameter in Driving Simulator Studies,” Driving Assessment 2011

How can such a massive response be managed? There is no single answer. However, one thing that can help is to make data about various aspects of the disaster, as well as the response effort, accessible to those conducting the response activities. This is the role of the Environmental Response Management Application (ERMA). ERMA is a web-based data visualization application. It visualizes geo-coded time series, without requiring users to know how to access specialized databases, or overlay data from these databases on virtual maps. ERMA was developed at UNH, under the guidance of the Coastal Response Research Center (CRRC).

Nancy Kinner is the co-director of the UNH Coastal Response Research Center. Building on Nancy’s experiences with ERMA, she and I are interested in exploring how a multi-touch table could be used to access and manipulate geo-coded time series.

Seeking UNH CS student

To further are effort, we are seeking a UNH CS student interested in developing a user interface on a multi-touch table. The interface would allow a human operator to access remote databases, manipulate the data (e.g. by sending it to Matlab for processing) and display the results on a virtual map or a graph. This work will be part of a team effort with two students working with Nancy on identifying data and manipulations of interest.

What should the user interface do?

The operator should be able to select data, e.g. from a website such as ERMA. Data types of interest include outputs from various sensors (temperature, pressure, accelerometers, etc.). Data manipulation will require some simple processing, such as setting beginning and end points for sensor readings. It will also require more complex processing of data, e.g. filtering.

What platform will be used?

The project will leverage Project54′s Microsoft Surface multi-touch table. Here is a video by UNH ECE graduate student Tim April introducing some of the interactions he has explored with the Surface.

What are the terms of this job?

We are interested in hiring an undergraduate or graduate UNH CS student for the 2011-2012 academic year, with the possibility of extending the appointment for the summer of 2012 and beyond, pending satisfactory performance and the availability of funding. The student will work up to 20 hours/week during the academic year and up to 40 hours a week during the summer break.

What are the required skills? And what new skills will I acquire?

Work on this ream-project will require object-oriented programming that is necessary to control the multi-touch table. You will explore the application of these skills to the design of surface user interfaces as well as experiments with human subjects – after all we will have to systematically test your creation! Finally, you will interact with students and faculty from at least two other disciplines (civil/environmental and electrical/computer engineering), which means you will gain valuable experience working on multi-disciplinary teams.

Interested? Have questions, ideas, suggestions?
Email me.

Under the exchange program five UNH ECE students spent the spring 2011 semester in Budapest. All five (picture above) were present at the information session to share their experiences with the eight juniors interested in the program (picture below).

The experiences we heard about were awesome, in fact life-changing. All five alumni of the exchange program agreed that spending a semester in Budapest was an excellent decision, with some calling it their “best decision.” The program was challenging, but that was one of its most important aspects because of the skills and confidence it built in each and every one of them. They all enjoyed their classes at BUTE, with one student describing a BUTE professor as the “best professor” he’s ever had – enthusiastic, knowledgeable, and helpful. For more on studying in Budapest, read the eceblogger posts by Carol Perkins, one of the five alumni of the program.

The juniors received instructions on administrative steps to take in order to participate in the program. These instructions were assembled by Kathy Reynolds and Caitlin Baldwin – thanks Kathy and Caitlin! Also thanks to József Porohnavec, a BUTE student spending a semester at UNH, for participating in the session.

As Kathy said in a follow-up email to the eight juniors: we can’t wait to hear their stories next fall when it is their turn to meet with the next group of students going to Budapest.