Josh and Shawn created a tangible user interface for the Microsoft Surface multitouch table.  Their interface allows users to play a game of air hockey on the Surface. Josh, a computer engineering senior, was in charge of creating the Arduino-based game controller. Shawn, a computer science senior, created the game on the Surface.

Here is a video introducing the work of Josh and Shawn:

Congratulations Josh and Shawn!

The Project54 lab was created in 1999 in partnership with the New Hampshire Department of Safety to improve technology for New Hampshire law enforcement. Project54’s in-car system integrates electronic devices in police cruisers into a single voice-activated system. Project54 also integrates cruisers into agency-wide communication networks. The Project54 system has been deployed in over 1000 vehicles in New Hampshire in over 180 state and local law enforcement agencies.

Research focus

Both the PhD and the MS student will focus on the relationship between various in-car user interface characteristics and the cognitive load of interacting with these interfaces, with the goal of designing interfaces that do not significantly increase driver workload. Work will involve developing techniques to estimate cognitive load using performance measures (such as the variance of lane position), physiological measures (such as changes in pupil diameter [1-5]) and subjective measures (such as the NASA-TLX questionnaire).

The PhD student will focus on spoken in-vehicle human-computer interaction, and will explore the use of human-human dialogue behavior [6-11] to guide the design process.

The work will utilize experiments in Project54’s world-class driving simulator laboratory which is equipped with two research driving simulators, three eye trackers and a physiological data logger.

Appointment

The PhD student will be appointed for four years, and the MS student for two years. Initial appointments will be for one year, starting between June and September 2012. Continuation of funding will be dependent on satisfactory performance. Appointments will be a combination of research and teaching assistantships. Compensation will include tuition, fees, health insurance and academic year and summer stipend.

How to apply

For application instructions, and for general information, email Andrew Kun, Project54 Principal Investigator at andrew.kun@unh.edu. Please attach a current CV.

References

[1] Oskar Palinko, Andrew L. Kun, “Exploring the Effects of Visual Cognitive Load and Illumination on Pupil Diameter in Driving Simulators,” ETRA 2012

[2] Andrew L. Kun, Zeljko Medenica, Oskar Palinko, Peter A. Heeman, “Utilizing Pupil Diameter to Estimate Cognitive Load Changes During Human Dialogue: A Preliminary Study,” AutomotiveUI 2011 Adjunct Proceedings

[3] Andrew L. Kun, Peter A. Heeman, Tim Paek, W. Thomas Miller, III, Paul A. Green, Ivan Tashev, Peter Froehlich, Bryan Reimer, Shamsi Iqbal, Dagmar Kern, “Cognitive Load and In-Vehicle Human-Machine Interaction,” AutomotiveUI 2011 Adjunct Proceedings

[4] Oskar Palinko, Andrew L. Kun, “Exploring the Influence of Light and Cognitive Load on Pupil Diameter in Driving Simulator Studies,” Driving Assessment 2011

[5] Oskar Palinko, Andrew L. Kun, Alexander Shyrokov, Peter Heeman, “Estimating Cognitive Load Using Remote Eye Tracking in a Driving Simulator,” ETRA 2010

[6] Andrew L. Kun, Alexander Shyrokov, and Peter A. Heeman, “Interactions between Human-Human Multi-Threaded Dialogues and Driving,” PUC Online First, to appear in PUC

[7] Andrew L. Kun, Zeljko Medenica, “Video Call, or Not, that is the Question,” to appear in CHI ’12 Extended Abstracts

[8] Fan Yang, Peter A. Heeman, Andrew L. Kun, “An Investigation of Interruptions and Resumptions in Multi-Tasking Dialogues,” Computational Linguistics, 37, 1

[9] Andrew L. Kun, Alexander Shyrokov, Peter A. Heeman, “Spoken Tasks for Human-Human Experiments: Towards In-Car Speech User Interfaces for Multi-Threaded Dialogue,” Automotive UI 2010

[10] Fan Yang, Peter A. Heeman, Andrew L. Kun, “Switching to Real-Time Tasks in Multi-Tasking Dialogue,” Coling 2008

[11] Alexander Shyrokov, Andrew L. Kun, Peter Heeman, “Experimental modeling of human-human multi-threaded dialogues in the presence of a manual-visual task,” SigDial 2007

1. Augmented reality (AR) on the cheap. Work in our lab has shown that in-vehicle AR navigation aids can effectively guide drivers, that they do not distract, and that drivers like them [1] – see video below. However, they’re expensive to make. In this senior project students will build a device that will augment the speech output of a personal navigation device/app (e.g. Google Navigation) with LED displays indicating upcoming turns. The device will be tested in driving simulator experiments.

2. Instrumented steering wheel. Today’s vehicles have myriad buttons, many on the steering wheel [2]. This project will build on our work with a push-to-talk glove [3, 4] to explore how drivers could interact with in-vehicle devices by tapping the steering wheel. Additionally, sensors on the steering wheel will produce feedback about the driver’s state (e.g. a stressed driver might squeeze the steering wheel much harder than a relaxed driver). Multiple driving simulator experiments will validate the design of the instrumented steering wheel.
3. Video call. Work in our lab has shown that video calling can be a real distraction from driving [5] – see video below. This project will explore how different topics of conversation (e.g. playing word games vs. arguing), different relationships between conversants (e.g. friends vs. strangers), and different driving conditions (e.g. city vs. highway) influence driver’s ability to operate a simulated vehicle while video calling.
4. Tangible user interfaces that support exploring large, time-sequence data sets. The Environmental Response Management Application (ERMA) is a web-based data visualization application. It visualizes geo-coded time series, without requiring users to know how to access specialized databases, or overlay data from these databases on virtual maps. ERMA was developed at UNH, under the guidance of the Coastal Response Research Center (CRRC). Nancy Kinner is the co-director of the UNH Coastal Response Research Center. Building on Nancy’s experiences with ERMA, she and I are interested in exploring how a tangible user interfaces utilizing a multi-touch table could be used to access and manipulate geo-coded time series. In this project students will develop a user interface on a multi-touch table. The interface will allow a human operator to access remote databases, manipulate the data (e.g. by sending it to Matlab for processing) and display the results on a virtual map or a graph.
5. Tangible user interfaces for children. How can we entertain and teach kids using technologies such as the Microsoft Surface and tangible interfaces? Students working on this project would seek opportunities to collaborate with other researchers on the UNH campus to further explore this question.
6. Your ideas related to user interfaces in vehicles and on multi-touch tables. Do you have an idea you’d like to explore? Tell me more about it!

References

[1] Zeljko Medenica, Andrew L. Kun, Tim Paek, Oskar Palinko, “Augmented Reality vs. Street Views: A Driving Simulator Study Comparing Two Emerging Navigation Aids,” MobileHCI 2011

[2] Dagmar Kern, Albrecht Schmidt, “Design Space for Driver-based Automotive User Interfaces,” AutomotiveUI 2009

[3] Oskar Palinko, Andrew L. Kun, “Prototype Wireless Push-to-Talk Glove,” IET 2008

[4] Oskar Palinko, Andrew L. Kun, “Comparison of the Effects of Two Push-to-Talk Button Implementations on Driver Hand Position and Visual Attention,” Driving Assessment 2009

[5] Andrew L. Kun, Zeljko Medenica, “Video Call, or Not, that is the Question,” to appear in CHI ’12 Extended Abstracts

Students and technical professionals can quickly gain an introduction to a variety of topics in pervasive/ubiquitous computing by watching online research videos. Of course, watching videos is no replacement for reading papers (yes, starting with Weiser’s) and using an excellent textbook, such as John Krumm‘s ubicomp fundamentals. Yet, videos can quickly convey many basic ubicomp ideas in an engaging manner.

Good places to look for videos are the usual suspects: Vimeo and YouTube. Additionally, many conferences now allow uploading videos as support material for papers, and conferences such as Ubicomp and Pervasive have tracks dedicated to video submissions (see e.g. Ubicomp 2012 and Pervasive 2012).

In my 2010 intro ubicomp course I used the following videos, grouped by general topic area:

Field studies:

1. Project54 police radio study (YouTube)
2. Tagging photographs using voice commands (YouTube)
3. Linking with Flickr using voice commands (YouTube)
4. AmbiKraf (YouTube)
5. HealthLine: Information Access for Community Health Workers in Developing Regions (YouTube)
6. LINC, An Inkable Digital Family Calendar (YouTube)
7. Rexplorer – an urban interactive game for tourists (YouTube)

Privacy:

1. Sharing data on public displays: academic transcript example (YouTube)
2. Sharing data on public displays: family pictures example (YouTube)
3. Sharing data on public displays: map example (YouTube)
4. Multi-user interaction using handheld projectors (YouTube)
5. Empathy mirror (YouTube – removed) (Website)
6. How to hack RFID-enabled credit cards for $8 (YouTube)
7. IBM RFID commercial (YouTube)
8. Minority report – mall scene (YouTube)
9. RFID parking access control long range system (YouTube)
10. Sensecam: Cambridge day out (YouTube)
11. The Ambient Clock (YouTube)
12. The RFID Ecosystem Project (YouTube)

User interfaces:

1. Natural User Interfaces: Utilizing physiological data (YouTube)
2. Skinput: Appropriating the body as an input surface (YouTube)
3. 10GUI: 10 finger multitouch user interface (YouTube)
4. BumpTop 3D multi-touch desktop (YouTube)
5. Ford SYNC and Pandora (YouTube)
6. Google Maps navigation (Beta) (YouTube)
7. Microsoft Courier in action (YouTube)
8. Microsoft Research: Project Gustav (YouTube)
9. Microsoft future vision: productivity (YouTube)
10. Microsoft’s vision of the future (Parody) (YouTube)
11. Microsoft future vision: Windows home concept (YouTube)
12. Microsoft future vision: manufacturing (YouTube)
13. Microsoft future vision: personal health (YouTube)
14. Microsoft future vision: banking (YouTube)
15. Microsoft future vision: retail (YouTube)
16. NanoTouch (YouTube)
17. Reboard (YouTube)
18. The invisible train (YouTube)
19. Searchvox (website)

Of course this list is now 2 years old – time to update!

Click image for source

This summer I’ll be teaching ECE 900 Electrical and Computer Engineering Seminar. The seminar introduces graduate students to the general tools of research. Students gain practical experience with framing research questions, setting goals, and proposing hypotheses. We also discuss ideas on how to read, write and review research publications, and on how to give oral presentations about such documents to different types of audiences. Finally, we explore best practices for success in graduate school.

A key outcome of the seminar is a research proposal. Proposals address the steps required to complete the research requirement of students’ graduate degrees (MS or PhD).

Interested? Please take a look at the facts below and/or send me email with questions and suggestions.

Who is this course for?
While ECE 900 is a requirement for UNH ECE graduate students, the course is open to graduate students from all UNH departments. I certainly hope non-ECE students will join us this summer.

Will the course be online?
Yes. Students resident at UNH can participate in person, remote students can participate online (no in-class meetings required).

How many credits?
During the academic year ECE 900 is offered both in the fall and spring. Material covered in the spring builds on that covered in the fall. Similarly, the summer course will run in two consecutive sections, each four weeks long. Section 1 will cover material covered in ECE 900 in the fall, while section 2 will cover the spring material. Students will earn 2 credits for completing each section, for a total of 4 credits for the summer.

What are the prerequisites?
Graduate standing.

Is there online material from previous years?
Yes, e.g. fall 2010 and spring 2011.

Schedule
ECE 900, Section 1 (fall material): June 11-July 6, Tuesdays and Thursdays 12-2 PM
ECE 900, Section 2 (spring material): July 9 – August 3, Tuesdays and Thursdays 12-2 PM

Under the exchange program five UNH ECE students spent the spring 2011 semester in Budapest. All five (picture above) were present at the information session to share their experiences with the eight juniors interested in the program (picture below).

The experiences we heard about were awesome, in fact life-changing. All five alumni of the exchange program agreed that spending a semester in Budapest was an excellent decision, with some calling it their “best decision.” The program was challenging, but that was one of its most important aspects because of the skills and confidence it built in each and every one of them. They all enjoyed their classes at BUTE, with one student describing a BUTE professor as the “best professor” he’s ever had – enthusiastic, knowledgeable, and helpful. For more on studying in Budapest, read the eceblogger posts by Carol Perkins, one of the five alumni of the program.

The juniors received instructions on administrative steps to take in order to participate in the program. These instructions were assembled by Kathy Reynolds and Caitlin Baldwin – thanks Kathy and Caitlin! Also thanks to József Porohnavec, a BUTE student spending a semester at UNH, for participating in the session.

As Kathy said in a follow-up email to the eight juniors: we can’t wait to hear their stories next fall when it is their turn to meet with the next group of students going to Budapest.

For the second year in a row the UNH ECE department has been selected by World Learning (WL) as a host institution for international exchange students participating in WL programs. WL is a non-profit organization that, according to its website, “provides education, exchange, and development programs that cultivate the global leadership and social innovation needed in a shrinking world.”

During the 2010-2011 academic year we hosted Angelica Sanabria from Honduras. Angelica participated in the WL UGRAD program. The picture to the left shows Angelica with her UGRAD Post image – UGRAD students use this image as a way to report on their travels around the US. Exploring the US, including by travel, is an integral part of the exchange program. During her time at UNH Angelica impressed me greatly as a smart and capable person. You can read more about her experiences at UNH on eceblogger.

During the 2011-2012 academic year, UNH ECE is hosting two students, smiling in the picture to the left. Huy Dong (left) has completed his freshman year at Vietnam National University, Ho Chi Minh City. Ivan Razumenic is a senior at the University of Belgrade, Serbia. Btw, Ivan is a blogger – take a look at his view of the WL FORECAST program and life in general on his blog.

I’m thrilled that WL has selected UNH as a host institution, and I’m really looking forward to working with both of our exchange students in the coming months. I am also greatful to the UNH CEPS Dean’s Office for their support in bringing in the exchange students.

Zeljko’s PhD committee includes Paul Green (UMTRI), Tim Paek (Microsoft Research), Nicholas Kirsch (UNH) and Tom Miller (UNH). Thanks to all for serving!

References

[1] Andrew L. Kun, Tim Paek, Zeljko Medenica, Nemanja Memarovic, Oskar Palinko, “Glancing at Personal Navigation Devices Can Affect Driving: Experimental Results and Design Implications,” Automotive UI 2009

[2] Zeljko Medenica, Andrew L. Kun, Tim Paek, Oskar Palinko, “Augmented Reality vs. Street Views: A Driving Simulator Study Comparing Two Emerging Navigation Aids,” to appear at MobileHCI 2011

Pervasive computing problems and ideas are often introduced using videos. An excellent exampe is the Microsoft Health Future Vision video (download, watch on YouTube). 

Let’s focus on three themes from the video that are relevant to the senior projects: interactions with multi-touch interfaces, interactions with tangible user interfaces, and data manipulation/fusion. Multi-touch surfaces appear throughout the video: in Sabine’s home, in the doctor’s office, and in the hospital lobby. Several of the multi-touch interfaces, such as Sabine’s remote control, and her virtual wallet (used in the lobby), are tangible interfaces. Finally, Dr. Kemp manipulates/fuses data when interacting with Alex (patient in bed) and especially during the meeting with Sabine and Wei Yu.

The two senior projects will leverage Project54′s Microsoft Surface multi-touch table. Here is a video by UNH ECE graduate student Tim April introducing some of the interactions he has explored with the Surface.

With all this in mind, here are the specifics on the two proposed projects.

Project 1: Mobile data fusion

This project will explore fusing data, such as images, video, sounds and outputs from various sensors (temperature, pressure, accelerometers, etc.). Data fusion will require some simple processing, such as setting beginning and end points for sounds, zooming images, etc. It will also require more complex digital signal processing of data, e.g. windowing and filtering (topics covered in ECE 714). Consequently, work on this project will focus on data processing as well as object-oriented programming that is necessary to control the multi-touch table.

This project will be tied to a collaborative effort with WPI on creating a fleet of UAVs. Thus, eventually, the data to process and display on the multi-touch will come from the UAVs. However, as a first step, data will be generated through games, similarly to work done by Jatin Matani and Trupti Telang.

Project 2: IR wallet

The Microsoft Surface uses infrared illumination and cameras to recognize interactions with its surface. It can also recognize 2D barcodes if they are visible in the IR part of the spectrum. The “IR wallet” project would result in a tangible user interface, similar to Sabine’s virtual wallet, that can display 2D barcodes in IR. These in turn will be picked up by the Microsoft Surface. Work on this project will focus on microcontroller-based design (e.g. with an Arduino board) and object-oriented programming for the Surface.

Interested? Have questions, ideas, suggestions? Email me.

Bryan spent time in the Project54 lab discussing various aspects of driving simulator and field studies. He then gave a thought-provoking talk reviewing results from multiple studies exploring driver workload and distraction. I expecially enjoyed his discussion of physiological measures that can be used to estimate workload. E.g. Bryan has found that heart rate is a robust estimate of workload and is often more useful than the often-used measure of heart rate variability. Bryan also discussed work on validating driving simulator results through field studies. His data indicate that driving simulator results can be used to predict relative changes in workload measures under different situations in real-life driving. However, the actual values of the measures collected in simulator and field studies often differ significantly.

For more pictures visit Flickr.