Smart Surfaces and the Future of Automotive Human Machine Interface (HMI)

We are truly in a golden age of Automotive. Technological innovations over the past fifteen years have created modern automobiles and have been key to not only improved reliability, but also improved safety that has reduced the death rate by approximately 25% per million miles driven.  And yet, despite these advances, distracted driving is the primary contributing factor in up to 58% of automobile accidents. 

Rethinking the Automotive Cabin
Beginning in the 1980s and 1990s, the automotive cabin morphed from utilitarian design of the 1950s into something more akin to that of an airplane cockpit.  Gone were the days where the only electronic device available to the driver was the radio; electronic door locks, power windows, power mirrors, CD changers, air-conditioning, cruise control, variable-timed wind-shield wipers, and even heated seat controls became standard, each getting its own set of buttons. 

Fast forward to today -- with Bluetooth, mapping software, large displays showing full Internet connectivity -- and in-auto functionality has grown beyond just the ability the road ahead. As a result, the auto industry is rethinking the Human Machine Interface (“HMI”). Newer designs are sleek and minimalist, aesthetically pleasing, and safe to operate while the auto is in motion.

The Smart Surface Within the Automobile
Automotive manufacturers initially took design influence from modern smartphones, creating touch displays with proprietary operating systems and various controls compartmentalized into menus and sub-menus. But these early adoptions failed for two main reasons:

• First, menus were difficult to navigate. Unlike a smartphone that has your undivided attention, the lack of a physical button with intent-driven tactile response frustrated drivers trying to multi-task.
  
  
• Second, existing display technology, while good enough for smart phones, was not adequate for the automobile. Resistive touch displays were washed out and difficult to read in direct sunlight and performance significantly degraded over the years.  Likewise, drivers found capacitive touch unaccommodating. In cold weather, the driver could not operate the HVAC without taking their gloves off. Environmental conditions such as moisture and EMI significantly degraded the user experience; technology in the automobile must remain responsive in any environment to prevent distracted driving and avoid hazards.

In response, industrial designers designed concept vehicles and renderings that re-imagined the dashboard. Stand-alone buttons had functionalities that were hidden behind luxury surfaces. Operations were available upon demand, but operation indicators disappeared when no longer needed. Designers imagined smooth gapless wood and carbon fiber panels that would reveal only relevant controls. These new interiors were beautiful and practical.

Aside from design update, by removing traditional buttons, designers eliminated the number one warranty expense in the modern automobile: mechanical button failure. Under even ideal conditions, a typical mechanical switch would fail after a few hundred thousand actuations. Other drawbacks to button controls included ingress from oil, dirt, and abuse.

Strain gauges were used in concept vehicles to demonstrate this new vision gapless surface. However, this option did not offer the sensitivity, repeatability, or reliability needed to move from concept to mass production. The vision existed, but enabling technology did not.

MEMS-based Sensing and NextInput
NextInput, Inc., a leader in MEMS-based sensing, is recognized within the automotive industry for its force sensing solutions. No longer are industrial designers forced to make compromises or design trade-off; NextInput’s patented “touch” and “touchless” sensing technologies are widely regarded as the only viable solution presently available and sensitive enough to detect intent though any material, while remaining robust enough to achieve AEC-Q100 qualification.

One innovative electric vehicle manufacturer has already passed into mass production with the NextInput solution, replacing traditional steering wheel button controls with NextInput solid state ingress-free solution. In fact, seven automotive OEMs have already designed-in NextInput solutions for a wide range of smart surface applications, from steering wheels to 3D-touch enabled displays and door panel controls. NextInput is now working to bring contextual awareness to the automotive cabin.

Contextual Awareness Goes into the Auto Cabin
Replace dashboard controls with a smart surface, and contextual awareness gives a driver quick and intuitive access. Like the smartphone, the modern automobile hosts a wide range of sensors. From inertial sensors, microphones, temperature sensors, GPS, infrared sensors, LiDAR systems, eye tracking, and now force sensing, information can be fused from dozens and dozens of sensors that detect, monitor, and even predict, driver behavior.

As an example, the car can detect when a passenger is listening to music on a smartphone. It can transfer the music to the car stereo and bring up audio controls on the smart surface. Likewise, on a very cold morning, the car can automatically turn on the HVAC and automatically display controls for the seat warmers. Now imagine hazardous road conditions: the music can be muted and a map could pop up showing an alternative route. Through Artificial Intelligence and Machine Learning, the automobile will learn not only how to keep passengers safe, but also predict what they may want next.

Safety Factors and Limitations of Voice
Some have predicted the Smart Surface as no longer necessary – with voice-assist being the best replacement for traditional button controls. The flawed thesis is that convenience of controlling the TV or the thermostat with one’s voice in the living room is directly applicable to the automobile cabin.  While it is merely a nuisance when voice-assist is misinterpreted in the safety of a home, it is a potential safety hazard while in motion in an auto.

• Linguistics have regional and educational differences that touch interfaces do not.
• Touch is the most intuitive of the human senses. Children learn to interact with their world through touch, long before they begin to speak.
• With voice, if the driver’s intent is not recognized the first time, or a child keeps changing the radio station from the backseat, the distraction inherently endangers both driver and those in close proximity.

 
Semi-Autonomous and Fully-Autonomous Driving
An additional benefit of the sensor-driven contextual awareness in the car cabin becomes evident with adoption of semi-autonomous driving technologies. The same sensors used to determine the driver’s intent can also be used to measure health and wellness of the driver, plus judge his/her incapacitation. 
For example, NextInput’s sensing solutions in the steering wheel and in the car seat can be used to determine whether the driver is alert and actively engaged or slumped over and not in control. If incapacitation is detected, the automobile can automatically take over, maneuver to safety and call for help; this versus instead of engaging the brakes because LiDAR detected an accident to be eminent.

Even as fully-autonomous driving becomes prevalent, the demand for smart surfaces will grow. As the cabin transforms into a productivity and entertainment space, consumers will demand an even higher degree of aesthetics as their interactions inside the cabin increase. The “touch and feel” experience of every interaction matters more now that focus is no longer external. The car cabin of the future will have everything the passenger desires all within an arm’s reach.

Conclusion
Over the next several years an entirely new automotive HMI will emerge as consumers fundamentally change the way they interact with automobile. Their interest in clean design styling will take influence from modern smart home environments. NextInput is on the forefront of this paradigm shift, having shipped over 10 million units of its production-ready force sensing solutions into automotive, mobile, and consumer applications.