Nokia : Multi-modality & voice UI project

Case Study

Nokia hired Eureka Time’s software experts to work on a R&D project to research the creation of context-aware phones. This multi-modality research included building text and speech capabilities.

We worked on multiple tasks to make the phones more context-aware, using C++, the QT framework and native APIs. We added Text-to-Speech (TTS) to the phones, developing a voice user interface that responded to audio prompts from the user.

 

Nokia-Multi-modality IT-Technology-Development-DSI-Mobile-Eureka-Time-Phone

 

In addition to TTS we also developed STT (Speech-to-Text) functionality, allowing the phone to capture and respond to speech input from the user. The STT first employed local ASR to try to determine what was said. If the local ASR was unsuccessful, a second remote Speech-to-Text (remote STT) was employed and the audio stream was sent to a cloud server for analysis.

 

These TTS and STT functions were one part of a larger multi-modal framework project, the objective of which was to create context-aware Nokia phones. The work we delivered could identify various contexts in which the phone was being used (in other words, what the user was doing with the phone). To achieve this, we used all of the phone’s sensors and the phone’s camera. By using the camera, we could determine if the user was looking at the phone or not.

 

Using information from sensors, we could decide whether to employ the voice user interface (voice UI). For example, if the phone was in motion but the user wasn’t looking at the phone, the user might be driving, and the voice UI engaged together with haptic output.

 

The aim of developing context-aware phones was to make interacting with the phone easier and more intuitive.

 

Our team also worked with Nokia on various embedded applications of their phones on separate occasions, including writing the gallery application and the phonebook/contacts application.

 

 

Eureka Time is a digital agency with headquarters in Paris, France and Barcelona, Spain. We work with brands from all over the world to help them achieve their digital goals. Our teams of experts can help your company with:

  • Technological development (creating websites, mobile applications, platforms, etc.).
  • Digital campaigns, content, and media
  • Influence strategies and promotional programs (community management, public relations, etc.).

To learn more about how we can help you or your company with our services, please contact us.

Read more about Eureka Time’s technological development projects here.

Discover other case studies & awards