Voice Input

How it works

• Voice Input
• Voice Input For Microsoft Word
• Talk To Text App Free

• Please note this feature is in BETA (meaning it might not work ;)
• Enable the Voice Input in my extension Options > Voice Input and it will add a microphone button in your Gmail webpage at the bottom of a new compose message (or reply area)
• Click the microphone button
• A prompt at the top should appear asking to use the microphone, click Allow
• Then start talking (try slow and loud at first). Google's speech recognition service will translate your spoken words into text and compose your message without typing! (It's much faster)
• If the words are not correct you can hover over them and choose from among other suggestions!
• Problems? Check your microphone settings

Google Input Tools remembers your corrections and maintains a custom dictionary for new or uncommon words and names. Type the way you want Get your message across in. 1- To mix your voice with your music on Skype or Google Voice and make your live podcast or own radio program. 2- To use 2x USB Headset on a computer, for VOIP applications or online educational program. 3- To Play Video Game in 5.1 and keep the MIC for communication with your team. Voice input definition: the control and operation of computer systems by spoken commands Meaning, pronunciation, translations and examples.

Speech recognition stopped

• Note that this happens frequently. It's an issue with Google and hopefully they will fix it. It happens when the speech recognition service is interrupted due to the network or is just unable to listen
• This also happens if you click outside of the voice input area
• Just click Continue (or hit Enter) and resume speaking

Voice commands

(English only)

Saying this..	Does this..
Comma	Inserts ( , )
Period	Inserts ( . )
Colon	Inserts ( : )
Question mark	Inserts ( ? )
Exclamation mark	Inserts ( ! )
New line	Inserts a new line
New paragraph	Inserts a new paragraph
Erase	Erases the last phrase spoken
Erase word	Erases the last word
Erase all	Erases everything in the input area
Stop recording	Stops speech recognition
Send Email	Sends the email
Message (while in subject field)	Puts the focus into the message body

Retrieved from 'https://jasonsavard.com/mediawiki/index.php?title=Voice_input&oldid=2576'

-->

Voice is one of the key forms of input on HoloLens. It allows you to directly command a hologram without having to use hand gestures. Voice input can be a natural way to communicate your intent. Voice is especially good at traversing complex interfaces, because it lets users cut through nested menus with one command.

Voice input is powered by the same engine that supports speech in all other Universal Windows Apps.On HoloLens, speech recognition will always function in the Windows display language configured in Settings.

Voice and gaze

When using voice commands, (head or eye) gaze is typically used as the targeting mechanism, whether with a cursor ('select') or to implicitly channel your command to an application that you are looking at.For this, it may not even be required to show any gaze cursor ('see it, say it').Of course, some voice commands don't require a target at all, such as 'go to start' or 'Hey Cortana.'

Device support

Feature	HoloLens (1st gen)	HoloLens 2	Immersive headsets
Voice input	✔️	✔️	✔️ (with microphone)

The 'select' command

HoloLens (1st gen)

Even without specifically adding voice support to your app, your users can activate holograms simply by saying the system voice command 'select'.This behaves the same as an air tap on HoloLens, pressing the select button on the HoloLens clicker, or pressing the trigger on a Windows Mixed Reality motion controller.You will hear a sound and see a tooltip with 'select' appear as confirmation. 'Select' is enabled by a low power keyword detection algorithm so it is always available for you to say at any time with minimal battery life impact, even with your hands at your side.

Voice Input

HoloLens 2
In order to use the 'select' voice command in HoloLens 2, you first need to bring up the gaze cursor to use as a pointer. The command to bring it up is easy to remember -- just say, 'select'.
To exit the mode, simply use your hands again, either by air tapping, approaching a button with your fingers, or using the system gesture.
Image: Say 'select' to use the voice command for selection

Hey Cortana

You can also say 'Hey Cortana' to bring up Cortana at anytime. You don't have to wait for her to appear to continue asking her your question or giving her an instruction - for example, try saying 'Hey Cortana, what's the weather?' as a single sentence.For more information about Cortana and what you can do, simply ask her!Say 'Hey Cortana, what can I say?' and she'll pull up a list of working and suggested commands.If you're already in the Cortana app you can also click the ? icon on the sidebar to pull up this same menu.

HoloLens-specific commands

• 'What can I say?'
• 'Go to Start' - instead of bloom to get to Start Menu
• 'Launch '
• 'Move here'
• 'Take a picture'
• 'Start recording'
• 'Stop recording'
• 'Show hand ray'
• 'Hide hand ray'
• 'Increase the brightness'
• 'Decrease the brightness'
• 'Increase the volume'
• 'Decrease the volume'
• 'Mute' or 'Unmute'
• 'Shut down the device'
• 'Restart the device'
• 'Go to sleep'
• 'What time is it?'
• 'How much battery do I have left?'

'See It, Say It'

HoloLens has a 'see it, say it' model for voice input, where labels on buttons tell users what voice commands they can say as well. For example, when looking at an app window in HoloLens (1st gen), a user can say the 'Adjust' command which they see in the App bar to adjust the position of the app in the world.
Image: A user can say the 'Adjust' command which they see in the App bar to adjust the position of the app

When apps follow this rule, users can easily understand what to say to control the system. To reinforce this, while gazing at a button in HoloLens (1st gen), you will see a 'voice dwell' tooltip that comes up after a second if the button is voice-enabled and displays the command to speak to 'press' it. To reveal voice tooltips in HoloLens 2, show the voice cursor by saying 'select' or 'What can I say' (See image).
Image: 'See it, say it' commands appear below the buttons Tipard all music converter 9 1 16 pm.

Voice commands for fast hologram manipulation

There are a number of voice commands you can say while gazing at a hologram to quickly perform manipulation tasks.These voice commands work on app windows as well as 3D objects you have placed in the world.

Hologram manipulation commands

• Face me
• Bigger | Enhance
• Smaller

On HoloLens 2, you can also create more natural interactions in combination with eye-gaze which implicitly provides contextual information about what you are referring to.For example, you could simply look at a hologram and say 'put this' and then look over where you want to place it and say 'over here'.Or you could look at a holographic part on a complex machine and say: 'give me more information about this'.

Discovering voice commands

Some commands, like the commands for fast manipulation above, can be hidden. To learn about what commands you can use, gaze at an object and say, 'what can I say?'. A list of possible commands pops up.You can also use the head gaze cursor to look around and reveal the voice tooltips for each button in front of you.

Doctor 1 8 0 – convert your documents using. If you want a complete list, just say, 'Show all commands' anytime.

Dictation

Rather than typing with air taps, voice dictation can be more efficient to enter text into an app. This can greatly accelerate input with less effort for the user.

Voice dictation starts by selecting the microphone button on the keyboard

Any time the holographic keyboard is active, you can switch to dictation mode instead of typing. Select the microphone on the side of the text input box to get started.

Adding voice commands to your app

Consider adding voice commands to any experience that you build. Voice is a powerful and convenient way control the system and apps. Because users speak with a variety of dialects and accents, proper choice of speech keywords will make sure that your users' commands are interpreted unambiguously.

Best practices

Below are some practices that will aid in smooth speech recognition.

• Use concise commands - When possible, choose keywords of two or more syllables. One-syllable words tend to use different vowel sounds when spoken by persons of different accents. Example: 'Play video' is better than 'Play the currently selected video'
• Use simple vocabulary - Example: 'Show note' is better than 'Show placard'
• Make sure commands are non destructive - Make sure any action that can be taken by a speech command is non destructive and can easily be undone in case another person speaking near the user accidentally triggers a command.
• Avoid similar sounding commands - Avoid registering multiple speech commands that sound very similar. Example: 'Show more' and 'Show store' can be very similar sounding.
• Unregister your app when not it use - When your app is not in a state in which a particular speech command is valid, consider unregistering it so that other commands are not confused for that one.
• Test with different accents - Test your app with users of different accents.
• Maintain voice command consistency - If 'Go back' goes to the previous page, maintain this behavior in your applications.
• Avoid using system commands - The following voice commands are reserved for the system. These should not be used by applications.
  • 'Hey Cortana'
  • 'Select'
  • 'Go to start'

Advantages of voice input

Voice input is a natural way to communicate our intents.Voice is especially good at interface traversals because it can help users cut through multiple steps of an interface (a user might say 'go back' while looking at a webpage, instead of having to go up and hit the back button in the app).This small time saving has a powerful emotional effect on user's perception of the experience and gives them a small amount superpower.Using voice is also a convenient input method when we have our arms full or are multi-tasking.On devices where typing on a keyboard is difficult, voice dictation can be an efficient alternative way to input text.Lastly, in some cases when the range of accuracy for gaze and gesture are limited, voice can help to disambiguate the user's intent.

How using voice can benefit the user

• Reduces time - it should make the end goal more efficient.
• Minimizes effort - it should make tasks more fluid and effortless.
• Reduces cognitive load - it's intuitive, easy to learn, and remember.
• It's socially acceptable - it should fit in with societal norms in terms of behavior.
• It's routine - voice can readily become a habitual behavior.

Challenges for voice input

While voice input is great for a lot of different applications, it also faces several challenges.Understanding both the advantages and challenges for voice input enables app developers to make smarter choices for how and when to use voice input and to create a great experience for their users.

Voice input for continuous input controlFine-grained control is one of them.For example, a user might want to change their volume in their music app.She can simply say 'louder', but it's not clear how much louder the system is supposed to make the volume.The user could say: 'Make it a little louder', but 'a little' is difficult to quantify.Moving or scaling holograms with voice is similarly difficult.

Reliability of voice input detectionWhile voice input systems become better and better, sometimes they may incorrectly hear and interpret a voice command.The key is to address this challenge in your application by providing feedback to the user when the system is listening and what the system understood to create clarity on potential issues in correctly understanding the user.

Voice input in shared spacesVoice may not be socially acceptable in spaces that you share with others.Here are a few examples:

• The user may not want to disturb others (e.g., in a quiet library or shared office)
• Users may feel awkward being seen talking to themselves in public,
• A user may feel uncomfortable dictating a personal or confidential message (including passwords) while others are listening

Voice input of unique or unknown wordsDifficulties for voice input also come when users are dictating words that may be unknown to the system, such as nicknames, certain slang words or abbreviations.

Learning voice commandsWhile the ultimate goal is to naturally converse with your system, often times apps still rely on specific pre-defined voice commands.A challenge associated with a big set of voice commands is how to teach them without overloading the user and how to help the user to retain them.

Voice feedback states

When Voice is applied properly, the user understands what they can say and get clear feedback the system heard them correctly. These two signals make the user feel confident in using Voice as a primary input. Below is a diagram showing what happens to the cursor when voice input is recognized and how it communicates that to the user.

1. Regular cursor state

2. Communicates voice feedback and then disappears

Top things users should know about 'speech' in mixed reality

• Say 'Select' while targeting a button (you can use this anywhere to click a button).
• You can say the label name of an app bar button in some apps to take an action. For example, while looking at an app, a user can say the command 'Remove' to remove the app from the world (this saves time from having to click it with your hand).
• You can initiate Cortana listening by saying 'Hey Cortana.' You can ask her questions ('Hey Cortana, how tall is the Eiffel tower'), tell her to open an app ('Hey Cortana, open Netflix'), or tell her to bring up the Start Menu ('Hey Cortana, take me home') and more.

Common questions and concerns users have about voice

• What can I say?
• How do I know the system heard me correctly?
  • The system keeps getting my voice commands wrong.
  • It doesn't react when I give it a voice command.
• It reacts the wrong way when I give it a voice command.
• How do I target my voice to a specific app or app command?
• Can I use voice to command things out the holographic frame on HoloLens?

Communication

For applications that want to take advantage of the customized audio input processing options provided by HoloLens, it is important to understand the various audio stream categories your app can consume. Windows 10 supports several different stream categories and HoloLens makes use of three of these to enable custom processing to optimize the microphone audio quality tailored for speech, communication and other which can be used for ambient environment audio capture (i.e. 'camcorder') scenarios.

• The AudioCategory_Communications stream category is customized for call quality and narration scenarios and provides the client with a 16kHz 24bit mono audio stream of the user's voice
• The AudioCategory_Speech stream category is customized for the HoloLens (Windows) speech engine and provides it with a 16kHz 24bit mono stream of the user's voice. This category can be used by 3rd party speech engines if needed.
• The AudioCategory_Other stream category is customized for ambient environment audio recording and provides the client with a 48kHz 24 bit stereo audio stream.

All this audio processing is hardware accelerated which means the features drain a lot less power than if the same processing was done on the HoloLens CPU. Avoid running other audio input processing on the CPU to maximize system battery life and take advantage of the built in, offloaded audio input processing.

Languages

HoloLens 2 supports multiple languages. Keep in mind that speech commands will always run in the system's display language even if multiple keyboards are installed or if apps attempt to create a speech recognizer in a different language.

Troubleshooting

If you're having any issues using 'select' and 'Hey Cortana', try moving to a quieter space, turning away from the source of noise, or by speaking louder. At this time, all speech recognition on HoloLens is tuned and optimized specifically to native speakers of United States English.

For the Windows Mixed Reality Developer Edition release 2017, the audio endpoint management logic will work fine (forever) after logging out and back in to the PC desktop after the initial HMD connection. Prior to that first sign out/in event after going through WMR OOBE, the user could experience various audio functionality issues ranging from no audio to no audio switching depending on how the system was set up prior to connecting the HMD for the first time.

Voice input in MRTK (Mixed Reality Toolkit) for Unity

Voice Input For Microsoft Word

With MRTK, you can easily assign voice command on any objects. Use MRTK's Speech Input Profile to define your keywords. By assigning SpeechInputHandler script, you can make any object respond to the keywords defined in the Speech Input Profile. SpeechInputHandler also provides speech confirmation label to improve the user's confidence.

Talk To Text App Free

See also