Unicorns Can Talk guide

Getting started

Install

The first start

1. Welcome. Two questions: start Unicorns Can Talk when you sign in, and start listening as soon as it opens. On a Mac it also asks for the microphone and for Accessibility (System Settings > Privacy & Security > Accessibility: switch on Unicorns Can Talk). Without Accessibility a Mac lets no program type into another, and the widget says so.
2. Model check. It measures the speech models on your computer, and if you read six short sentences aloud, how accurately each one hears you, then picks the most accurate one that is fast enough. You can skip it; the recommended model is used.
3. Download. The chosen model is downloaded once (about 660 MB for the recommended one). From then on everything works offline.

The unicorn widget appears near the edge of the screen, and a little unicorn in the system tray (Windows) or menu bar (Mac). Right-click either for the menu.

Dictating

Click in the program you want to write in, then click the widget (or say your wake phrase) and speak. Words are typed as you speak: the newest one or two are held back for a moment until the recogniser is sure of them. Click the widget again to pause. The widget never takes the cursor away from your document, so you can click it at any time.

• Idle: not listening
• Listening
• Paused
• Busy (proofreading, finishing a phrase)
• Loading the model
• Error (the widget says what)

Colours of the Standard skin; other skins show the same states

Drag the widget to move it; it remembers where. Right-click it for the menu: Dictation box, Transcribe a recording…, Settings…, About and Quit. Only one copy of Unicorns Can Talk runs at a time.

Speak in phrases, the way you would dictate to a person. Commands are recognised when you say them as their own phrase, with a short pause before and after, so “scratch that idea” in the middle of a sentence is typed as words.

Spoken punctuation

By default you say the punctuation (Settings > General > Punctuation: “Say punctuation aloud”). Capitals after a full stop, spaces, and no space before a comma are handled for you. To type one of these words as a word, say literal first: “literal comma”. The other choice, automatic punctuation, lets the recogniser punctuate for you.

All of these can be changed or added to in Settings > Words > Spoken punctuation.

Numbers, money and spelling

• Numbers are written as digits (“twenty five” → 25) or, below a number you choose, as words. Digits are always used before units (“5 percent”, “3 km”) and after words like section, clause or page, and you can add your own such words.
• Digits spoken one at a time (“nine eight two two…”) are joined into one number.
• Layout: Western (1,234,567), Indian (12,34,567) or no grouping, the separator and decimal mark, from how many digits to group, and whether years are left alone.
• Money: “rupees one lakh” becomes ₹ 1 lakh (with or without a space after the symbol); other currencies can be added.
• Spelling: British or American, with your own exceptions and pairs.
• Vocabulary: your own replacements for names and terms the recogniser gets wrong.

All in Settings > Numbers and Settings > Words.

Voice commands

Every phrase below can be changed, added to or switched off in Settings > Voice commands, and each shortcut's keys too. Keys are stored the Windows way and pressed the Mac way on a Mac (Ctrl becomes ⌘, Home becomes ⌘←, and so on), so one settings folder works on both.

Correcting

Moving the cursor

Dictation and the dictation box

Proofreading (Word on Windows)

Shortcuts (any program)

Word (Windows)

These act only when Microsoft Word is the program in front; anywhere else the words are simply typed.

Your own commands and text blocks

In Settings > Voice commands you can add a command that types a snippet, presses keys (one combination after another, like Alt+N, D) or does both. Settings > Text blocks holds longer texts, such as a standard letter or a signature, of any length (pasted in one go), each with the phrases that insert it and how it joins the text around it: after a space, as a new paragraph, or exactly where the cursor is. “Scratch that” takes a block back out.

Wake words

Switch on Wake words in Settings > General to start dictating hands-free. While dictation is off the microphone stays open only to hear your phrases (“start dictation”, “hey dictation”, or your own); nothing is typed or kept. Say one on its own, with a pause either side. “Stop dictation” goes back to waiting. The widget shows a dim ring while it waits.

The dictation box

Some programs and web pages do not cope well with typed dictation. Say “open a dictation box” (or choose it from the menu): a small window takes your dictation instead, with every editing command working inside it. Its Paste button, or “paste the dictation box”, pastes the text into the program in front, then clears the box if you like. It never takes the cursor from your program.

Transcribing a recording

Menu > Transcribe a recording…: choose an audio or video file (wav, mp3, m4a, flac, ogg, opus, wma, mp4 and more), up to two hours long, and where the text should go: a Markdown file, a Word document, or the dictation box. Progress is shown; Cancel keeps what was done so far. The recogniser's own punctuation is used (people in a meeting do not say “full stop”), unless you tick that the speaker dictated it. On a fast computer two hours take about three minutes.

Paragraphs start where the speaker paused for a while (2 seconds by default; you choose it) at the end of a sentence, so a thinking pause in the middle of a sentence does not split it. Very long paragraphs are also broken after about 90 words.

Proofreading the transcript (Windows, with the proofreading model downloaded): choose a level in the same window and the local language model corrects the text before it is saved, with the same instructions and safeguards as in Word. The file notes how many changes were made. Cancel while it proofreads keeps the rest as transcribed.

Proofreading (Word on Windows)

Off by default. When you pause or stop dictating (or say “proofread that”), a language model on your computer reads what you dictated in Word and fixes it, in one sweep, so it never fights your cursor while you dictate. Choose how much it may change:

1. Punctuation: commas, full stops, capitals and spaces only; no word changes.
2. Grammar: also agreement, tense, articles, misheard words and stumbles.
3. Structure: also run-on sentences and awkward phrasing.
4. Custom: your own instructions, with your own limit on how much may change.

Edits are shown as tracked changes (recommended: accept or reject each one), as highlights, or applied silently. “Undo proofread” and “clear highlights” do what they say. Answers that change too much, change numbers or add chatter are refused and the text is left as dictated. You can read and edit what the model is told for each level, and try it on sample text, in Settings > Proofreading. The model is downloaded only when you switch proofreading on:

How careful they are, measured on 24 test texts (including some that are already correct): at levels 1 and 2 the 3B model made 4 unwanted changes and the 7B model 1, out of about 70 corrections needed. Choose the 7B if you have 16 GB of memory and don't mind waiting twice as long. Whatever the model, numbers, currencies, titles and times are never changed, and tracked changes let you reject any edit.

Skins

Settings > Appearance changes how the widget looks: Standard, HAL (a red eye that glows with your voice), Rainbow Unicorn (eyes and mouth that move as you speak), Old-school Winamp and First-generation iPod. Every skin shows every state. Make your own from shapes and pictures in a skin.json file, and add it from a folder or a .zip.

Settings, page by page

Right-click the widget or the tray/menu bar unicorn, then Settings…. Changes apply when you press OK; nothing is saved half-way.

Speech models

Choose in Settings > General. The model check measures them on your own computer. Models are downloaded once and checked against a fingerprint.

Privacy and your files

Your voice is turned into text on your computer and is never recorded, stored or sent. The internet is used only to download models you choose. Logs, if switched on, hold timings and lengths, not words, unless you also tick “include the words”.

The settings files are the same on both, so you can copy the folder from one computer to the other. Uninstalling on Windows asks before deleting this folder (the answer is No unless you say so).

How it works

• Listening. The microphone is read in 30 ms pieces. A small neural speech detector (Silero) decides when you start and stop speaking.
• Recognising, live. While you speak, the growing phrase is re-recognised about once a second, and words are typed as soon as they stop changing; the newest are held back a little. This gives a word-error rate of about 3% with words appearing around 1.4 s after you say them.
• Commands or text. A finished phrase is first compared with the commands (whole phrase, tolerant of small mishearings). Otherwise it goes through the text rules in a fixed order: spelling, your vocabulary, numbers and money, then punctuation, spacing and capitals.
• Typing safely. Text is typed (or pasted, for long text) into the program in front. The app remembers what it typed and reads the two characters before the cursor, so it knows whether to add a space or a capital, and so “scratch that” deletes only what it typed, never your own writing.

How it was built

Unicorns Can Talk is written in Python, about 13,000 lines plus more than 900 automated tests, in small steps, each one tested in real use before the next. The layers:

In order

Troubleshooting

• Nothing is typed (Mac). Switch on the app in System Settings > Privacy & Security > Accessibility. After installing a new version, switch it off and on again there.
• Capitals or words change by themselves (Mac). macOS can correct spelling and capitalise words in some programs after the text is typed. Switch off “Correct spelling automatically” and “Capitalise words automatically” in System Settings > Keyboard > Text Input > Edit.
• Nothing is heard. Check the microphone in Settings > General, and on a Mac in Privacy & Security > Microphone.
• It is slow. Run “Check this computer and pick the best model…” in Settings > General, or choose a smaller model.
• A word keeps coming out wrong. Add it to Settings > Words > Vocabulary.
• A command fires in the middle of a sentence. Settings > General: commands only at a pause; or change the command's phrase.
• Something else. Run the program with --self-test: it checks everything it needs and says what is missing.