Skip to content
Draco Fortis
Contact
All work

Audio Annotation

Recording, checking and annotating speech data

A web application where contributors record sentences in their own dialect, check each other's recordings and mark who speaks when in conversations. The checked data trains speech models.

Industry
Speech technology
Type
Web application
What we did
Design, frontend and backend development
Audio Annotation's speaker timeline: a conversation's waveform split into coloured blocks, one colour for each of three speakers, with a label showing speaker 1 from 0:08 to 0:12

  1. Correct or start again. Earlier annotations are loaded, so a contributor can correct them instead of starting from nothing.

  2. Speakers. Pick a speaker, then drag across the waveform where they talk.

  3. Segments. Each block is one stretch of speech, in its speaker's colour.

  4. Times. Who speaks, from when to when.

  5. Zoom. Zoom into the waveform to place the edges precisely.

01Scenario

A company working on natural language processing and voice recognition was building models for speech generation and for speaker diarization, which tells apart who speaks when. To train them it needed a varied, reliable set of recordings, checked by people.

02Solution

Recording and checking

Contributors read set sentences aloud in their own dialect, so the models learn how speech differs between regions.

A recording is not used straight away. Other contributors listen to it and check that the words match the text. They can:

  • correct the text to match what was said,
  • or reject the recording if it strays too far from the text or the sound is poor.

Only recordings that pass this check go into the training data.

Audio Annotation's review screen: the sentence a contributor read, the waveform of the recording, the dialect region they chose, blurred here, and buttons to accept or reject it

  1. The question. Was the sentence spoken as written?

  2. Sentence. What the contributor read. The pencil lets the reviewer correct the text to match the audio.

  3. Recording. The audio, played back while it is checked.

  4. Dialect. The region the contributor chose for their dialect, blurred here.

  5. Verdict. Reject or accept the recording.

Speaker annotation

For speaker diarization, contributors listen to conversations between two or more people and mark on a timeline the stretches where each one speaks. Where earlier annotations exist, they check and correct them.

The checked data then feeds two kinds of model:

  • Speech generation: recordings in many dialects help models produce natural intonation and dialect-specific expressions.
  • Speaker separation: annotated conversations teach models to tell speakers apart, for uses such as transcription and voice assistants.
Screen recording of a contributor marking where each speaker talks on a timeline

03Outcome

Checked data
Every recording is heard by another person before it is used, and annotations are checked and corrected the same way.
One place
Recording, review and annotation all happen in one web application.
Many dialects
Contributors choose their dialect region, so the data covers how speech varies between regions.

Next case study

Balloon Live

Hot air balloon pilots declare goals and drop markers on their phones, and the signed flight data reaches the scorers without SD cards.

Tell us what you need.

A few lines about the problem are enough to start. We reply by email, with questions or a time to talk.

Describe your project

or write to info@dracofortis.com