Understanding the Workflow

Last updated 06/18/2021

Hotpepper’s workflow is informed by Deepgram best practices for labeling audio, which require that all audio files be reviewed by four independent users. Hotpepper implements the idea of independent reviewers through the concept of levels, each of which represents a queue of unlabeled files. For a file to be considered completed, it must pass through all four levels of labeling and be reviewed and marked as complete by four independent reviewers, one at each level.

Understanding the Hotpepper Workflow

When users select a level, the system automatically assigns them a file from the corresponding queue and takes them to the labeling view for the assigned file.

While reviewing, users can pause their work before completion. If a user previously paused a file at the chosen level, the system automatically loads the paused file into the labeling view, so they can finish their work.

At each level, users perform different operations on the file. Deepgram recommends the following ordering of operations:

LevelOperation
L1Transcribe what you hear. Create new paragraphs each time the speaker changes.
L2Correct the transcript. Add speaker labels (for example, "[Speaker 1:]" and "[Speaker 2:]") at the beginning of each paragraph.
L3Review the transcript and speaker labels. Make any necessary corrections.
L4Review the transcript and speaker labels. Make any necessary corrections.

Once a user has finished labeling a file and marked it as done, the file is added to the queue for the next labeling level, where it will no longer be available to the user.

Throughout the workflow, administrators control the process, including availability and prioritization of files for labeling, as well as who may use the system. To learn more about administrator operations, visit Hotpepper User Guide: Managing Datasets and Hotpepper User Guide: Managing Users.