Admin: Review & Correct Detail
  • 18 Minutes to read
  • Dark
    Light

Admin: Review & Correct Detail

  • Dark
    Light

On the Detail Review & Correct page you can see dictation transcripts or name of a shortcut that was run, and you can also listen to and correct dictation transcripts. For dictations you can also view a timeline of events on the dictation server.

To view the event timeline for dictations, see Dictation Event Timeline.

 You can open all items and view the Detail Review & Correct page, whether or not text and/or audio is available to view/hear.

 

Listen to Audio & View Transcripts

By listening to audio recorded by users and reviewing their transcripts, you can troubleshoot issues with audio equipment, and shortcuts.

 

Step 1. Find an item

  1. Go to the Review & Correct page in nVoq.Administrator. 
  • Optionally, click the Manage icon in the sidebar to hide the sidebar while you work.

 

 

  1. Select the appropriate organization from the organization menu at the top of the page.
  • Navigate this menu by using the Expand All button to open all organizations, by clicking the > arrow next to each organization level to open it, or by typing an organization name in the Filter text field.

 

 

  1. From the Account drop-down menu, select an account or choose "All" at the top of the list to view activity. 
  • Optionally use the Find Account field at the top of the menu to locate a specific account.

RandC-find-account

You can hover your mouse over the account in the list to see the username in a tooltip if it's cut off on the menu.
account-list-hover-username

 

  1. Optionally, filter results in this list to view specific items, activity, event status, date range, etc.

 

 

  1. Click the refresh iconToolbar-Refresh-iconin the blue bar to update the list to include the selected filtered results. 

 

 

  1. A summary table of all items appears on the page. 
You can click on any column header to sort information to find an item.

The first letter of the Item ID tells you what kind of recording it is:

D = Dictation
M = Microphone Setup recording
S = Voice Shortcut recording
U = Popup shortcut
B = Sub-shortcut
K = Keyword used to start or stop a hands free dictation or shortcut

Due to clock differences between the computer running the voice client and the nVoq servers, hands free keyword items on the Review & Correct page may appear out of sequence with transcription (or shortcut) results.
The list on this page can display up to 10,000 transactions. If you see an item count of 10,000 on a page, there are likely more transactions that are not included in the list. You could change the date range or filters to find specific items that may not be included in the list.

 

 

Step 2. View the Transcript & Listen to the Recording

  1. Click the Actions icon (Accounts-actions-icon) and select Open Item to open the Detail Review & Correct screen (or you can double-click on the item in the list).

RandC-open-item

 

 

  1. View information about the transaction on the Detail Review & Correct screen:

    Review Tab

    • Make sure you're on the Review tab. (The detail view will automatically open to the Review tab.)
      RandC-review-tab-17-1-0

      External ID

    • External ID is a text field where you can enter information about the transaction. This field may already be populated with information supplied by the API for partner clients or the SayIt client.

      Original Tab

    • The Original tab will show (for dictations) the dictation transcript as it was heard by the dictation server, or (for shortcuts) the name of the shortcut that was launched, or (for microphone setup recordings) the text that was presented as part of microphone setup.
      • If you used the Look for Word feature on the list view to find a specific word or phrase in dictations, the word or phrase for which you searched will be highlighted.
      • This tab will be empty if a no words were understood by the dictation or shortcut server or (in the case of a voice shortcut) if the words spoken do not match to an existing shortcut name.

RandC-view-original-tab

If text and/or audio has been deleted per the Keep Text and Keep Audio settings (on the Group Client Settings tab of the Organizations page) there will be NO TRANSCRIPT here and/or you will not be able to play back the audio.


Substitution Tab

  • The Substitution tab displays the dictation transcript as it appeared after substitutions were applied. The text in the Substitutions tab is the dictation transcript as it was returned to the user unless the user had substitutions turned off.
    • This tab is blank for shortcuts.
    • There are "built-in" substitutions that will show up as substitutions here, though they are not listed on the Substitutions page in nVoq.Administrator.
    • If you used the Look for Word feature on the list view to find a specific word or phrase in dictations, the word or phrase for which you searched will be highlighted.

RandC-view-substitution-tab

This user has a substitution that replaces a dictation of "UNIX time" to "UNIX Epoch time" in the transcript.

 

 

  1. Listen to the recording of the dictation, voice shortcut, or microphone setup on the Detail Review & Correct screen:
  • Click the play button  on the audio player to hear the recording.
    RandC-audio-player
    • Click the Audio Command Shortcuts link below the player (or Ctrl+M on your keyboard) to open a list of available hotkeys for navigating audio. Use the Esc key or just click on the background of the page to dismiss the window. Hotkeys for audio playback include:
      • Ctrl+P: Play, Stop
      • Ctrl+R: Rewind 1.5 seconds (works when in both Play or Stop mode)
      • Ctrl+F: Fast Forward 1.5 seconds (works when in both Play or Stop mode)
  • For dictations:
    • Listen for audio quality issues.
      Is the audio too loud, too quiet, or is there a hum or other noise? This could be caused by the user recording with a non-approved microphone. 
    • Listen to see if the beginning or end of the transcript was cut off.
      This is generally caused by a poorly-timed dictation button press by the user.
    • Listen to see if the transcript matches what was dictated.
      If not, you can optionally correct the dictation (to add to user sentence modeling) or reject the dictation (if there are audio issues).
  • For voice shortcuts:
    • Listen to see if the name of the shortcut that ran (in the Original tab) matches the name of the shortcut that was spoken.
      If not, this was a mis-match. You can optionally check to see if the user said a shortcut name that matched an existing shortcut name in the grammar file.
    • Listen to see if the beginning or end of the shortcut name was cut off.
      This is generally caused by a poorly-timed voice shortcut button press by the user.
  • For microphone setup: 
    • In the case of microphone setup, the text you see is the actual text that the user was presented to read as part of the Microphone Setup Wizard.
      It is not necessarily what the user said in the recording.
Trouble playing audio? See Browser Support with Audio Players.
Transcripts and audio recordings are displayed on the Review & Correct page until they are deleted from the system. The length of time that this information is saved is configurable on the Organizations page at the Group level on the Client Settings tab under Data PersistenceIf a group is configured to discard text and/or audio, those files are not available for review.

 

  1. Decide what you want to do next:
  • To view or listen to another item, click the Previous or Next button at the lower left of the page to listen to another recording.
  • To return to the list of items on the Review & Correct page, click the Close button in the lower right.
  • To correct a dictation transcript, see Correct a Dictation Transcript.
  • To reject a dictation recording with poor audio quality or words cut off at the beginning or end so that it does not get used for accuracy scoring, see Mark a Dictation as Poor Quality.

  

 

Correct a Dictation Transcript

When you correct a dictation transcript you create an accuracy score for the dictation and user sentence modeling that is applied overnight to the user's account to help improve dictation accuracy.

  1. Find the Item you want to correct. 

a)  We recommend that you FILTER the results (using the Type drop-down menu) to view Dictation items only.

nVoq.Administrator Review & Correct allow all types of transactions to be corrected (not just dictations), however only corrected dictations will be applied as sentence modeling to improve accuracy. Correcting shortcut transcripts of any kind will NOT improve shortcut or dictation recognition.

b)  After you select filter, click the refresh icon in the blue bar Toolbar-Refresh-icon to update the list to include the selected filter results.

c)  We also recommend that you SORT by the Word Count column (by clicking on the column to sort in ascending order) to see longer dictations.

You should select transcripts of at least 3 words or more if you want the correction to apply to sentence modeling, but we recommend that you focus on correcting longer dictations (10-40 or so words) so the dictation engine is given more context. 

 

  1. View the transcript and listen to the recording. 

a) Click the Actions icon (Accounts-actions-icon) and select Open Item to open the Detail Review & Correct screen (or double-click on the item).
RandC-open-item

 

b) Make sure you're on the Review tab. (The detail view will automatically open to the Review tab.)
RandC-review-tab-17-1-0

 

c) Click the play button  on the audio player to listen to the recording.
RandC-audio-player

  • Click the Audio Command Shortcuts link below the player (or Ctrl+M on your keyboard) to open a list of available hotkeys for navigating audio. Use the Esc key or just click on the background of the page to dismiss the window. Hotkeys for audio playback include: 
    • Ctrl+P: Play, Stop 
    • Ctrl+R: Rewind 1.5 seconds (works when in both Play or Stop mode
    • Ctrl+F: Fast Forward 1.5 seconds (works when in both Play or Stop mode)
Transcripts and audio recordings are displayed on the Review & Correct page until they are deleted from the system. The length of time that this information is saved is configurable on the Organizations page at the Group level on the Client Settings tab under Data Persistence. If text and/or audio has been deleted per the Keep Text and Keep Audio settings on the Group Client Settings tab, there will be no transcript and/or you will not be able to playback the audio.

 

 

  1. Determine whether or not the recording is suitable for accuracy scoring.
  • How does the audio sound? Is it too loud or too quiet? Is it distorted or truncated? Is there popping, clipping, or static? 
    • If accuracy is good, don't worry about this.
    • If accuracy is bad, this could be the culprit. See Audio Level. Click the Poor Quality button. 
Poor audio quality that has affected the dictation accuracy will also affect the accuracy score if the item is corrected. See Mark a Dictation as Poor Quality.
  • Are any words cut off from the beginning or end of the recording?  If yes, click the Poor Quality button. 
  • Is the audio quality acceptable?  If yes, proceed to the next step.

 

 

  1. If the audio quality is good and there are no words cut off from the recording, click on the Original tab (it may be already showing by default) and then click Copy Text to Corrected button to copy the text from the Original tab over to the Corrected Text box.
You must be viewing the Original tab in order to copy text from that tab. The "Copy Text to Corrected" button will copy text from whichever tab you are viewing when the button is clicked.

RC_Why-Original-Tab

  • The Original tab will show the dictation transcript as it was heard by the dictation server. This tab will be empty if no words were understood by the dictation.

RandC-view-original-tab

  • The Substitution tab displays the a dictation transcript as it appeared after substitutions were applied. The text in the Substitutions tab is the dictation transcript as it was returned to the user unless the user had substitutions turned off.
    • There are "built-in" substitutions that will show up as substitutions here, though they are not listed on the Substitutions page in nVoq.Administrator. 

RandC-view-substitution-tab

 

 

  1. Press play on the audio player and pause as needed while you edit the text in the Corrected Text box.

The text in the Corrected Text box should reflect what the user actually said, including any words that may be used to elicit a substitution.

  • Include punctuation that the user dictated as punctuation marks (not the words). If the use said, "period" and meant to get a punctuation mark  (.) instead of the word "period", include that punctuation mark in the Corrected Text field.
  • For most substitutions, enter the words that the user dictated to elicit the substitutions in the Corrected Text field, like "Number Next" or "Caps On" / "Caps Off" .
  • Unlike other substitutions, "New Line" and "New Paragraph" commands are NOT returned as words on the Original tab transcript. Instead these appear as one or two line carriage returns.  This is fine to keep in the Corrected Text field. If you are correcting a dictation in which the "new line" or "new paragraph" command was spoken but did not work, correct the transcript by adding 1 carriage return for the "new line" command, or 2 carriage returns for the "new paragraph" command in the Corrected Text field.
EXCEPTION: Do NOT include “Stop Dictation” commands in the Corrected Text if you hear them in the recording. 

 

 

  1. When you've completed the correction in the Corrected Text box (OR if no corrections were needed), click the Save button.
If you've made a mistake, click the Clear Corrected button. See Undo a Correction.

 

  1. You will now see Accuracy scores and a Status of RandC-icon-corrected  at the top of the Detail Review & Correct screen.

RandC-accuracy-corrected-status-17-0-0

 

Optionally, click on the Original Transcript Difference tab to view the differences between the original transcript and the corrected transcript.

RandC-original-accuracy-difference

 

You can also click on the Substituted Transcript Difference tab to view the differences between the substitution transcript and the corrected transcript.

RandC-substituted-accuracy-difference

  • The corrected transcript will automatically be added to this user's account sentence modeling as part of a system job that runs overnight. 
  • Any unknown words in the sentence model will be ignored by the dictation server, but you can add unknown words to vocabulary to allow them to be recognized by the dictation server in future dictations. 

You can optionally apply the user's corrected transcript sentence model to an entire organization.

 

  1. Optionally, move on to the next recording by clicking the Next or Previous buttons, or click the Close button to return to the list of items on the Review & Correct page and choose another dictation. 
 

 

Undo a Correction

If you make a mistake in your correction, click the Clear Corrected button to reset a corrected transcript back to a New state. 

  • If you click the Clear Corrected button BEFORE YOU SAVE, all information that you entered on this page (in the External ID field, in the Corrected field, etc.) will be deleted.
  • If you click the Clear Corrected button AFTER YOU SAVE, only correction statistics associated with this item or a "poor quality" status will be cleared from the item. (It will NOT clear other saved information.)
  • Clicking the Clear Corrected button automatically saves the item with the cleared status.
You cannot clear items that have already been applied as sentence modeling.

RandC-clear-corrected-17-1-0

 

 

Mark a Dictation as Poor Quality

If poor audio quality has affected dictation accuracy, the audio recording and dictation transcript are not suitable for accuracy scoring.

  • If this occurs, click the Poor Quality button to reject the audio recording. 
  • Doing so will give the item a Status ofRandC-icon-rejected and clear all correction statistics associated with this item (if any). Any associated accuracy score is removed from the summary table as well as any text in the Corrected Text field.
  • Clicking the Poor Quality button will automatically save the item with the poor quality status.

RandC-reject-poor-quality-17-1-0

Dictations may be marked as Poor Quality for the following reasons:

  • The audio recording quality is poor. This can be caused by using a microphone with poor recording quality, static, a hum, poor connection, or audio level that is too low or too high.
These issues may be caused by use of a microphone that is not approved or recommended by nVoq.
  • The recording volume is too loud and there is clipping in the recording. 
  • The recording volume is too low and inaudible. 
  • Words are cut off from the beginning or end of the recording.

See Audio Level

 

Email Transaction Information

Click the email icon in the blue toolbar Toolbar-Envelope-icon to send an email to nVoq Support (or whoever you want) with your questions or comments about a particular transaction that includes everything they might need to know about the transaction.

  • When you click the icon, an email opens in your default email client that includes the following information about the transaction: environment, ID, username, account first and last name, audio length, status, submit time, audio level, accuracy percentage, word count, review date, reviewed by name, external ID, and audio observations. 
  • If the transaction is a dictation, additional information from the Timeline tab will also be included: client product, client vendor, client version, the UUID of the dictation, microphone used, queue depth, queue time, topic, total time, and upload delay. 
  • You will need to manually complete the TO field of the email.

 

Audio Observations

This field is for use by nVoq staff.

In the Audio Observations field a system-level administrator can add tags relating to audio or accuracy issues in order to identify at-risk users, or to identify transactions that represent an ideal experience. The field is available at the bottom of the Detail Review & Correct page for all types of transactions, including voice and popup shortcuts.

  • This field and tags added to it are visible to anyone who has access to the Detail Review & Correct page. Tags can also be viewed on the Review & Correct page list view by adding a custom column. Tags can only be added by system-level administrators.
  • When a system-level administrator clicks into this field, a menu appears with default tags that can be select, including: Accented speaker, Audio level, Background noise, Exemplary, Microphone, Speech, Vocabulary, and Other. Select menu items with a mouse click, or by using the up and down arrow keys to navigate and the enter key to select.
    RandC-audio-observations-menu

  • Custom tags can also be entered manually by typing in a word or phrase into the field and then clicking either the enter key or the comma key on the keyboard to create the tag.
    RandC-audio-observations-add-custom

  • Once created, tags are not editable. If you need to change a tag, delete it (by clicking the X next to it) and add a new one.
    RandC-audio-observations-tags



Audio Level

Audio Level provides a visual indicator of the relative quality of the audio of dictations. This is a quick way to check if the audio is within normal range.

Because audio statistics are not calculated for voice shortcuts, the Audio Level indicator will display 0 for voice shortcuts.

Click on the Audio Level bar to see detailed information about the relative audio quality of the recording.

RandC-audio-level-17-0-0

The audio level calculation is based on average of the energy levels and the amount of clipping detected in the recording. Decibels are not used to calculate the audio level. For more information about audio clipping, see Troubleshooting: Audio Clipping (video)

The audio modal window provides additional details about the audio quality:

  • The Hint provides the additional information about the audio level signal.
  • Audio is below acceptable levels. 
  • Audio is starting to get too quiet.
  • Audio is in normal range
  • Audio is too loud
  • Average Root Mean Square (RMS) Energy is a statistical measure of the varying magnitude of the audio signal from a given baseline. An RMS range between 500 and 5000 returns the best accuracy. Below 500, the signal slowly degrades and reduces accuracy. Above 5000 there are increasing chances of errors.
  • Average Energy is the amplitude or loudness of the recording. It is calculated as an average across the entire recording. Energy level that is too low will result in poor recognition. A low energy level occurs when a microphone is too far away from a user's mouth or a user is speaking too softly. Some microphones let you configure the input level. Low energy level can occur when the input level is set too low.
  • Clipping means the signal is too loud, resulting in distortion, and the signal is difficult to moderate with normalization. Any number above 0 indicates clipping. Clipping can occur when a microphone is too close to a user's mouth or a user is talking too loudly. Some microphones let you configure the input level. Clipping can occur when the input level is set too high.

Isolated cases do not indicate an issue. However, a large number of audio recordings that indicate clipping, in combination with recordings that have a high RMS, for example over 5000, may indicate a problem with the user's microphone setup.

 

   

Download Audio and Download Text

The Download Audio and Download Text links allow you to download the audio and text pairs for dictations and voice shortcuts. 

If you are an administrator at the company, division, or group level, you may not see these links on the Item Detail window. (Whether you do or not is a tenant level setting.)

RandC-download-audio-text

To download audio and/or text files:

  1. Click the Download Audio or Download Text link.
  2. The audio or text file will be downloaded to your computer.

If a dictation transcript was corrected, the corrected text is downloaded from the Download Text link. If the audio was removed from the database, only the text is downloaded.

 

Supported file types are .ogg and .wav for audio, and .txt for the text file.

The default naming convention is to name the file by lastname, firstname_JobID.filetype.

For example: Minerva, Dottie-D4741741.txt  or  Minerva, Dottie-D4741741.ogg.

If you are not prompted to download the file, check to see if the file was automatically downloaded, or check your browser's configuration.

 

 

Download Active Shortcuts

The Active Shortcuts is a "grammar file" that contains a list of all of shortcut names, including alternatives, that were available at the time the user attempted to run a voice shortcut. This file is available until it is removed by the "orphan cleaning" job (usually after 14 days). After the file has been removed, the Download Active Shortcuts link will no longer appear for the item.

This link is not available when Popup shortcuts are used because there is no grammar file produced.

RandC-download-active-shortcuts

To view the grammar file:

  1. Click the Download Active Shortcuts link.
  2. The grammar file will be downloaded to your computer. One of two types of files will be downloaded:
  • A regular voice shortcut will produce a .grxml (eXtensible Markup Language) file.
  • A Fast Shortcut will provide a .jsgf (JSpeech Grammar Format) file.  

 

An example XML grammar file is shown below:

<?xml version="1.0" encoding="UTF-8" ?>
 <grammar xml:lang="en-US" version="1.0" root="ROOT" xmlns="http://www.w3.org/2001/06/grammar">
  <rule id="ROOT" scope="public">
   <one-of>
    <item>Account Change Request</item>
    <item>BRS Notepad</item>
    <item>Chat Fillers</item>
    <item>Mammogram Results Letter</item>
    <item>Price Adjustment Request</item>
    <item>Support Ticket Status Email</item>
    <item>Ticket Detail Template</item>
    <item>Training Request Email</item>
   </one-of>
  </rule>
</grammar>


 An example JSGF file is shown below:

#JSGF v1.0;
 grammar quicknotes;
public <quicknotes> = /22/account change request | /11/brs notepad | /12/chat fillers | /24/mammogram results letter | /24/price adjustment request | /27/support ticket status email | /22/ticket detail template | /22/training request email;

 

Browser Support with Audio Players

  • If you are unable to listen to OGG audio on Windows, the work-around is to download the audio in Review & Correct and play the audio in the stand alone version of Windows Media Player.
  • QuickTime v.7.7.6 no longer supports legacy codecs in Firefox.
  • If you are using Firefox and QuickTime does not launch, download the audio in Review & Correct and play the audio in the stand alone version of QuickTime. Alternatively, use an older version of QuickTime with Firefox. See http://www.oldversion.com/windows/quicktime/.