ISV Training: Troubleshooting Accuracy & Latency Issues

  • 7 minute(s) read
Prev Next

When it comes to dictation, the two most common issues users may experience are latency and accuracy.

Latency

  • Look at https://status.nvoq.com/ to see if there is a known problem with nVoq servers. On this page you can see the current health and system uptime for the past 90 days for any nVoq URL address (including the nVoq.Administraror) as well as upgrade information, maintenance notifications, outage incidents, and post-mortem reports. You can even sign up to receive emails or text messages so that you can be notified when upgrades or issues occur.

  • You can also review system health information on the Review & Correct page to see where in the process the issue might be occurring. 
    • To see system health information:
      1. Go to the Review & Correct page. 
      2. Select "Dictation" from the Type menu to only include dictation items in the list view.
      3. Click the the Customize Columns menu (tool icon) to open the Customize Columns window.
      4. In the Customize Columns window select the following columns:  ID, Submit time, Account, Audio length, Status, Queue depth, Total time, and Upload delay. Optionally uncheck all other columns to make the report easier to view. Click Save.
        system-health-how-to-view
    • Look at is differences between Audio Length and Total Time. The Total Time is the elapsed time between the recording being submitted and the time item it was completed. This is measured from when the user starts recording to when they received their final transcript. The Audio Length is the length of each recording in seconds. We expect Total Time to be a little longer than the Audio Length because of added processing time by the dictation server, but if you see a large difference between the two, it could mean that our servers are not processing the audio optimally. In this case, please report the issue to nVoq Support so we can investigate. If you see a large discrepancy here, compare it to other users in that same group.

    • Large positive values (like thousands of milliseconds) in the Upload Delay column could indicate client network issues. Upload Delay is the delta for Audio set to done - (Audio Length *1000). Normally the value of the Upload Delay is negative, which means the dictation was processed in less time than the length of the audio. A large positive value, like a 5-digit number, indicates that it took much more time to process the dictation than the length of the audio. For example, an upload delay value of 20,000 would mean a 20 second delay. A positive upload delay value of four digits represents a delay of at least one second (e.g. a value of 3000 is a delay of 3 seconds). Positive upload delay values may indicate slowness on the customer’s internal system (like their WiFi), but it could be a problem on our end. If you see a large number, you should compare it to other users in that same group. (Is it happening for just one user? Or the whole group?) The best course of action to take when you see this is to contact the user’s IT department so they can investigate network speed. You should also pull the user’s log file and send that to nVoq Support.

Accuracy

  • If dictated words or phrases are being understood by the dictation engine, but don't look right, that's a formatting issue. For example, if you say a blood pressure, and it returns as "one hundred over sixty" instead of "100/60," that's formatting. An administrator is going to have to look into these issues. If some of a user's dictated words or phrases are not understood by the dictation engine and incorrect words or phrases return in the transcript, that's a recognition issue. For information about what can be done to resolve these different types of issues, see Dictation Accuracy Troubleshooting

  • Audio quality can also affect accuracy, and not all microphones are created equal. Even if the audio sounds fine to you on the Review & Correct page, the audio sent to the server may not be high enough quality. ISV Partners can use the microphone parameter in the STARTDICTATION WebSocket method so that the client can inform the server of which microphone was used to record a dictation. This information can then appear on the Timeline for dictation events on the Review & Correct page. Then, if you notice that a user is getting poor quality transcripts when they dictate, you can review the Timeline for dictation events to see the microphone was used on the dictation. Is it on the nVoq list of recommended microphones? If not, tell the user to try switching out with a microphone from the list and see if dictation results are improved.

Tier 1 Dictation Accuracy Troubleshooting

To enhance the user experience, nVoq must prioritize resolving high-frequency accuracy issues. ISV partners can best assist users by reviewing transcripts for misrecognitions, making corrections, and promptly adding vocabulary entries or substitutions as needed. Taking these steps enables immediate corrections and helps minimize repeated errors.

Required Steps for Tier 1 Support Before Escalating to nVoq Tier 2

nVoq providers Tier 2 support. Our direct support contact information should only be used by your staff who have completed Tier 1 troubleshooting. In the case of accuracy concerns, that includes the following steps:

  • Confirm that the misrecognition occurs across two or more users with at least four occurrences of the misrecognition per user. 
  • Use the Look for Word filter on the Review & Correct page to find multiple instances of the misrecognition. 
  • Confirm that audio quality is sufficient, including appropriate speech pace, clear pronunciation, adequate dictation volume, and minimal background noise.
  • Verify that the user is dictating using the correct dictation topic associated with their medical specialty. You can find that information on the Detail Review & Correct page "Timeline" tab.
  • Correct the transcripts in Review & Correct. Note: In some cases, adding corrections through user sentence modeling may resolve the issue without escalation.
  • Ensure the misrecognition can be reproduced by the ISV partner using a test account and the same dictation topic as the users experiencing the issue.
  • Confirm that the issue persists even after a vocabulary or substitution entry has been added and properly tested,
    — or —
     Determine that the issue cannot be resolved by adding a vocabulary or substitution entry.

If after the recommended steps have been completed and the misrecognition persists, escalate to nVoq Tier 2 Support for additional investigation. Include the following information when reporting potential accuracy defects:

  • Submit a support request to nVoq Tier 2 Support through the FreshDesk portal. (Instructions for doing that are available on this site.)
  • Provide a clear description of the recognition issue.
  • Include dictation IDs that demonstrate the problem, including the dates when the affected dictations were submitted.
  • Include the Tenant and usernames of impacted users.
  • Ensure dictation transcripts have been corrected in Review & Correct.
  • Include a description of all steps completed in the Tier 1 support process. 

nVoq Tier 2 Support will perform internal testing to determine whether the reported misrecognition is a defect. If the misrecognition is determined not to be a defect, nVoq will provide recommendations via the Support portal to improve dictation accuracy.

Definitions for Accuracy Troubleshooting Support

Defect

A defect is any unintended behavior or error within nVoq’s speech-to-text platform that causes it to operate incorrectly. This may include issues such as inaccurate transcription of dictated speech, misinterpretation of standard terms, or incorrect formatting of recognized terms. Defects typically require investigation by nVoq’s Tier 2 Support and Linguistics teams.

Out-of-Vocabulary (OOV) Word Support 

At times, a dictated word may not be recognized if it does not exist in nVoq’s dictionary. nVoq conducts a monthly analysis of all corrections made through Review & Correct to identify out-of-vocabulary (OOV) terms. Any terms detected during this process are automatically added to the nVoq dictionary. Note: Proper names such as patient names, locations, clinic names, or medical company names should be entered as vocabulary terms at the Organizational level. 

  • Do not escalate OOV terms to nVoq Tier 2 Support. As this process is automated, escalating to nVoq Tier 2 Support is unnecessary. Note: The Word Finder tool in nVoq Administrator can be used to determine if the term already exists in nVoq’s dictionary.

One-Off Account-Level Misrecognitions

One-off account-level misrecognitions are isolated cases where nVoq’s speech-to-text platform inaccurately transcribed dictated speech for a specific user, without indicating a broader or systemic issue affecting other users. These misrecognitions are often caused by factors such as individual speaking style, pronunciation, background noise, microphone quality, or account-specific configuration settings. They are not considered defects in the platform itself and are typically resolved through user training, vocabulary or substitution entries, or transcript corrections at the account level. 

  • Do not escalate one-off account-level misrecognitions to nVoq Tier 2 Support. By not escalating one-off misrecognitions, nVoq can focus efforts on the issues users experience most frequently, resulting in a better overall experience. Instead, use Review & Correct to correct the dictation transcript. This will trigger sentence modeling to improve accuracy for the user's account. You may also consider adding vocabulary and/or substitutions to better accommodate the user’s unique speech-to-text patterns.
back_pg Next_pg