Admin: Note Assist QA

Using Note Assist QA you can run clinical documentation downloaded from your system against a Note Assist audit to identify gaps in required information. This powerful feature helps you quickly spot documentation issues and determine which clinicians may benefit from targeted coaching.

Run Note Assist QA

1. Run a batch file of narratives from your EMR against a Note Assist audit to see if existing narratives match audit categories.
   Information from these audits is not saved in nVoq Administrator and does not count towards Note Assist usage.
   1. Do a bulk export of patient narratives from your EHR -OR- create your own audit file using our template.
      
      
   2. Prepare exported data for the audit:
      1. Pull out any narratives you do not want to include in the audit.
         1. In Microsoft Excel, select the entire spreadsheet by clicking on the Select All button. That's the arrow in the top left corner of the spreadsheet (between the first column and the first row).
            
         2. Add filters to each column header. Click the Sort & Filter button on the toolbar, then select the Filter option.
            
         3. On the column that displays the narrative type, click the drop-down arrow next to the header to open the filter, and uncheck the box next to any of the listed narrative types you do not want to include in the audit. Then click the OK button.
            
         4. With only the narrative types you want in the file and the entire spreadsheet still selected, click Crtl+C on your keyboard (to copy selected narratives), and Ctrl+N (to create a new spreadsheet), and then Ctrl+V (to paste copied data into the new spreadsheet). This new spreadsheet is the one you will run in Note Assist QA.
            
         5. Name your new file and save it in a HIPAA-compliant location. This may be the hard drive on your computer. Do not save it to a shared network folder.
            
            
      2. Ensure that your file has the three required headers/columns:
         1. Your file must include - in any order - three column headers: Narrative Date, Narrative, and Name.
            1. The Narrative Date column includes the date that the narrative was created. The header name is not case-sensitive. Other header names, such as "Narrative Signed Date" may also be accepted. The information in this column needs to be a date, but many formats of date (including dates with time and milliseconds) are accepted.
            2. The Narrative column includes the actual patient narrative text. The header name is not case-sensitive. Other header names, such as "Narrative Text" may also be accepted.
            3. The Name column includes the name of the clinician who created the narrative. The header name is not case-sensitive. Other header names, such as "Worker" may also be accepted. The clinician named here does not need to have a corresponding nVoq account.
         2. Your file can have any additional columns that might be helpful for you when analyzing the data. These will be ignored by the system, but will still appear in the output file.
            
            • All columns must have a header. 
            • Combined column headers (merged cells) should be split out so that every column has its own header. 
         3. Save your file.
      3. Eliminate any hidden formulas that may be in your file from the extraction process from your EHR to prevent errors during processing.
         1. Select the first column that has no data.
            
         2. With the column selected, press Ctrl+Shift+Right key on your keyboard.
            
         3. Click Delete on the Excel toolbar to delete all of the columns that may have hidden data.
            
         4. Select the first row that has no data.
            
         5. With the row selected, press Ctrl+Shift+Down key on your keyboard.
            
         6. Click Delete on the Excel toolbar to delete all of the rows that may have hidden data.
            
         7. Save your file.
      4. Make sure all narrative cells have data in them. (You may have some that are empty.)
         1. Type the word "BLANK" in each empty narrative cell to show that noting was recorded by the clinician and to ensure you do not receive the "missing value" error from nVoq Administrator.
            To quickly do this:
            1. Click the Sort & Filter button in the Excel toolbar, then select Custom Sort.
               
            2. In the Sort window that opens, select Narrative from the Column menu, and make sure the Order menu has A to Z selected. Then click OK.
               
            3. Scroll down to the empty narrative cells. Type "BLANK" in the first empty narrative cell, then select that cell and the rest of the empty narrative cells, and use Ctrl+D to fill down.
               
         2. Save your file.
            
            
   3. Run the audit in nVoq Administrator: 
      1. On the Note Assist page Audits tab, click the actions menu next to the audit you want to run a batch file against and select Run Note Assist QA.
         OR Double click on the audit to open the Edit Note Assist Audit page, then click the Run Note Asssit QA iconin the blue toolbar.
         
         
         
      2. On the Run Note Assist QA page, click on the Choose a file link in the dotted line box and navigate to your saved file, OR drag your file from an open directory into the dotted line box. 
         • The maximum file size is 6MB.
         • Run the audit against one file at a time.
         • Only the first sheet in the file will be included in the audit run; any additional sheets will be ignored.
      3. The system will check the file for missing required columns, column names, or data, and to make sure it does not exceed the maximum file size.
         • If no issues exist, go to step 3d.
         • If information is missing or the file is too large that information will be presented below the file box. Click the X next to the file name to remove it from the page, fix the errors in the file, and then try the upload again.
      4. When your file looks good, click the Run button.
   4. When processing is complete your output file is downloaded automatically to your local Downloads folder. 
      • The output file name is the same as the original file name prepended with the current date. 
      • If the file was unable to run you will see a red popup notification explaining why the batch request was unable to run, and that information is also available in the notifications list.
        
        
        

Analyze Audit Results

1. Go to your Downloads folder to open the output file. The output file includes:
   1. The Narrative, Narrative Date, and Name columns
   2. Any extra columns that were included in the original file
   3. A column for the Audit Name against which the batch was run
   4. A column for each category in the audit. Each category cell shows either TRUE (if the category was included in the Narrative text) or FALSE (if the category was missing from the Narrative text) for each narrative in the report.
2. Click the Enable Edit button near the top of the Excel file.
3. Rename the sheet that includes the results to Audit Results.
   1. Double-click on the text on the tab.
   2. Type over the original tab name.
4. Click on the New Sheet icon near the sheet tab to create a second sheet.
5. Name the new sheet Analysis.
6. Copy the data from the Audit Results sheet to the Analysis sheet. This gives you a place to analyze the data without changing the formatting of the original results.
   1. Select the entire Audit Results sheet by clicking on the Select All button in the top left corner of the spreadsheet (between the first column and first row).
   2. Press Ctrl+C on your keyboard to copy the selected data.
   3. Click the Analysis tab.
   4. Place your cursor in cell A1 and then press Ctrl+V to paste the audit results into the Analysis sheet.
7. Make the category data numerical.
   1. Select all of the category columns.
      
   2. Press Ctrl+H on your keyboard to open the Find and Replace window.
   3. In the Find and Replace window on the Replace tab, type FALSE in the Find What box, and 0 (zero) in the Replace With box. Then click the Replace All button. Click OK to close the message that tells you how many replacements were made.
      
   4. Still in the Find and Replace window on the Replace tab, type TRUE in the Find What box, and type 1 (one) in the Replace With box. Then click the Replace All button. Click OK to close the message that tells you how many replacements were made.
      
   5. Close the Find and Replace window.
      
      
8. Add highlighting to category data cells to make it easier to differentiate the results.
   1. Select all of the category columns. (They may still be selected from the previous step.)
   2. In the Excel toolbar Styles section, click Conditional Formatting, select Highlight Cell Rules, and then select Text that Contains.
      
   3. In the Text That Contains window, type 0 (zero) in the Format cells that contain the text box, and select Light Red Fill with Dark Red Text on the with menu (this may already be selected by default). Click the OK button.
      
   4. With category columns still selected, again click Conditional Formatting and select Highlight Cell Rules, and then select Text that Contains.
      
   5. This time type 1 (one) in the Format cells that contain the text box, and select Green Fill with Dark Green Text on the with menu. Click the OK button.
      
      
9. Calculate the total "score" for each narrative.
   1. In the first blank column header to the right, type Total.
      
   2. Select all of the numerical column results for the first narrative (in Row 2).
      
   3. Click the AutoSum button (in the Editing section of the toolbar).
      
   4. Double-click the lower right corner of the cell showing the sum. This will carry the AutoSum function down through the remaining rows.
      
      
10. Put a filter on all of the data in the sheet so you can filter on anything as you analyze results.
    1. Select all columns with data.
    2. Click Sort & Filter (in the Editing section of the toolbar), and select Filter.
       
    3. This adds the filter drop-down to all selected columns.
       
       
11. Calculate the sums and percentages of each category that was met in the audit.
    1. In one cell type Total Narratives Audited.
       
    2. In the cell next to that, enter the total number of narratives that were audited. (The total number of narratives audited is one less than the total number of rows of data on your spreadsheet.)
       
    3. Below your Total Narratives Audited, add three columns titles: Category, Number of Narratives that Met Category, and Percentage of Narratives that Met Category.
       
    4. Below the Category column, copy and paste the name of each category in a separate cell.
       
    5. Place your cursor in the first cell below the Number of Narratives that Met Category column heading and enter =SUM( then click on the letter at the top of the corresponding category column. Finally add ) (closing parenthesis) to your formula, then press Enter on your keyboard. You'll end up with something like this: =SUM(F:F) 
    6. Repeat for each category below the Number of Narratives that Met Category heading.
       
    7. Place your cursor in the first cell below the Percentage of Narratives that Met Category column heading and enter = (equal sign) then click on the cell that includes the Number of Narratives that Met Category for that row. Enter / (forward slash) and then click on the number of Total Narratives Audited and press Enter on your keyboard. So you will have something like this: =Q3/Q1
    8. With your cursor on the cell of the formula you just entered, click the % button (in the Number section of the Excel toolbar) to change the result in that cell from a decimal to a percentage.
       
    9. Repeat the previous two steps for each category below the Percentage of Narratives that Met Category heading.
       
       
12. Create a chart of the data from the sums and percentages of each category met.
    1. Select the cells that include the data you just created (below the Total Narratives Audited) and click the Quick Analysis icon that appears in the lower right corner of the selected cells.
       
    2. On the menu that opens, select Charts.
       
    3. Click More to open the Insert Chart window and see all of the charts you have available.
       
    4. In the Insert Chart window, select the chart you want to create (in this example I have selected Clustered Column), then click OK.
       
    5. Drag the chart anywhere you want it on the sheet, and resize by dragging the chart area handles.
       
    6. Double-click the Chart Title text and type over it to add an appropriate chart title.
13. Calculate the number of categories met, number of total narratives, and percentage of narratives by number of categories met.
    1. Add three columns with the following headers: Number of Categories Met, Total Narratives, and Percentage of Narratives by Number of Categories Met.
       
    2. Under the Number of Categories Met column enter the total number of categories in this audit in the first cell, then count down to 1 in each cell under that. For the final cell (below 1) type NONE.
       
    3. Select the first cell below Total Narratives and enter =COUNTIF( then click on the letter of the Total column, add a comma (,) and then click on the cell that includes the first Number of Categories Met. Finally add ) (closing parenthesis) and press Enter, so you end up with something like =COUNTIF(N:N,P14)
    4. Repeat that process for each cell below to ensure the formula incorporates the cell to the left.
       When you get to the final cell (next to NONE) instead of selecting the cell, type 0 (zero) in the formula. (For example, =COUNTIF(N:N,0)
       
    5. To make sure the math is correct, sum up the Total Narratives column and make sure you get the same as the Total Narratives Audited: select all of the totals in the column you just created and then click the AutoSum button. It should be the same as the Total Narratives Audited.
    6. Place your cursor in the first cell below the Percentage of Narratives by Number of Categories Met column heading and enter = (equal sign). Click on the cell that includes the Total Narratives number for that row, enter / (forward slash) and then click on the cell that includes the Total Narratives Audited number and press Enter on your keyboard. For example, =Q3/Q1 
    7. With your cursor on the cell of the formula you just entered, click the % button (in the Excel toolbar) to change the result in that cell from a decimal to a percentage.
       
    8. Repeat the previous step for each of the number of categories met rows below the Percentage of Narratives by Number of Categories Met heading. (Copying and pasting the formula does not work here.)
    9. To make sure your math is correct use AutoSum again to sum the percentages you just added and make sure you get 100%.
       
       
14. Create a chart of the data from number of categories met, number of total narratives, and percentage of narratives by number of categories met.
    1. Select the cells that include the data you just created (do not include the sums at the bottom) and click the Quick Analysis icon that appears in the lower right corner of the selected cells.
       
    2. On the menu that opens, select Charts.
       
    3. Click More to open the Insert Chart window.
       
    4. On the Insert Chart window, click the All Charts tab.
       
    5. Select Pie, then click the OK button.
       
    6. Drag the chart anywhere you want it on the sheet, and resize by dragging the chart area handles.
       
    7. Add an appropriate chart title by double clicking on the Chart Title text, selecting it, and typing over it.
       
       

Download the Note Assist QA Audit Template

On the Run Note Assist QA page, use the Download template xlsx file link to create your own file.

• The template includes the columns required for Note Assist QA: 
  • Narrative - The clinical note text that will be run against the selected audit.
  • Narrative Date - The date that the narrative was created.
  • Name - The name of the person who created the narrative. This name does not need to have a corresponding nVoq account.