Ai Scorecards
    • 23 Feb 2024
    • 11 Minutes to read
    • Dark
      Light
    • PDF

    Ai Scorecards

    • Dark
      Light
    • PDF
    • New

    Article Summary

    The content discusses the benefits of quality assurance in improving customer service, client retention, and employee motivation. It introduces Ai scorecards as a tool to automate and streamline the grading process for contact center calls. The setup is minimal, with out-of-the-box templates provided by Dialpad. The scorecards can be customized with traditional or Ai questions, and can be applied to specific contact centers or coaching teams. The process of creating a scorecard involves selecting the desired features, adding questions, and publishing it. Grading an interaction involves reviewing the Ai-assigned grade and answering any traditional questions. Supervisors can search for graded interactions based on various criteria and delete grades if permitted. Scores can be shared with agents in real-time or scheduled for later sharing. The content also mentions user permissions, scorecard analytics, and FAQs related to Ai scorecards.

    Let's talk about Quality Assurance (QA). Continual coaching and actively monitoring your team's performance leads to heightened customer service, enhanced client retention, and employee motivation.

    Ai scorecards removes the manual legwork and ensure all of your contact center calls are graded with an objective and actionable grading tool. 

    Not only that, there’s minimal set-up required — Dialpad provides out-of-the-box templates, letting you get started right away!

    The icing on the cake? You can do all of this within the Dialpad app!

    Let's dive into the details.

    Who can use this feature

    Ai scorecards can be used in contact centers and coaching teams, for users with Ai Contact Center and Ai Sales accounts on the Pro and Enterprise level. 

    Contact us today to enable Ai scorecards for your team (there is an additional fee for this feature). 

    Create a scorecard

    Create a traditional Ai scorecard, or add Ai questions to your scorecard. When Ai questions are used, every call is graded by Dialpad Ai, removing the need for managers to spend hours a week manually reviewing calls or shadowing agents.

    Note
    To use Ai Questions, you'll need to ensure that Dialpad Ai is turned on.

    Ai must be enabled at the company level before you can configure Ai preferences for offices, contact centers, individual users, and coaching groups. 

    If you don’t see the Ai option in your settings, reach out to your admin. 

    To create an Ai scorecard, navigate to Admin Settings from Dialpad.com

    1. Select Ai Scorecards
    2. Select Create Scorecard
    3. Enter the name and description of your scorecard
    4. Select which contact center or coaching team can use the scorecard
      1. Scorecards can be applied to a single contact center or coaching team, or multiple
      2. Unless otherwise specified, a scorecard is assigned to all contact centers and coaching teams
    5. Select if you'd like to use Ai questions
      1. If you want a scorecard with Ai questions and traditional questions, you must select Ai Questions when creating the scorecard
    6. Select Create scorecard
    7. Add questions, pressing Save after each question
      1. Enter the value points, skipping options, mandatory fields, and failure notes
      2. If you're using Ai questions, review and add trigger words/phrases
    8. Once you've finished adding questions, select Publish
    9. When prompted, select Yes, publish now to complete the scorecard creation process

    That's it! Your scorecard is now active and will appear in your scorecard inventory. 

    Question content

    Every question must have the following components:

    1. Question title:  Question or statement to be considered when grading the call
    2. Response type: Yes/no, agreement (Likert), frequency (Likert), importance (Likert), likelihood (Likert), or quality (Likert)
      1. Ai questions can only be yes/no. If you choose a different response type, the question will revert to a traditional (non-Ai) question and will not be scored by Ai
    3. Responses: Responses are auto-filled to match the response type chosen but are customizable 
    4. Points: Point value given for each response

    Get granular by adding any of the following optional question features:

    1. Add a comment: Open the text box allowing graders to add comments
      • The comment field can be marked as required or optional.
      • A comment field can be added for each question — all grading comments appear on the detailed scorecard exports.
    2. Allow question to be skipped: Marks the question as N/A without affecting the agent's overall score
    3. Automatically fail entire scorecard for certain responses:  When selected, this feature automatically fails the entire scorecard, giving the agent a 0% grade
      • A scorecard can have multiple questions marked to be automatically failed.
    4. Save question as a template: Saves the Ai question, allowing you to re-use it in different scorecards

    Notes:
    Each scorecard must have a unique name.
     

    Once a scorecard has been created, it cannot be edited.

    Archive a scorecard

    Workflows adapt, and your scorecard needs will change as your company grows. Ai scorecards can be archived, which maintains the card's analytics data.

    To archive a scorecard, head to your Admin Settings from Dialpad.com 

    1. Navigate to Ai Scorecards 
    2. Select the scorecard you would like to archive
    3. Select Archive 
    4. When prompted, select Yes, Archive to archive the scorecard —remember this action is not reversible


    Archived scorecards can be reviewed at any time and easily using the Archived filter.

    Duplicate a scorecard 

    Need to create a copy of a scorecard? Not to worry, duplicating Ai Scorecards is a piece of cake. 

    To duplicate an Ai scorecard, head to your Admin Settings from Dialpad.com

    1. Navigate to Ai Scorecards
    2. Select the scorecard you want to duplicate
    3. Select Duplicate
    4. Review the copy of the scorecard, then select Publish
    5. When prompted, select Yes, publish now

    Grade an interaction

    Ai questions are automatically graded with Dialpad Ai pre-selecting answers based on the conversation, all the supervisors, contact center admins, or coaches need to do is review the Ai-assigned grade, and review any traditional (non-Ai) questions. 

    Currently, only contact center and coaching team calls can be graded — you cannot grade department or mainline calls. 

    To grade a call:

    1. Navigate to Call History 
    2. Choose the call you want to grade
    3. Select Grade Call from the menu options
      • This will open the call summary view
    4. Select the desired scorecard, then select Next
      1. If the scorecard has any Ai questions, you'll see what percentage was completed by Ai
      2. Scorecards with the highest completion rate are listed first
      3. You'll only see the scorecards that have been applied to the contact center or coaching team that the call belongs to
    5. Fill out the scorecard, reviewing and updating Ai questions as needed
      1. On the left, the Moments sidebar tells you how many times each Ai question was triggered. Click on each Ai question to see where in the transcript the question was triggered. See why Ai suggested a response by clicking on the Suggested by Ai answer on the scorecard. This will take you to the exact moment in the conversation where the trigger word was mentioned. 
      2. If the trigger word was mentioned multiple times, you can click on the Suggested by Ai answer multiple times to navigate through each moment of the transcript.
    6. Select Submit grade, or Submit & share

    Once the grade has been submitted, the supervisor can view the grade details in the Web Call History view.

    Each scorecard has an Overall Searchable comment field, which also displays the detailed exports. Comments entered here are searchable in the call history as well as in analytics. 

    If a call has screen capture enabled, you can listen to the recording and view the screen capture within the call summary view. 

    Choose the screen you’d like to watch and press play (that’s the purple triangle), and you’ll be able to obtain the complete perspective of the call.

    Finding graded interactions

    Supervisors can search for graded interactions based on contact centers or agents.

    To easily find graded interactions:

    1. Navigate to Call History
    2. Select More Filters
    3. Select Ai Scorecards
    4. Choose your desired filters, then select Apply

    From the Ai Scorecard search filter, you can search by the following criteria:

    • Graded/Not Graded by (anyone): Displays calls that have been graded by you or a fellow supervisor, admin, or coach.  
      • Your ability to search for grades submitted by another user depends on your permissions.
    • Ai Scorecard: Displays calls graded by a specific scorecard.
    • Grade Score: Displays calls graded with a specific score range.
    • Search Overall Call Feedback: Displays calls based on the overall feedback noted. The search query must match the feedback entered when the grade was submitted.
    • Date Graded: Displays calls graded within a certain date range.
    • Grader: Displays calls graded by a specific grader.

    Graders can view all of their grades scheduled to be sent by selecting View Schedule on any call they have graded. 

    A pop-up will appear, noting which graded scorecards will be sent out on what date. 

    Select the drop-down menu to change between all agents or specific agents. 

    Note:
    Regardless of the selected search filter, all search filters are based on the date of the call.

    Delete a grade

    Need to delete a grade? No problem! However, what you are able to delete depends on your permissions. 

    • Office administrators can delete any grade in their office.
    • Supervisors can only delete grades that they submitted.

    Once a score has been deleted, it will be removed from the Web Call History and Scorecard Analytics data.

    To delete a grade, navigate to Admin Settings from Dialpad.com

    1. Navigate to Call History
    2. Select More Filters
    3. Select the Ai Scorecards 
    4. Choose your desired filter, then select Apply
    5. Select the grade to view the scorecard
    6. Select the trashcan icon at the bottom of the scorecard
    7. Select Confirm to confirm deletion 

    Share scores with agents

    Feedback is crucial to prevent habit formation. Supervisors can grade a call and share the score with the agent in real-time, or schedule the grade to be shared at a later date.

    Scores can be shared with the agent via email or Dialbot alert.

    To share a graded call:

    1. Select Share grade
    2. Choose to Share now or Share later
      1. Scheduled sharing sends the grade on the date of your choice, up to 30 days out
    3. Select your sharing method
      1. Grades can be shared via email, Dialbot alert or both
    4. Select Share now

    Once shared, the agent is notified of the grade, the grader, and the call. 

    Dialbot_share_score.png

    From the Call Summary page, the agent can view the details of the grade, broken down by question.

    Who can do what 

    There are 4 different Ai Scorecard permission levels:

    • Supervisor 
      • Coaches have the same QA Scorecard permissions as supervisors.
    • Contact center admin
    • Office admin
    • QA grader 

    Let's take a deeper look at user permissions and who can grade which calls. 

    PermissionRole
    Contact Center SupervisorContact Center AdminOffice Admin
    QA Grader
    Delete grade submitted by another supervisor  
    Delete grade submitted by another contact center admin  
    View basic scorecard details for grades submitted by another supervisor/admin (overall score, who graded it, and when)
    View basic scorecard details and per-question responses for grades submitted by another supervisor/admin 
    Create, duplicate, publish & archive Scorecards
    View analytics
    Export scorecard reports
    Grade contact center or coach team call that the user is monitoring
    Grade contact center or coaching team call the user is not monitoring  
    Edit grade of self-graded call 
    Edit grade submitted by another supervisor****
    Edit grade submitted by another contact center admin****
    Delete grade submitted by user (their own grade)
    Notes: 
    Users can have multiple permission types.
     
    You can only edit grades for calls that you have graded.
     
    Users can always view the full grades and scorecard details for interactions that they have graded. 

    QA Grader

    The QA Grader role contains three permission types:

    1. Scorecard access: All coaching teams - Provides the grader access to all coaching team calls across the entire company
    2. Scorecard access: Contact centers (office) - Provides the grader access to all contact centers calls within their office
    3. Scorecard access: Contact centers (company) - Provides the grader access to all contact center calls across the whole company and can only be assigned by the company (super) admin

    Scorecard analytics

    Dialpad provides supervisors and admins a clear overview of their team's and agent's performance with in-depth scorecard analytics. 

    Learn all about scorecard analytics in this handy Help Center article

    Frequently asked questions 

    How are scores calculated when a question is skipped?

    When a grader skips a question (if applicable), that response is excluded from the grade calculation. 

    For example, when John grades Mary’s call, he skipped 3 questions, totaling 30 points. The scorecard is worth 100 points and the agent received 60 points for the first 3 questions, 0/10 points on question 4, and then skipped questions 5 (5 pts), 6 (10 pts), 7 (5pts), and 8 (10 pts), totaling 30 points skipped. This results in a score of 60/70, which works out to 85.71 => 86%.

    Is screen recording available while grading a call?

    Not at this time.

    Is Ai scorecards automatically included in my plan?

    There is an additional monthly fee to use the Ai scorecard feature. 

    What languages do Ai scorecards support?

    Currently, Ai scorecards are available only in English. But stay tuned, we're adding more functionality soon! 

    Why don't I see Ai scorecards on the mobile app?

    Currently, Ai scorecards are available only on the Dialpad browser app and our desktop app. You will not see Ai scorecards on your Dialpad mobile apps. 

    Can I restrict visibility to my scorecards (so only I can see them)?

    Currently, there is no way to restrict visibility to Ai scorecards. When you create an Ai scorecard, it will be visible to all other supervisors, contact center admins, office admins, and QA graders.

    How do I add Ai scorecards to my account?

    There are 2 options to add Ai scorecards to your Dialpad account:

    1. Contact your Customer Support Manager 
    2. Contact Customer Support 

    Can I use Ai scorecards on any call?

    No. Ai scorecards can be used on contact center and coaching team calls only. 

    Coaching Team meeting calls must be graded with a 'traditional' scorecard, as AI-powered scorecards are not yet available for meetings.  

    Can I use Ai scorecards during my free trial?

    Not at this time. Ai scorecards are available on paid Ai sales and contact center plans. 

    Can I edit trigger words or phrases on a published scorecard?

    Yes, absolutely!

    Can I edit questions on a published Ai scorecard?

    Not at this time. You can only edit trigger words and phrases, not the scorecard questions. 

    Why aren’t all of my trigger words and phrases being answered by Ai? 

    The Ai QA question engine runs in real-time and matches against words and phrases immediately after they are spoken. Occasionally, these immediate words are updated a few seconds later, once more context is available (usually due to punctuation, time, date, currency, or  number formatting)

    Transcript adjustments sometimes lead to Ai questions being missed, even if the phrase is present in the final call transcript.

    For example, the real-time sentence “Thank you for calling office three six five support how may I help” is formatted to “Thank you for calling Office365 support. How may I help?” after the call is completed. 

    The phrase “How may I help” was missed as a separate sentence because is it transcribed in real-time as “how may I help”.

    Can we have Ai turned on to use Ai scorecards but not have the audio recorded (due to recording laws)?

    No. The Ai scorecards feature is built off of Ai scorecards and therefore still requires call recording.

    Why don't I see the option to rate a call in the web call history?

    The option to rate/grade a call (Ai scorecards) will only populate in the web call history if it's a recorded contact center call.

    If certain area codes should not be recorded, is it mandatory to add them to the exception list?

    Yes! Please see Add All-Party Consent States to Exception List for more information. 

    Once I’ve created a new Ai scorecard, can it analyze my agents' historical calls?

    No.  Once a Ai scorecard has been created and published it will begin to analyze all net new calls from the time the scorecard was published.


    Want to learn more?
    Check out Dialpad University's free courses covering our products and services, as well as insightful tips and best practices.

    Was this article helpful?