A lot of call scoring starts as a checklist. Ours did too, and it did not hold up. What works better is a written rubric, with evidence behind every score and a person to settle disputes.
Why a checklist stops working
Our first version scored seven skills as yes or no checkboxes. Did the rep handle the objection? Did they set a next step? Run that across real calls and the problem shows up quickly: a rep who acknowledges a budget objection and moves on did part of the job, and a rep who reframes it and checks the buyer accepted the reframe did a lot more. A checkbox gives both the same mark.
It also hides progress. A rep who gets steadily better inside one skill shows no movement until they cross the line, so the score cannot tell a manager whether coaching worked. We replaced the checkboxes with a graded scale. The rest of this guide is the method we run now.
Step 1: write the skills down
Start with the skills a good call in your sales motion needs, each written as one behavior you can hear on the recording. If two managers would read the description differently, rewrite it.
Big Sister scores 18 skills on every conversation. Some of the names you will see in the product are Ice Break, Agenda and Consent, Discovery Depth, Qualification Coverage, Empathy Signal, Next Step and Post-Sell. A few, such as Opinion Safety, are reverse-scored. A default set works out of the box, and it can be tuned to a team's industry and sales motion, because discovery looks different on a two-week deal and on a nine-month one.
If you are building your own, eight to twelve skills is a fine start. Leave out anything you cannot point to in a transcript, such as "confidence" or "energy", unless you can say what it sounds like.
Step 2: score each skill 1 to 5 against written examples
Each skill gets a 1 to 5 scale, and each level gets a written description and, ideally, a real quote from one of your calls. Here is our Next Step scale, from what 10,373 sales conversations showed about next steps:
- 1 the conversation ended with nothing
- 2 "I'll call you" or "reach out if you have questions"
- 3 an action with no date: "let's talk next week"
- 4 the seller named a date and time, the buyer did not confirm
- 5 date and time named, buyer said yes out loud
The written bar is what makes two scorers land in the same place. Without it, a 3 from one manager and a 3 from another mean different things, and the team stops believing the number.
Step 3: tie every score to the transcript
Every skill score should point to the line that earned it, with a timestamp. In Big Sister each score links to that moment in the transcript, so a rep can read why they got a 2 on Discovery Depth and a manager can check it in seconds.
This changes the coaching conversation. "Your discovery needs work" becomes "at 14:20 the buyer mentioned a second team and we moved on", which is something both people can replay and agree on.
Step 4: roll the skills into one 0-100 number
A single number lets you rank calls, compare reps and track a team week over week. Big Sister rolls the 18 skill scores into one 0-100 Sales Score per conversation, then per rep and per team.
Keep the breakdown one click away from the number. Use the Sales Score to decide where to look and the skill scores to decide what to coach. A rep at 58 with a 1 on Next Step needs a different conversation than a rep at 58 with a 2 on Qualification Coverage.
Step 5: let reps dispute, and have a person decide
Any rubric will be wrong sometimes, and reps know it. If they can't challenge a score, they stop reading the scores. In Big Sister any rep can dispute a score, and a person settles the dispute, looking at the same line in the transcript.
Disputes are also the cheapest way to find unclear wording in your rubric. When the same skill keeps getting challenged, the level descriptions usually need another example. Even trained raters disagree more than most managers expect when they score the same calls; we published our own numbers and their caveats in Sales skill has never had a real standard.
Step 6: score every call, reps and AI agents alike
Spot-checking a few calls a month shows you those calls well. Patterns in a rep's discovery or a team's next steps only show up when every call is scored the same way.
If your team also puts AI agents in front of buyers, score them on the same skills as the reps. Big Sister scores human reps and AI agents against the same 18-skill standard, so one report covers both and nobody argues about whose rubric applies.
Step 7: practice the weak moment
The replay coach is there so a rep can practice the same day a call is scored. It takes a weak moment from a real call and lets the rep replay it by voice against an AI version of the buyer, then scores each skill again. Replay scores stay with the rep and their manager and never change the Sales Score.
How to start by hand this week
You do not need a tool to start. Two people and a spreadsheet will tell you a lot in a week.
- Pick five recent calls per rep. Mix good and bad outcomes, and use the recordings you already have.
- Pick six skills from your sales process. For each, write one line on what it is and a 1 to 5 scale with a real quote for each level.
- Have two people score the same calls separately, then compare. Wherever they are two or more points apart, rewrite that level's description and add an example.
- Write the timestamp next to every score. If you can't find the line, leave that skill unscored for the call.
- Convert to 0-100 if you want one number. A simple way: take the average skill score, subtract 1, divide by 4 and multiply by 100, so an average of 3 is 50.
- Review with each rep line by line, and let them dispute. Note which skills get disputed most.
After two weeks you will know whether the rubric holds. The hard part is doing it for every call, every week, without the scorers drifting apart. That is the part we built Big Sister for: it reads calls and meetings from Gong, Fireflies, Granola, tl;dv, Fathom, Otter or any source by API key, and phone calls from Ringostat, and scores every one on 18 skills. The free plan scores 5 of your own calls a month, with no card.
Choosing a tool? We compared the options from each vendor's own site: Gong alternatives for teams of 5 to 30 reps, Big Sister vs Gong and Big Sister vs Hyperbound.
FAQ
How do you score a sales call?
Score each skill you expect on the call from 1 to 5 against a written example for every level, attach the transcript line that earned each score, and roll the skill scores into one number. Big Sister scores 18 skills and rolls them into a 0-100 Sales Score per call, per rep and per team.
What skills should a sales call scorecard include?
The skills your sales process depends on, each written as one behavior you can hear on the call. Big Sister's 18 skills include Ice Break, Agenda and Consent, Discovery Depth, Qualification Coverage, Empathy Signal, Next Step and Post-Sell, and the set can be tuned to a team's industry and sales motion.
Why score 1 to 5 instead of yes or no?
Almost nothing in a sales call is binary. A 1 to 5 scale with a written bar for each level shows partial work and shows a rep improving inside a skill, which a yes or no checkbox hides.
What is a Sales Score?
A single 0-100 number for a conversation, a rep or a team, built from 18 skill scores of 1 to 5. Each skill score links to the moment in the transcript that earned it, so the number can always be opened up and checked.
Should AI agents be scored the same way as human reps?
If an AI agent talks to your buyers, it is doing part of the sales job, so it makes sense to hold it to the same written standard. Big Sister scores human reps and AI agents against the same 18 skills.
Val Yaromenko is CEO and co-founder of Big Sister AI. Read The Manifesto.