Microsoft Copilot is Microsoft’s AI assistant family, and Microsoft 365 Copilot is the paid version that works inside Word, Excel, PowerPoint, Outlook and Teams using an organisation’s own files and emails. Proving its value means tracking active use per seat, time saved on specific tasks, the quality of what it produces and whether that justifies the licence cost. Microsoft’s own usage reports give a starting point, but they do not measure value by themselves.
Which Copilot are you measuring?
The name covers several products. Copilot in Windows and the free Copilot app are general assistants. Microsoft 365 Copilot is licensed per user and grounded in Microsoft Graph, meaning it can read the user’s emails, chats, meetings and documents that they already have permission to see. Copilot Studio lets organisations build their own agents, and there are role-based versions for sales and service.
For a marketing team, the measurable investment is usually Microsoft 365 Copilot seats. The rest of this article focuses on that.
Why are seat licences hard to justify without data?
Copilot seats are a recurring cost on top of the existing Microsoft 365 subscription. Finance teams increasingly ask for evidence before renewing. Anecdotes such as “it summarised a meeting nicely” do not survive a budget review.
The other difficulty is that use is uneven. Some people use Copilot many times a day, others open it twice and forget. Averages hide both groups, so measurement must work at the level of people and tasks.
What do Microsoft’s built-in reports show?
Admins can see Copilot usage in the Microsoft 365 admin center under the usage reports, showing active users and activity by app. The Copilot Dashboard in Viva Insights adds readiness, adoption and impact views, including estimated hours of Copilot-assisted work, for organisations that meet its licensing requirements.
These tools answer “who uses it and where”. They do not answer “did the work get better or cheaper”. You need your own task-level measures for that.
Which KPIs prove value?
A useful set mixes adoption, efficiency and quality:
- Active users per week as a share of licensed seats.
- Actions per active user by app: Teams meeting recaps, Outlook drafts, Word drafts, Excel analyses, PowerPoint creation.
- Time per task before and after for three to five defined tasks.
- Quality score from a reviewer on a sample of Copilot-assisted outputs.
- Error incidents: wrong facts, wrong recipients, confidential content surfaced to the wrong person.
- Cost per active seat and cost per hour saved.
Cost per active seat is the number finance cares about. If half the seats are idle, the effective price per real user doubles.
How do you measure time saved honestly?
Pick tasks that repeat often and have a clear start and end. In marketing, good candidates are writing a campaign brief from meeting notes, producing a weekly performance summary from an Excel export, drafting a client update email and building a first-draft slide deck from a Word document.
Time a sample of each task without Copilot, then with it, done by the same people. Include review and correction time in the “with” figure. A short self-logging form in Microsoft Forms, filled in right after the task, is enough for a pilot. It is less precise than observation, but far better than a survey months later.
What should the dashboard look like?
One Power BI or Excel page, refreshed monthly, covers most needs:
- Seat utilisation: licensed, active, and inactive for 30 days.
- Use by app and by team.
- Time per task, before and after, for the selected tasks.
- Quality scores from spot checks.
- Incidents and their causes.
- Cost per active seat and estimated hours saved.
Add a short note each month on what changed, such as training sessions or new prompts shared. It explains jumps in the lines.
Which risks should the metrics surface?
Copilot can only see what the user can see, which means oversharing problems in SharePoint and Teams become visible quickly. A user asking about salaries may find a spreadsheet that was shared too widely years ago. Track these findings as incidents, and treat them as a reason to fix permissions with tools such as Microsoft Purview sensitivity labels, not as a reason to blame the assistant.
Accuracy is the other risk. Meeting recaps can attribute statements to the wrong person, and financial summaries can misread a column. Spot-check quality scores catch this before it reaches clients.
How Alien Road applies Microsoft Copilot measurement
In a typical engagement, discovery starts with the admin center data: how many seats exist, how many are active, and which apps see use. Interviews with a few heavy and light users then identify the tasks where Copilot helps and where it does not.
Setup focuses on three to five repeatable marketing tasks. The team writes shared prompt templates for them, runs a short training session and sets up the self-logging form for timing. Permissions on the main SharePoint sites are reviewed before rollout to reduce oversharing surprises.
Measurement uses the dashboard above. For example, a monthly report might show seat utilisation, median time for the weekly reporting task before and after, and quality scores from a reviewer. Where seats stay idle for two months, the report recommends reassigning them, which often improves the return more than any prompt tip.
How do you raise adoption without forcing it?
Usage grows when people see Copilot save them time on their own work. Short sessions that show one task, such as turning a Teams meeting into action items, work better than general training. Share prompts that worked inside a Teams channel so others can copy them.
Ask light users what stopped them. Common answers are not knowing what to ask, poor results on the first try, or worries about accuracy. Each has a practical fix: templates, examples and a review habit.
Where to start this week
Open the Copilot usage report in the Microsoft 365 admin center and list inactive seats. Then choose three tasks your marketing team repeats weekly and ask five people to time them, with and without Copilot, over the next two weeks. Those numbers, plus seat utilisation, are enough for a first value review and an informed decision about renewals.
Keep reading: AI Marketing Agency · AI Marketing