Most chat tools report volume: messages sent, sessions started, minutes engaged. All of it goes up whether the tool is helping or not — a bot that answers badly and gets asked five follow-ups looks busier than one that answers correctly first time.
Deflection is the number
The useful measure is what share of conversations finished without a human. A conversation counts as handled if nobody on your team was pulled in and the assistant didn't hit a gap in its knowledge. Everything else was escalated.
That single percentage tells you whether the thing is earning its keep. Eighty-something percent means most of your routine questions are gone. Forty percent means your knowledge base has holes.
Turn it into time
Multiply handled conversations by the minutes a person would have spent on each — six is a defensible, conservative figure for a short enquiry. That gives you hours saved, which is the version of the number a business owner can act on.
A lowballed estimate nobody disputes is worth more than a flattering one that gets waved away.
Then close the gaps
The escalated conversations are the valuable ones. Each is a question your content didn't answer. Add the answer, and next month that question is handled too — deflection is a number you improve deliberately, not one you hope for.