Documentation
This is doc description
mconnect.ai provides comprehensive analytics and insights to help you understand your agent's performance, conversation patterns, and areas for improvement. The Analytics section offers visual graphs and charts, while the Insights section provides detailed information about individual conversations.
Overview
The Analytics page displays various metrics and visualizations to help you monitor your agent's performance over time. You can filter data by selecting a date range using the date picker at the top of the page.
Analytics Graphs
Conversation Volume
The Conversation Volume section provides an overview of conversation activity during the selected time period.
Top-Level Metrics
| Metric | Description |
|---|---|
| Total Chats | The total number of unique chat sessions initiated during the selected time period. |
| Total Messages | The total number of messages exchanged (both user and agent messages) during the selected time period. |
| Total Tokens | The total number of tokens consumed by the agent during conversations in the selected time period. |
Goal Completion Score
The Goal Completion Score measures how successfully the agent achieves its intended objectives in conversations.
Top-Level Metrics
| Aspect | Details |
|---|---|
| Score Range | 0.0 (0%) to 1.0 (100%) |
| Interpretation | Higher scores indicate better goal achievement |
| Charts | - Detailed Goal Completion Chart: Shows goal completion trends over time - Goal Completion Distribution Chart: Displays the distribution of goal completion scores across conversations |
How to Use: Monitor this score to understand if your agent is meeting user expectations and achieving its intended purpose. Lower scores may indicate a need to refine the agent's instructions or knowledge base.
Fallback Score
The Fallback Score measures how often the agent provides fallback responses based on criteria defined by the agent owner.
Top-Level Metrics
| Aspect | Details |
|---|---|
| Score Range | 0.0 (0%) to 1.0 (100%) |
| Interpretation | Lower scores indicate fewer fallback responses (better performance) |
| Fallback Examples | "I don't know", "I'm not sure", or similar responses indicating the agent cannot adequately address the query |
| Charts | - Detailed Fallback Chart: Shows fallback frequency trends over time - Fallback Distribution Chart: Displays the distribution of fallback scores across conversations |
How to Use: Track this score to identify when your agent is unable to answer questions. High fallback scores may indicate gaps in your knowledge base or the need to improve agent instructions.
Attention Score
The Attention Score measures chats that require agent owner attention, typically because they may represent potential future leads or require human intervention.
Top-Level Metrics
| Aspect | Details |
|---|---|
| Score Range | 0.0 (0%) to 1.0 (100%) |
| Interpretation | Higher scores indicate chats that immediately require agent owner attention |
| Criteria | Defined by the agent owner in the agent's analytics settings |
| Charts | - Detailed Attention Chart: Shows attention-worthy chat trends over time - Attention Distribution Chart: Displays the distribution of attention scores across conversations |
How to Use: Use this score to identify high-value conversations that may need follow-up, such as potential leads, complex queries, or conversations requiring human intervention.
Topic Cloud
The Topic Cloud displays the distribution of topics discussed in conversations during the selected time period.
Top-Level Metrics
| Aspect | Details |
|---|---|
| Visualization | Word cloud format where topic size represents frequency |
| Purpose | Identify the most common conversation themes and trending topics |
| Use Cases | - Content strategy planning - Knowledge base gap identification - Understanding user interests and needs |
How to Use: Review the topic cloud to understand what users are asking about most frequently. This can help you prioritize knowledge base updates and identify areas where your agent may need more information.
Insights
The Insights section provides detailed information about individual conversations, allowing you to review specific interactions and their performance metrics.
Accessing Insights
Navigate to the Insights tab in your agent's navigation menu. You can filter insights by:
- Date Range: Select a from date and to date to view insights within a specific time period
- Pagination: Navigate through multiple pages of insights
Insight Information
Each insight card displays the following information:
| Field | Description |
|---|---|
| Chat ID | Unique identifier for the conversation, with a lock/unlock icon indicating visibility (public/private) |
| Created By | The user who initiated the conversation |
| Goal Completion Score | Score indicating how well the agent achieved its objectives (0.0 to 1.0) |
| Attention Score | Score indicating if the conversation requires attention (0.0 to 1.0) |
| Fallback Score | Score indicating how often fallback responses were used (0.0 to 1.0) |
| Updated Time | Last update timestamp for the conversation |
| Attention Summary | A brief summary of the conversation's key points |
Score Color Coding
Scores are color-coded to quickly identify performance:
| Score Type | Green | Red |
|---|---|---|
| Goal Completion Score | Score ≥ base score (meeting or exceeding expectations) | Score < base score (below expectations) |
| Attention Score | Score ≥ base score (attention-worthy) | Score < base score (normal) |
| Fallback Score | Score ≤ base score (fewer fallbacks, better) | Score > base score (more fallbacks, needs improvement) |
Base Scores: Each agent has configurable base scores defined in the agent's analytics settings. These serve as benchmarks for comparison.
Insight Detail View
Clicking on an insight card opens a detailed view showing:
- Full Conversation Context: Complete information about the conversation
- Attention Reason: Detailed explanation of why the conversation received its attention score
- All Metrics: Complete breakdown of all scores with visual indicators
- Chat Link: Direct link to view the conversation in the chat interface
Using Insights
Insights help you:
- Identify Patterns: Review multiple insights to identify common issues or successful patterns
- Quality Assurance: Ensure your agent is performing as expected across different conversation types
- Follow-up Actions: Identify conversations that require human follow-up or attention
- Continuous Improvement: Use insights to refine agent instructions, update knowledge bases, and improve overall performance
Best Practices
For Analytics
- Regular Monitoring: Review analytics regularly to track trends and identify issues early
- Date Range Selection: Use appropriate date ranges to get meaningful insights (e.g., weekly, monthly comparisons)
- Score Interpretation: Understand that scores are relative to your agent's configuration and use case
- Trend Analysis: Look for trends over time rather than focusing on individual data points
For Insights
- Review High-Attention Insights: Prioritize reviewing insights with high attention scores
- Investigate Low Goal Completion: Examine insights with low goal completion scores to identify improvement opportunities
- Monitor Fallback Patterns: Review insights with high fallback scores to identify knowledge gaps
- Use Filters: Leverage date range filters to focus on specific time periods or events
Configuration
Setting Base Scores
Base scores for Goal Completion, Attention, and Fallback can be configured in the agent's settings:
- Navigate to your agent's Settings page
- Go to the Analytics section
- Set your desired base scores for:
- Goal Completion Score (default: 0.5)
- Attention Score (default: 0.5)
- Fallback Score (default: 0.5)
These base scores serve as benchmarks for color coding and comparison in the Insights section.
Setting Score Criteria
You can also configure the criteria for how scores are calculated by setting:
- Goal Completion Instruction: Instructions that define what constitutes successful goal completion
- Attention Instruction: Criteria for identifying conversations that require attention
- Fallback Instruction: Criteria for determining when fallback responses should be used
These instructions help the system accurately calculate and assign scores to conversations.