Skip to main content

Token Consumption Feed

Token consumption is a critical metric in systems where resources are billed or allocated based on token usage, especially in environments involving AI models. Monitoring token consumption allows organizations to keep track of resource usage, optimize costs, and ensure that their agents are operating efficiently. The Token Consumption Feed provides detailed insights into how tokens are being consumed across various projects, models, and agents, offering transparency into usage patterns and helping identify areas for cost optimization.

Key Metrics in the Token Consumption Feed​

The Token Consumption Feed provides a range of important metrics related to token usage:

  • Model Provider: Identifies the provider of the model being used. This is important for understanding which model providers are contributing to token consumption and can help optimize costs across providers.
  • Model Name: Specifies the name of the model being used in the token-consuming operation. Knowing which specific model is in use helps track consumption patterns for different models and aids in performance optimization.
  • Project: Shows the project associated with the token consumption activity. This metric enables users to track how resources are consumed within different projects, helping identify areas where resource usage may need to be controlled or optimized.
  • Agent Name: Displays the name of the agent that performed the token-consuming operation. This is useful for understanding how different agents contribute to overall token consumption and for optimizing agent performance.
  • Total Spend: Represents the total dollar amount for the operation. This metric provides an overview of the total cost for a given action or series of actions, allowing for easy cost analysis.
  • Input Spend: Reflects the dollar amount to process the input during the operation. This is critical for understanding the cost associated with feeding data into models or systems.
  • Output Spend: Captures the dollar amount used to process the output generated by the model or agent. This helps track the resource cost associated with the results of the operation.
  • Total Tokens: This is the total number of tokens consumed, combining both input and output tokens. It provides a holistic view of the resource usage in the operation.
  • Input Tokens: Displays the number of tokens consumed specifically for input data.
  • Output Tokens: Shows the number of tokens consumed for output processing. This helps to understand the cost associated with the generation or delivery of results.

Filtering Options​

To ensure you can tailor the report to your specific needs, the Token Consumption Feed provides filtering options by the following parameters:

  • Date: Filter by specific time periods to track token consumption trends over time, analyze spikes, or review historical consumption patterns.
  • Project: Focus on specific projects to analyze token consumption within a given scope or initiative, enabling targeted optimization.
  • Model Provider: Filter based on the model provider to assess how different providers are contributing to token consumption. This is particularly useful for managing costs across multiple providers.
  • Source: Narrow down the report based on the source of the token-consuming actions.
  • Credential Type: Filter based on the type of credential used for the operation.

These filtering options allow for more granular analysis and insights, ensuring you can focus on specific areas of token consumption and optimize where necessary.

Report Refresh and Export Features​

The Token Consumption Feed can be refreshed at any time to provide real-time insights into ongoing token usage. This allows for up-to-date monitoring of resource consumption, helping you stay on top of costs and performance in real time.

In addition, the report can be exported to CSV, making it easy to share or analyze the token consumption data offline. The CSV export feature is especially valuable for users who need to conduct further analysis in external tools.