LLM Applications
In recent years, LLM (Large Language Model) has been leading a technological revolution with its amazing language generation and understanding capabilities, and many enterprises are incorporating LLM into their businesses. LLM typically involves complex natural language processing tasks, but model operation is like a black box, making it difficult for external understanding and verification, and enterprises struggle to effectively monitor and optimize LLM applications.
For LLM, observability represents understanding the model's decision-making process from the outside. Through Prompt input/output interaction information, Token consumption, model response time, anomalies and errors, Agent workflow, and user feedback, we can achieve deep insights into the model's internal working mechanisms.
LLM observability not only improves model transparency but also helps identify and correct potential issues promptly, ensuring model safety and accuracy, reducing errors, increasing user trust, thus promoting artificial intelligence technology on a safe and sustainable development path.
LLM Application List
The LLM application list page displays information for all created LLM applications, including application name, average latency, user count, Session count, Traces count, and total Token usage. Members with permissions can perform edit and delete operations on applications.
Overview
Clicking on an application in the "Application List" takes you to the "Overview" page by default.
The top 4 charts show trend graphs for the current application's Session count, Trace count, average Trace response time, and Token usage.
The bottom 2 tables show performance data at the Trace level, specifically the Top 10 Traces by average response time and Top 10 Traces by total Token usage. Clicking on a Trace in the tables opens a new page locating that "Trace Details" page.
Session Analysis
Sessions
Sessions List
The top search box allows quick finding of corresponding Session data through Session ID.
The bottom data list displays, for each Session in the selected time period, the last access time, initial start time, duration, average response time, Traces count, and total Token usage. The data list is sorted by last access time in descending order by default.
Session Details
Clicking on a Session in the Sessions list opens a drawer page showing Session detail data.
The top shows current Session related information, such as start time, duration, Session ID, Traces count, Token usage, etc.
The bottom switches between different tabs to show different detailed information:
- Traces: Shows the Traces data information contained in the current Session, including each Trace's start time, type, response time, Token input count, Token return count, total Token usage, input content, and return content.
- Properties: View the associated field information collected for the current Session.
Traces
Traces List
The top search box allows quick finding of corresponding Trace data through Trace ID.
The bottom data list displays, for each Trace in the selected time period, the start time, type, response time, Token input count, Token return count, total Token usage, Input, and Output content. The data list is sorted by time in descending order by default.
Trace Details
Clicking on a Trace in the Traces list opens a drawer page showing Trace detail data.
The top shows current Trace related information, such as start time, response time, Trace ID, type, Token input count, Token return count, total Token usage, Session source, etc.
The bottom switches between different tabs to show different detailed information:
- Event Content: The left side shows the Input, Output content, and source data contained in the current Trace; the right side shows a tree diagram of different observation items contained in the current Trace and their response times and Token usage.
- Waterfall: Shows the execution distribution of different observation items contained in the current Trace.
- Properties: View the associated field information collected for the current Trace.