Ch 8 — Observability & Debugging
Under the Hood
10 Steps
1
Click Next to dive into the internals of LLM observability — callback handlers, run trees, context propagation, and how every span gets created.
Internals
BaseCallbackHandler
The event system that powers all tracing
1
webhook
Callback Events
on_llm_start, on_chain_start, on_tool_start…
fired by
settings
CallbackManager
Dispatches events to all registered handlers
received by
hub
LangSmith Handler
Converts events into Run objects → API
2
account_tree CallbackManager.get_child() — how parent→child propagation works
Propagation
Run ID & Parent-Child Linking
How the trace tree forms automatically
family_restroom
ParentRunManager
Holds run_id + parent_run_id
.get_child()
child_care
Child Manager
New run_id, parent_run_id = parent's id
inherits
content_copy
Handlers + Tags
Inheritable handlers propagate down
Data Format
Run (Span) Object
The exact fields stored for every operation
3
fingerprint
Identity
id, trace_id, parent_run_id, dotted_order
+
data_object
Payload
run_type, inputs, outputs, events, extra
+
timer
Metrics
start/end_time, tokens, cost, status
4
send Async batched POST — how runs reach the LangSmith API
Transport
Async Run Submission
Non-blocking delivery to the tracing backend
queue
Background Queue
Runs buffered in-memory, not blocking .invoke()
batched
cloud_upload
POST /runs/batch
Multipart payload: create + update in one call
stored
storage
LangSmith Backend
Indexed by trace_id, project, tags
Decorator
@traceable Internals
How non-LangChain code gets traced
5
code
@traceable
Wraps function, creates RunTree on call
uses
account_tree
RunTree
Tree node: id, parent, inputs, outputs
context
stacks
Context Var
Thread-local stack for auto-nesting
6
lan Distributed tracing — cross-service context propagation
Distributed
Cross-Service Tracing
Linking spans across microservices
output
run_tree.to_headers()
Serializes trace context into HTTP headers
HTTP
dns
Remote Service
Receives langsmith-trace + baggage headers
restores
input
tracing_context(parent=headers)
Reconstructs parent link on server side
Langfuse
@observe() Decorator Internals
How Langfuse builds the observation tree
7
code
@observe()
Wraps function, pushes onto observation stack
manages
stacks
OTEL Context
start_as_current_observation() sets active span
creates
layers
Trace / Span / Gen
Outermost = Trace, nested = Span or Generation
8
payments Cost calculation internals — model registry → token pricing
Cost Engine
Token Counting & Cost Calculation
How platforms compute cost per run
token
Usage from API
prompt_tokens, completion_tokens from LLM response
×
menu_book
Model Registry
Maps model name → $/input_token, $/output_token
=
calculate
Run Cost
prompt_cost + completion_cost = total_cost
Evals
Evaluation Pipeline Internals
How experiments and evaluators execute
9
dataset
Dataset Examples
List of {input, reference_output} dicts
map()
play_arrow
Target Function
Your agent/chain — called per example
scored
functions
Evaluator Fn
(run, example) → {key, score, comment}
Walkthrough
Full Trace: End-to-End Event Flow
Every callback event for one agent.invoke()
10
play_circle
on_chain_start
Root chain begins — trace_id assigned
→
psychology
on_llm_start/end
LLM call — tokens, latency captured
→
build
on_tool_start/end
Tool execution — input/output captured
→
stop_circle
on_chain_end
Root chain ends — trace complete