Ch 7 — Sampling & Elicitation

Server-initiated requests: asking the LLM to think and the user to decide
IndexUnder the Hood →
High Level
-
Click play or press Space to begin...
Step- / 8
AThe Reverse FlowNormally clients call servers — but sometimes servers need to call back
1
dns
ServerNeeds LLM help
or user input
requests
hub
HostMediates all
reverse requests
routes
smart_toy
LLM / UserProvides response
back to server
2
arrow_downward Sampling: server asks the LLM to generate text
BSamplingServer requests the host's LLM to generate a completion
dns
ServerSends messages +
model preferences
createMessage
hub
Host ReviewsMay modify, approve,
or reject
sends to
3
psychology
LLMGenerates response
returned to server
4
arrow_downward Why sampling? Agentic workflows inside servers
CSampling Use CasesWhy a server would need to call the LLM
auto_fix_high
Multi-StepServer runs agentic
loop with LLM
translate
SummarizeServer asks LLM
to process data
rule
ClassifyServer uses LLM
for decisions
5
arrow_downward Elicitation: server asks the user for input
DElicitationServer requests structured input directly from the user
dns
ServerNeeds a decision
or confirmation
elicit
hub
Host Shows UIPresents form
to the user
responds
6
person
User AnswersFills in form
or cancels
7
arrow_downward Host as gatekeeper: security for reverse requests
EHost as GatekeeperThe host mediates all server-initiated requests
shield
ReviewHost inspects
the request
decide
edit_note
ModifyMay change messages
or parameters
or
block
RejectCan deny the
request entirely
8
arrow_downward Sampling vs elicitation: when to use which
FSampling vs ElicitationTwo reverse flows for different needs
psychology
SamplingAsk the LLM
to think
vs
person
ElicitationAsk the user
to decide
1
Detail