+ ~j@^RIHt^RIt^RIt^RIt^RIt^RIt^RIt^RIH t H t ^RIH t ^RI H t ^RIHtHtHtHtHtHt^RIHt^RIHt^R IHtHt^R IHtHt^R IH t H!t!^R IH"t"^R I#H$t$]'d^RI%H&t&]'!]PP!R^44t)]!R4t*] !RR44t+] !RR44t,] !RR44t-] !RR44t.]+],,]-,].,t/!RR]]*,4t0!RR4t1!RR4t2!RR 4t3!R!R"4t4!R#R$4t5!R%R&4t6!R'R(4t7]'d]8]9]&,]8]:] 3,3,t;]PxR),!R*4t=] R+R,l4t>R-R/R.R/llt?R#)0) annotationsN)Callable Generator)contextmanager) dataclass) TYPE_CHECKINGAnyGenericLiteralTypeVaroverload)trace)llm) function_toolutils) trace_typestracer) NOT_GIVEN NotGivenOr)is_given) SpeechHandle)AgentLIVEKIT_EVALS_VERBOSERun_Tc0]tRt^'t$R]R&RtR]R&RtR#)ChatMessageEventzllm.ChatMessageitemmessageLiteral['message']typeN__name__ __module__ __qualname____firstlineno____annotations__r __static_attributes__r!t/Users/mitch_tango/dev/rabbit-r1-livekit/agent/.venv/lib/python3.14/site-packages/livekit/agents/voice/run_result.pyrr's (D (r)rc0]tRt^-t$R]R&RtR]R&RtR#)FunctionCallEventzllm.FunctionCallr function_callLiteral['function_call']r r!Nr"r!r)r*r,r,-s %4D "4r)r,c0]tRt^3t$R]R&RtR]R&RtR#)FunctionCallOutputEventzllm.FunctionCallOutputrfunction_call_outputLiteral['function_call_output']r r!Nr"r!r)r*r0r03s  ,BD )Br)r0cD]tRt^9t$R]R&R]R&R]R&RtR]R &R tR #) AgentHandoffEventllm.AgentHandoffr Agent | None old_agentr new_agent agent_handoffLiteral['agent_handoff']r r!Nr"r!r)r*r4r49s %4D "4r)r4c]tRt^DtRR/RRllt]RRl4t]PRRl4t ]R R l4t R R lt R Rlt RRlt RRltRRltRRltRRltRRltRRltRtR#) RunResult user_inputNc$V^8dQhRRRRRR/#)r=z str | None output_typeztype[Run_T] | NonereturnNoner!)formats"r* __annotate__RunResult.__annotate__Es# > >j >FX >]a >r)c \4Vn\PR,!4VnWnW n.VnRVnRVn R#N) set_handlesasyncioFuture _done_fut _user_input _output_type_recorded_items _final_output_RunResult__last_speech_handle)selfr=r@s&$$r*__init__RunResult.__init__EsB:=%  -/%'/1+/9=!r)cV^8dQhRR/#)r?rAlist[RunEvent]r!)rCs"r*rDrEQs $ $ $r)c VP#)ar List of recorded run events in chronological order. This surface is intended for assertions in tests. Events may include `ChatMessageEvent`, `FunctionCallEvent`, `FunctionCallOutputEvent`, and `AgentHandoffEvent`. Use `RunResult.events` when validating what happened in a run instead of depending on lower-level session internals, room state, or raw media artifacts. )rOrRs&r*eventsRunResult.eventsPs###r)cV^8dQhRR/#)r?rA RunAssertr!)rCs"r*rDrE`s r)c \'dARP\VP44p\ RVP RV R24\ V4#)zy Provides an assertion helper for verifying the run events. Returns: RunAssert: Assertion interface for run events. z z + RunResult( user_input=`z` events: z ))lk_evals_verbosejoin_format_eventsrYprintrMr\)rR events_strs& r*expectRunResult.expect_sZ  !~dkk'BCJ ""&"2"2!34##-,/ r)cV^8dQhRR/#)r?rArr!)rCs"r*rDrEts""e"r)c VPP4'g \R4hVP'g \R4hVP#)z Returns the final output of the run after completion. Raises: RuntimeError: If the run is not complete or no output is set. Returns: Run_T: The final result output. z3cannot retrieve final_output, RunResult is not donezno final output)rLdone RuntimeErrorrPrXs&r* final_outputRunResult.final_outputssG~~""$$TU U!!!01 1!!!r)cV^8dQhRR/#)r?rAboolr!)rCs"r*rDrEs%%d%r)c 6VPP4#)z=Indicates whether the run has finished processing all events.)rLrgrXs&r*rgRunResult.dones~~""$$r)cV^8dQhRR/#)r?rAz'Generator[None, None, RunResult[Run_T]]r!)rCs"r*rDrEs))B)r)c @aRV3RllpV!4P4#)cV^8dQhRR/#)r?rAzRunResult[Run_T]r!)rCs"r*rD)RunResult.__await__..__annotate__s  #3 r)cd<"\P!SP4GRjxL S#L5irG)rJshieldrLrXsr* _await_impl(RunResult.__await__.._await_impls&..0 0 0K 1s $0.0) __await__)rRrusf r*rwRunResult.__await__s   }&&((r)c(V^8dQhRRRRRRRR/#) r?rr5r7r6r8rrArBr!)rCs"r*rDrEs,22'24@2MR2 2r)c VPP4'dR#\WVR7pVPVPP R7pVP PWT4R#)N)rr7r8 created_at)rLrgr4_find_insertion_indexrr|rOinsert)rRrr7r8eventindexs&$$$ r*_agent_handoffRunResult._agent_handoffsW >>   !tIV**ejj6K6K*L ##E1r)c V^8dQhRRRR/#)r?rz llm.ChatItemrArBr!)rCs"r*rDrEs66 66r)c VPP4'dR#RpVPR8Xd\VR7pM;VPR8Xd\ VR7pMVPR8Xd \ VR7pVeDVP VPPR7pVPPW24R#R#)Nr)rr-r1r{) rLrgr rr,r0r}rr|rOr~)rRrrrs&& r* _item_addedRunResult._item_addeds >>   !% 99 !$$/E YY/ )%40E YY0 0+6E  ..%**:O:O.PE  ' ' 5 r)c V^8dQhRRRR/#)r?handleSpeechHandle | asyncio.TaskrArBr!)rCs"r*rDrEs < <$? >    &! fl + +  + +D,<,< =  !:!:;r)c V^8dQhRRRR/#)r?rrrArlr!)rCs"r*rDrEs  &A d r)c WP9dR#VPPV4VPVP4\ V\ 4'dVP VP4R#)FT)rIdiscardremove_done_callbackrrr_remove_item_added_callbackrrs&&r*_unwatch_handleRunResult._unwatch_handlesY  & f%##D$=$=> fl + +  . .t/?/? @r)c V^8dQhRRRR/#)r?rz"SpeechHandle | asyncio.Task | NonerArBr!)rCs"r*rDrEs+MRVr)c \V\4'dWn\;QJd&RVP4F 'dK RM RM!RVP44'dVP 4R#R#)c3@"TFqP4xK R#5irG)rg).0rs& r* 1RunResult._mark_done_if_needed..s9={{}}=sFTN)rrrQallrI _mark_doners&&r*rRunResult._mark_done_if_neededsM fl + +(. % 394==933394==9 9 9 OO  :r)cV^8dQhRR/#r?rArBr!)rCs"r*rDrEs;;D;r)c  \P!\P4;_uu_4VPf'VP P R4RRR4R#VPPp\V\4'gVP'dn\WP4'gSVP P\RVPP R\V4P 244M=WnVP P R4MVP PV4RRR4R# +'giR#;i)NzExpected output of type , got ) contextlibsuppressrJInvalidStateErrorrQrL set_result_maybe_run_final_outputr BaseExceptionrN set_exceptionrhr#r rP)rRris& r*rRunResult._mark_dones  !:!: ; ;((0))$/< ;  44LLLlM::$$$Z FWFW-X-XNN00$6t7H7H7Q7Q6RS##' #5#>#>"?A*6&NN--d3,,\:%< ; ; ;s+E"=E B*E E% c V^8dQhRRRR/#)r?r|floatrAintr!)rCs"r*rDrEs  5 S r)c \\\VP444F:pVPV,PP V8:gK1V^,u# ^#)z Returns the index to insert an item by creation time. Iterates in reverse, assuming items are sorted by `created_at`. Finds the position after the last item with `created_at <=` the given timestamp. )reversedrangelenrOrr|)rRr|is&$ r*r}RunResult._find_insertion_indexsP%D$8$8 9:;A##A&++66*D1u <r))__last_speech_handlerLrPrIrNrOrM)r#r$r%r&rSpropertyrY functoolscached_propertyrcrirgrwrrrrrrr}r(r!r)r*r<r<Ds >4 > $ $&""$%)26 < ;*  r)r<c ]tRt^tRRlt]RRl4t]RRl4tRRltR R ltR8R R llt]RR /RRll4t ]RRl4t ]RRl4t ]RRl4t ]RRl4t RR /RRllt ]R] /RRll4t ]R] R] /R R!ll4t ]R"] R#] /R$R%ll4t ]R&] /R'R(ll4t R] R] R] R"] R#] R&] /R)R*llt R9R+R,llt R-R.lt R] R] /R/R0lltR] /R1R2lltR"] R#] /R3R4lltR&] /R5R6lltR7tR #):r\cV^8dQhRR/#)r? run_resultr<r!)rCs"r*rDRunAssert.__annotate__s  9 r)c 6VPVn^VnR#)rN)rY _events_list_current_index)rRrs&&r*rSRunAssert.__init__s&--r)c V^8dQhRRRR/#)r?rrrA EventAssertr!)rCs"r*rDrs9999r)c R#rGr!)rRrs&&r* __getitem__RunAssert.__getitem__s69r)c V^8dQhRRRR/#)r?sslicerAEventRangeAssertr!)rCs"r*rDrs<>> result.expect[0].is_message(role="user") >>> result.expect[-1].is_message(role="assistant") # Full range access >>> result.expect[:].contains_function_call(name="foo") # Partial range access >>> result.expect[0:2].contains_message(role="assistant") znth(z) out of range (total events: )rz# indices must be int or slice, not ) rrrrrr_raise_with_debug_infor TypeErrorr r#)rRrrYs&& r*rrs4 c5 ! !&&s+F#F#6 6 c3  Qws4,,--5s4#4#455++3%=c$BSBS>T=UUVW,t005tA ADz""##FtCyGYGYFZ [  r)cV^8dQhRR/#)r?rArr!)rCs"r*rDr!s r)c RpVP\VP48dVPR4WP,pV#)Tz&Expected another event, but none left.)rrrrrR__tracebackhide__rs& r*_current_eventRunAssert._current_event!sC    #d&7&7"8 8  ' '(P Q(() r)Nc$V^8dQhRRRRRR/#)r?rstrr int | NonerArBr!)rCs"r*rDr*s'SScS*SPTSr)c RpVf VPMTpRP\VPVR74p\ V R2V,4h)T )selected_indexz Context around failure: )rr_r`rAssertionError)rRrrr marker_indexrbs&&& r*r RunAssert._raise_with_debug_info*sM .3mt** YY~d.?.?P\]^ y(CDzQRRr)r c V^8dQhRRRR/#)r?r rBrArr!)rCs"r*rDr2sBB$B+Br)c R#rGr!rRr s&$r* next_eventRunAssert.next_event1s?Br)c V^8dQhRRRR/#)r?r rrAChatMessageAssertr!)rCs"r*rDr5sOO"4O9JOr)c R#rGr!rs&$r*rr4sLOr)c V^8dQhRRRR/#)r?r r.rAFunctionCallAssertr!)rCs"r*rDr8VV":V?QVr)c R#rGr!rs&$r*rr7SVr)c V^8dQhRRRR/#)r?r r2rAFunctionCallOutputAssertr!)rCs"r*rDr;scc"AcF^cr)c R#rGr!rs&$r*rr:s`cr)c V^8dQhRRRR/#)r?r r:rAAgentHandoffAssertr!)rCs"r*rDr>rr)c R#rGr!rs&$r*rr=rr)c V^8dQhRRRR/#)r?r zSLiteral['message', 'function_call', 'function_call_output', 'agent_handoff'] | NonerAzdEventAssert | ChatMessageAssert | FunctionCallAssert | FunctionCallOutputAssert | AgentHandoffAssertr!)rCs"r*rDr@s ***   *r)c bRpVP4pV;P^, unVe"VP4PV8XgKOTR8XdTP 4#TR8XdTP 4#TR8XdTP 4#TR8XdTP4#T#)a Advance to the next event, optionally filtering by type. Args: type (str, optional): Event type to match. Returns: EventAssert or subclass: Assertion object for the matched event. Example: >>> result.expect.next_event(type="function_call").is_function_call(name="foo") Trr-r1r9)rrrr is_messageis_function_callis_function_call_outputis_agent_handoff)rRr r ev_asserts&$ r*rr@s0!++-I   1 $ |y055= 9 '') ) _ $--/ / + +446 6 _ $--/ /r)rolec$V^8dQhRRRRRR/#)r?r rrNotGivenOr[llm.ChatRole]rAzChatMessageAssert | Noner!)rCs"r*rDrms$'')'1I' !'r)c R#rGr!)rRr rs&$$r*skip_next_event_ifRunAssert.skip_next_event_ifls$'r)name argumentsc(V^8dQhRRRRRRRR/#) r?r r.rNotGivenOr[str]rNotGivenOr[dict[str, Any]]rAzFunctionCallAssert | Noner!)rCs"r*rDrrs4(('( ( . ( # (r)c R#rGr!)rRr rrs&$$$r*rrqs%(r)outputis_errorc(V^8dQhRRRRRRRR/#) r?r r2r rr NotGivenOr[bool]rAzFunctionCallOutputAssert | Noner!)rCs"r*rDr{s4.... . # . ) .r)c R#rGr!)rRr r r s&$$$r*rrzs+.r)new_agent_typec$V^8dQhRRRRRR/#)r?r r:rNotGivenOr[type[Agent]]rAzAgentHandoffAssert | Noner!)rCs"r*rDrs$((/(AX( "(r)c R#rGr!)rRr rs&$$r*rrs%(r)c8V^8dQhRRRRRRRRR RR R R R RR/#)r?r zLLiteral['message', 'function_call', 'function_call_output', 'agent_handoff']rrrrrrr r r rrrAz]ChatMessageAssert | AgentHandoffAssert | FunctionCallAssert | FunctionCallOutputAssert | Noner!)rCs"r*rDrs]:1:1[:1' :1  :1 . :1 :1#:10:1 :1r)c RpRp VR8Xd"VP4PVR7p MwVR8Xd"VP4PW4R7p MOVR8Xd"VP4PWVR7p M'VR 8Xd!VP4P VR 7p V;P ^, unV # \ dR#i;i) a Conditionally skip the next event if it matches criteria. Args: type (str): Type of event to check. role (ChatRole, optional): Required role for message events. name (str, optional): Required function name for calls. arguments (dict, optional): Required args for function calls. output (str, optional): Required output for function call outputs. is_error (bool, optional): Required error flag for call outputs. new_agent_type (type, optional): Required agent class for handoffs. Returns: EventAssert or None: The skipped event assertion if matched. Example: >>> skipped = result.expect.skip_next_event_if(type="message", role="assistant") TNrrr-rrr1r r r9r)rrrrrrrrh) rRr rrrr r rrevs &$$$$$$$ r*rrsF!  y ((*5545@(((*;;;[//((*BB&Bd(((*;;>;Z   1 $ I  sB;C CCc V^8dQhRRRR/#)r?countrrAr\r!)rCs"r*rDrss9r)c Rp\V4FXpVP\VP48dVP RV RV R24V;P^, unKZ V#)z Skip a specified number of upcoming events without assertions. Args: count (int): Number of events to skip. Returns: RunAssert: Self for chaining. Example: >>> result.expect.skip_next(2) TzTried to skip z event(s), but only z were available.)rrrrr)rRrrrs&& r* skip_nextRunAssert.skip_nextsm!uA""c$*;*;&<<++$UG+?sBRS   1 $    r)cV^8dQhRR/#rr!)rCs"r*rDrsr)c RpVP\VP48dGVPVP,pVPR\ V4P 24R#R#)z Assert that there are no further events. Raises: AssertionError: If unexpected events remain. Example: >>> result.expect.no_more_events() Tz$Expected no more events, but found: N)rrrrr r#rs& r*no_more_eventsRunAssert.no_more_eventssa!   T%6%6!7 7%%d&9&9:E  ' '6tE{7K7K6LM  8r)c$V^8dQhRRRRRR/#r?rrrrrArr!)rCs"r*rDrs/NNN. N  Nr)c 8RpVR,PWR7#)a1 Assert existence of a function call event matching criteria. Args: name (str, optional): Function name to match. arguments (dict, optional): Arguments to match. Returns: FunctionCallAssert: Assertion for the matching call. Example: >>> result.expect.contains_function_call(name="foo") TNNNr)contains_function_call)rRrrrs&$$ r*r' RunAssert.contains_function_calls"&!Aw--4-MMr)c V^8dQhRRRR/#r?rrrArr!)rCs"r*rDrs 33'3  3r)c 8RpVR,PVR7#)z Assert existence of a message event matching criteria. Args: role (ChatRole, optional): Role to match. Returns: ChatMessageAssert: Assertion for the matching message. Example: >>> result.expect.contains_message(role="user") Tr&r)contains_messagerRrrs&$ r*r,RunAssert.contains_messages""!Aw''T'22r)c$V^8dQhRRRRRR/#r?r rr r rArr!)rCs"r*rDrs/WW W# W " Wr)c 8RpVR,PWR7#)aL Assert existence of a function call output event matching criteria. Args: output (str, optional): Output string to match. is_error (bool, optional): Error flag to match. Returns: FunctionCallOutputAssert: Assertion for the matching output. Example: >>> result.expect.contains_function_call_output(is_error=True) Tr&r)contains_function_call_outputrRr r rs&$$ r*r2'RunAssert.contains_function_call_outputs"&!Aw44F4VVr)c V^8dQhRRRR/#r?rrrArr!)rCs"r*rDr.s MM!8M Mr)c 8RpVR,PVR7#)a Assert existence of an agent handoff event matching criteria. Args: new_agent_type (type, optional): Expected new agent class. Returns: AgentHandoffAssert: Assertion for the matching handoff. Example: >>> result.expect.contains_agent_handoff(new_agent_type=MyAgent) Tr&r)contains_agent_handoffrRrrs&$ r*r8 RunAssert.contains_agent_handoff.s"!Aw--^-LLr))rrrG))r#r$r%r&rSr rrrrrrrr!r'r,r2r8r(r!r)r*r\r\s 99 <<* XSBBB OO VV cc VV* *X'LU''(!* ( 1: ((.#, . &/ ..([d((:1*3 :1 !* :1 1: :1#,:1&/:13<:1x0$N!*N1: N,3*33(W#,W&/ W,M;DMMr)r\c]tRtRtRRRlltRRltRRltR]R ]/R R lltR ]R ]/RRllt R]/RRllt R]/RRllt Rt R#)riAc$V^8dQhRRRRRR/#)r?rRunEventparentr\rrr!)rCs"r*rDEventAssert.__annotate__Bs!h #r)c *WnW nW0nR#rG_event_parent_indexrRrr?rs&&&&r*rSEventAssert.__init__B   r)c V^8dQhRRRR/#r?rrrArBr!)rCs"r*rDr@GHHcHdHr)c VRpVPPWPR7R#TrNrDrrErRrrs&& r*_raiseEventAssert._raiseG!  ++G;;+Gr)cV^8dQhRR/#)r?rAr>r!)rCs"r*rDr@Ksxr)c VP#rGrCrXs&r*rEventAssert.eventK {{r)rrc$V^8dQhRRRRRR/#r$r!)rCs"r*rDr@Ns/%J%J%J. %J  %Jr)c  Rp\VP\4'gVPR4\VP\4'gQh\ V4'd\VPP P V8wd7VPRV RVPP P R24\ V4'd\P!VPP P4pVP4FCwrVWT9gWE,V8wgKVPRV RV RVPV4 24KE \VPVPVP4#) a Verify this event is a function call with matching details. Args: name (str, optional): Expected function name. arguments (dict, optional): Expected call arguments. Returns: FunctionCallAssert: Assertion for the function call. Raises: AssertionError: If the event is not a function call or details mismatch. Example: >>> ev_assert.is_function_call(name="foo", arguments={"x": 1}) TzExpected FunctionCallEventzExpected call name '', got ''z For key 'z ', expected r)rrCr,rPrrrjsonloadsritemsgetrrDrE)rRrrractualrvalues&$$ r*rEventAssert.is_function_callNs ,!$++'899 KK4 5$++'89999 D>>dkk..33t; KK.tfHT[[=M=M=R=R>> ev_assert.is_function_call_output(output="OK", is_error=False) Tz Expected FunctionCallOutputEventzExpected output 'rZr[zExpected is_error=r) rrCr0rPrrr r rrDrEr3s&$$ r*r#EventAssert.is_function_call_outputus&!$++'>?? KK: ;$++'>???? F   0 0 7 76 A KK+F88DKK>> ev_assert.is_message(role="assistant") TzExpected ChatMessageEventzExpected role 'rZr[) rrCrrPrrrrrDrEr-s&$ r*rEventAssert.is_messages !$++'788 KK3 4$++'78888 D>>dkk..33t; KK/$x 8H8H8M8M7NaP Q dllDKKHHr)rc V^8dQhRRRR/#r6r!)rCs"r*rDr@s JJ!8J Jr)c Rp\VP\4'gVPR4\VP\4'gQh\ V4'dp\VPP V4'gJVPRVP R\VPP 4P R24\VPVPVP4#)aD Verify this event is an agent handoff. Args: new_agent_type (type, optional): Expected new agent class. Returns: AgentHandoffAssert: Assertion for the handoff. Raises: AssertionError: If the event is not an agent handoff or type mismatch. Example: >>> ev_assert.is_agent_handoff(new_agent_type=MyAgent) TzExpected AgentHandoffEventzExpected new_agent 'rZr[) rrCr4rPrr8r#r rrDrEr9s&$ r*rEventAssert.is_agent_handoffs$!$++'899 KK4 5$++'89999 N # #Jt{{7L7Ln,],] KK&~'>'>&?xT[[MbMbHcHlHlGmmno "$++t||T[[IIr)rCrErDN)) r#r$r%r&rSrPrrrrrrr(r!r)r*rrAsk H%J!*%J1: %JNP+4PS\P@IYI6J;DJJr)rcz]tRtRtRRltR]R]/RRlltR]/R R lltR ]R ]/R RlltR]/RRllt Rt R#)ric$V^8dQhRRRRRR/#)r?rYrVr?r\rngrr!)rCs"r*rDEventRangeAssert.__annotate__s!~yur)c *WnW nW0nR#rG_eventsrD_rng)rRrYr?rps&&&&r*rSEventRangeAssert.__init__s   r)rrc$V^8dQhRRRRRR/#r$r!)rCs"r*rDrqs* * * *. *  *r)c Rp\VP4FwrE\YPPVPP ;'g^V,4p\ P!\4;_uu_4VPWR7uuRRR4u# VPPRVP: 24\R4h +'giK;i)a Assert that a function call matching criteria exists in the event range. Args: name (str, optional): Expected function name. arguments (dict, optional): Expected call arguments. Returns: FunctionCallAssert: Assertion for the matched function call. Raises: AssertionError: If no matching function call is found in range. Example: >>> result.expect[0:3].contains_function_call(name="foo") TrNz8No FunctionCallEvent satisfying criteria found in range unreachable) enumeratertrrDrustartrrrrrrh)rRrrridxr candidates&$$ r*r''EventRangeAssert.contains_function_calls,! .GC#B tyy7K7K!s6RSI$$^44 11t1Q54/ ++Ftyym T =)) 544 :C C rc V^8dQhRRRR/#r*r!)rCs"r*rDrqs **'*  *r)c Rp\VP4Fwr4\Y@PVPP ;'g^V,4p\ P!\4;_uu_4VPVR7uuRRR4u# VPPRVP: 24\R4h +'giK;i)aR Assert that a message matching criteria exists in the event range. Args: role (ChatRole, optional): Expected sender role. Returns: ChatMessageAssert: Assertion for the matched message. Raises: AssertionError: If no matching message is found in range. Example: >>> result.expect[:2].contains_message(role="assistant") TrNz5No ChatMessageEvent matching criteria found in range ry) rzrtrrDrur{rrrrrrh)rRrrr|rr}s&$ r*r,!EventRangeAssert.contains_messages(! .GC#B tyy7K7K!s6RSI$$^44 +++654/ ++CDII= Q =)) 544rr r c$V^8dQhRRRRRR/#r0r!)rCs"r*rDrqs* * *  *# * " *r)c Rp\VP4FwrE\YPPVPP ;'g^V,4p\ P!\4;_uu_4VPWR7uuRRR4u# VPPRVP: 24\R4h +'giK;i)a Assert that a function call output matching criteria exists in the event range. Args: output (str, optional): Expected output text. is_error (bool, optional): Expected error flag. Returns: FunctionCallOutputAssert: Assertion for the matched output. Raises: AssertionError: If no matching output is found in range. Example: >>> result.expect[1:4].contains_function_call_output(is_error=True) TrNz>> result.expect[0:3].contains_agent_handoff(new_agent_type=MyAgent) TrNz6No AgentHandoffEvent matching criteria found in range ry) rzrtrrDrur{rrrrrrh)rRrrr|rr}s&$ r*r8'EventRangeAssert.contains_agent_handoff:s$! .GC#B tyy7K7K!s6RSI$$^44 111P54/ ++DTYYM R =)) 544rrsN) r#r$r%r&rSrr'r,r2r8r(r!r)r*rrs] *!* *1: *D**3*@ *#, *&/ *D*;D**r)rcp]tRtRtRRltRRltRRlt]P!R4R R l4t R t R #) riYc$V^8dQhRRRRRR/#)r?rrr?r\rrr!)rCs"r*rDChatMessageAssert.__annotate__Zs". #r)c *WnW nW0nR#rGrBrFs&&&&r*rSChatMessageAssert.__init__ZrHr)c V^8dQhRRRR/#rJr!)rCs"r*rDr_rKr)c VRpVPPWPR7R#rMrNrOs&& r*rPChatMessageAssert._raise_rRr)cV^8dQhRR/#)r?rArr!)rCs"r*rDrcs'r)c VP#rGrUrXs&r*rChatMessageAssert.eventcrWr)judge_evaluationc$V^8dQhRRRRRR/#)r?llm_vzllm.LLMintentrrArr!)rCs"r*rDrgs'|||S|=N|r)c a"Rp\P!4pVPPPpVP \ PR4VP \ PSP4VP \ PR4VP \ P\P!RVRV/44V'gVPR4\R4hV'gVPR4\R4h\ R R l4p\"P$!4pVP'R R R 7VP'RRV RV 2R 7RpRp /p R.p \(;QJdV3RlV 4F 'gK RM RM!V3RlV 44'gRV R&SP+VV.RRRRR//V R7RjxL p V P,e V P,p V P.'gK8V P.P0'gKVV P.P0^,p V P2pKL|D}T'gTPR4\5T\64'gQh\8P:!YhR7wrT!T/TBGRjxL wppTP \ P<T'*4TP \ P>T4T 'dTPA\ PBT PD\ PFT PH\ PJT PD\ PLT PH\ PNT PP/4T'gTPRT 24T#\R'd4^RI*H+pT!TPYR R!4^R"R#7p\[R$T R%T R&24T#5i)'a9 Evaluate whether the message fulfills the given intent. Args: llm_v (llm.LLM): LLM instance for judgment. intent (str): Description of the expected intent. Returns: ChatMessageAssert: Self for chaining further assertions. Example: >>> await msg_assert.judge(llm, intent="should ask for size") TjudgerrrzThe chat message is empty.ryz(Intent is required to judge the message.c$V^8dQhRRRRRR/#)r?successrlreasonrrAztuple[bool, str]r!)rCs"r*rD-ChatMessageAssert.judge..__annotate__s" # # #c #>N #r)c"W3#5i)z Determines whether the message correctly fulfills the given intent. Args: success: Whether the message satisfies the intent. reason: A concise explanation justifying the result. r!)rrs&&r* check_intent-ChatMessageAssert.judge..check_intents? "ssystemanYou are a test evaluator for conversational agents. You will be shown a message and a target intent. Determine whether the message accomplishes the intent. Only respond by calling the `check_intent(success: bool, reason: str)` function with your final judgment. Be strict: if the message does not clearly fulfill the intent, return `success = False` and explain why.)rcontentuserzCCheck if the following message fulfills the given intent. Intent: z Message: Nzgpt-5c3@<"TFqSP9xK R#5irG)model)rexcluded_modelrs& r*r*ChatMessageAssert.judge..scGb^U[[0GbsFg temperaturer functionrr)chat_ctxtools tool_choice extra_kwargsz0LLM did not return any arguments for evaluation.)fncjson_argumentszJudgement failed: )shortenrz\nz...)width placeholderz- Judgment succeeded for `z`: ``).rget_current_spanrCr text_content set_attributerATTR_GEN_AI_OPERATION_NAMEATTR_GEN_AI_REQUEST_MODELrATTR_FUNCTION_TOOL_NAMEATTR_FUNCTION_TOOL_ARGSr\dumpsrPrhrr ChatContext add_messageanychatusagedelta tool_callsrrr llm_utilsprepare_function_argumentsATTR_FUNCTION_TOOL_IS_ERRORATTR_FUNCTION_TOOL_OUTPUTset_attributesATTR_GEN_AI_USAGE_INPUT_TOKENS prompt_tokensATTR_GEN_AI_USAGE_OUTPUT_TOKENScompletion_tokens#ATTR_GEN_AI_USAGE_INPUT_TEXT_TOKENS$ATTR_GEN_AI_USAGE_OUTPUT_TEXT_TOKENS%ATTR_GEN_AI_USAGE_INPUT_CACHED_TOKENSprompt_cached_tokensr^textwraprreplacera)rRrrr current_span msg_contentrrrrrexcluded_models_temperaturechunktoolfnc_args fnc_kwargsrrr print_msgs&f$ r*rChatMessageAssert.judgefsv!--/ kk&&33 "";#I#I7S"";#H#H%++V"";#F#FHZ[""  / / JJ&)[A B  KK4 5}- - KKB C}- -  #  #??${   "8$(M+  !% ,0 '.i#scGbcssscGbccc*-L '!::.Z&.9QR% &  + +% {{& ;;;{{%%%{{--a0 NN  +  KKJ K)S))))(CC !-h E* EEE"";#J#JPWKX"";#H#H&Q   ' '>>@S@S??AXAXCCUEXEXDDeF]F]EEuGaGa   KK,VH5 6   ( 3 3D% @X]^I .ykfXQG H svDP A?P P P>!PI"#I $I"'-PP5+P I""P+A P8J;9APBP)!P 5PrlN) r#r$r%r&rSrPrrstart_as_current_spanrr(r!r)r*rrYs4 H !!"45|6|r)rc.]tRtRtRRltRRltRtR#)ric$V^8dQhRRRRRR/#)r?rr,r?r\rrr!)rCs"r*rDFunctionCallAssert.__annotate__"/3r)c *WnW nW0nR#rGrBrFs&&&&r*rSFunctionCallAssert.__init__rHr)cV^8dQhRR/#)r?rAr,r!)rCs"r*rDr(r)c VP#rGrUrXs&r*rFunctionCallAssert.eventrWr)rlNr#r$r%r&rSrr(r!r)r*rr r)rc.]tRtRtRRltRRltRtR#)ric$V^8dQhRRRRRR/#)r?rr0r?r\rrr!)rCs"r*rD%FunctionCallOutputAssert.__annotate__s#5yQTr)c *WnW nW0nR#rGrBrFs&&&&r*rS!FunctionCallOutputAssert.__init__rHr)cV^8dQhRR/#)r?rAr0r!)rCs"r*rDrs.r)c VP#rGrUrXs&r*rFunctionCallOutputAssert.eventrWr)rlNrr!r)r*rrrr)rc.]tRtRtRRltRRltRtR#)ric$V^8dQhRRRRRR/#)r?rr4r?r\rrr!)rCs"r*rDAgentHandoffAssert.__annotate__rr)c *WnW nW0nR#rGrBrFs&&&&r*rSAgentHandoffAssert.__init__rHr)cV^8dQhRR/#)r?rAr4r!)rCs"r*rDrrr)c VP#rGrUrXs&r*rAgentHandoffAssert.eventrWr)rlNrr!r)r*rrrr)r MockToolsagents_mock_toolsc$V^8dQhRRRRRR/#)r?agentz type[Agent]mockszdict[str, Callable]rAzGenerator[None, None, None]r!)rCs"r*rDrD s#**k**=*B]*r)c#"\P/4p/VCW/Cp\PV4pRx\PV4R# \PT4i;i5i)z Temporarily assign a set of mock tool callables to a specific Agent type within the current context. Usage: with mock_tools(MyAgentClass, {"tool_name": mock_fn}): # inside this block, MyAgentClass will see the given mocks N)_MockToolsContextVarr_rHreset)rrcurrentupdatedtokens&& r* mock_toolsr s[#&&r*G''%'G $ $W -E* ""5)""5)s2A*AA*A''A*rc$V^8dQhRRRRRR/#)r?rYrVrrrAz list[str]r!)rCs"r*rDrDs">jT]r)c .p\V4Fwr4RpVe W18XdRMRp\V\\\34'dDVP P RR0R mR7pV RV RVPP RV R2pMC\V\4'd$V RV R VP R VP R2pM V RV RV 2pVPV4K V#) z>>>z T) exclude_noneexclude_defaultsexcludez [z] z(item=rz] AgentHandoffEvent(old_agent=z , new_agent=>idr call_idr|) rzrrr,r0r model_dump __class__r#r4r7r8append)rYrlinesrrprefix item_reprlines&$ r*r`r`sEf%  %1UuF e.0ACZ[ \ \ --!!%?.I XRs"U__%=%=$>fYKqQD 0 1 1("QC "__-\%//9J!M  XRs"UG,D T)&, Lr))@ __future__rrJr contextvarsrr\oscollections.abcrrr dataclassesrtypingrr r r r r opentelemetryrrrrrr telemetryrrtypesrrr speech_handlerrrrgetenvr^rrr,r0r4r>r<r\rrrrrrdictr rr ContextVarrrr`r!r)r*r s" /%! 3+)'ryy!8!<= )) )  55 5  CC C  55 5 / /2I IL] ]ddNSMSMl LJLJ^F*F*RJJZT%[$sH}"556I"--k:;NO**"4r)