+ ~j(:^RIHtRtRtRt!RR]4tR#))PushToHubMixincRp\VP44pTPP pTR8Xd2^RIHpT!4'd^RIp\TR4PeR#T# \d\RT 24hi;i)zw Infers the device type from the model parameters. Args: model: The model instance. Returns: The device type. z { "RMSNorm": { "cuda": "kernels-community/layer_norm:LlamaRMSNorm", ... }, ... } zPCannot determine model device, please provide a device to the mapping. Example: cuda)is_torch_availableNversionrocm) next parameters StopIteration ValueErrordevicetypeutilsrtorchgetattrhip)modelEXAMPLE_MAPPINGparamdev_typerrs& u/Users/mitch_tango/dev/rabbit-r1-livekit/agent/.venv/lib/python3.14/site-packages/transformers/utils/kernel_config.py infer_devicers O U%%'( ||  H6.    ui(,,8 O!  ^_n^o p   s A--Bc^RIHpVR9d\RV 24hVPR4^,pVPR4^,pVW5!VVR7//W@&R#)LayerRepositoryBOnly cuda, rocm, xpu, npu, neuron and tpu devices supported, got: :)repo_id layer_nameNrrxpunpuneurontpu)kernelsrr split)r r repo_namemodecompatible_mappingrrepo_layer_namers&&&&& radd_to_mappingr,:sl' DD]^d]efggooc*1-Oooc"1%G /* &"c^RIHp^RIHpVR9d\ RV 24hVP R4^,pVP R4^,pVP R4R ,p VW6!V!V4V VR7//W@&R#) r)Path)LocalLayerRepositoryrr/) repo_path package_namer Nr!)pathlibr/r&r0r r') r r r(r)r*r/r0r+r2repo_package_names &&&&& radd_to_mapping_localr7Ks, DD]^d]efggooc*1-O$Q'I!,R0 &y/.* &"r-cNa]tRt^`toRtR RltR RltRtRtR Rlt Rt Vt R#) KernelConfigz] Kernel configuration class. This class is used to configure the kernel mapping for a model. Nc:VeTM/Vn/VnW nR#Nkernel_mappingregistered_layer_namesuse_local_kernel)selfr=r?s&&&r__init__KernelConfig.__init__es 0>0JnPR&(# 0r-cH^RIHpVWW!VVVR7//VPV&R#)rr)rr revisionN)r&rr=)r@rregistered_namer r r)rDrs&&&&&&& r update_kernelKernelConfig.update_kerneljs4+ o#)%0 O,r-cVP4F2wr#\VR4'gKVPVPV&K4 R#)kernel_layer_nameN) named_moduleshasattrrIr>)r@rnamemodules&& rstore_registered_layer_names)KernelConfig.store_registered_layer_namesws;!//1LDv2334:4L4L++D12r-c RpVPV4\VP\4'g%\ RV R\ VP4 24hVPP 4Fwr4W0PP49d\ RV R24h\V\4'd"RV9gRV9d\ RV R V 24hKi\V\4'dfVP 4FOwrVVR 9d\ R V 24h\V\4'dRV9g RV9gK@\ RV R V 24h K\ R V RV 24h R #)a Validates the kernel_mapping to ensure that: 1. Each layer_name in the mapping is registered in the model (i.e., the model contains a module with a matching kernel_layer_name). 2. Each kernel value is either a string of the form 'org/repo:layer_name' or a dict mapping device types ("cuda", "rocm", "xpu", "npu") to such strings. 3. Each device key in a dict is one of "cuda", "rocm", "xpu", or "npu". 4. Each repo_name is a valid repository and layer name in the format 'org/repo:layer_name' (i.e., a string containing both a slash and a colon). 5. If a local path is detected, it should be in the format '/abs/path:layer_name'. The absolute path must include the `package_name`, like "/home/user/layer_norm". Args: model: The model instance whose modules are checked for registered kernel_layer_name attributes. Raises: ValueError: If a layer_name is not registered in the model, if a device is not supported, or if a repo_name is not a valid 'org/repo:layer_name' string. a For single device form remote { "RMSNorm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, For multiple devices form remote { "RMSNorm": { "cuda": "kernels-community/layer_norm:LlamaRMSNorm", "rocm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, ... } For single device form local { "RMSNorm": "/abs/path:LlamaRMSNorm", ... }, For multiple devices form local { "RMSNorm": { "cuda": "/abs/path:LlamaRMSNorm", "rocm": "/abs/path:LlamaRMSNorm", ... }, ... } z7Kernel mapping must be a dict of the following format: z, got: zLayer z[ is not registered in the model, please register it first using use_kernel_forward_from_hubr1rzKernel mapping for 'zl' must be a valid repo name with a layer name (e.g., 'org/repo:layer_name' or '/abs/path:layer_name'), got: rz'Kernel mapping must follow the format: Nr!) rN isinstancer=dictr ritemsr>valuesstr)r@rMAPPING_FORMATr kernelr r(s&& rsanitize_kernel_mapping$KernelConfig.sanitize_kernel_mapping|s # H ))%0$--t44I.IYY`aefjfyfyaz`{| #'"5"5";";"= J!JZZabhai!jkk-#>r-c  d^RIHp/p\V4pVPP 4FwrgVP 'd VP M VPpV'dWP,p\V\4'd3Tp VP'g\WeWV4K\WeWV4K\V\4'gKVP 4F;wrW8wdK VP'g\WjWV4K.\WjWV4K= K W@nR#)a Transforms a simple kernel_mapping of the form: { "RMSNorm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, or for local path: { "RMSNorm": "/home/user/liger_kernels:LigerRMSNorm", ... }, into a nested mapping: { "RMSNorm": { "cuda": { Mode.INFERENCE: LayerRepository( repo_id="kernels-community/layer_norm", layer_name="LlamaRMSNorm", ) } } } or for local path: { "RMSNorm": { "cuda": { Mode.INFERENCE: LocalLayerRepository( repo_path=Path("/home/user/liger_kernels"), package_name="liger_kernels", layer_name="LigerRMSNorm", ) } } } that's compatible with the kernels library. The device is inferred from the model's parameters if not provided. The Mode is inferred from the model's training state. )ModeN)r&r[rr=rStrainingTRAINING INFERENCE TORCH_COMPILErQrUr?r,r7rR) r@rcompiler[r*current_devicer rWr)r(r s &&& rcreate_compatible_mapping&KernelConfig.create_compatible_mappingsb !%e,"&"5"5";";"= J$)NNN4==D000&#&&" ,,,":yPbc(YVhiFD)))/%F/ 000&z9L^_,ZRde *8#>*1r-r<)NFr;)F) __name__ __module__ __qualname____firstlineno____doc__rArFrNrXrb__static_attributes____classdictcell__) __classdict__s@rr9r9`s.1  M QlfJ1J1r-r9N)rrrr,r7r9r-rrms(#%P"*y1>y1r-