o ,h$@s@ddlmZddlZddlZddlmZddlmZmZm Z ddl m Z ddl m Z ddlmZddlmZdd lmZmZmZmZdd lmZdd lmZGd d d eZeZeeddZeeddZ ej!ddZ"eej#e eddeej$ddZ%ddZ&eddZ'ddZ(ddZ)d d!Z*dS)")contextmanagerN) DispatchKey)_ConstantFunction flat_apply to_graphable strict_mode)autograd_not_implemented)HigherOrderOperator)FakeTensorMode)get_proxy_slotPreDispatchTorchFunctionModeProxyTorchDispatchModetrack_tensor_tree)_pytree)"is_traceable_wrapper_subclass_typecs(eZdZfddZfddZZS)ExportTracepointcstddS)N_export_tracepoint)super__init__)self __class__Q/var/www/html/scripts/venv/lib/python3.10/site-packages/torch/_export/wrappers.pyrszExportTracepoint.__init__cstj|i|SN)r__call__)rargskwargsrrrrzExportTracepoint.__call__)__name__ __module__ __qualname__rr __classcell__rrrrrs rcOs<t|jj||f\}}|jdt||}t||d|jdS)N call_functionconstanttracer)pytreetree_mapr' unwrap_proxy create_proxyrr)moderrp_argsp_kwargsproxyrrrexport_tracepoint_dispatch_mode%s r0cOs,| |WdS1swYdSrr)r,rrrrr"export_tracepoint_fake_tensor_mode.s$r1cOsR||}||}|t|i||WdS1s"wYdSr)unwrap_tensorsredispatch_to_nextr)ctxrrunwrapped_argsunwrapped_kwargsrrrexport_tracepoint_functional4s   $r7T)deferred_errorcOs|Srr)rrrrrexport_tracepoint_cpuCsr9cst|tjjs JdksJtjj|}fddddfdd}fdd }|j|d d }|j|d d }||fS) NcsB|vr|d|ksJ|d|ksJ||d|<dS)Nin_specout_spec)r;r<r)pathr;r<)module_call_specsrrupdate_module_call_signaturesMsz6_wrap_submodule..update_module_call_signaturescSs:|D]}t|tjttttfs|dustd|qdS)NzGOnly Tensors or scalars are supported as pytree flattened inputs, got: ) isinstancetorchTensorstrintfloatboolAssertionError) flat_argsarrrcheck_flattenedSsz(_wrap_submodule..check_flattenedcsBt||f\}}|t|dd}t||\}}||fS)Nmodule_call_inputskindr=r( tree_flattenrtree_unflatten)modulerrrHr;)rJr=rrpre_hookZs z!_wrap_submodule..pre_hookcsPt||f\}}t|\}}|t|dd}||t||S)Nmodule_call_outputsrLrN)rQrrres_r;flat_resr<)rJr=r?rr post_hookas   z"_wrap_submodule..post_hookT) with_kwargs) r@rAnnModulefx graph_module _get_attrregister_forward_pre_hookregister_forward_hook)modr=r> submodulerRrW pre_handle post_handler)rJr>r=r?r_wrap_submoduleHs  rdc csVg}z|D] }|t|||qdVW|D]}|qdS|D]}|q#wr)extendrdremove)fpreserve_signaturemodule_call_signatureshandlesr=handlerrr_wrap_submodulesns  rlcCsdd}||_|S)NcWs t||Srr)rrrrrcall|s z'_mark_strict_experimental..call)r)clsrmrrr_mark_strict_experimental{srocCs`|d}t|j|rt|j||ksJ|d|diS||}t|j|||d|diS)a This is a wrapper utility method on top of tracer to cache the already registered subclass spec attribute. This is useful because Subclass.__init__ will be same for each subclass. By default, fx will create multiple attributes/proxies for given attribute. 0get_attrr)hasattrrootgetattrr+get_fresh_qualnamesetattr)r'namespecfx_namequalnamerrr'_register_subclass_spec_proxy_in_tracers  r{cs2dd}|stdjdfdd}|S)a) Experimental decorator that makes subclass to be traceable in export with pre-dispatch IR. To make your subclass traceble in export, you need to: 1. Implement __init__ method for your subclass (Look at DTensor implementation) 2. Decorate your __init__ method with _mark_constructor_exportable_experimental 3. Put torch._dynamo_disable decorator to prevent dynamo from peeking into its' impl Example: class FooTensor(torch.Tensor): @staticmethod def __new__(cls, elem, *, requires_grad=False): # ... return torch.Tensor._make_subclass(cls, elem, requires_grad=requires_grad) @torch._dynamo_disable @mark_subclass_constructor_exportable_experimental def __init__(self, elem, ...): # ... cSst|o|jdkS)Nr)callabler )fnrrr_is_initrzCmark_subclass_constructor_exportable_experimental.._is_initztorch._export.wrappers.mark_constructor_exportable_experimental can only be applied on subclass tensor.__init__But, you are adding it on z which is not supported. If __init__ doesn't exist on your subclass, please add it. Look at DTensor.__init__ implementation for examplecstt|ds"jdsJjdtd }td|d|i|tjs0dStj }dd|D}t|dksKJdt|t|dkrSdS|d}|j |d}t t |dd|f\}}d jd } | } tj| |d | d i} ttjfd d|} tjjtt|\} }t|jd}t||}dt|| g| Ri}t||dddS)Nrrz5Applying mark_constructor_exportable_experimental on z is not valid as it is not a traceable tensor subclass. Please look at DTensor.__init__ implementation as an example of proper usage of this API.cSsg|] }t|tr|qSr)r@r ).0r,rrr s zVmark_subclass_constructor_exportable_experimental..wrapper..z6Expected only one PreDispatchTorchFunctionMode, found rU.rqrcs t|jSr)r r/)xr'rrs zTmark_subclass_constructor_exportable_experimental..wrapper.._const_func_specr$r%) rtyper"endswithlen RuntimeErrorrA_C_is_torch_function_mode_enabled overrides _get_current_function_mode_stackr'rtuplejoinlowersplitrurvrsr+r( tree_map_onlyrButilsrrOrr r{rr)rrobj_nametorch_function_mode_stackpre_dispatch_tf_modesr,subclassrHr;constructor_spec_namerz spec_proxyflat_proxy_argsrU func_spec!fxable_constructor_call_spec_namefunc_spec_proxy inner_proxyconstructor_subclassrrwrappers^         zBmark_subclass_constructor_exportable_experimental..wrapper)rr )rr~rrrr1mark_subclass_constructor_exportable_experimentals Gr)+ contextlibrrAtorch._custom_opstorch._Cr"torch._higher_order_ops.flat_applyrrr#torch._higher_order_ops.strict_modertorch._higher_order_ops.utilsr torch._opsr torch._subclasses.fake_tensorr "torch.fx.experimental.proxy_tensorr r rr torch.utilsrr(torch.utils._python_dispatchrrrpy_implr0r1py_functionalize_implr7AutogradCPUr9rdrlror{rrrrrs>              &