%ɷi~dZddlZddlZddlZddlmZddlZddlmZm Z m Z m Z m Z m Z mZmZmZddZdZded efd Z dd eed ed eefdZy)z A collection of utilities for ensuring that training can always occur. Heavily influenced by the [toma](https://github.com/BlackHC/toma) library. N)Optional) is_cuda_availableis_hpu_availableis_mlu_availableis_mps_availableis_musa_availableis_neuron_availableis_npu_availableis_sdaa_availableis_xpu_availablec|rtjtrtjj yt rtjj ytrtjj ytrtjj ytrtjj ytdrtjj yt!rtj"j yt%ryt'rtj(j yy)z Clears the device cache by calling `torch.{backend}.empty_cache`. Can also run `gc.collect()`, but do note that this is a *considerable* slowdown and should be used sparingly. z2.0) min_versionN)gccollectr torchxpu empty_cachermlur sdaar musar npurmpsrcudarr neurongarbage_collections J/var/www/html/venv/lib/python3.12/site-packages/accelerate/utils/memory.pyclear_device_cacher(s                e ,           " ct|ts t|}tt|D]}d||< t d|S)aN Releases memory from `objects` by setting them to `None` and calls `gc.collect()` and `torch.cuda.empty_cache()`. Returned objects should be reassigned to the same variables. Args: objects (`Iterable`): An iterable of objects Returns: A list of `None` objects to replace `objects` Example: ```python >>> import torch >>> from accelerate.utils import release_memory >>> a = torch.ones(1000, 1000).cuda() >>> b = torch.ones(1000, 1000).cuda() >>> a, b = release_memory(a, b) ``` NTr) isinstancelistrangelenr)objectsis rrelease_memoryr(FsE, gt $w- 3w<  $/ Nr exceptionreturncgd}ttr,tjdk(rt fd|DSy)z Checks if `exception` relates to CUDA out-of-memory, XPU out-of-memory, CUDNN not supported, or CPU out-of-memory Args: exception (`Exception`): An exception )z out of memory.z(cuDNN error: CUDNN_STATUS_NOT_SUPPORTED.z*DefaultCPUAllocator: can't allocate memoryz1FATAL ERROR :: MODULE:PT_DEVMEM Allocation failedrc3@K|]}|jdvyw)rN)args).0errr)s r z+should_reduce_batch_size..ssC3)..++CsF)r" RuntimeErrorr%r-any)r) _statementss` rshould_reduce_batch_sizer4ds:K )\*s9>>/Ba/GC{CCC r functionstarting_batch_sizereduce_batch_size_fncftjt|S|fdfd}|S)a A basic decorator that will try to execute `function`. If it fails from exceptions related to out-of-memory or CUDNN, the batch size is multiplied by 0.9 and passed to `function` `function` must take in a `batch_size` parameter as its first argument. Args: function (`callable`, *optional*): A function to wrap starting_batch_size (`int`, *optional*): The batch size to try and fit into memory Example: ```python >>> from accelerate.utils import find_executable_batch_size >>> @find_executable_batch_size(starting_batch_size=128) ... def train(batch_size, model, optimizer): ... ... >>> train(model, optimizer) ``` )r6c$tdzS)Ng?)int) batch_sizesrr7z8find_executable_batch_size..reduce_batch_size_fnsZ#-.J r c 6tdttjjj }t |t |dzkrbdjt|dd|ddDcgc] \}}|d|c}}}tdjdjd|d  d k(r td  g|i|Scc}}w#t$r*}t|rtd nYd}~nd}~wwxYwZ) NTrrz, =zBatch size was passed into `zS` as the first argument when called.Remove this as the decorator already does so: `(z)`rz-No executable batch size found, reached zero.)rr#inspect signature parameterskeysr%joinzip TypeError__name__r1 Exceptionr4) r-kwargsparamsargvaluearg_strer;r5r7s r decoratorz-find_executable_batch_size..decorators1d3g''1<<AACD v;#d)a- (iiCqr TXYZY[T\D] ^jc5C5%!1 ^_G.x/@/@.ABBBJBSBSATTUV]U^^`b Q"#RSS  C  C$$ D- DD) functoolspartialfind_executable_batch_size)r5r6r7rNr;s` ` @rrQrQws=>  !;Qdee$J#  . r )F)NN)__doc__rOrr?typingrrimportsrrrrr r r r r rr(rGboolr4callabler:rQr rrYs      #<< d($("/3Ax AA#8,Ar