{Kg{,ddlmZddlmZddlmZddlZddlm Z dddd Z dd Z Gd d eZ d Z GddeZdZdZdddZddZGddeZdZy))Counter)suppress) NamedTupleN) is_scalar_nanFreturn_inverse return_countsc`|jtk(rt|||St|||S)aHelper function to find unique values with support for python objects. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : ndarray Values to check for unknowns. return_inverse : bool, default=False If True, also return the indices of the unique values. return_counts : bool, default=False If True, also return the number of times each unique item appears in values. Returns ------- unique : ndarray The sorted unique values. unique_inverse : ndarray The indices to reconstruct the original array from the unique array. Only provided if `return_inverse` is True. unique_counts : ndarray The number of times each of the unique values comes up in the original array. Only provided if `return_counts` is True. r)dtypeobject_unique_python _unique_np)valuesr r s Y/home/alanp/www/video.onchill/myenv/lib/python3.12/site-packages/sklearn/utils/_encode.py_uniquer s:>||v >   ~] ctj|||}d\}}|r|^}}|r|^}}|s|r|d}|jrit|dr[tj|tj }|d|dz}|r||||kD<|r#tj ||d||<|d|dz}|f}|r||fz }|r||fz }t|dk(r|dS|S)zHelper function to find unique values for numpy arrays that correctly accounts for nans. See `_unique` documentation for details.r)NNrNr)npuniquesizer searchsortednansumlen)rr r uniquesinversecountsnan_idxrets rrr3sii~]G!OGV"&#'!*|| gbk2//'2662-GaK( )0GGg% &  ffVGH%56F7OMgk*F *C z yX]3q6++rc,eZdZUdZeed<eed<dZy) MissingValuesz'Data class for missing data informationrnonecg}|jr|jd|jr|jtj|S)z3Convert tuple to a list where None is always first.N)r$appendrr)selfoutputs rto_listzMissingValues.to_listbs6 99 MM$  88 MM"&& ! rN)__name__ __module__ __qualname____doc__bool__annotations__r)rrr#r#\s1 I Jrr#c|Dchc]}| t|s|}}|s|tddfSd|vr*t|dk(rtdd}ntdd}n tdd}||z }||fScc}w)a.Extract missing values from `values`. Parameters ---------- values: set Set of values to extract missing from. Returns ------- output: set Set with missing values extracted. missing_values: MissingValues Object with missing value information. NF)rr$rT)rr#r)rvaluemissing_values_setoutput_missing_valuesr(s r_extract_missingr5ls""!%U]mE6J6 }U;;; !! ! "a '$1e$$G !%2d$F ! -$U C( (F ( (('s A3A3c(eZdZdZfdZdZxZS)_nandictz!Dictionary with support for nans.c|t|||jD]\}}t|s||_yyN)super__init__itemsr nan_value)r'mappingkeyr2 __class__s rr;z_nandict.__init__s6 !!--/JCS!!&*rc^t|drt|r |jSt|)Nr=)hasattrrr=KeyErrorr'r?s r __missing__z_nandict.__missing__' 4 %-*<>> !smr)r*r+r,r-r;rE __classcell__r@s@rr7r7s+rr7ctt|Dcic]\}}|| c}}}tj|Dcgc]}|| c}Scc}}wcc}w)z,Map values based on its position in uniques.)r7 enumeraterarray)rrivaltablevs r_map_to_integerrPsT 9W+=>+=Cc1f+=> ?E 88v.v!U1Xv. //?.s A Ac t|}t|\}}t|}|j|j t j ||j}|f}|r|t||fz }|r|t||fz }t|dk(r|dS|S#t$r1tdtd|DD}td|wxYw)Nr c34K|]}|jywr9)r,).0ts r z!_unique_python..sL/K!q~~/Ksc32K|]}t|ywr9)type)rTrOs rrVz!_unique_python..s2KFq47FszPEncoders require their input argument must be uniformly strings or numbers. Got rr) setr5sortedextendr)rrKr TypeErrorrP _get_countsr)rr r uniques_setmissing_valuesrtypesr!s rrrs  &k &6{&C# ^%~--/0((7&,,7 *C 022 FG,..X]3q6++  Ls2KF2K/KLL '',g /   s A$B"":CT) check_unknownc|jjdvr t||S|r%t ||}|rt dt |tj||S#t$r}t dt |d}~wwxYw)aHelper function to encode values into [0, n_uniques - 1]. Uses pure python method for object dtype, and numpy method for all other dtypes. The numpy method has the limitation that the `uniques` need to be sorted. Importantly, this is not checked but assumed to already be the case. The calling method needs to ensure this for all non-object values. Parameters ---------- values : ndarray Values to encode. uniques : ndarray The unique values in `values`. If the dtype is not object, then `uniques` needs to be sorted. check_unknown : bool, default=True If True, check for values in `values` that are not in `unique` and raise an error. This is ignored for object dtype, and treated as True in this case. This parameter is useful for _BaseEncoder._transform() to avoid calling _check_unknown() twice. Returns ------- encoded : ndarray Encoded values OUSz%y contains previously unseen labels: N) r kindrPrC ValueErrorstr_check_unknownrr)rrraediffs r_encoderjs:||E! O"673 3 !&'2D #HT !TUUw// ODSVHMN N Os A## B,BBcXd}|jjdvr t|}t|\}}t|t\|z }|jxr j }|j xr j }fd} |rT|s|s|r*t j|D cgc] } | |  c} }n$t jt|t}t|}|r|jd|r|jt jnt j|} t j| |d}|rG|jrt j ||}n$t jt|t}t j"|j%rSt j"|} | j%r.|jr|rt j"|} d|| <|| }t|}|r||fS|Scc} w)a Helper function to check for unknowns in values to be encoded. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : array Values to check for unknowns. known_values : array Known values. Must be unique. return_mask : bool, default=False If True, return a mask of the same shape as `values` indicating the valid values. Returns ------- diff : list The unique values present in `values` and not in `know_values`. valid_mask : boolean array Additionally returned if ``return_mask=True``. Nrccj|vxs-jxr|duxsjxr t|Sr9)r$rr)r2missing_in_uniquesr^s ris_validz _check_unknown..is_validsG$)%**"TM)&)))!%(  rrRT assume_uniquer)r rdrYr5rr$rrKonesrr.listr&r setdiff1drisinisnanany)r known_values return_mask valid_mask values_setmissing_in_valuesri nan_in_diff none_in_diffrnr2 unique_values diff_is_nanis_nanrmr^s @@rrgrgs2J ||E![ (8(D% %,' *:;*G' 'K''++J4F4J4J0J (--M6H6M6M2M   {lXXF&KF5xF&KL WWS[= Dz  KK   KK  &) ||MGenerate items without nans. Stores the nan counts separately. nan_countrrN)rrBr)r'r<items rrz_NaNCounter._generate_itemsGs?D & 4-!" NNa N sAAc^t|drt|r |jSt|)Nr)rBrrrCrDs rrEz_NaNCounter.__missing__QrFr)r*r+r,r-r;rrErGrHs@rrrAs.6 rrcp|jjdvrnt|}tjt |tj }t|D]%\}}tt5||||<ddd'|St|d\}}tj||d}tj|drtj|drd|d<tj|||} tj|tj }|| ||<|S#1swYxYw)zGet the count of each of the `uniques` in `values`. The counts will use the order passed in by `uniques`. For non-object dtypes, `uniques` is assumed to be sorted and `np.nan` is at the end. OUrRNT)r ror)r rdrrzerosrint64rJrrCrrtrur zeros_like) rrcounterr(rLrr~runiques_in_valuesunique_valid_indicess rr]r]Ws  ||D f%#g,bhh7 )GAt(##DMq $#* &vTBM6dK xx b!"rxx '< $"??='BS:TU ]]7"(( 3F &'; rsr#',5&R&,R J  #)L t  0 ,4/3'0TRj',r