{Kgh<dZddlZddlmZddlmZGddZy)zA Loss functions for linear models with raw_prediction = X @ coef N)sparse) squared_normc~eZdZdZdZd dZdZdZdZ ddZ dd Z dd Z dd Z dd Z y)LinearModelLossaGeneral class for loss functions with raw_prediction = X @ coef + intercept. Note that raw_prediction is also known as linear predictor. The loss is the average of per sample losses and includes a term for L2 regularization:: loss = 1 / s_sum * sum_i s_i loss(y_i, X_i @ coef + intercept) + 1/2 * l2_reg_strength * ||coef||_2^2 with sample weights s_i=1 if sample_weight=None and s_sum=sum_i s_i. Gradient and hessian, for simplicity without intercept, are:: gradient = 1 / s_sum * X.T @ loss.gradient + l2_reg_strength * coef hessian = 1 / s_sum * X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity Conventions: if fit_intercept: n_dof = n_features + 1 else: n_dof = n_features if base_loss.is_multiclass: coef.shape = (n_classes, n_dof) or ravelled (n_classes * n_dof,) else: coef.shape = (n_dof,) The intercept term is at the end of the coef array: if base_loss.is_multiclass: if coef.shape (n_classes, n_dof): intercept = coef[:, -1] if coef.shape (n_classes * n_dof,) intercept = coef[n_features::n_dof] = coef[(n_dof-1)::n_dof] intercept.shape = (n_classes,) else: intercept = coef[-1] Note: If coef has shape (n_classes * n_dof,), the 2d-array can be reconstructed as coef.reshape((n_classes, -1), order="F") The option order="F" makes coef[:, i] contiguous. This, in turn, makes the coefficients without intercept, coef[:, :-1], contiguous and speeds up matrix-vector computations. Note: If the average loss per sample is wanted instead of the sum of the loss per sample, one can simply use a rescaled sample_weight such that sum(sample_weight) = 1. Parameters ---------- base_loss : instance of class BaseLoss from sklearn._loss. fit_intercept : bool c ||_||_yN) base_loss fit_intercept)selfr r s e/home/alanp/www/video.onchill/myenv/lib/python3.12/site-packages/sklearn/linear_model/_linear_loss.py__init__zLinearModelLoss.__init__Es"*Nc|jd}|jj}|jr|dz}n|}|jjrt j |||f|d}|St j |||}|S)aAllocate coef of correct shape with zeros. Parameters: ----------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. dtype : data-type, default=None Overrides the data type of coef. With dtype=None, coef will have the same dtype as X. Returns ------- coef : ndarray of shape (n_dof,) or (n_classes, n_dof) Coefficients of a linear model. F)shapedtypeorderrr)rr n_classesr is_multiclassnp zeros_like)r Xr n_featuresrn_dofcoefs r init_zero_coefzLinearModelLoss.init_zero_coefIs~ WWQZ NN,,   NEE >> ' '==9e*11rcP|jdk(r||zn t|}d|z|zS)z5Compute L2 penalty term l2_reg_strength/2 *||w||_2^2.rg?)r$r)r r'l2_reg_strengthnorm2_ws r l2_penaltyzLinearModelLoss.l2_penaltys.'.||q'8'G#l7>S_$w..rc||j||\}} }n|j|\}} |jj||d|} t j | |} | |j ||zS)aCompute the loss as weighted average over point-wise losses. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Weighted average of losses per sample, plus penalty. Ny_truer+ sample_weight n_threads)r')r,r(r lossraverager0) r rryr4r.r5r+r'r&r6s r r6zLinearModelLoss.losssN  !151J1J4QR1S .GY!%!6!6t!< GY~~"") #  zz$ 6doog???rc|j|jjc\}} } | t|jz} ||j ||\} } }n|j |\} } |jj||||\}}||ntj|}|j|z }||j| |z }||z}|jjs\tj|| j}|j|z|| zz|d| |jr|j|d<||fStj| | f| jd}|j|z|| zz|ddd| f<|jr|jd|dddf<|j d k(r|j#d }||fS) a\Computes the sum of loss and gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Weighted average of losses per sample, plus penalty. gradient : ndarray of shape coef.shape The gradient of the loss. Nr2rr!rrrraxisrr#)rr rintr r,r( loss_gradientrsumr0r empty_likerr*emptyr$ravel)r rrr8r4r.r5r+ n_samplesrrrr'r&r6grad_pointwisesw_sumgrads r r?zLinearModelLoss.loss_gradientsT./WWdnn6N6N*JS!3!344  !151J1J4QR1S .GY!%!6!6t!< GY#~~;;)' < n ,3 9NxxzF" 99& ~~++==W]];D !n 47P PD* !!)--/RTz88Y.gmm3OD#1#3#3a#7/G:S#SDKZK !!,00a08QU yyA~zzz,TzrcF|j|jjc\}} } | t|jz} ||j ||\} } }n|j |\} } |jj||||}||ntj|}||z}|jjsZtj|| j}|j|z|| zz|d| |jr|j|d<|Stj| | f| jd}|j|z|| zz|ddd| f<|jr|jd|dddf<|jd k(r|j!d S|S) aComputes the gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. Nr2r:r!rr;rr<rr#)rr rr>r r,r(gradientrr@rrArr*rBr$rC)r rrr8r4r.r5r+rDrrrr'r&rErFrGs r rIzLinearModelLoss.gradient5sN./WWdnn6N6N*JS!3!344  !151J1J4QR1S .GY!%!6!6t!< GY00)' 1  ,3 9N& ~~++==W]];D !n 47P PD* !!)--/RK88Y.gmm3OD#1#3#3a#7/G:S#SDKZK !!,00a08QU yyA~zzz,, rc |j\} } | t|jz} | |j||\} }} n|j |\} }|j j || ||\}}|| ntj|}||z}||z}tj|dkdkD}tj|}|j js|"tj|| j}n|}|j|z|| zz|d| |jr|j|d<|$tj| | f| j}n|}|r|||fSt!j"|rC|jt!j$|df| | fz|zj'|d| d| f<n5|dddf|z}tj(|j||d| d| f<|dkDr%|j+dd| | z| d zxx|z cc<|jr;|j|z}||dddf<||dddf<|j|d <nt,|||fS) aComputes gradient and hessian w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. gradient_out : None or ndarray of shape coef.shape A location into which the gradient is stored. If None, a new array might be created. hessian_out : None or ndarray A location into which the hessian is stored. If None, a new array might be created. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessian : ndarray Hessian matrix. hessian_warning : bool True if pointwise hessian has more than half of its elements non-positive. Nr2rg?r:r!rrr)r!r!)rr>r r,r(r gradient_hessianrr@meanabsrrArr*rBrissparse dia_matrixtoarraydotr%NotImplementedError)r rrr8r4r.r5 gradient_out hessian_outr+rDrrr'r&rEhess_pointwiserFhessian_warningrGhessWXXhs r rLz LinearModelLoss.gradient_hessian~sj!" :S!3!344  !151J1J4QR1S .GY!%!6!6t!< GY)-)H)H)' *I* & ,3 9N& &  ''.A"56=/~~++#}}T?# !n 47P PD* !!)--/R"xxuenGMMJ"T?22 q!CC'''+Iy3I  ') [j[+:+-.$AtG,q013R[j[+:+-." R 8zE)eai8$%!!SS>) "SbS"W "R"W -113V & %T?**rc Bjjjc\}tjzj \}} |nt jjjsJjj|| |\} } | z} | z} t jj} j| zzz| djr| j| d<| jtjrtj| df||fzn| ddt j fzjrMt j"t j$jdt j&fd} | | fSjj)|| |\} | z} t j*fjd } | jzzz| dddf<jr| jd| dddf<f d } j,d k(r| j/d | fS| | fS)aComputes gradient and hessp (hessian product function) w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessp : callable Function that takes in a vector input of shape of gradient and and returns matrix-vector product with hessian. Nr2r:r!rrKr<ctj|}tjrj|dzz|dn2tj j j|dg|d|dxxx|dzz cccjr(|dxxx|dzz ccc|dz|dzz|d<|S)Nr!)rrArrOr*linalg multi_dotr ) sretrhXhX_sum hessian_sumr.rr s r hesspz7LinearModelLoss.gradient_hessian_product..hesspRsmmA&??1%'(ssb1[j>.A'BC $')yy':':ACCQ{ ^;T'UC $KZ Oa n$DD %% $"6$$q*~5 ae8KKCG rrr;cV |jdfd} jr|dddf}|ddddf}nd}|jz|z}| |zjdddtj fz }| z} | ddtj fz}t j fjd}|jz z |zz|ddd f< jr|jd z |dddf<jdk(r|jdS|S)Nr!rr#rrr<r;) r%r r*r@rnewaxisrBrr$rC)r_ s_intercepttmp hess_prodrrr.rrrprobar4r rFr's r rdz7LinearModelLoss.gradient_hessian_product..hessps8IIy"oSI9%%"#ArE(K!SbS& A"#K!##g + ))q)1!RZZ-@@u  ,=BJJ77CHHi%7w}}TWX -0UUQY&,@?UVCV,V ![j[.)%%'*wwAw'?Iae$99>$???55$$rrr#)rr rr>r r,rr@rrLrArr*rrOrPrfsqueezeasarray atleast_1dgradient_probarBr$rC)r rrr8r4r.r5rDr&r+rErVrGrdrarbrcrrrrjrFr's``` `` @@@@@@@@@r gradient_hessian_productz(LinearModelLoss.gradient_hessian_product s@./WWdnn6N6N*JS!3!344-1-F-FtQ-O*N+3 9N~~++-1^^-L-L-+# .M. *NN f $N f $N==W]];D !n 47P PD* !!)--/R),,.Kq!%%~q&9)YAWX $ArzzM2Q6!!BJJrvv1v~$>?v.  \U{w%)NN$A$A-+# %B% !NE f $N88Y.gmm3OD#1#3#3a#7/G:S#SDKZK !!,00a08QU . % %.yyA~zzz,e33U{rr )Nr"rN)Nr"rNNN)Nr"r)__name__ __module__ __qualname____doc__rrr(r,r0r6r?rIrLrorr rr s7r+8%"N2@/4@vLfG\I+XNOWrr)rsnumpyrscipyr utils.extmathrrrtrr rxs!(U U r