tKg+PBdZddlmZddlZddlmZddlmZm Z m Z ddl m Z ddl mZmZej ej$ZdZ d"d Zd Zd Zd#d Zd$d Zd%dZdZdZd&dZd&dZdZdZ dZ!dZ"dZ#dZ$dZ%d%dZ&dZ'dZ(dZ)d'dZ*d'dZ+d Z,d!Z-y)(z+Functions used by least-squares algorithms.)copysignN)norm) cho_factor cho_solve LinAlgError)issparse)LinearOperatoraslinearoperatorc\tj||}|dk(r tdtj||}tj|||dzz }|dkDr tdtj||z||zz }|t ||z }||z }||z } || kr|| fS| |fS)aqFind the intersection of a line with the boundary of a trust region. This function solves the quadratic equation with respect to t ||(x + s*t)||**2 = Delta**2. Returns ------- t_neg, t_pos : tuple of float Negative and positive roots. Raises ------ ValueError If `s` is zero or `x` is not within the trust region. rz `s` is zero.z#`x` is not within the trust region.)npdot ValueErrorsqrtr) xsDeltaabcdqt1t2s ^/home/alanp/www/video.onchill/myenv/lib/python3.12/site-packages/scipy/optimize/_lsq/common.pyintersect_trust_regionrs q! AAv(( q! A q! uaxA1u>?? !ac A hq!n A QB QB Bw2v 2v c d} ||z} ||k\rt|z|dz} |d| kD} nd} | r(|j||z  } t| |kr| ddfSt| |z }| r| d| ||\}}| |z }nd}|| s|dk(rtd|z||zdz}n|}t |D]u}||ks||kDrtd|z||zdz}| || ||\}}|dkr|}||z }t|||z }|||z|z|z z}t j |||zksun|j| |dz|zz  } | |t| z z} | |d zfS) aSolve a trust-region problem arising in least-squares minimization. This function implements a method described by J. J. More [1]_ and used in MINPACK, but it relies on a single SVD of Jacobian instead of series of Cholesky decompositions. Before running this function, compute: ``U, s, VT = svd(J, full_matrices=False)``. Parameters ---------- n : int Number of variables. m : int Number of residuals. uf : ndarray Computed as U.T.dot(f). s : ndarray Singular values of J. V : ndarray Transpose of VT. Delta : float Radius of a trust region. initial_alpha : float, optional Initial guess for alpha, which might be available from a previous iteration. If None, determined automatically. rtol : float, optional Stopping tolerance for the root-finding procedure. Namely, the solution ``p`` will satisfy ``abs(norm(p) - Delta) < rtol * Delta``. max_iter : int, optional Maximum allowed number of iterations for the root-finding procedure. Returns ------- p : ndarray, shape (n,) Found solution of a trust-region problem. alpha : float Positive value such that (J.T*J + alpha*I)*p = -J.T*f. Sometimes called Levenberg-Marquardt parameter. n_iter : int Number of iterations made by root-finding procedure. Zero means that Gauss-Newton step was selected as the solution. References ---------- .. [1] More, J. J., "The Levenberg-Marquardt Algorithm: Implementation and Theory," Numerical Analysis, ed. G. A. Watson, Lecture Notes in Mathematics 630, Springer Verlag, pp. 105-116, 1977. c|dz|z}t||z }||z }tj|dz|dzz  |z }||fS)zFunction of which to find zero. It is defined as "norm of regularized (by alpha) least-squares solution minus `Delta`". Refer to [1]_. r )rr sum)alphasufrrdenomp_normphi phi_primes rphi_and_derivativez2solve_lsq_trust_region..phi_and_derivativejsU 1u cEk"unVVC1Huax/0069 I~rrFggMbP??r )EPSrrmaxranger abs)nmufrVr initial_alphartolmax_iterr(r# threshold full_rankp alpha_upperr&r' alpha_lowerr"itratios rsolve_lsq_trust_regionr>9sb  b&C Av!GadN bEI%   UU26]N 7e c19 s)e#K+Ca?YdY&  I-12DEK'+ *Cc)IJHo ; %+"5 +kK.G#-MNE+E35AY 7Ki+uu}5  #+&.. 66#; %  sadUl# $$A aA eR!V rc& t|\}}t||f| }tj|||dzkr|dfS |d|dzz}|d|dzz}|d|dzz}|d|z} |d|z} tj | | zd||z | zzd|zd| |z| zz| | z g} tj | } tj| tj| } |tjd| zd| dzzz d| dzz d| dzzz fz}d tj||j|zd ztj||z} tj| }|d d |f}|d fS#t$rYOwxYw) azSolve a general trust-region problem in 2 dimensions. The problem is reformulated as a 4th order algebraic equation, the solution of which is found by numpy.roots. Parameters ---------- B : ndarray, shape (2, 2) Symmetric matrix, defines a quadratic term of the function. g : ndarray, shape (2,) Defines a linear term of the function. Delta : float Radius of a trust region. Returns ------- p : ndarray, shape (2,) Found solution. newton_step : bool Whether the returned solution is the Newton step which lies within the trust region. r T)rr)rr+)r+r+rr+r*axisNF) rrr rrarrayrootsrealisrealvstackr!argmin)BgrRlowerr9rrrrfcoeffstvalueis rsolve_trust_region_2drRs. a=5 5z1 % % 66!Q<5!8 #d7N $ $%(A $%(A $%(A !u A !u A XX aa!eai!a%qb1fqj)9A26BDF A "))A, A  1q5A1H-AqDQAX/FGHHA "&&QUU1XA. .1 =E %A !Q$A e8O)    s aA q! A  RVVAHa  HA q! A ~ EE"I RVVAq\ "&&A, 2 .   T 1% %A rvvb4i,, ,A!Qw!t rc||g}|dk7r'd|z|z }||cxkr|krnn|j|tj|}|||z|zz|z}tj|}||||fS)zMinimize a 1-D quadratic function subject to bounds. The free term `c` is 0 by default. Bounds must be finite. Returns ------- t : float Minimum point. y : float Minimum value. rg)appendr asarrayrH) rrlbubrrOextremumy min_indexs rminimize_quadratic_1drj.s} RAAv!8a<  2  HHX  1 A QUQY!A ! I Y<9 %%rc|jdk(rF|j|}tj||}|t|tj||z|z }nW|j|j}tj|dzd}| |tj||dzzdz }tj||}d|z|zS)aCompute values of a quadratic function arising in least squares. The function is 0.5 * s.T * (J.T * J + diag) * s + g.T * s. Parameters ---------- J : ndarray, sparse matrix or LinearOperator, shape (m, n) Jacobian matrix, affects the quadratic term. g : ndarray, shape (n,) Gradient, defines the linear term. s : ndarray, shape (k, n) or (n,) Array containing steps as rows. diag : ndarray, shape (n,), optional Addition diagonal part, affects the quadratic term. If None, assumed to be 0. Returns ------- values : ndarray with shape (k,) or float Values of the function. If `s` was 2-D, then ndarray is returned, otherwise, float is returned. r+r rrAr*)ndimrr Tr!)r\rJrr]Jsrls revaluate_quadraticrpEs. vv{ UU1X FF2rN   D!$ $A UU133Z FF2q5q !   q!t !, ,A q! A 7Q;rc>tj||k\||kzS)z$Check if a point lies within bounds.)r all)rrerfs r in_boundsrsos 6617qBw' ((rctj|}||}tj|}|jtjtj d5tj ||z ||z ||z ||z ||<dddtj|}|tj||tj|jtzfS#1swY_xYw)aCompute a min_step size required to reach a bound. The function computes a positive scalar t, such that x + s * t is on the bound. Returns ------- step : float Computed step. Non-negative value. hits : ndarray of int with shape of x Each element indicates whether a corresponding variable reaches the bound: * 0 - the bound was not hit. * -1 - the lower bound was hit. * 1 - the upper bound was hit. ignore)overN) r nonzero empty_likefillinferrstatemaximumminequalsignastypeint)rrrerfnon_zero s_non_zerostepsmin_steps rstep_size_to_boundrts$zz!}H8J MM! E JJrvv ( #**b1fh%7*%D&(1fh%7*%DFh $vve}H RXXeX.1B1B31GG GG $ #s %,C//C8ctj|t}|dk(rd|||k<d|||k\<|S||z }||z }|tjdtj|z}|tjdtj|z}tj ||tj ||kz} d|| <tj ||tj ||kz} d|| <|S)aDetermine which constraints are active in a given point. The threshold is computed using `rtol` and the absolute value of the closest bound. Returns ------- active : ndarray of int with shape of x Each component shows whether the corresponding constraint is active: * 0 - a constraint is not active. * -1 - a lower bound is active. * 1 - a upper bound is active. dtyperr)r+)r zeros_likerr|r/isfiniteminimum) rrerfr5active lower_dist upper_distlower_thresholdupper_threshold lower_active upper_actives rfind_active_constraintsrs]]1C (F qyqBwqBw RJaJRZZ266":66ORZZ266":66OKKO2::j/#JJLLF<KKO2::j/#JJLLF< Mrc :|j}t||||}tj|d}tj|d}|dk(r?tj||||||<tj||||||<np|||tj dtj ||zz||<|||tj dtj ||zz ||<||k||kDz}d||||zz||<|S)zShift a point to the interior of a feasible region. Each element of the returned vector is at least at a relative distance `rstep` from the closest bound. If ``rstep=0`` then `np.nextafter` is used. r)r+rr*)copyrr r~ nextafterr|r/) rrerfrstepx_newr lower_mask upper_mask tight_boundss rmake_strictly_feasiblers FFHE $QB 6F&"%J&!$J zLLJJHjLLJJHj ^"RZZ266"Z.3I%JJKj ^"RZZ266"Z.3I%JJKjBJ52:.LL!1B|4D!DEE, Lrctj|}tj|}|dktj|z}||||z ||<d||<|dkDtj|z}||||z ||<d||<||fS)a4Compute Coleman-Li scaling vector and its derivatives. Components of a vector v are defined as follows:: | ub[i] - x[i], if g[i] < 0 and ub[i] < np.inf v[i] = | x[i] - lb[i], if g[i] > 0 and lb[i] > -np.inf | 1, otherwise According to this definition v[i] >= 0 for all i. It differs from the definition in paper [1]_ (eq. (2.2)), where the absolute value of v is used. Both definitions are equivalent down the line. Derivatives of v with respect to x take value 1, -1 or 0 depending on a case. Returns ------- v : ndarray with shape of x Scaling vector. dv : ndarray with shape of x Derivatives of v[i] with respect to x[i], diagonal elements of v's Jacobian. References ---------- .. [1] M.A. Branch, T.F. Coleman, and Y. Li, "A Subspace, Interior, and Conjugate Gradient Method for Large-Scale Bound-Constrained Minimization Problems," SIAM Journal on Scientific Computing, Vol. 21, Number 1, pp 1-23, 1999. rr)r+)r ones_likerr)rrJrerfr_dvmasks rCL_scaling_vectorrs< QA q B ER[[_ $Dh4 AdGBtH ER[[_ $Dg4 AdGBtH b5Lrct|||r|tj|fStj|}tj|}|j }tj |t }||z}tj||d||z||z ||<||||k||<||z}tj||d||z||z ||<||||kD||<||z}||z }tj||||z d||z} ||tj| d||z| z z||<| ||kD||<tj|} d| |<|| fS)z3Compute reflective transformation and its gradient.rr r)) rsr rrrrboolr|r remainder) rhrerf lb_finite ub_finiter g_negativerrrOrJs rreflective_transformationrsB",,q/!! BI BI Aq-J  z !Djj4!bh,4"89AdGwD)Jt : !Djj4!bh,4"89AdGwD)Jt y D RA QtWr$x'QtW5AhAq1T7{Q77AdG1T7{Jt QAAjM a4Krc Btdjddddddy)Nz${:^15}{:^15}{:^15}{:^15}{:^15}{:^15} Iterationz Total nfevCostCost reduction Step norm OptimalityprintformatrUrrprint_header_nonlinearr!s% 0 6+|V5E| -.rc f|d}n|d}|d}n|d}tdj||||||y)N ^15.2ez"{:^15}{:^15}{:^15.4e}{}{}{:^15.2e}r) iterationnfevcostcost_reductionrX optimalitys rprint_iteration_nonlinearr'sN!*62  (  . 6)T4Z )*rc @tdjdddddy)Nz{:^15}{:^15}{:^15}{:^15}{:^15}rrrrrrrUrrprint_header_linearr8s# * 6+v'7 !rcX|d}n|d}|d}n|d}t|d|d|||dy)Nrrz^15z^15.4e)r)rrrrXrs rprint_iteration_linearr>sQ!*62  (  YsOD=(8 JvCV WXrczt|tr|j|S|jj |S)z4Compute gradient of the least-squares cost function.) isinstancer rmatvecrmr)r\rMs r compute_gradrPs,!^$yy|sswwqzrc.t|rFtj|jdj dj dz}ntj|dzddz}| d||dk(<ntj ||}d|z |fS)z5Compute variables scale based on the Jacobian matrix.r rrAr*r+)rr rdpowerr!ravelr|)r\ scale_inv_old scale_invs rcompute_jac_scalerXs{JJqwwqz~~1~56<<>C FF1a4a(#- $% )q.!JJy-8 y=) ##rcrtfd}fd}fd}tj|||S)z#Return diag(d) J as LinearOperator.c,j|zSN)matvecrr\rs rrz(left_multiplied_operator..matvecks188A;rcVddtjfj|zSr)r newaxismatmatXr\rs rrz(left_multiplied_operator..matmatns#BJJ!((1+--rcHj|jzSr)rrrs rrz)left_multiplied_operator..rmatvecqsyyQ''rrrrr r shaper\rrrrs`` rleft_multiplied_operatorrgs8A.( !''&") ++rcrtfd}fd}fd}tj|||S)z#Return J diag(d) as LinearOperator.cRjtj|zSr)rr rrs rrz)right_multiplied_operator..matvec|sxx a((rcVj|ddtjfzSr)rr rrs rrz)right_multiplied_operator..matmats$xxAam,,--rc,j|zSrrrs rrz*right_multiplied_operator..rmatvecs199Q<rrrrs`` rright_multiplied_operatorrxs8A).  !''&") ++rc|tj\}fd}fd}t|z|f||S)zReturn a matrix arising in regularized least squares as LinearOperator. The matrix is [ J ] [ D ] where D is diagonal matrix with elements from `diag`. cVtjj||zfSr)r hstackr)rr\r]s rrz(regularized_lsq_operator..matvecs#yy!((1+tax011rcF|d}|d}j||zzSrr)rx1x2r\r]r1s rrz)regularized_lsq_operator..rmatvecs0 rU qrUyy}tby((r)rr)r rr )r\r]r0rrr1s`` @rregularized_lsq_operatorrs? A 77DAq2) 1q5!*VW EErc |r t|ts|j}t|r2|xj|j |j dzc_|St|trt||}|S||z}|S)zhCompute J diag(d). If `copy` is False, `J` is modified in place (unless being LinearOperator). clip)mode)rr rrdatatakeindicesrr\rrs rright_multiplyrsw  Jq.1 FFH{ !&&&00 H A~ & %a + H Q Hrc`|r t|ts|j}t|rH|xjt j |t j|jzc_|St|trt||}|S||ddt jfz}|S)zhCompute diag(d) J. If `copy` is False, `J` is modified in place (unless being LinearOperator). N) rr rrrr repeatdiffindptrrrrs r left_multiplyrs  Jq.1 FFH{ "))Arwwqxx011 H A~ & $Q * H Qq"**}  HrcN|||zkxr|dkD}||||zzk}|r|ry|ry|ryy)z8Check termination condition for nonlinear least squares.rTr r NrU) dFFdx_normx_normr=ftolxtolftol_satisfiedxtol_satisfieds rcheck_terminationrsB$(]3ut|Nttf}55N.  rc|dd|dz|dzzz}t||tk<|dz}||d|z z}t||d|fS)z`Scale Jacobian and residuals for a robust loss function. Arrays are modified in place. r+r r*F)r)r,r)r\rMrhoJ_scales rscale_for_robust_loss_functionrsa !fq3q6zAqD((G GGcM OGQ' A G% 0! 33r)Ng{Gz? )NN)rr)g|=)T).__doc__mathrnumpyr numpy.linalgr scipy.linalgrrr scipy.sparserscipy.sparse.linalgr r finfofloatepsr,rr>rRrZrarjrprsrrrrrrrrrrrrrrrrrrrUrrrs1;;!@bhhuo $NAE/1od0f:0f&.$T) H:$N6)XD. *"! Y$ $+"+"F, $ $  4r