Performs backtracking line search to find a step size that satisfies
the Armijo condition (sufficient decrease). This follows the textbook
backtracking scheme in Nocedal & Wright, "Numerical Optimization"
(2nd ed.), Algorithm 3.1.
The Armijo condition ensures that the function value decreases sufficiently:
f(x + α * d) <= f(x) + c * α * ∇f(x)^T * d
where:
α is the step size
d is the search direction (typically -gradient)
c is the Armijo parameter (typically chosen around 1e-4 per Nocedal & Wright)
β is the backtracking contraction factor (Algorithm 3.1 suggests β = 0.5)
Initial step size selection:
If initialStepSize is explicitly provided in options, it is used as-is.
Otherwise, the initial step size is automatically scaled by the gradient norm:
α₀ = 1.0 / ||∇f(x)||
This prevents steps from being too large when gradients are large, improving
convergence performance. If the gradient norm is very small (< 1e-10) or the
computed step size is not finite, the default value of 1.0 is used.
Performs backtracking line search to find a step size that satisfies the Armijo condition (sufficient decrease). This follows the textbook backtracking scheme in Nocedal & Wright, "Numerical Optimization" (2nd ed.), Algorithm 3.1.
The Armijo condition ensures that the function value decreases sufficiently: f(x + α * d) <= f(x) + c * α * ∇f(x)^T * d
where:
Initial step size selection:
initialStepSizeis explicitly provided in options, it is used as-is.