User Guide
To get started, check out the Examples. The Advanced Examples demonstrate ways to improve performance for a variety of different problem types.
Custom linear operators for QP
The QPSolver
interface does not require an AbstractMatrix
for P
and M
. They only must support the following operations:
P
must supportLinearAlgebra.mul!
(three argument)M
must supportsize
,LinearAlgebra.mul!
(three argument),LinearAlgebra.adjoint
, andLinearAlgebra.mul!
(three and five argument) forM'
.
The package LinearMaps.jl
provides much of this functionality, especially for the common case in which M
contains an identity block (e.g., see the Markowitz Portfolio Optimization example).
Algorithm parameters
The GeNIOS
algorithm has the following tunable parameters, which are tunable either through the JuMP interface or via the SolverOptions
object, passed to solve!
:
ρ0
- ADMM penalty parameter initial value (updated by the algorithm)
- Default value:
1.0
relax::Bool
- Toggles if over-relaxation is used. Empirically, over-relaxation has been
- default value:
false
α
- Over-relaxation parameter. Note that
relax
option (inSolverOptions
, see Algorithm parameters) must betrue
, which is the default to use over-relaxation. For QPs, we recommend a parameter of1.5
to1.8
, which has empirically been shown to help convergence in the literature. - Default value:
1.2
- Over-relaxation parameter. Note that
logging::Bool
- Toggles if the iterates are logged (objective value, iteration time, linear system solve time, primal and dual residuals, dual gap (if applicable))
- default value:
true
use_dual_gap::Bool
- (
MLSolver
only) Toggle if dual gap is used as the convergence criterion - default value:
false
- (
dual_gap_tol::Real
- (
MLSolver
only) Dual gap convergence tolerance - default value:
1e-4
- (
max_iters::Int
- Maximum number of allowed solver iterations
- default value:
1000
max_time_sec::Real
- Maximum allowed solve time
- default value:
1200.0
print_iter::Int
- Printing frequency. Every
print_iter
iterations, the solver prints the objective value, convergence criteria,ρ
value, and time. TheMLSolver
also prints the root mean squared error (square root of the average per-sample loss) - default value:
25
- Printing frequency. Every
rho_update_iter::Int
- Frequency at which the parameter
ρ
is updated - default value:
50
- Frequency at which the parameter
sketch_update_iter::Int
- Frequency at which the preconditioner is updated
- default value:
20
verbose::Bool
- Printing toggle
- default value:
true
multithreaded::Bool
- Multithreading toggle
- default value:
false
linsys_max_tol::Real
- Maximum relative error tolerance for the linear system solve. This criterion mostly matters for the first iteration; we use a criterion based on the square root of the residuals as soon as they are sufficiently small
- default value:
1e-1
eps_abs::Real
- Absolute stopping tolerance (applied to residuals)
- default value:
1e-4
eps_rel::Real
- Relative stopping tolerance (applied to residuals)
- default value:
1e-4
eps_inf::Real
- Infeasibility detection tolerance
- default value:
1e-8
norm_type::Real
- Norm to use for convergence criteria
- default value:
2
infeas_check_iter
- Frequency at which infeasibility is checked
- default value:
25
num_threads
- Number of threads used for BLAS operations
- default value:
Sys.CPU_THREADS
linsys_exponent
- Parameter that controls the inexactness in the x-subproblem solve.
- default value:
1.2
Nyström PCG Parameters
A subset of the SolverOptions
parameters deal with the preconditioner directly:
precondition::Bool
- Toggles if the linear system solve for the $x$-update is preconditioned
- default value:
true
update_preconditioner::Bool
- Preconditioner update toggle. If the Hessian is known to be a constant (e.g., in the lasso problem), this should be set to
false
. - default value:
true
- Preconditioner update toggle. If the Hessian is known to be a constant (e.g., in the lasso problem), this should be set to
init_sketch_size
::Int- Initial sketch size used for the preconditioner. Note that the solver will use the smaller of this and the side length divided by 10.
- default value:
50
use_adaptive_sketch::Bool
- If true, the solver resketches the matrix and increases the sketch size until a pre-defined tolerance is met.
- default value:
false
adaptive_sketch_tol::Real
- If using an adaptive sketch, increases the sketch rank until the estimated spectral norm is under this tolerance times $n^2$
- default value:
eps()