### This file is generated automatically by the packages `mkdoc.pl'.

adsmax
@c adsmax ../inst/adsmax.m
ADSMAX  Alternating directions method for direct search optimization.
       [x, fmax, nf] = ADSMAX(FUN, x0, STOPIT, SAVIT, P) attempts to
       maximize the function FUN, using the starting vector x0.
       The alternating directions direct search method is used.
       Output arguments:
              x    = vector yielding largest function value found,
              fmax = function value at x,
              nf   = number of function evaluations.
       The iteration is terminated when either
              - the relative increase in function value between successive
                iterations is <= STOPIT(1) (default 1e-3),
              - STOPIT(2) function evaluations have been performed
                (default inf, i.e., no limit), or
              - a function value equals or exceeds STOPIT(3)
                (default inf, i.e., no test on function values).
       Progress of the iteration is not shown if STOPIT(5) = 0 (default 1).
       If a non-empty fourth parameter string SAVIT is present, then
       `SAVE SAVIT x fmax nf' is executed after each inner iteration.
       By default, the search directions are the co-ordinate directions.
       The columns of a fifth parameter matrix P specify alternative search
       directions (P = EYE is the default).
       NB: x0 can be a matrix.  In the output argument, in SAVIT saves,
           and in function calls, x has the same shape as x0.
       ADSMAX(fun, x0, STOPIT, SAVIT, P, P1, P2,...) allows additional
       arguments to be passed to fun, via feval(fun,x,P1,P2,...).
    Reference:
    N. J. Higham, Optimization by direct search in matrix computations,
       SIAM J. Matrix Anal. Appl, 14(2): 317-333, 1993.
    N. J. Higham, Accuracy and Stability of Numerical Algorithms,
       Second edition, Society for Industrial and Applied Mathematics,
       Philadelphia, PA, 2002; sec. 20.5.
battery
@c battery ../inst/battery.m
battery.m: repeatedly call bfgs using a battery of 
start values, to attempt to find global min
of a nonconvex function

INPUTS:
func: function to mimimize
args: args of function
minarg: argument to minimize w.r.t. (usually = 1)
startvals: kxp matrix of values to try for sure (don't include all zeros, that's automatic)
max iters per start value
number of additional random start values to try

OUTPUT: theta - the best value found - NOT iterated to convergence
bfgsmin_example
@c bfgsmin_example ../inst/bfgsmin_example.m
usage: bfgsmin_example (to run) or edit bfgsmin_example (to examine)

Shows how to call bfgsmin. There are two objective functions, the first
supplies the analytic gradient, and the second does not. The true minimizer
is at "location", a 50x1 vector (0.00, 0.02, 0.04 ..., 1.00).
Note that limited memory bfgs is faster when the dimension is high.
Also note that supplying analytic derivatives gives a speedup.

Six examples are given:
Example 1: regular bfgs, analytic gradient
Example 2: same as Example 1, but verbose
Example 3: limited memory bfgs, analytic gradient
Example 4: regular bfgs, numeric gradient
Example 5: limited memory bfgs, numeric gradient
Example 6: regular bfgs, analytic gradient, minimize wrt second argument
bfgsmin
@c bfgsmin ../inst/bfgsmin.m
bfgsmin: bfgs or limited memory bfgs minimization of function

Usage: [x, obj_value, convergence, iters] = bfgsmin(f, args, control)

The function must be of the form
[value, return_2,..., return_m] = f(arg_1, arg_2,..., arg_n)
By default, minimization is w.r.t. arg_1, but it can be done
w.r.t. any argument that is a vector. Numeric derivatives are
used unless analytic derivatives are supplied. See bfgsmin_example.m
for methods.

Arguments:
* f: name of function to minimize (string)
* args: a cell array that holds all arguments of the function
	The argument with respect to which minimization is done
	MUST be a vector
* control: an optional cell array of 1-8 elements. If a cell
  array shorter than 8 elements is provided, the trailing elements
  are provided with default values.
	* elem 1: maximum iterations  (positive integer, or -1 or Inf for unlimited (default))
	* elem 2: verbosity
		0 = no screen output (default)
		1 = only final results
		2 = summary every iteration
		3 = detailed information
	* elem 3: convergence criterion
		1 = strict (function, gradient and param change) (default)
		0 = weak - only function convergence required
	* elem 4: arg in f_args with respect to which minimization is done (default is first)
	* elem 5: (optional) Memory limit for lbfgs. If it's a positive integer
		then lbfgs will be use. Otherwise ordinary bfgs is used
	* elem 6: function change tolerance, default 1e-12
	* elem 7: parameter change tolerance, default 1e-6
	* elem 8: gradient tolerance, default 1e-5

Returns:
* x: the minimizer
* obj_value: the value of f() at x
* convergence: 1 if normal conv, other values if not
* iters: number of iterations performed

Example: see bfgsmin_example.m
brent_line_min
@c brent_line_min ../inst/brent_line_min.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{s},@var{v},@var{n}]} brent_line_min ( @var{f},@var{df},@var{args},@var{ctl} )
Line minimization of f along df

Finds minimum of f on line @math{ x0 + dx*w | a < w < b } by
bracketing. a and b are passed through argument ctl.

@subheading Arguments
@itemize @bullet
@item @var{f}     : string : Name of function. Must return a real value
@item @var{args}  : cell   : Arguments passed to f or RxC    : f's only argument. x0 must be at @var{args}@{ @var{ctl}(2) @}
@item @var{ctl}   : 5      : (optional) Control variables, described below.
@end itemize

@subheading Returned values
@itemize @bullet
@item @var{s}   : 1        : Minimum is at x0 + s*dx
@item @var{v}   : 1        : Value of f at x0 + s*dx
@item @var{nev} : 1        : Number of function evaluations
@end itemize

@subheading Control Variables
@itemize @bullet
@item @var{ctl}(1)       : Upper bound for error on s              Default=sqrt(eps)
@item @var{ctl}(2)       : Position of minimized argument in args  Default= 1
@item @var{ctl}(3)       : Maximum number of function evaluations  Default= inf
@item @var{ctl}(4)       : a                                       Default=-inf
@item @var{ctl}(5)       : b                                       Default= inf
@end itemize

Default values will be used if ctl is not passed or if nan values are
given.
@end deftypefn
cdiff
@c cdiff ../inst/cdiff.m
c = cdiff (func,wrt,N,dfunc,stack,dx) - Code for num. differentiation
  = "function df = dfunc (var1,..,dvar,..,varN) .. endfunction

Returns a string of octave code that defines a function 'dfunc' that
returns the derivative of 'func' with respect to it's 'wrt'th
argument.

The derivatives are obtained by symmetric finite difference.

dfunc()'s return value is in the same format as that of  ndiff()

func  : string : name of the function to differentiate

wrt   : int    : position, in argument list, of the differentiation
                 variable.                                Default:1

N     : int    : total number of arguments taken by 'func'. 
                 If N=inf, dfunc will take variable argument list.
                                                        Default:wrt

dfunc : string : Name of the octave function that returns the
                  derivatives.                   Default:['d',func]

stack : string : Indicates whether 'func' accepts vertically
                 (stack="rstack") or horizontally (stack="cstack")
                 arguments. Any other string indicates that 'func'
                 does not allow stacking.                Default:''

dx    : real   : Step used in the symmetric difference scheme.
                                                 Default:10*sqrt(eps)

See also : ndiff, eval, todisk
cg_min
@c cg_min ../inst/cg_min.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{x0},@var{v},@var{nev}]} cg_min ( @var{f},@var{df},@var{args},@var{ctl} )
NonLinear Conjugate Gradient method to minimize function @var{f}.

@subheading Arguments
@itemize @bullet
@item @var{f}   : string   : Name of function. Return a real value 
@item @var{df}  : string   : Name of f's derivative. Returns a (R*C) x 1 vector 
@item @var{args}: cell     : Arguments passed to f.@*
@item @var{ctl}   : 5-vec    : (Optional) Control variables, described below
@end itemize

@subheading Returned values
@itemize @bullet
@item @var{x0}    : matrix   : Local minimum of f
@item @var{v}     : real     : Value of f in x0
@item @var{nev}   : 1 x 2    : Number of evaluations of f and of df
@end itemize

@subheading Control Variables
@itemize @bullet
@item @var{ctl}(1)       : 1 or 2 : Select stopping criterion amongst :
@item @var{ctl}(1)==0    : Default value
@item @var{ctl}(1)==1    : Stopping criterion : Stop search when value doesn't
improve, as tested by @math{ ctl(2) > Deltaf/max(|f(x)|,1) }
where Deltaf is the decrease in f observed in the last iteration
(each iteration consists R*C line searches).
@item @var{ctl}(1)==2    : Stopping criterion : Stop search when updates are small,
as tested by @math{ ctl(2) > max @{ dx(i)/max(|x(i)|,1) | i in 1..N @}}
where  dx is the change in the x that occured in the last iteration.
@item @var{ctl}(2)       : Threshold used in stopping tests.           Default=10*eps
@item @var{ctl}(2)==0    : Default value
@item @var{ctl}(3)       : Position of the minimized argument in args  Default=1
@item @var{ctl}(3)==0    : Default value
@item @var{ctl}(4)       : Maximum number of function evaluations      Default=inf
@item @var{ctl}(4)==0    : Default value
@item @var{ctl}(5)       : Type of optimization:
@item @var{ctl}(5)==1    : "Fletcher-Reves" method
@item @var{ctl}(5)==2    : "Polak-Ribiere" (Default)
@item @var{ctl}(5)==3    : "Hestenes-Stiefel" method
@end itemize

@var{ctl} may have length smaller than 4. Default values will be used if ctl is
not passed or if nan values are given.
@subheading Example:

function r=df( l )  b=[1;0;-1]; r = -( 2*l@{1@} - 2*b + rand(size(l@{1@}))); endfunction @*
function r=ff( l )  b=[1;0;-1]; r = (l@{1@}-b)' * (l@{1@}-b); endfunction @*
ll = @{ [10; 2; 3] @}; @*
ctl(5) = 3; @*
[x0,v,nev]=cg_min( "ff", "df", ll, ctl ) @*

Comment:  In general, BFGS method seems to be better performin in many cases but requires more computation per iteration
See also http://en.wikipedia.org/wiki/Nonlinear_conjugate_gradient.
@seealso{bfgsmin}
@end deftypefn
cpiv_bard
@c cpiv_bard ../inst/cpiv_bard.m
[lb, idx, ridx, mv] = cpiv_bard (v, m[, incl])

v: column vector; m: matrix; incl (optional): index. length (v)
must equal rows (m). Finds column vectors w and l with w == v + m *
l, w >= 0, l >= 0, l.' * w == 0. Chooses idx, w, and l so that
l(~idx) == 0, l(idx) == -inv (m(idx, idx)) * v(idx), w(idx) roughly
== 0, and w(~idx) == v(~idx) + m(idx, ~idx).' * l(idx). idx indexes
at least everything indexed by incl, but l(incl) may be < 0. lb:
l(idx) (column vector); idx: logical index, defined above; ridx:
~idx & w roughly == 0; mv: [m, v] after performing a Gauss-Jordan
'sweep' (with gjp.m) on each diagonal element indexed by idx.
Except the handling of incl (which enables handling of equality
constraints in the calling code), this is called solving the
'complementary pivot problem' (Cottle, R. W. and Dantzig, G. B.,
'Complementary pivot theory of mathematical programming', Linear
Algebra and Appl. 1, 102--125. References for the current
algorithm: Bard, Y.: Nonlinear Parameter Estimation, p. 147--149,
Academic Press, New York and London 1974; Bard, Y., 'An eclectic
approach to nonlinear programming', Proc. ANU Sem. Optimization,
Canberra, Austral. Nat. Univ.).
curvefit_stat
@c curvefit_stat ../inst/curvefit_stat.m
-*- texinfo -*-
@deftypefn {Function File} {@var{info} =} curvefit_stat (@var{f}, @var{p}, @var{x}, @var{y}, @var{settings})

Frontend for computation of statistics for fitting of values,
computed by a model function, to observed values.

Please refer to the description of @code{residmin_stat}. The only
differences to @code{residmin_stat} are the additional arguments
@var{x} (independent values) and @var{y} (observations), that the
model function @var{f}, if provided, has a second obligatory argument
which will be set to @var{x} and is supposed to return guesses for
the observations (with the same dimensions), and that the possibly
user-supplied function for the jacobian of the model function has
also a second obligatory argument which will be set to @var{x}.

@seealso {residmin_stat}
@end deftypefn
d2_min
@c d2_min ../inst/d2_min.m
[x,v,nev,h,args] = d2_min(f,d2f,args,ctl,code) - Newton-like minimization

Minimize f(x) using 1st and 2nd derivatives. Any function w/ second
derivatives can be minimized, as in Newton. f(x) decreases at each
iteration, as in Levenberg-Marquardt. This function is inspired from the
Levenberg-Marquardt algorithm found in the book "Numerical Recipes".

ARGUMENTS :
f    : string : Cost function's name

d2f  : string : Name of function returning the cost (1x1), its
                differential (1xN) and its second differential or it's
                pseudo-inverse (NxN) (see ctl(5) below) :

                [v,dv,d2v] = d2f (x).

args : list   : f and d2f's arguments. By default, minimize the 1st
    or matrix : argument.

ctl  : vector : Control arguments (see below)
     or struct

code : string : code will be evaluated after each outer loop that
                produced some (any) improvement. Variables visible from
                "code" include "x", the best parameter found, "v" the
                best value and "args", the list of all arguments. All can
                be modified. This option can be used to re-parameterize 
                the argument space during optimization

CONTROL VARIABLE ctl : (optional). May be a struct or a vector of length
---------------------- 5 or less where NaNs are ignored. Default values
                       are written <value>.
FIELD  VECTOR
NAME    POS

ftol, f N/A    : Stop search when value doesn't improve, as tested by

                  f > Deltaf/max(|f(x)|,1)

            where Deltaf is the decrease in f observed in the last
            iteration.                                     <10*sqrt(eps)>

utol, u N/A    : Stop search when updates are small, as tested by

                  u > max { dx(i)/max(|x(i)|,1) | i in 1..N }

            where  dx is the change in the x that occured in the last
            iteration.                                              <NaN>

dtol, d N/A    : Stop search when derivative is small, as tested by

                  d > norm (dv)                                     <eps>

crit, c ctl(1) : Set one stopping criterion, 'ftol' (c=1), 'utol' (c=2)
                 or 'dtol' (c=3) to the value of by the 'tol' option. <1>

tol, t  ctl(2) : Threshold in termination test chosen by 'crit'  <10*eps>

narg, n ctl(3) : Position of the minimized argument in args           <1>
maxev,m ctl(4) : Maximum number of function evaluations             <inf>
maxout,m       : Maximum number of outer loops                      <inf>
id2f, i ctl(5) : 0 if d2f returns the 2nd derivatives, 1 if           <0>
                 it returns its pseudo-inverse.

verbose, v N/A : Be more or less verbose (quiet=0)                    <0>
dcdp
@c dcdp ../inst/dcdp.m
function prt = dcdp (f, p, dp, func[, bounds])

This is an interface to __dfdp__.m, similar to dfdp.m, but for
functions only of parameters 'p', not of independents 'x'. See
dfdp.m.

dfpdp is more general and is meant to be used instead of dcdp in
optimization.
de_min
@c de_min ../inst/de_min.m
de_min: global optimisation using differential evolution

Usage: [x, obj_value, nfeval, convergence] = de_min(fcn, control)

minimization of a user-supplied function with respect to x(1:D),
using the differential evolution (DE) method based on an algorithm
by  Rainer Storn (http://www.icsi.berkeley.edu/~storn/code.html)
See: http://www.softcomputing.net/tevc2009_1.pdf


Arguments:  
---------------
fcn        string : Name of function. Must return a real value
control    vector : (Optional) Control variables, described below
        or struct

Returned values:
----------------
x          vector : parameter vector of best solution
obj_value  scalar : objective function value of best solution
nfeval     scalar : number of function evaluations
convergence       : 1 = best below value to reach (VTR)
                    0 = population has reached defined quality (tol)
                   -1 = some values are close to constraints/boundaries
                   -2 = max number of iterations reached (maxiter)
                   -3 = max number of functions evaluations reached (maxnfe)

Control variable:   (optional) may be named arguments (i.e. "name",value
----------------    pairs), a struct, or a vector, where
                    NaN's are ignored.

XVmin        : vector of lower bounds of initial population
               *** note: by default these are no constraints ***
XVmax        : vector of upper bounds of initial population
constr       : 1 -> enforce the bounds not just for the initial population
const        : data vector (remains fixed during the minimization)
NP           : number of population members
F            : difference factor from interval [0, 2]
CR           : crossover probability constant from interval [0, 1]
strategy     : 1 --> DE/best/1/exp           7 --> DE/best/1/bin
               2 --> DE/rand/1/exp           8 --> DE/rand/1/bin
               3 --> DE/target-to-best/1/exp 9 --> DE/target-to-best/1/bin
               4 --> DE/best/2/exp           10--> DE/best/2/bin
               5 --> DE/rand/2/exp           11--> DE/rand/2/bin
               6 --> DEGL/SAW/exp            else  DEGL/SAW/bin
refresh      : intermediate output will be produced after "refresh"
               iterations. No intermediate output will be produced
               if refresh is < 1
VTR          : Stopping criterion: "Value To Reach"
               de_min will stop when obj_value <= VTR.
               Use this if you know which value you expect.
tol          : Stopping criterion: "tolerance"
               stops if (best-worst)/max(1,worst) < tol
               This stops basically if the whole population is "good".
maxnfe       : maximum number of function evaluations
maxiter      : maximum number of iterations (generations)

      The algorithm seems to work well only if [XVmin,XVmax] covers the 
      region where the global minimum is expected.
      DE is also somewhat sensitive to the choice of the
      difference factor F. A good initial guess is to choose F from
      interval [0.5, 1], e.g. 0.8.
      CR, the crossover probability constant from interval [0, 1]
      helps to maintain the diversity of the population and is
      rather uncritical but affects strongly the convergence speed.
      If the parameters are correlated, high values of CR work better.
      The reverse is true for no correlation.
      Experiments suggest that /bin likes to have a slightly
      larger CR than /exp.
      The number of population members NP is also not very critical. A
      good initial guess is 10*D. Depending on the difficulty of the
      problem NP can be lower than 10*D or must be higher than 10*D
      to achieve convergence.

Default Values:
---------------
XVmin = [-2];
XVmax = [ 2];
constr= 0;
const = [];
NP    = 10 *D
F     = 0.8;
CR    = 0.9;
strategy = 12;
refresh  = 0;
VTR   = -Inf;
tol   = 1.e-3;
maxnfe  = 1e6;
maxiter = 1000;


Example to find the minimum of the Rosenbrock saddle:
----------------------------------------------------
Define f as:
                   function result = f(x);
                     result = 100 * (x(2) - x(1)^2)^2 + (1 - x(1))^2;
                   end
Then type:

	ctl.XVmin = [-2 -2];
	ctl.XVmax = [ 2  2];
	[x, obj_value, nfeval, convergence] = de_min (@f, ctl);

Keywords: global-optimisation optimisation minimisation
deriv
@c deriv ../inst/deriv.m
-*- texinfo -*-
@deftypefn {Function File} {@var{dx} =} deriv (@var{f}, @var{x0})
@deftypefnx {Function File} {@var{dx} =} deriv (@var{f}, @var{x0}, @var{h})
@deftypefnx {Function File} {@var{dx} =} deriv (@var{f}, @var{x0}, @var{h}, @var{O})
@deftypefnx {Function File} {@var{dx} =} deriv (@var{f}, @var{x0}, @var{h}, @var{O}, @var{N})
Calculate derivate of function @var{f}.

@var{f} must be a function handle or the name of a function that takes @var{x0}
and returns a variable of equal length and orientation. @var{x0} must be a
numeric vector or scalar.

@var{h} defines the step taken for the derivative calculation. Defaults to 1e-7.

@var{O} defines the order of the calculation. Supported values are 2 (h^2 order)
or 4 (h^4 order). Defaults to 2.

@var{N} defines the derivative order. Defaults to the 1st derivative of the
function. Can be up to the 4th derivative.

Reference: Numerical Methods for Mathematics, Science, and Engineering by
John H. Mathews.
@end deftypefn
dfdp
@c dfdp ../inst/dfdp.m
function prt = dfdp (x, f, p, dp, func[, bounds])
numerical partial derivatives (Jacobian) df/dp for use with leasqr
--------INPUT VARIABLES---------
x=vec or matrix of indep var(used as arg to func) x=[x0 x1 ....]
f=func(x,p) vector initialsed by user before each call to dfdp
p= vec of current parameter values
dp= fractional increment of p for numerical derivatives
     dp(j)>0 central differences calculated
     dp(j)<0 one sided differences calculated
     dp(j)=0 sets corresponding partials to zero; i.e. holds p(j) fixed
func=function (string or handle) to calculate the Jacobian for,
     e.g. to calc Jacobian for function expsum prt=dfdp(x,f,p,dp,'expsum')
bounds=two-column-matrix of lower and upper bounds for parameters
     If no 'bounds' options is specified to leasqr, it will call
     dfdp without the 'bounds' argument.
----------OUTPUT VARIABLES-------
prt= Jacobian Matrix prt(i,j)=df(i)/dp(j)
================================

dfxpdp is more general and is meant to be used instead of dfdp in
optimization.
dfpdp
@c dfpdp ../inst/dfpdp.m
function jac = dfpdp (p, func[, hook])

Returns Jacobian of func (p) with respect to p with finite
differencing. The optional argument hook is a structure which can
contain the following fields at the moment:

hook.f: value of func(p) for p as given in the arguments

hook.diffp: positive vector of fractional steps from given p in
finite differencing (actual steps may be smaller if bounds are
given). The default is .001 * ones (size (p)).

hook.diff_onesided: logical vector, indexing elements of p for
which only one-sided differences should be computed (faster); even
if not one-sided, differences might not be exactly central if
bounds are given. The default is false (size (p)).

hook.fixed: logical vector, indexing elements of p for which zero
should be returned instead of the guessed partial derivatives
(useful in optimization if some parameters are not optimized, but
are 'fixed').

hook.lbound, hook.ubound: vectors of lower and upper parameter
bounds (or -Inf or +Inf, respectively) to be respected in finite
differencing. The consistency of bounds is not checked.
dfxpdp
@c dfxpdp ../inst/dfxpdp.m
function jac = dfxpdp (x, p, func[, hook])

Returns Jacobian of func (p, x) with respect to p with finite
differencing. The optional argument hook is a structure which can
contain the following fields at the moment:

hook.f: value of func(p, x) for p and x as given in the arguments

hook.diffp: positive vector of fractional steps from given p in
finite differencing (actual steps may be smaller if bounds are
given). The default is .001 * ones (size (p));

hook.diff_onesided: logical vector, indexing elements of p for
which only one-sided differences should be computed (faster); even
if not one-sided, differences might not be exactly central if
bounds are given. The default is false (size (p)).

hook.fixed: logical vector, indexing elements of p for which zero
should be returned instead of the guessed partial derivatives
(useful in optimization if some parameters are not optimized, but
are 'fixed').

hook.lbound, hook.ubound: vectors of lower and upper parameter
bounds (or -Inf or +Inf, respectively) to be respected in finite
differencing. The consistency of bounds is not checked.
expfit
@c expfit ../inst/expfit.m
USAGE  [alpha,c,rms] = expfit( deg, x1, h, y )

Prony's method for non-linear exponential fitting

Fit function:   \sum_1^{deg} c(i)*exp(alpha(i)*x)

Elements of data vector y must correspond to
equidistant x-values starting at x1 with stepsize h

The method is fully compatible with complex linear
coefficients c, complex nonlinear coefficients alpha
and complex input arguments y, x1, non-zero h .
Fit-order deg  must be a real positive integer.

Returns linear coefficients c, nonlinear coefficients
alpha and root mean square error rms. This method is
known to be more stable than 'brute-force' non-linear
least squares fitting.

Example
   x0 = 0; step = 0.05; xend = 5; x = x0:step:xend;
   y = 2*exp(1.3*x)-0.5*exp(2*x);
   error = (rand(1,length(y))-0.5)*1e-4;
   [alpha,c,rms] = expfit(2,x0,step,y+error)

 alpha =
   2.0000
   1.3000
 c =
   -0.50000
    2.00000
 rms = 0.00028461

The fit is very sensitive to the number of data points.
It doesn't perform very well for small data sets.
Theoretically, you need at least 2*deg data points, but
if there are errors on the data, you certainly need more.

Be aware that this is a very (very,very) ill-posed problem.
By the way, this algorithm relies heavily on computing the
roots of a polynomial. I used 'roots.m', if there is
something better please use that code.

Demo for a complex fit-function:
deg= 2; N= 20; x1= -(1+i), x= linspace(x1,1+i/2,N).';
h = x(2) - x(1)
y= (2+i)*exp( (-1-2i)*x ) + (-1+3i)*exp( (2+3i)*x );
A= 5e-2; y+= A*(randn(N,1)+randn(N,1)*i); % add complex noise
[alpha,c,rms]= expfit( deg, x1, h, y )
fmins
@c fmins ../inst/fmins.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{x}] =} fmins (@var{f},@var{X0},@var{options},@var{grad},@var{P1},@var{P2}, ...)

Find the minimum of a funtion of several variables.
By default the method used is the Nelder&Mead Simplex algorithm

Example usage:
  fmins(inline('(x(1)-5).^2+(x(2)-8).^4'),[0;0])

@strong{Inputs}
@c use @asis and explicite @var in @table to avoid makeinfo warning
@c `unlikely character , in @var' for `P1, P2, ...'.
@table @asis 
@item @var{f} 
A string containing the name of the function to minimize
@item @var{X0}
A vector of initial parameters fo the function @var{f}.
@item @var{options}
Vector with control parameters (not all parameters are used)
@verbatim
options(1) - Show progress (if 1, default is 0, no progress)
options(2) - Relative size of simplex (default 1e-3)
options(6) - Optimization algorithm
   if options(6)==0 - Nelder & Mead simplex (default)
   if options(6)==1 - Multidirectional search Method
   if options(6)==2 - Alternating Directions search
options(5)
   if options(6)==0 && options(5)==0 - regular simplex
   if options(6)==0 && options(5)==1 - right-angled simplex
      Comment: the default is set to "right-angled simplex".
        this works better for me on a broad range of problems,
        although the default in nmsmax is "regular simplex"
options(10) - Maximum number of function evaluations
@end verbatim
@item @var{grad}
Unused (For compatibility with Matlab)
@item @var{P1}, @var{P2}, ...
Optional parameters for function @var{f} 

@end table
@end deftypefn
gjp
@c gjp ../inst/gjp.m
m = gjp (m, k[, l])

m: matrix; k, l: row- and column-index of pivot, l defaults to k.

Gauss-Jordon pivot as defined in Bard, Y.: Nonlinear Parameter
Estimation, p. 296, Academic Press, New York and London 1974. In
the pivot column, this seems not quite the same as the usual
Gauss-Jordan(-Clasen) pivot. Bard gives Beaton, A. E., 'The use of
special matrix operators in statistical calculus' Research Bulletin
RB-64-51 (1964), Educational Testing Service, Princeton, New Jersey
as a reference, but this article is not easily accessible. Another
reference, whose definition of gjp differs from Bards by some
signs, is Clarke, R. B., 'Algorithm AS 178: The Gauss-Jordan sweep
operator with detection of collinearity', Journal of the Royal
Statistical Society, Series C (Applied Statistics) (1982), 31(2),
166--168.
jacobs
@c jacobs ../inst/jacobs.m
-*- texinfo -*-
@deftypefn {Function File} {Df =} jacobs (@var{x}, @var{f})
@deftypefnx {Function File} {Df =} jacobs (@var{x}, @var{f}, @var{hook})
Calculate the jacobian of a function using the complex step method.

Let @var{f} be a user-supplied function. Given a point @var{x} at
which we seek for the Jacobian, the function @command{jacobs} returns
the Jacobian matrix @code{d(f(1), @dots{}, df(end))/d(x(1), @dots{},
x(n))}. The function uses the complex step method and thus can be
applied to real analytic functions.

The optional argument @var{hook} is a structure with additional options. @var{hook}
can have the following fields:
@itemize @bullet
@item
@code{h} - can be used to define the magnitude of the complex step and defaults
to 1e-20; steps larger than 1e-3 are not allowed.
@item
@code{fixed} - is a logical vector internally usable by some optimization
functions; it indicates for which elements of @var{x} no gradient should be
computed, but zero should be returned.
@end itemize

For example:

@example
@group
f = @@(x) [x(1)^2 + x(2); x(2)*exp(x(1))];
Df = jacobs ([1, 2], f)
@end group
@end example
@end deftypefn
leasqr
@c leasqr ../inst/leasqr.m
-*- texinfo -*-
@deftypefn  {Function File} {} leasqr (@var{x}, @var{y}, @var{pin}, @var{F})
@deftypefnx {Function File} {} leasqr (@var{x}, @var{y}, @var{pin}, @var{F}, @var{stol})
@deftypefnx {Function File} {} leasqr (@var{x}, @var{y}, @var{pin}, @var{F}, @var{stol}, @var{niter})
@deftypefnx {Function File} {} leasqr (@var{x}, @var{y}, @var{pin}, @var{F}, @var{stol}, @var{niter}, @var{wt})
@deftypefnx {Function File} {} leasqr (@var{x}, @var{y}, @var{pin}, @var{F}, @var{stol}, @var{niter}, @var{wt}, @var{dp})
@deftypefnx {Function File} {} leasqr (@var{x}, @var{y}, @var{pin}, @var{F}, @var{stol}, @var{niter}, @var{wt}, @var{dp}, @var{dFdp})
@deftypefnx {Function File} {} leasqr (@var{x}, @var{y}, @var{pin}, @var{F}, @var{stol}, @var{niter}, @var{wt}, @var{dp}, @var{dFdp}, @var{options})
@deftypefnx {Function File} {[@var{f}, @var{p}, @var{cvg}, @var{iter}, @var{corp}, @var{covp}, @var{covr}, @var{stdresid}, @var{Z}, @var{r2}] =} leasqr (@dots{})
Levenberg-Marquardt nonlinear regression.

Input arguments:

@table @var
@item x
Vector or matrix of independent variables.

@item y
Vector or matrix of observed values.

@item pin
Vector of initial parameters to be adjusted by leasqr.

@item F
Name of function or function handle. The function must be of the form
@code{y = f(x, p)}, with y, x, p of the form @var{y}, @var{x}, @var{pin}.

@item stol
Scalar tolerance on fractional improvement in scalar sum of squares, i.e.,
@code{sum ((@var{wt} .* (@var{y}-@var{f}))^2)}.  Set to 0.0001 if
empty or not given;

@item niter
Maximum number of iterations.  Set to 20 if empty or not given.

@item wt
Statistical weights (same dimensions as @var{y}).  These should be
set to be proportional to @code{sqrt (@var{y}) ^-1}, i.e., the
covariance matrix of the data is assumed to be proportional to
diagonal with diagonal equal to @code{(@var{wt}.^2)^-1}.  The constant of
proportionality will be estimated.  Set to @code{ones (size
(@var{y}))} if empty or not given.

@item dp
Fractional increment of @var{p} for numerical partial derivatives.  Set
to @code{0.001 * ones (size (@var{pin}))} if empty or not given.

@itemize @bullet
@item dp(j) > 0 means central differences on j-th parameter p(j).
@item dp(j) < 0 means one-sided differences on j-th parameter p(j).
@item dp(j) = 0 holds p(j) fixed, i.e., leasqr won't change initial guess: pin(j)
@end itemize

@item dFdp
Name of partial derivative function in quotes or function handle. If
not given or empty, set to @code{dfdp}, a slow but general partial
derivatives function. The function must be of the form @code{prt =
dfdp (x, f, p, dp, F [,bounds])}.  For backwards compatibility, the
function will only be called with an extra 'bounds' argument if the
'bounds' option is explicitly specified to leasqr (see dfdp.m).

@item options
Structure with multiple options. The following fields are recognized:

@table @asis
@item @qcode{fract_prec}
Column vector (same length as @var{pin})
of desired fractional precisions in parameter estimates.
Iterations are terminated if change in parameter vector (chg)
relative to current parameter estimate is less than their
corresponding elements in 'fract_prec', i.e.,
@code{all (abs (chg) < abs (options.fract_prec .* current_parm_est))} on two
consecutive iterations. Defaults to @code{zeros (size (@var{pin}))}.

@item @qcode{max_fract_change}
Column vector (same length as @var{pin}) of maximum fractional step
changes in parameter vector.
Fractional change in elements of parameter vector is constrained to
be at most 'max_fract_change' between sucessive iterations, i.e.,
@code{abs (chg(i)) = abs (min([chg(i), options.max_fract_change(i) * current param estimate]))}.
Defaults to @code{Inf * ones (size (@var{pin}))}.

@item @qcode{inequc}
Cell-array containing up to four entries,
two entries for linear inequality constraints and/or one or two
entries for general inequality constraints.  Initial parameters
must satisfy these constraints.  Either linear or general
constraints may be the first entries, but the two entries for
linear constraints must be adjacent and, if two entries are given
for general constraints, they also must be adjacent.  The two
entries for linear constraints are a matrix (say m) and a vector
(say v), specifying linear inequality constraints of the form
`m.' * parameters + v >= 0'. If the constraints are just bounds,
it is suggested to specify them in 'options.bounds' instead,
since then some sanity tests are performed, and since the
function 'dfdp.m' is guarantied not to violate constraints during
determination of the numeric gradient only for those constraints
specified as 'bounds' (possibly with violations due to a certain
inaccuracy, however, except if no constraints except bounds are
specified). The first entry for general constraints must be a
differentiable vector valued function (say h), specifying general
inequality constraints of the form `h (p[, idx]) >= 0'; p is the
column vector of optimized paraters and the optional argument idx
is a logical index. h has to return the values of all constraints
if idx is not given, and has to return only the indexed
constraints if idx is given (so computation of the other
constraints can be spared). If a second entry for general
constraints is given, it must be a function (say dh) which
returnes a matrix whos rows contain the gradients of the
constraint function h with respect to the optimized parameters.
It has the form jac_h = dh (vh, p, dp, h, idx[, bounds]); p is
the column vector of optimized parameters, and idx is a logical
index --- only the rows indexed by idx must be returned (so
computation of the others can be spared). The other arguments of
dh are for the case that dh computes numerical gradients: vh is
the column vector of the current values of the constraint
function h, with idx already applied. h is a function h (p) to
compute the values of the constraints for parameters p, it will
return only the values indexed by idx. dp is a suggestion for
relative step width, having the same value as the argument 'dp'
of leasqr above. If bounds were specified to leasqr, they are
provided in the argument bounds of dh, to enable their
consideration in determination of numerical gradients. If dh is
not specified to leasqr, numerical gradients are computed in the
same way as with 'dfdp.m' (see above). If some constraints are
linear, they should be specified as linear constraints (or
bounds, if applicable) for reasons of performance, even if
general constraints are also specified.

@item @qcode{bounds}
Two-column-matrix, one row for each
parameter in @var{pin}. Each row contains a minimal and maximal value
for each parameter. Default: [-Inf, Inf] in each row. If this
field is used with an existing user-side function for 'dFdp'
(see above) the functions interface might have to be changed.

@item @qcode{equc}
Equality constraints, specified the same
way as inequality constraints (see field 'options.inequc').
Initial parameters must satisfy these constraints.
Note that there is possibly a certain inaccuracy in honoring
constraints, except if only bounds are specified.
@emph{Warning}: If constraints (or bounds) are set, returned guesses
of @var{corp}, @var{covp}, and @var{Z} are generally invalid, even if
no constraints
are active for the final parameters. If equality constraints are
specified, @var{corp}, @var{covp}, and @var{Z} are not guessed at all.

@item @qcode{cpiv}
Function for complementary pivot algorithm
for inequality constraints. Defaults to cpiv_bard.  No different
function is supplied.

@end table

For backwards compatibility, @var{options} can also be a matrix whose
first and second column contains the values of @qcode{fract_prec} and
@qcode{max_fract_change}, respectively.

@end table

Output:

@table @var
@item f
Column vector of values computed: f = F(x,p).

@item p
Column vector trial or final parameters, i.e, the solution.

@item cvg
Scalar: = 1 if convergence, = 0 otherwise.

@item iter
Scalar number of iterations used.

@item corp
Correlation matrix for parameters.

@item covp
Covariance matrix of the parameters.

@item covr
Diag(covariance matrix of the residuals).

@item stdresid
Standardized residuals.

@item Z
Matrix that defines confidence region (see comments in the source).

@item r2
Coefficient of multiple determination, intercept form.

@end table

Not suitable for non-real residuals.

References:
Bard, Nonlinear Parameter Estimation, Academic Press, 1974.
Draper and Smith, Applied Regression Analysis, John Wiley and Sons, 1981.

@end deftypefn
LinearRegression
@c LinearRegression ../inst/LinearRegression.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{p},@var{e_var},@var{r},@var{p_var},@var{y_var}] =} LinearRegression (@var{F},@var{y})
@deftypefnx {Function File} {[@var{p},@var{e_var},@var{r},@var{p_var},@var{y_var}] =} LinearRegression (@var{F},@var{y},@var{w})


general linear regression

determine the parameters p_j  (j=1,2,...,m) such that the function
f(x) = sum_(i=1,...,m) p_j*f_j(x) is the best fit to the given values y_i = f(x_i)

parameters:  
@itemize
@item @var{F} is an n*m matrix with the values of the basis functions at
the support points. In column j give the values of f_j at the points
x_i  (i=1,2,...,n)
@item @var{y} is a column vector of length n with the given values
@item @var{w} is n column vector of of length n vector with the weights of data points
@end itemize

return values:
@itemize
@item @var{p} is the vector of length m with the estimated values of the parameters
@item @var{e_var} is the estimated variance of the difference between fitted and measured values
@item @var{r} is the weighted norm of the residual
@item @var{p_var} is the estimated variance of the parameters p_j
@item @var{y_var} is the estimated variance of the dependend variables
@end itemize

 Caution:  
 do NOT request @var{y_var} for large data sets, as a n by n matrix is
 generated

@c Will be cut out in optims info file and replaced with the same
@c refernces explicitely there, since references to core Octave
@c functions are not automatically transformed from here to there.
@c BEGIN_CUT_TEXINFO
@seealso{regress,leasqr,nonlin_curvefit,polyfit,wpolyfit,expfit}
@c END_CUT_TEXINFO
@end deftypefn
line_min
@c line_min ../inst/line_min.m
[a,fx,nev] = line_min (f, dx, args, narg, h, nev_max) - Minimize f() along dx

INPUT ----------
f    : string  : Name of minimized function
dx   : matrix  : Direction along which f() is minimized
args : cell    : Arguments of f
narg : integer : Position of minimized variable in args.  Default=1
h    : scalar  : Step size to use for centered finite difference
approximation of first and second derivatives. Default=1E-3.
nev_max : integer : Maximum number of function evaluations.  Default=30

OUTPUT ---------
a    : scalar  : Value for which f(x+a*dx) is a minimum (*)
fx   : scalar  : Value of f(x+a*dx) at minimum (*)
nev  : integer : Number of function evaluations

(*) The notation f(x+a*dx) assumes that args == {x}.

Reference: David G Luenberger's Linear and Nonlinear Programming
linprog
@c linprog ../inst/linprog.m
-*- texinfo -*-
@deftypefn{Function File} {@var{x} =} linprog (@var{f}, @var{A}, @var{b})
@deftypefnx{Function File} {@var{x} =} linprog (@var{f}, @var{A}, @var{b}, @var{Aeq}, @var{beq})
@deftypefnx{Function File} {@var{x} =} linprog (@var{f}, @var{A}, @var{b}, @var{Aeq}, @var{beq}, @var{lb}, @var{ub})
@deftypefnx{Function File} {[@var{x}, @var{fval}] =} linprog (@dots{})
Solve a linear problem.

Finds

@example
min (f' * x)
@end example

(both f and x are column vectors) subject to

@example
@group
A   * x <= b
Aeq * x  = beq
lb <= x <= ub
@end group
@end example

If not specified, @var{Aeq} and @var{beq} default to empty matrices.

If not specified, the lower bound @var{lb} defaults to minus infinite
and the upper bound @var{ub} defaults to infinite.

@c Will be cut out in optims info file and replaced with the same
@c refernces explicitely there, since references to core Octave
@c functions are not automatically transformed from here to there.
@c BEGIN_CUT_TEXINFO
@seealso{glpk}
@c END_CUT_TEXINFO
@end deftypefn
mdsmax
@c mdsmax ../inst/mdsmax.m
MDSMAX  Multidirectional search method for direct search optimization.
       [x, fmax, nf] = MDSMAX(FUN, x0, STOPIT, SAVIT) attempts to
       maximize the function FUN, using the starting vector x0.
       The method of multidirectional search is used.
       Output arguments:
              x    = vector yielding largest function value found,
              fmax = function value at x,
              nf   = number of function evaluations.
       The iteration is terminated when either
              - the relative size of the simplex is <= STOPIT(1)
                (default 1e-3),
              - STOPIT(2) function evaluations have been performed
                (default inf, i.e., no limit), or
              - a function value equals or exceeds STOPIT(3)
                (default inf, i.e., no test on function values).
       The form of the initial simplex is determined by STOPIT(4):
         STOPIT(4) = 0: regular simplex (sides of equal length, the default),
         STOPIT(4) = 1: right-angled simplex.
       Progress of the iteration is not shown if STOPIT(5) = 0 (default 1).
       If a non-empty fourth parameter string SAVIT is present, then
       `SAVE SAVIT x fmax nf' is executed after each inner iteration.
       NB: x0 can be a matrix.  In the output argument, in SAVIT saves,
           and in function calls, x has the same shape as x0.
       MDSMAX(fun, x0, STOPIT, SAVIT, P1, P2,...) allows additional
       arguments to be passed to fun, via feval(fun,x,P1,P2,...).

This implementation uses 2n^2 elements of storage (two simplices), where x0
is an n-vector.  It is based on the algorithm statement in [2, sec.3],
modified so as to halve the storage (with a slight loss in readability).

References:
[1] V. J. Torczon, Multi-directional search: A direct search algorithm for
    parallel machines, Ph.D. Thesis, Rice University, Houston, Texas, 1989.
[2] V. J. Torczon, On the convergence of the multidirectional search
    algorithm, SIAM J. Optimization, 1 (1991), pp. 123-145.
[3] N. J. Higham, Optimization by direct search in matrix computations,
    SIAM J. Matrix Anal. Appl, 14(2): 317-333, 1993.
[4] N. J. Higham, Accuracy and Stability of Numerical Algorithms,
       Second edition, Society for Industrial and Applied Mathematics,
       Philadelphia, PA, 2002; sec. 20.5.
minimize
@c minimize ../inst/minimize.m
[x,v,nev,...] = minimize (f,args,...) - Minimize f

ARGUMENTS
f    : string  : Name of function. Must return a real value
args : list or : List of arguments to f (by default, minimize the first)
       matrix  : f's only argument

RETURNED VALUES
x   : matrix  : Local minimum of f. Let's suppose x is M-by-N.
v   : real    : Value of f in x0
nev : integer : Number of function evaluations 
    or 1 x 2  : Number of function and derivative evaluations (if
                derivatives are used)


Extra arguments are either a succession of option-value pairs or a single
list or struct of option-value pairs (for unary options, the value in the
struct is ignored).

OPTIONS : DERIVATIVES   Derivatives may be used if one of these options
---------------------   uesd. Otherwise, the Nelder-Mean (see
                        nelder_mead_min) method is used.

'd2f', d2f     : Name of a function that returns the value of f, of its
                 1st and 2nd derivatives : [fx,dfx,d2fx] = feval (d2f, x)
                 where fx is a real number, dfx is 1x(M*N) and d2fx is
                 (M*N)x(M*N). A Newton-like method (d2_min) will be used.

'hess'         : Use [fx,dfx,d2fx] = leval (f, args) to compute 1st and
                 2nd derivatives, and use a Newton-like method (d2_min).

'd2i', d2i     : Name of a function that returns the value of f, of its
                 1st and pseudo-inverse of second derivatives : 
                 [fx,dfx,id2fx] = feval (d2i, x) where fx is a real
                 number, dfx is 1x(M*N) and d2ix is (M*N)x(M*N).
                 A Newton-like method will be used (see d2_min).

'ihess'        : Use [fx,dfx,id2fx] = leval (f, args) to compute 1st
                 derivative and the pseudo-inverse of 2nd derivatives,
                 and use a Newton-like method (d2_min).

           NOTE : df, d2f or d2i take the same arguments as f.

'order', n     : Use derivatives of order n. If the n'th order derivative
                 is not specified by 'df', 'd2f' or 'd2i', it will be
                 computed numerically. Currently, only order 1 works.

'ndiff'        : Use a variable metric method (bfgs) using numerical
                 differentiation.

OPTIONS : STOPPING CRITERIA  Default is to use 'tol'
---------------------------
'ftol', ftol   : Stop search when value doesn't improve, as tested by

             ftol > Deltaf/max(|f(x)|,1)

                where Deltaf is the decrease in f observed in the last
                iteration.                                 Default=10*eps

'utol', utol   : Stop search when updates are small, as tested by

             tol > max { dx(i)/max(|x(i)|,1) | i in 1..N }

                where  dx is the change in the x that occured in the last
                iteration.

'dtol',dtol    : Stop search when derivatives are small, as tested by

             dtol > max { df(i)*max(|x(i)|,1)/max(v,1) | i in 1..N }

                where x is the current minimum, v is func(x) and df is
                the derivative of f in x. This option is ignored if
                derivatives are not used in optimization.

MISC. OPTIONS
-------------
'maxev', m     : Maximum number of function evaluations             <inf>

'narg' , narg  : Position of the minimized argument in args           <1>
'isz'  , step  : Initial step size (only for 0 and 1st order method)  <1>
                 Should correspond to expected distance to minimum
'verbose'      : Display messages during execution

'backend'      : Instead of performing the minimization itself, return
                 [backend, control], the name and control argument of the
                 backend used by minimize(). Minimimzation can then be
                 obtained without the overhead of minimize by calling, if
                 a 0 or 1st order method is used :

             [x,v,nev] = feval (backend, args, control)
                  
                 or, if a 2nd order method is used :

             [x,v,nev] = feval (backend, control.d2f, args, control)
nelder_mead_min
@c nelder_mead_min ../inst/nelder_mead_min.m
[x0,v,nev] = nelder_mead_min (f,args,ctl) - Nelder-Mead minimization

Minimize 'f' using the Nelder-Mead algorithm. This function is inspired
from the that found in the book "Numerical Recipes".

ARGUMENTS
---------
f     : string : Name of function. Must return a real value
args  : list   : Arguments passed to f.
     or matrix : f's only argument
ctl   : vector : (Optional) Control variables, described below
     or struct

RETURNED VALUES
---------------
x0  : matrix   : Local minimum of f
v   : real     : Value of f in x0
nev : number   : Number of function evaluations

CONTROL VARIABLE : (optional) may be named arguments (i.e. "name",value
------------------ pairs), a struct, or a vector of length <= 6, where
                   NaN's are ignored. Default values are written <value>.
 OPT.   VECTOR
 NAME    POS
ftol,f  N/A    : Stopping criterion : stop search when values at simplex
                 vertices are all alike, as tested by 

                  f > (max_i (f_i) - min_i (f_i)) /max(max(|f_i|),1)

                 where f_i are the values of f at the vertices.  <10*eps>

rtol,r  N/A    : Stop search when biggest radius of simplex, using
                 infinity-norm, is small, as tested by :

             ctl(2) > Radius                                     <10*eps>

vtol,v  N/A    : Stop search when volume of simplex is small, tested by
           
             ctl(2) > Vol

crit,c ctl(1)  : Set one stopping criterion, 'ftol' (c=1), 'rtol' (c=2)
                 or 'vtol' (c=3) to the value of the 'tol' option.    <1>

tol, t ctl(2)  : Threshold in termination test chosen by 'crit'  <10*eps>

narg  ctl(3)  : Position of the minimized argument in args            <1>
maxev ctl(4)  : Maximum number of function evaluations. This number <inf>
                may be slightly exceeded.
isz   ctl(5)  : Size of initial simplex, which is :                   <1>

               { x + e_i | i in 0..N } 

               Where x == args{narg} is the initial value 
                e_0    == zeros (size (x)), 
                e_i(j) == 0 if j != i and e_i(i) == ctl(5)
                e_i    has same size as x

               Set ctl(5) to the distance you expect between the starting
               point and the minimum.

rst   ctl(6)   : When a minimum is found the algorithm restarts next to
                 it until the minimum does not improve anymore. ctl(6) is
                 the maximum number of restarts. Set ctl(6) to zero if
                 you know the function is well-behaved or if you don't
                 mind not getting a true minimum.                     <0>

verbose, v     Be more or less verbose (quiet=0)                      <0>
nmsmax
@c nmsmax ../inst/nmsmax.m
NMSMAX  Nelder-Mead simplex method for direct search optimization.
       [x, fmax, nf] = NMSMAX(FUN, x0, STOPIT, SAVIT) attempts to
       maximize the function FUN, using the starting vector x0.
       The Nelder-Mead direct search method is used.
       Output arguments:
              x    = vector yielding largest function value found,
              fmax = function value at x,
              nf   = number of function evaluations.
       The iteration is terminated when either
              - the relative size of the simplex is <= STOPIT(1)
                (default 1e-3),
              - STOPIT(2) function evaluations have been performed
                (default inf, i.e., no limit), or
              - a function value equals or exceeds STOPIT(3)
                (default inf, i.e., no test on function values).
       The form of the initial simplex is determined by STOPIT(4):
          STOPIT(4) = 0: regular simplex (sides of equal length, the default)
          STOPIT(4) = 1: right-angled simplex.
       Progress of the iteration is not shown if STOPIT(5) = 0 (default 1).
          STOPIT(6) indicates the direction (ie. minimization or 
                  maximization.) Default is 1, maximization.
                  set STOPIT(6)=-1 for minimization
       If a non-empty fourth parameter string SAVIT is present, then
       `SAVE SAVIT x fmax nf' is executed after each inner iteration.
       NB: x0 can be a matrix.  In the output argument, in SAVIT saves,
           and in function calls, x has the same shape as x0.
       NMSMAX(fun, x0, STOPIT, SAVIT, P1, P2,...) allows additional
       arguments to be passed to fun, via feval(fun,x,P1,P2,...).
References:
N. J. Higham, Optimization by direct search in matrix computations,
   SIAM J. Matrix Anal. Appl, 14(2): 317-333, 1993.
C. T. Kelley, Iterative Methods for Optimization, Society for Industrial
   and Applied Mathematics, Philadelphia, PA, 1999.
nonlin_curvefit
@c nonlin_curvefit ../inst/nonlin_curvefit.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{p}, @var{fy}, @var{cvg}, @var{outp}] =} nonlin_curvefit (@var{f}, @var{pin}, @var{x}, @var{y})
@deftypefnx {Function File} {[@var{p}, @var{fy}, @var{cvg}, @var{outp}] =} nonlin_curvefit (@var{f}, @var{pin}, @var{x}, @var{y}, @var{settings})
Frontend for nonlinear fitting of values, computed by a model
function, to observed values.

Please refer to the description of @code{nonlin_residmin}. The
differences to @code{nonlin_residmin} are the additional arguments
@var{x} (independent values, mostly, but not necessarily, an array of
the same dimensions or the same number of rows as @var{y}) and
@var{y} (array of observations), the returned value @var{fy} (final
guess for observed values) instead of @var{resid}, that the model
function has a second obligatory argument which will be set to
@var{x} and is supposed to return guesses for the observations (with
the same dimensions), and that the possibly user-supplied function
for the jacobian of the model function has also a second obligatory
argument which will be set to @var{x}.

@c The following block will be cut out in the package info file.
@c BEGIN_CUT_TEXINFO

Also, if the setting @code{user_interaction} is given, additional
information is passed to these functions. Type @code{optim_doc
("Common optimization options")} for this setting.

@c END_CUT_TEXINFO

@seealso {nonlin_residmin}
@end deftypefn
nonlin_min
@c nonlin_min ../inst/nonlin_min.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{p}, @var{objf}, @var{cvg}, @var{outp}] =} nonlin_min (@var{f}, @var{pin})
@deftypefnx {Function File} {[@var{p}, @var{objf}, @var{cvg}, @var{outp}] =} nonlin_min (@var{f}, @var{pin}, @var{settings})
Frontend for nonlinear minimization of a scalar objective function.

The functions supplied by the user have a minimal interface; any
additionally needed constants can be supplied by wrapping the user
functions into anonymous functions.

The following description applies to usage with vector-based
parameter handling. Differences in usage for structure-based
parameter handling will be explained separately.

@var{f}: objective function. It gets a column vector of real
parameters as argument. In gradient determination, this function may
be called with an informational second argument, whose content
depends on the function for gradient determination.

@var{pin}: real column vector of initial parameters.

@var{settings}: structure whose fields stand for optional settings
referred to below. The fields can be set by @code{optimset()}.

The returned values are the column vector of final parameters
@var{p}, the final value of the objective function @var{objf}, an
integer @var{cvg} indicating if and how optimization succeeded or
failed, and a structure @var{outp} with additional information,
curently with possible fields: @code{niter}, the number of
iterations, @code{nobjf}, the number of objective function calls
(indirect calls by gradient function not counted), @code{lambda}, the
lambda of constraints at the result, and @code{user_interaction},
information on user stops (see settings). The backend may define
additional fields. @var{cvg} is greater than zero for success and
less than or equal to zero for failure; its possible values depend on
the used backend and currently can be @code{0} (maximum number of
iterations exceeded), @code{1} (success without further specification
of criteria), @code{2} (parameter change less than specified
precision in two consecutive iterations), @code{3} (improvement in
objective function less than specified), @code{-1} (algorithm aborted
by a user function), or @code{-4} (algorithm got stuck).

@c The following block will be cut out in the package info file.
@c BEGIN_CUT_TEXINFO

For settings, type @code{optim_doc ("nonlin_min")}.

For desription of structure-based parameter handling, type
@code{optim_doc ("parameter structures")}.

For description of individual backends (currently only one), type
@code{optim_doc ("scalar optimization")} and choose the backend in
the menu.

@c END_CUT_TEXINFO

@end deftypefn
nonlin_residmin
@c nonlin_residmin ../inst/nonlin_residmin.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{p}, @var{resid}, @var{cvg}, @var{outp}] =} nonlin_residmin (@var{f}, @var{pin})
@deftypefnx {Function File} {[@var{p}, @var{resid}, @var{cvg}, @var{outp}] =} nonlin_residmin (@var{f}, @var{pin}, @var{settings})
Frontend for nonlinear minimization of residuals returned by a model
function.

The functions supplied by the user have a minimal
interface; any additionally needed constants (e.g. observed values)
can be supplied by wrapping the user functions into anonymous
functions.

The following description applies to usage with vector-based
parameter handling. Differences in usage for structure-based
parameter handling will be explained separately.

@var{f}: function returning the array of residuals. It gets a column
vector of real parameters as argument. In gradient determination,
this function may be called with an informational second argument,
whose content depends on the function for gradient determination.

@var{pin}: real column vector of initial parameters.

@var{settings}: structure whose fields stand for optional settings
referred to below. The fields can be set by @code{optimset()}.

The returned values are the column vector of final parameters
@var{p}, the final array of residuals @var{resid}, an integer
@var{cvg} indicating if and how optimization succeeded or failed, and
a structure @var{outp} with additional information, curently with the
fields: @code{niter}, the number of iterations and
@code{user_interaction}, information on user stops (see settings).
The backend may define additional fields. @var{cvg} is greater than
zero for success and less than or equal to zero for failure; its
possible values depend on the used backend and currently can be
@code{0} (maximum number of iterations exceeded), @code{2} (parameter
change less than specified precision in two consecutive iterations),
or @code{3} (improvement in objective function -- e.g. sum of squares
-- less than specified), or @code{-1} (algorithm aborted by a user
function).

@c The following block will be cut out in the package info file.
@c BEGIN_CUT_TEXINFO

For settings, type @code{optim_doc ("nonlin_residmin")}.

For desription of structure-based parameter handling, type
@code{optim_doc ("parameter structures")}.

For description of individual backends (currently only one), type
@code{optim_doc ("residual optimization")} and choose the backend in
the menu.

@c END_CUT_TEXINFO

@seealso {nonlin_curvefit}
@end deftypefn
nrm
@c nrm ../inst/nrm.m
-*- texinfo -*-
@deftypefn {Function File} {@var{xmin} =} nrm (@var{f},@var{x0})
Using @var{x0} as a starting point find a minimum of the scalar
function @var{f}.  The Newton-Raphson method is used.
@end deftypefn
optim_doc
@c optim_doc ../inst/optim_doc.m
-*- texinfo -*-
@deftypefn {Function File} {} optim_doc ()
@deftypefnx {Function File} {} optim_doc (@var{keyword})
Show optim package documentation.

Runs the info viewer Octave is configured with on the documentation
in info format of the installed optim package. Without argument, the
top node of the documentation is displayed. With an argument, the
respective index entry is searched for and its node displayed.

@end deftypefn
optim_problems
@c optim_problems ../inst/optim_problems.m
Problems for testing optimizers. Documentation is in the code.
poly_2_ex
@c poly_2_ex ../inst/poly_2_ex.m
 ex = poly_2_ex (l, f)       - Extremum of a 1-var deg-2 polynomial

l  : 3 : Values of variable at which polynomial is known.
f  : 3 : f(i) = Value of the degree-2 polynomial at l(i).

ex : 1 : Value for which f reaches its extremum

Assuming that f(i) = a*l(i)^2 + b*l(i) + c = P(l(i)) for some a, b, c,
ex is the extremum of the polynome P.

This function will be removed from future versions of the optim
package since it is not related to optimization.
polyconf
@c polyconf ../inst/polyconf.m
[y,dy] = polyconf(p,x,s)

  Produce prediction intervals for the fitted y. The vector p 
  and structure s are returned from polyfit or wpolyfit. The 
  x values are where you want to compute the prediction interval.

polyconf(...,['ci'|'pi'])

  Produce a confidence interval (range of likely values for the
  mean at x) or a prediction interval (range of likely values 
  seen when measuring at x).  The prediction interval tells
  you the width of the distribution at x.  This should be the same
  regardless of the number of measurements you have for the value
  at x.  The confidence interval tells you how well you know the
  mean at x.  It should get smaller as you increase the number of
  measurements.  Error bars in the physical sciences usually show 
  a 1-alpha confidence value of erfc(1/sqrt(2)), representing
  one standandard deviation of uncertainty in the mean.

polyconf(...,1-alpha)

  Control the width of the interval. If asking for the prediction
  interval 'pi', the default is .05 for the 95% prediction interval.
  If asking for the confidence interval 'ci', the default is
  erfc(1/sqrt(2)) for a one standard deviation confidence interval.

Example:
 [p,s] = polyfit(x,y,1);
 xf = linspace(x(1),x(end),150);
 [yf,dyf] = polyconf(p,xf,s,'ci');
 plot(xf,yf,'g-;fit;',xf,yf+dyf,'g.;;',xf,yf-dyf,'g.;;',x,y,'xr;data;');
 plot(x,y-polyval(p,x),';residuals;',xf,dyf,'g-;;',xf,-dyf,'g-;;');
polyfitinf
@c polyfitinf ../inst/polyfitinf.m
function [A,REF,HMAX,H,R,EQUAL] = polyfitinf(M,N,K,X,Y,EPSH,MAXIT,REF0)

  Best polynomial approximation in discrete uniform norm

  INPUT VARIABLES:

  M       : degree of the fitting polynomial
  N       : number of data points
  X(N)    : x-coordinates of data points
  Y(N)    : y-coordinates of data points
  K       : character of the polynomial:
                  K = 0 : mixed parity polynomial
                  K = 1 : odd polynomial  ( X(1) must be >  0 )
                  K = 2 : even polynomial ( X(1) must be >= 0 )
  EPSH    : tolerance for leveling. A useful value for 24-bit
            mantissa is EPSH = 2.0E-7
  MAXIT   : upper limit for number of exchange steps
  REF0(M2): initial alternating set ( N-vector ). This is an
            OPTIONAL argument. The length M2 is given by:
                  M2 = M + 2                      , if K = 0
                  M2 = integer part of (M+3)/2    , if K = 1
                  M2 = 2 + M/2 (M must be even)   , if K = 2

  OUTPUT VARIABLES:

  A       : polynomial coefficients of the best approximation
            in order of increasing powers:
                  p*(x) = A(1) + A(2)*x + A(3)*x^2 + ...
  REF     : selected alternating set of points
  HMAX    : maximum deviation ( uniform norm of p* - f )
  H       : pointwise approximation errors
	R		: total number of iterations
  EQUAL   : success of failure of algorithm
                  EQUAL=1 :  succesful
                  EQUAL=0 :  convergence not acheived
                  EQUAL=-1:  input error
                  EQUAL=-2:  algorithm failure

  Relies on function EXCH, provided below.

  Example: 
  M = 5; N = 10000; K = 0; EPSH = 10^-12; MAXIT = 10;
  X = linspace(-1,1,N);   % uniformly spaced nodes on [-1,1]
  k=1; Y = abs(X).^k;     % the function Y to approximate
  [A,REF,HMAX,H,R,EQUAL] = polyfitinf(M,N,K,X,Y,EPSH,MAXIT);
  p = polyval(A,X); plot(X,Y,X,p) % p is the best approximation

  Note: using an even value of M, e.g., M=2, in the example above, makes
  the algorithm to fail with EQUAL=-2, because of collocation, which
  appears because both the appriximating function and the polynomial are
  even functions. The way aroung it is to approximate only the right half
  of the function, setting K = 2 : even polynomial. For example: 

N = 10000; K = 2; EPSH = 10^-12; MAXIT = 10;  X = linspace(0,1,N);
for i = 1:2
    k = 2*i-1; Y = abs(X).^k;
    for j = 1:4
        M = 2^j;
        [~,~,HMAX] = polyfitinf(M,N,K,X,Y,EPSH,MAXIT);
        approxerror(i,j) = HMAX;
    end
end
disp('Table 3.1 from Approximation theory and methods, M.J.D.POWELL, p. 27');
disp(' ');
disp('            n          K=1          K=3'); 
disp(' '); format short g;
disp([(2.^(1:4))' approxerror']);

  ALGORITHM:

  Computation of the polynomial that best approximates the data (X,Y)
  in the discrete uniform norm, i.e. the polynomial with the  minimum
  value of max{ | p(x_i) - y_i | , x_i in X } . That polynomial, also
  known as minimax polynomial, is obtained by the exchange algorithm,
  a finite iterative process requiring, at most,
     n
   (   ) iterations ( usually p = M + 2. See also function EXCH ).
     p
  since this number can be very large , the routine  may not converge
  within MAXIT iterations . The  other possibility of  failure occurs
  when there is insufficient floating point precision  for  the input
  data chosen.

  CREDITS: This routine was developed and modified as 
  computer assignments in Approximation Theory courses by 
  Prof. Andrew Knyazev, University of Colorado Denver, USA.

  Team Fall 98 (Revision 1.0):
          Chanchai Aniwathananon
          Crhistopher Mehl
          David A. Duran
          Saulo P. Oliveira

  Team Spring 11 (Revision 1.1): Manuchehr Aminian

  The algorithm and the comments are based on a FORTRAN code written
  by Joseph C. Simpson. The code is available on Netlib repository:
  http://www.netlib.org/toms/501
  See also: Communications of the ACM, V14, pp.355-356(1971)

  NOTES:

  1) A may contain the collocation polynomial
  2) If MAXIT is exceeded, REF contains a new reference set
  3) M, EPSH and REF can be altered during the execution
  4) To keep consistency to the original code , EPSH can be
  negative. However, the use of REF0 is *NOT* determined by
  EPSH< 0, but only by its inclusion as an input parameter.

  Some parts of the code can still take advantage of vectorization.  

  Revision 1.0 from 1998 is a direct human translation of 
  the FORTRAN code http://www.netlib.org/toms/501
  Revision 1.1 is a clean-up and technical update.  
  Tested on MATLAB Version 7.11.0.584 (R2010b) and 
  GNU Octave Version 3.2.4
powell
@c powell ../inst/powell.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{p}, @var{obj_value}, @var{convergence}, @var{iters}, @var{nevs}] =} powell (@var{f}, @var{p0}, @var{control})
Multidimensional minimization (direction-set method). Implements a direction-set (Powell's) method for multidimensional minimization of a function without calculation of the gradient [1, 2]

@subheading Arguments

@itemize @bullet
@item
@var{f}: name of function to minimize (string or handle), which should accept one input variable (see example for how to pass on additional input arguments)

@item
@var{p0}: An initial value of the function argument to minimize

@item
@var{options}: an optional structure, which can be generated by optimset, with some or all of the following fields:
@itemize @minus
@item
MaxIter: maximum iterations  (positive integer, or -1 or Inf for unlimited (default))
@item
TolFun: minimum amount by which function value must decrease in each iteration to continue (default is 1E-8)
@item
MaxFunEvals: maximum function evaluations  (positive integer, or -1 or Inf for unlimited (default))
@item
SearchDirections: an n*n matrix whose columns contain the initial set of (presumably orthogonal) directions to minimize along, where n is the number of elements in the argument to be minimized for; or an n*1 vector of magnitudes for the initial directions (defaults to the set of unit direction vectors)
@end itemize
@end itemize

@subheading Examples

@example
@group
y = @@(x, s) x(1) ^ 2 + x(2) ^ 2 + s;
o = optimset('MaxIter', 100, 'TolFun', 1E-10);
s = 1;
[x_optim, y_min, conv, iters, nevs] = powell(@@(x) y(x, s), [1 0.5], o); %pass y wrapped in an anonymous function so that all other arguments to y, which are held constant, are set
%should return something like x_optim = [4E-14 3E-14], y_min = 1, conv = 1, iters = 2, nevs = 24
@end group

@end example

@subheading Returns:

@itemize @bullet
@item
@var{p}: the minimizing value of the function argument
@item
@var{obj_value}: the value of @var{f}() at @var{p}
@item
@var{convergence}: 1 if normal convergence, 0 if not
@item
@var{iters}: number of iterations performed
@item
@var{nevs}: number of function evaluations
@end itemize

@subheading References

@enumerate
@item
Powell MJD (1964), An efficient method for finding the minimum of a function of several variables without calculating derivatives, @cite{Computer Journal}, 7 :155-162

@item
Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (1992). @cite{Numerical Recipes in Fortran: The Art of Scientific Computing} (2nd Ed.). New York: Cambridge University Press (Section 10.5)
@end enumerate
@end deftypefn
residmin_stat
@c residmin_stat ../inst/residmin_stat.m
-*- texinfo -*-
@deftypefn {Function File} {@var{info} =} residmin_stat (@var{f}, @var{p}, @var{settings})
Frontend for computation of statistics for a residual-based
minimization.

@var{settings} is a structure whose fields can be set by
@code{optimset}. With @var{settings} the computation of certain
statistics is requested by setting the fields
@code{ret_<name_of_statistic>} to @code{true}. The respective
statistics will be returned in a structure as fields with name
@code{<name_of_statistic>}. Depending on the requested statistic and
on the additional information provided in @var{settings}, @var{f} and
@var{p} may be empty. Otherwise, @var{f} is the model function of an
optimization (the interface of @var{f} is described e.g. in
@code{nonlin_residmin}, please see there), and @var{p} is a real
column vector with parameters resulting from the same optimization.

Currently, the following statistics (or general information) can be
requested:

@code{dfdp}: Jacobian of model function with respect to parameters.

@code{covd}: Covariance matrix of data (typically guessed by applying
a factor to the covariance matrix of the residuals).

@code{covp}: Covariance matrix of final parameters.

@code{corp}: Correlation matrix of final parameters.

@c The following block will be cut out in the package info file.
@c BEGIN_CUT_TEXINFO

For further settings, type @code{optim_doc ("residmin_stat")}.

For desription of structure-based parameter handling, type
@code{optim_doc ("parameter structures")}.

For backend information, type @code{optim_doc ("residual
optimization")} and choose the backends type in the menu.

@c END_CUT_TEXINFO

@seealso {curvefit_stat}
@end deftypefn
rosenbrock
@c rosenbrock ../inst/rosenbrock.m
Rosenbrock function - used to create example obj. fns.

Function value and gradient vector of the rosenbrock function
The minimizer is at the vector (1,1,..,1),
and the minimized value is 0.
samin_example
@c samin_example ../inst/samin_example.m
samin_example: example script that contains examples of how to call
samin for minimization using simulated annealing.
Edit the script to see how samin may be used.

usage: samin_example
test_min_1
@c test_min_1 ../inst/test_min_1.m
Test an optimization function with the same synopsis as bfgs.m 
test_min_2
@c test_min_2 ../inst/test_min_2.m
test_min_2                   - Test that bfgs works

Defines some simple functions and verifies that calling

bfgs on them returns the correct minimum.

Sets 'ok' to 1 if success, 0 otherwise
test_min_3
@c test_min_3 ../inst/test_min_3.m
ok                     - Test that bfgs works with extra
                         arguments 

Defines some simple functions and verifies that calling
bfgs on them returns the correct minimum.

Sets 'ok' to 1 if success, 0 otherwise
test_min_4
@c test_min_4 ../inst/test_min_4.m
test_bfgs              - Test that bfgs works

Check that bfgs treats struct options correctly

Sets 'ok' to 1 if success, 0 otherwise
test_minimize_1
@c test_minimize_1 ../inst/test_minimize_1.m
ok = test_minimize           - Test that minimize works
test_nelder_mead_min_1
@c test_nelder_mead_min_1 ../inst/test_nelder_mead_min_1.m
Checks wether the function 'nelder_mead_min' works, by making it minimize a
quadratic function.
test_nelder_mead_min_2
@c test_nelder_mead_min_2 ../inst/test_nelder_mead_min_2.m
Checks wether the function 'nelder_mead_min' accepts options properly
test_wpolyfit
@c test_wpolyfit ../inst/test_wpolyfit.m
Tests for wpolyfit.

Test cases are taken from the NIST Statistical Reference Datasets
   http://www.itl.nist.gov/div898/strd/
vfzero
@c vfzero ../inst/vfzero.m
-*- texinfo -*-
@deftypefn  {Function File} {} vfzero (@var{fun}, @var{x0})
@deftypefnx {Function File} {} vfzero (@var{fun}, @var{x0}, @var{options})
@deftypefnx {Function File} {[@var{x}, @var{fval}, @var{info}, @var{output}] =} vfzero (@dots{})
A variant of @code{fzero}. Finds a zero of a vector-valued
multivariate function where each output element only depends on the
input element with the same index (so the Jacobian is diagonal).

@var{fun} should be a handle or name of a function returning a column
vector.  @var{x0} should be a two-column matrix, each row specifying
two points which bracket a zero of the respective output element of
@var{fun}.

If @var{x0} is a single-column matrix then several nearby and distant
values are probed in an attempt to obtain a valid bracketing.  If
this is not successful, the function fails. @var{options} is a
structure specifying additional options. Currently, @code{vfzero}
recognizes these options: @code{"FunValCheck"}, @code{"OutputFcn"},
@code{"TolX"}, @code{"MaxIter"}, @code{"MaxFunEvals"}. For a
description of these options, see optimset.

On exit, the function returns @var{x}, the approximate zero and
@var{fval}, the function value thereof. @var{info} is a column vector
of exit flags  that can have these values:

@itemize 
@item 1 The algorithm converged to a solution.

@item 0 Maximum number of iterations or function evaluations has been
reached.

@item -1 The algorithm has been terminated from user output function.

@item -5 The algorithm may have converged to a singular point.
@end itemize

@var{output} is a structure containing runtime information about the
@code{fzero} algorithm.  Fields in the structure are:

@itemize
@item iterations Number of iterations through loop.

@item nfev Number of function evaluations.

@item bracketx A two-column matrix with the final bracketing of the
zero along the x-axis.

@item brackety A two-column matrix with the final bracketing of the
zero along the y-axis.
@end itemize
@end deftypefn
wpolyfit
@c wpolyfit ../inst/wpolyfit.m
-*- texinfo -*-
@deftypefn {Function File} {[@var{p}, @var{s}] =} wpolyfit (@var{x}, @var{y}, @var{dy}, @var{n})
Return the coefficients of a polynomial @var{p}(@var{x}) of degree
@var{n} that minimizes
@c ########################################################
@c These lines must be written without space at start to work around
@c a bug in html generation.
@iftex
@tex
$$
\sum_{i=1}^N (p(x_i) - y_i)^2
$$
@end tex
@end iftex
@ifnottex
@code{sumsq (p(x(i)) - y(i))},
@end ifnottex
@c ########################################################
to best fit the data in the least squares sense.  The standard error
on the observations @var{y} if present are given in @var{dy}.

The returned value @var{p} contains the polynomial coefficients 
suitable for use in the function polyval.  The structure @var{s} returns
information necessary to compute uncertainty in the model.

To compute the predicted values of y with uncertainty use
@example
[y,dy] = polyconf(p,x,s,'ci');
@end example
You can see the effects of different confidence intervals and
prediction intervals by calling the wpolyfit internal plot
function with your fit:
@example
feval('wpolyfit:plt',x,y,dy,p,s,0.05,'pi')
@end example
Use @var{dy}=[] if uncertainty is unknown.

You can use a chi^2 test to reject the polynomial fit:
@example
p = 1-chi2cdf(s.normr^2,s.df);
@end example
p is the probability of seeing a chi^2 value higher than that which 
was observed assuming the data are normally distributed around the fit.
If p < 0.01, you can reject the fit at the 1% level.

You can use an F test to determine if a higher order polynomial 
improves the fit:
@example
[poly1,S1] = wpolyfit(x,y,dy,n);
[poly2,S2] = wpolyfit(x,y,dy,n+1);
F = (S1.normr^2 - S2.normr^2)/(S1.df-S2.df)/(S2.normr^2/S2.df);
p = 1-f_cdf(F,S1.df-S2.df,S2.df);
@end example
p is the probability of observing the improvement in chi^2 obtained
by adding the extra parameter to the fit.  If p < 0.01, you can reject 
the lower order polynomial at the 1% level.

You can estimate the uncertainty in the polynomial coefficients 
themselves using
@example
dp = sqrt(sumsq(inv(s.R'))'/s.df)*s.normr;
@end example
but the high degree of covariance amongst them makes this a questionable
operation.
@end deftypefn

@deftypefn {Function File} {[@var{p}, @var{s}, @var{mu}] =} wpolyfit (...)

If an additional output @code{mu = [mean(x),std(x)]} is requested then 
the @var{x} values are centered and normalized prior to computing the fit.
This will give more stable numerical results.  To compute a predicted 
@var{y} from the returned model use
@code{y = polyval(p, (x-mu(1))/mu(2)}
@end deftypefn

@deftypefn {Function File} {} wpolyfit (...)

If no output arguments are requested, then wpolyfit plots the data,
the fitted line and polynomials defining the standard error range.

Example
@example
x = linspace(0,4,20);
dy = (1+rand(size(x)))/2;
y = polyval([2,3,1],x) + dy.*randn(size(x));
wpolyfit(x,y,dy,2);
@end example
@end deftypefn

@deftypefn {Function File} {} wpolyfit (..., 'origin')

If 'origin' is specified, then the fitted polynomial will go through
the origin.  This is generally ill-advised.  Use with caution.

Hocking, RR (2003). Methods and Applications of Linear Models.
New Jersey: John Wiley and Sons, Inc.

@c Will be cut out in optims info file and replaced with the same
@c refernces explicitely there, since references to core Octave
@c functions are not automatically transformed from here to there.
@c BEGIN_CUT_TEXINFO
@seealso{polyfit}
@c END_CUT_TEXINFO
@seealso{polyconf}
@end deftypefn
wrap_f_dfdp
@c wrap_f_dfdp ../inst/wrap_f_dfdp.m
[ret1, ret2] = wrap_f_dfdp (f, dfdp, varargin)

f and dftp should be the objective function (or "model function" in
curve fitting) and its jacobian, respectively, of an optimization
problem. ret1: f (varagin{:}), ret2: dfdp (varargin{:}). ret2 is
only computed if more than one output argument is given. This
manner of calling f and dfdp is needed by some optimization
functions.
wsolve
@c wsolve ../inst/wsolve.m
[x,s] = wsolve(A,y,dy)

Solve a potentially over-determined system with uncertainty in
the values. 

    A x = y +/- dy

Use QR decomposition for increased accuracy.  Estimate the 
uncertainty for the solution from the scatter in the data.

The returned structure s contains

   normr = sqrt( A x - y ), weighted by dy
   R such that R'R = A'A
   df = n-p, n = rows of A, p = columns of A

See polyconf for details on how to use s to compute dy.
The covariance matrix is inv(R'*R).  If you know that the
parameters are independent, then uncertainty is given by
the diagonal of the covariance matrix, or 

   dx = sqrt(N*sumsq(inv(s.R'))')

where N = normr^2/df, or N = 1 if df = 0.

Example 1: weighted system

   A=[1,2,3;2,1,3;1,1,1]; xin=[1;2;3]; 
   dy=[0.2;0.01;0.1]; y=A*xin+randn(size(dy)).*dy;
   [x,s] = wsolve(A,y,dy);
   dx = sqrt(sumsq(inv(s.R'))');
   res = [xin, x, dx]

Example 2: weighted overdetermined system  y = x1 + 2*x2 + 3*x3 + e

   A = fullfact([3,3,3]); xin=[1;2;3];
   y = A*xin; dy = rand(size(y))/50; y+=dy.*randn(size(y));
   [x,s] = wsolve(A,y,dy);
   dx = s.normr*sqrt(sumsq(inv(s.R'))'/s.df);
   res = [xin, x, dx]

Note there is a counter-intuitive result that scaling the
uncertainty in the data does not affect the uncertainty in
the fit.  Indeed, if you perform a monte carlo simulation
with x,y datasets selected from a normal distribution centered
on y with width 10*dy instead of dy you will see that the
variance in the parameters indeed increases by a factor of 100.
However, if the error bars really do increase by a factor of 10
you should expect a corresponding increase in the scatter of 
the data, which will increase the variance computed by the fit.
