NLinRegress Routines

Summary

Non-linear regression with lower and upper bounds.

Declaration

Function NLinRegress(X, Y: TVec; RegFun: TRegressFun; DeriveProc: TDeriveProc; B, BLowerB, BUpperB: TVec; Method: TOptMethod; out StopReason: TOptStopReason; Weights: TVec = nil; YCalc: TVec = nil; SoftSearch: boolean = false; MaxIter: Integer = 500; Tol: TSample = 0.00000001; GradTol: TSample = 0.00000001): Integer;

Parameter	Description
X	Vector of dependent variable.
Y	Vector of independent variable.
RegFun	Regression function.
DeriveProc	Procedure to calculate the derivatives of RegFun. You can define the exact derivative or use NumericDerive routine as numerical approximation.
B	Holds initial estimate for regression parameters. After the call to NLinRegress b returns calculated regression parameters.
BLowerB	Holds lower bounds for regression parameters. If there are no lower bounds, set BLowerB values to -INF.
BUpperB	Holds upper bounds for regression parameters. If there are no upper bounds, set BUpperB values to +INF.
OptMethod	Defines which optimization method will be used to find regression parameters (see MtxVec.hlp TOptMethod type to learn more about this).
StopReason	Returns why regression parameters search stopped (see MtxVec.hlp TOptStopReason type to learn more about different stop reasons).
Weights	Weights (optional).
YCalc	Returns calculated values (optional).
SoftSearch	If true, internal line search algoritm will use soft line search method. Set this parameter to true if you're using numerical approximation for derivative. If this parameter is set to false, internal line search algorithm will use exact line search method. Set this parameter to false if you're using exact derivative.
MaxIter	Maximum allowed numer of allowed iterations.
Tol	Desired regression parameters tolerance.
GradTol	Minimum allowed gradient C-Norm.

Result

Number of iterations needed to calculate regression parameters with specified tolerance.

Description

General non-linear regression with lower and upper bounds for regression coefficients.

Declaration

Function NLinRegress(X, Y: TVec; RegFun: TRegressFun; DeriveProc: TDeriveProc; B: TVec; Method: TOptMethod; out StopReason: TOptStopReason; Weights: TVec = nil; YCalc: TVec = nil; SoftSearch: boolean = false; MaxIter: Integer = 500; Tol: TSample = 0.00000001; GradTol: TSample = 0.00000001): Integer;

Summary

General non-linear regression.

Parameter	Description
X	Vector of dependent variable.
Y	Vector of independent variable.
RegFun	Regression function.
DeriveProc	Procedure to calculate the derivatives of RegFun. You can define the exact derivative or use NumericDerive routine as numerical approximation.
B	Holds initial estimate for regression parameters. After the call to NLinRegress b returns calculated regression parameters.
OptMethod	Defines which optimization method will be used to find regression parameters (see MtxVec.hlp TOptMethod type to learn more about this).
StopReason	Returns why regression parameters search stopped (see MtxVec.hlp TOptStopReason type to learn more about different stop reasons).
Weights	Weights (optional).
YCalc	Returns calculated values (optional).
SoftSearch	If true, internal line search algoritm will use soft line search method. Set this parameter to true if you're using numerical approximation for derivative. If this parameter is set to false, internal line search algorithm will use exact line search method. Set this parameter to false if you're using exact derivative.
MaxIter	Maximum allowed numer of allowed iterations.
Tol	Desired regression parameters tolerance.
GradTol	Minimum allowed gradient C-Norm.

Result

Number of iterations needed to calculate regression parameters with specified tolerance.

Description

The routine fits equations to data by minimizing the sum of squared residuals :

SS = Sum [y(k) - ycalc(k)]^2 ,

where y(k) and ycalc(k) are respectively the observed and calculated value of the dependent variable for observation k. ycalc(k) is a function of the regression parameters b(0), b(1) ... Here the observed values obey the following (non-linear) equation:

y(k) = RegFun[x(k), b(0), b(1), ... ]

i.e

Y = RegFun[X,b(0),b(1), ...]

where RegFun is the regression function and b(0),..b(i) are the regression parameters.

See Also
NumericDerive

Example 1

The following example uses data from NIST study involving circular interference transmittance. The response variable is transmittance, and the predictor variable is wavelength. First we setup the regression function Eckerle4 with three regression parameters (b0,b1,b2). Then we setup data and specify initial estimate for regression parameters (see below):

  Uses MtxExp, Math387, Regress, Optimization, MtxVecTee;
  // function definition
  function Eckerle4(B: TVec; X: TSample): TSample;
  begin
    Result := B[0]/B[1] * Exp(-0.5*Sqr((X-B[2])/B[1]));
  end;
  procedure Example;
  var x,y,b,yhat: Vector;
    StopReason: TOptStopReason;
  begin
    x.SetIt(false,[400.0, 405.0, 410.0, 415.0,
                   420.0, 425.0, 430.0, 435.0,
                   436.5, 438.0, 439.5, 441.0,
                   442.5, 444.0, 445.5, 447.0,
                   448.5, 450.0, 451.5, 453.0,
                   454.5, 456.0, 457.5, 459.0,
                   460.5, 462.0, 463.5, 465.0,
                   470.0, 475.0, 480.0, 485.0,
                   490.0, 495.0, 500.0]);
    y.SetIt(false,[0.0001575, 0.0001699, 0.0002350, 0.0003102,
                   0.0004917, 0.0008710, 0.0017418, 0.0046400,
                   0.0065895, 0.0097302, 0.0149002, 0.0237310,
                   0.0401683, 0.0712559, 0.1264458, 0.2073413,
                   0.2902366, 0.3445623, 0.3698049, 0.3668534,
                   0.3106727, 0.2078154, 0.1164354, 0.0616764,
                   0.0337200, 0.0194023, 0.0117831, 0.0074357,
                   0.0022732, 0.0008800, 0.0004579, 0.0002345,
                   0.0001586, 0.0001143, 0.0000710]);
    b.SetIt(false,[1.0, 10.0, 500.0]); // initial estimates
    NLinRegress(x,y,Eckerle4,nil,b,optMarquardt, StopReason,
                nil,yhat,false,300,1e-8,1e-10);
    DrawValues(x,y,Series1,false); // draw data
    DrawValues(x,yhat,Series2,false); // draw fitted values
  end;

  #include "MtxVecCpp.h"
  #include "Regress.hpp"
  #include "MtxVecTee.hpp"
  double __fastcall Exkerle4(TVec * B, double x)
  {
      double sqrterm = (x-B[2])/B[1])*(x-B[2])/B[1]);
      return B[0]/B[1] * Exp(-0.5*(sqrterm));
  }
  void __fastcall(TLineSeries* series1, TLineSeries* series2)
  {
    Vector x,y,b,yhat;
    TOptStopReason stopreason;

    x->SetIt(false,OPENARRAY(TSample,(400.0, 405.0, 410.0, 415.0,
                   420.0, 425.0, 430.0, 435.0,
                   436.5, 438.0, 439.5, 441.0,
                   442.5, 444.0, 445.5, 447.0,
                   448.5, 450.0, 451.5, 453.0,
                   454.5, 456.0, 457.5, 459.0,
                   460.5, 462.0, 463.5, 465.0,
                   470.0, 475.0, 480.0, 485.0,
                   490.0, 495.0, 500.0)));
    y->SetIt(false,OPENARRAY(TSample,(0.0001575, 0.0001699, 0.0002350, 0.0003102,
                   0.0004917, 0.0008710, 0.0017418, 0.0046400,
                   0.0065895, 0.0097302, 0.0149002, 0.0237310,
                   0.0401683, 0.0712559, 0.1264458, 0.2073413,
                   0.2902366, 0.3445623, 0.3698049, 0.3668534,
                   0.3106727, 0.2078154, 0.1164354, 0.0616764,
                   0.0337200, 0.0194023, 0.0117831, 0.0074357,
                   0.0022732, 0.0008800, 0.0004579, 0.0002345,
                   0.0001586, 0.0001143, 0.0000710)));
    b->SetIt(false,OPENARRAY(TSample,(1.0, 10.0, 500.0))); // initial estimates
    NLinRegress(x,y,Eckerle4,NULL,b,optMarquardt, StopReason,
                NULL,yhat,false,300,1e-8,1e-10);
    DrawValues(x,y,Series1,false); // draw data
    DrawValues(x,yhat,Series2,false); // draw fitted values
  }