RidgeRegress Routine

Summary

Regression by using "ridge" regression method.

Declaration

Procedure RidgeRegress(y: TVec; A: TMtx; k: TSample; b: TVec; Normalize: Boolean = true);

Description

Calculates regression coefficients b by using "ridge regression" method. When data suffers from multicollinearity one of the available methods which can still be used is ridge regression. In this case least squares estimates are unbiased, but their variances are large so they may be far from the true value. By adding a degree of bias to the regression estimates, ridge regression reduces the standard errors. It is hoped that the total effect will be to give more reliable estimates. Ridge regression uses following model:

y = A*b ,

where y is vector of observations, A matrix of independent variables and b are regression coefficients. Additional k parameter is the so called "ridge parameter". Regression coefficients are then calculated from the following formula:

b = inv(AT*A + k*I) * (AT*y),

where I is the identity matrix, A matrix of independent variables and AT=Transp(A).

Note: The routine does NOT calculate optimal k value for ridge regression.

Example 1

An example of ridge regression method.

  Uses MtxExp, Regress;
  procedure Example;
  var y,b: Vector;
    A : Matrix;
  begin
    y.SetIt(false,[-2.5, 0.1, 6.1]);
    A.SetIt(3,2,false,[1.0, 2.5,
                     3.2, -1.5,
                     0.4, 0.7]);
    RidgeRegress(y, A, 0.0, b);
    // b = (-6.8889789853, -6.450976395)
  end;

  #include "MtxVecCpp.h"
  #include "Regress.hpp"
  #include "Math387.hpp"
  void __fastcall Example()
  {
    Vector y,b;
    Matrix A;
    y->SetIt(false,OPENARRAY(TSample,(-2.5, 0.1, 6.1])));
    A->SetIt(3,2,false,OPENARRAY(TSample,(1.0, 2.5,
                     3.2, -1.5,
                     0.4, 0.7)));
    RidgeRegress(y,A,0.0,b);
    // b = (-6.8889789853, -6.450976395)
  }

  using Dew.Math;
  using Dew.Stats;
  using Dew.Stats.Units;
  namespace Dew.Examples
  {
    private void Example()
    {
      Vector y = new Vector(0);
      Vector b = new Vector(0);
      Matrix A = new Matrix(0,0);
      y.SetIt(false,new double[] {-2.5, 0.1, 6.1});
      A.SetIt(3,2,false, new double[] {1.0, 2.5,
                     3.2, -1.5,
                     0.4, 0.7});
      Regress.RidgeRegress(y,A,0.0,b);
      // b = (-6.8889789853, -6.450976395)
    }
  }