Machine Learning Library
Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
CSVM< Type > Class Template Reference

Support Vector Machine algorithm for binary classifcation. More...

#include <CSVM.h>

Inheritance diagram for CSVM< Type >:

Public Member Functions

 CSVM (TSVMType=L1SVM)
 
 ~CSVM ()
 
void setMaxIterations (long lMax)
 
void classify (const CVector< Type > &tVec, Type *ptClass, Type *ptMargin) const
 
void classify (const CVector< Type > &tVec, Type &rtClass, Type &rtMargin) const
 
void classify (Type *ptVec, Type *ptClass, Type *ptMargin) const
 
int classify (const CVector< Type > &tVec) const
 
string className () const
 
bool isA (const char *acClass) const
 
void train (const CDataset< Type > &tTrainSet)
 
void setRegularization (Type tC)
 
void setWeight (Type tWPos, Type tWNeg)
 
void setTerminationError (Type tMinError)
 
void trainChunking (CDataset< Type > &rtTrainset, int iStartSetSize, int iMaxChunkSize)
 
void setKernel (CKernel< Type > *ptKernel)
 
const CKernel< Type > * kernel ()
 
void setCacheSize (Type tSize)
 
CDataset< Type > getSupportData (bool bDropNonSV)
 
CDataset< Type > getNonSupportData () const
 
CDenseVector< Type > getWeightVector () const
 
Type bias () const
 
bool serialize (fstream &stream, IO_MODE mode)
 
bool serialize2 (CArchiv &tA)
 

Protected Member Functions

void analyticSolve (int i1, int i2)
 
Type l1Regularizer (Type tClass)
 
Type l2Regularizer (Type tClass)
 
void updateGradient (int &a, double &tDeltaA, int &b, double &tDeltaB)
 
void init ()
 
void optimizeWorkingSet ()
 
bool checkShrinking (int, int)
 
void rebuildGradient ()
 
Type calculateBias ()
 
bool selectPair (int *piI1, int *piI2, Type &tError)
 

Protected Attributes

vector< CSVCandidate< Type > > tWorkingSet
 
CKernelCache< Type > * ptCache
 
CKernel< Type > * ptKernel
 
Type tC
 
Type tWeightPos
 
Type tWeightNeg
 
Type tTerminationError
 
Type tBias
 
long lMaxIterations
 
long lNextShrink
 
bool bUnshrinked
 
bool bFoldKernel
 
int iNumIteration
 
int iNumActiveItems
 
TSVMType tSVMType
 
CDenseVector< Type > tWeight
 
int iCacheSize
 
vector< int > tTransformedAccess
 

Detailed Description

template<class Type>
class CSVM< Type >

Support Vector Machine algorithm for binary classifcation.

Solves the binary SVM training by decomposition. This implementation allows L1 and L2 penalties, class specific regularization, kernel caching and shrinking. The implementation is strongly inspired by libsvm but is much more readable. You can find more information about the implementation details in papers about Sequential Minimal Optimization and libsvm.

Author
Thorsten Twellmann (ttwel.nosp@m.lma@.nosp@m.TechF.nosp@m.ak.U.nosp@m.ni-Bi.nosp@m.elef.nosp@m.eld.d.nosp@m.e)

Constructor & Destructor Documentation

template<class Type >
CSVM< Type >::CSVM ( TSVMType  = L1SVM)

Constructor - Creates a new instance of a SVM with given penalty metric

Parameters
tSVMTypeType of svm (default: L1SVM).
template<class Type >
CSVM< Type >::~CSVM ( )

Destructor

Member Function Documentation

template<class Type >
void CSVM< Type >::analyticSolve ( int  i1,
int  i2 
)
protected

Derive analytical solution with respect to i1 and i2

Parameters
i1Workingset index
i2Workingset index
template<class Type >
Type CSVM< Type >::bias ( ) const
inline

Return bias value

Returns
Bias

References CSVM< Type >::tBias.

template<class Type >
Type CSVM< Type >::calculateBias ( )
protected

Calculate bias for trained svm

Returns
New bias value
template<class Type >
bool CSVM< Type >::checkShrinking ( int  ,
int   
)
protected
template<class Type >
void CSVM< Type >::classify ( const CVector< Type > &  tVec,
Type *  ptClass,
Type *  ptMargin 
) const

Classifies a given pattern.

Parameters
tVecVector containing the pattern to classify.
ptClassPointer to variable which will be filled with the classlable.
ptMarginPointer to variable which will be filled with calculated margin.
template<class Type >
void CSVM< Type >::classify ( const CVector< Type > &  tVec,
Type &  rtClass,
Type &  rtMargin 
) const
template<class Type >
void CSVM< Type >::classify ( Type *  ptVec,
Type *  ptClass,
Type *  ptMargin 
) const

Classifies a given pattern.

Parameters
ptVecArray containing the pattern to classify.
ptClassPointer to variable which will be filled with the classlable.
ptMarginPointer to variable which will be filled with calculated margin.
template<class Type >
int CSVM< Type >::classify ( const CVector< Type > &  tVec) const
template<class Type >
string CSVM< Type >::className ( ) const
inline
template<class Type >
CDataset<Type> CSVM< Type >::getNonSupportData ( ) const
template<class Type >
CDataset<Type> CSVM< Type >::getSupportData ( bool  bDropNonSV)

Returns the svm solution as a dataset. The input vector of each item contains the training pattern. The output vector contains the class label (element 0) and the lagrange coefficient (element 1). By setting the parameter bDropNonSV=true, training vectors with lagrange coefficient equal to zero will be deleted.

Parameters
bDropNonSVIf true, non-support vectors will be deleted
Returns
Dataset containing support data
template<class Type >
CDenseVector<Type> CSVM< Type >::getWeightVector ( ) const
inline

References CSVM< Type >::tWeight.

template<class Type >
void CSVM< Type >::init ( )
protected

Initialize algorithm with given dataset

Parameters
tDataset
template<class Type >
bool CSVM< Type >::isA ( const char *  acClass) const
inline
template<class Type >
const CKernel<Type>* CSVM< Type >::kernel ( )
inline

Return const pointer to kernel object

Returns
Const pointer to kernel object

References CSVM< Type >::ptKernel.

template<class Type >
Type CSVM< Type >::l1Regularizer ( Type  tClass)
inlineprotected
template<class Type >
Type CSVM< Type >::l2Regularizer ( Type  tClass)
inlineprotected
template<class Type >
void CSVM< Type >::optimizeWorkingSet ( )
protected

Train the algorithm with the current trainset

template<class Type >
void CSVM< Type >::rebuildGradient ( )
protected

Recalculate the gradient

\[ g(i) = \sum_{j=1}^{N} y_i \alpha_i K_{i,j} - 1 \]

for all training examples.

template<class Type >
bool CSVM< Type >::selectPair ( int *  piI1,
int *  piI2,
Type &  tError 
)
protected
template<class Type >
bool CSVM< Type >::serialize ( fstream &  stream,
IO_MODE  mode 
)
template<class Type >
bool CSVM< Type >::serialize2 ( CArchiv tA)
template<class Type >
void CSVM< Type >::setCacheSize ( Type  tSize)
inline

Set new cache size (in MByte)

Parameters
tSizeCache size

References CSVM< Type >::iCacheSize.

template<class Type >
void CSVM< Type >::setKernel ( CKernel< Type > *  ptKernel)

Use a new kernel object. The SVM becomes the owner of the kernel object and will delete it in destruction.

Parameters
ptKernelPointer to new kernel object
template<class Type >
void CSVM< Type >::setMaxIterations ( long  lMax)
inline

Set the maximal number of iterations

Parameters
lMaxMaximal number of iterations

References CSVM< Type >::lMaxIterations.

template<class Type >
void CSVM< Type >::setRegularization ( Type  tC)

Sets regularization paramert C. C will be the upper bound for the lagrange coefficients of the svm in case of L1 penalty and an additional term on the kernel in case of L2 penalty.

Parameters
tCNew regularization value.
template<class Type >
void CSVM< Type >::setTerminationError ( Type  tMinError)
inline

Sets the termination error. SVM training will terminate if the maximal difference between two lagrange gradients will drop below this threshold.

Parameters
tMinErrorTermination error.

References CSVM< Type >::tTerminationError.

template<class Type >
void CSVM< Type >::setWeight ( Type  tWPos,
Type  tWNeg 
)

Set class-specific regularization values. In the case of L1 penalty, the lagrange coefficients $ \alpha_i$ are upper bounded by the regularization values $ C^+ = w^+ C $ and $ C^- = w^- C $. Typical selections are e.g $ \frac{C^+}{C^-}= \frac{n^-}{n+} $ (according Morik, Brockhausen and Joachims 1999), leading to $ w^+ = \frac{n^-}{N} $ and $ w^- = \frac{n^+}{N} $. In the case of L2 penalty, the regularization values $ C^+ = \frac{1}{w^+ C} $ and $ C^- = \frac{1}{w^- C} $ are added to the diagonal of the kernel matrix.

Parameters
tWPosWeight $ w^+ $ for class $ y=+1 $
tWNegWeight $ w^- $ for class $ y=-1 $
template<class Type >
void CSVM< Type >::train ( const CDataset< Type > &  tTrainSet)
template<class Type >
void CSVM< Type >::trainChunking ( CDataset< Type > &  rtTrainset,
int  iStartSetSize,
int  iMaxChunkSize 
)
template<class Type >
void CSVM< Type >::updateGradient ( int &  a,
double &  tDeltaA,
int &  b,
double &  tDeltaB 
)
protected

Update the gradient of the active items $ g_i $ using

\[ g(i) = g(i) + \Delta\alpha_a * K_{i,a} + \Delta\alpha_b * K_{i,b} \]

Parameters
aWorkingset index
dDeltaAAlpha change of a
bWorkingset index
dDeltaBAlpha change of b

Member Data Documentation

template<class Type >
bool CSVM< Type >::bFoldKernel
protected

If true, the linear kernel is folded to a single vector describing the hyperplane

template<class Type >
bool CSVM< Type >::bUnshrinked
protected
template<class Type >
int CSVM< Type >::iCacheSize
protected
template<class Type >
int CSVM< Type >::iNumActiveItems
protected

Current number of active items

template<class Type >
int CSVM< Type >::iNumIteration
protected

Current number of iterations

template<class Type >
long CSVM< Type >::lMaxIterations
protected

Max number of iterations

Referenced by CSVM< Type >::setMaxIterations().

template<class Type >
long CSVM< Type >::lNextShrink
protected
template<class Type >
CKernelCache<Type>* CSVM< Type >::ptCache
protected

Cache for the kernelmatrix

template<class Type >
CKernel<Type>* CSVM< Type >::ptKernel
protected

Kernel object

Referenced by CSVM< Type >::kernel().

template<class Type >
Type CSVM< Type >::tBias
protected

Bias value

Referenced by CSVM< Type >::bias().

template<class Type >
Type CSVM< Type >::tC
protected

Regularization parameter for L1-SVM

template<class Type >
TSVMType CSVM< Type >::tSVMType
protected

Type of error penalty

template<class Type >
Type CSVM< Type >::tTerminationError
protected

Error threshold

Referenced by CSVM< Type >::setTerminationError().

template<class Type >
vector<int> CSVM< Type >::tTransformedAccess
protected
template<class Type >
CDenseVector<Type> CSVM< Type >::tWeight
protected

speed-up for linear kernel

Referenced by CSVM< Type >::getWeightVector().

template<class Type >
Type CSVM< Type >::tWeightNeg
protected
template<class Type >
Type CSVM< Type >::tWeightPos
protected
template<class Type >
vector< CSVCandidate<Type> > CSVM< Type >::tWorkingSet
protected

current workingset


The documentation for this class was generated from the following file: