Preamble

• If something needs to be clarified or fixed in this documentation, the developers are interested in hearing about it.


• The ECOC PAK C++ library is actually an Armadillo C++ Linear Algebra Library extension. So in order to use efficiently the ECOC PAK as a C++ library you must know the basic Armadillo usage. Armadillo documentation is available at Armadillo C++ Linear Algebra Library documentation, reading this documentation is highly recommended.


• u32 is an Armadillo typedef for unsigned int and s32 is an Armadillo typedef for int.


• The ECOC PAK C++ library uses as SVM classifier the implementation of the LIBSVM. It is strongly recommended to visit LIBSVM site as well.

[index] Enumerations

Codings Enumeration · Decodings Enumeration · Classifiers Enumeration · Decomposition Criteria Enumeration · ECOC One Modes Coding

[index] Codings

One-versus-One or All-Pairs Coding · One-versus-All Coding · Discriminant Error Correcting Output Codes (DECOC) Coding · Discriminant Error Correcting Output Codes with Subclasses (subDECOC) Coding · Dense Random Coding · Sparse Random Coding · ECOC One Coding · Forest ECOC Coding · Custom Coding

[index] Setting Classifier Arguments

Support Vector Machine · Discrete AdaBoost · Sum of Error Squares

Armadillo Basics

Real Valued 2D Matrices (mat) · Integral Valued 2D Matrices (imat, umat) · Real Valued 1D Vectors (rowvec, colvec) · Integral Valued 1D Vectors (irowvec, icolvec, urowvec, ucolvec)

---

---

Enumerations

---

---

Available Codings Enumeration

The type of coding for the ECOC One initial coding is specified by the init_coding_strategy input argument.

Available options:


  'ONE_VS_ONE'    -> One versus One Coding.
  'ONE_VS_ALL'    -> One versus All Coding.
  'DECOC'         -> Discriminant Error Correcting.
  'SUBDECOC'      -> DECOC with subclasses.
  'DENSE_RANDOM'  -> Dense Random Coding.
  'SPARSE_RANDOM' -> Sparse Random Coding.
  'ECOC_ONE'      -> ECOC One Coding.
  'FOREST_ECOC'   -> Forest ECOC.
  'CUSTOM_CODING' -> Custom Coding.

Available Decodings Enumeration

The type of decoding is specified by the decoding_strategy input argument.

Available options:


  'HAMMING'                            -> Hamming Decoding.
  'EUCLIDEAN'                          -> Euclidean Decoding
  'LAPLACIAN'                          -> Laplacian Decoding.
  'HAMMING_ATTENUATED'                 -> Hamming Attenuated Decoding.
  'EUCLIDEAN_ATTENUATED'               -> Euclidean Attenuated Decoding.
  'LINEAR_LOSS_WEIGHTED_DECODING'      -> Linear Loss Weighted Decoding.
  'EXPONENTIAL_LOSS_WEIGHTED_DECODING' -> Exponential Loss Weighted Decoding.
  'LINEAR_LOSS_BASED_DECODING'         -> Linear Loss Based Decoding.
  'EXPONENTIAL_LOSS_BASED_DECODING'    -> Exponential Loss Based Decoding.
  'BETA_DENSITY_DECODING'              -> Beta-density Decoding.
  'PROBABILISTIC_BASED_DECODING'       -> Probabilistic Based Decoding.
  'INVERSE_HAMMING_DECODING'           -> Inverse Hamming Decoding.
  'CUSTOM_DECODING'                    -> Custom Decoding.

Available Classifiers Enumeration

The type of classifier is specified by the classifier_type input argument.

Available options:


  'NCC'               -> Nearest Class Centroid Classifier.
  'FLDA'              -> Fisher Linear Discriminant Analysis Followed by NCC.
  'SVM'               -> LIBSVM's Support Vector Machine.
  'ADABOOST'          -> Discrete AdaBoost.
  'LEAST_SQUARES'     -> Sum of Least Error Squares Classifier.
  'CUSTOM_CLASSIFIER' -> Custom Classifier.

Available Decomposition Criteria for DECOC/subDECOC Enumeration

The type of decomposition criterion for the DECOC/subDECOC, Forest ECOC and ECOC One is specified by the criterion_option input argument.

Available options:


  'FQMI'             -> Fast Quadratic Mutual Information.
  'FLDR'             -> Fisher's Linear Discriminant Ratio.
  'CUSTOM_CRITERION' -> Custom Decomposition Criterion.

ECOC One Modes Enumeration

The type of ECOC One mode is specified by the ecocone_mode input argument.

Available options:


  'PAIR'        -> Add only pair of classes.
  'ALL_CLASSES' -> Use SFFS to determine new binary problem.

Codings

---

---

One-versus-One or All-Pairs Coding · One-versus-All Coding · Discriminant Error Correcting Output Codes (DECOC) Coding · Discriminant Error Correcting Output Codes with Subclasses (subDECOC) Coding · Dense Random Coding · Sparse Random Coding · ECOC One Coding · Forest ECOC Coding · Custom Coding

---

One-Versus-One or All-Pairs Coding

---

Training - Testing Mode:

• Description: In the One-Versus-One Coding all pairs of classes are considered separately creating a coding matrix of K*(K-1)/2 columns (i.e., classifiers), where K is the number of classes. Each column consists of only a (+1,-1) pair and the rest entries of the column are 0.


• Associated Function: one_vs_one().
  
  
• Interface:

  u32
  one_vs_one
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const int decoding_strategy,
    const int classifier_type,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // compute misclassified test samples
  const u32 missed = one_vs_one
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       decoding_strategy,
                       classifier_type,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the One-Versus-One Coding.


• Associated Function: cvonevsone().
  
  
• Interface:

  double
  cvonesvsone
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = onevsonecv
                              (
                              samples,
                              labels,
                              n_folds,
                              decoding_strategy,
                              classifier_type,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

One-Versus-All Coding

---

Training - Testing Mode:

• Description:In the One-Versus-All Coding each class is considered against all the other classes creating a coding matrix of K columns (i.e., classifiers), where K is the number of classes. Each column consists of only a +1 and the rest entries of the column are -1.


• Associated Function: one_vs_all().
  
  
• Interface:

  u32
  one_vs_all
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const int decoding_strategy,
    const int classifier_type,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // compute misclassified test samples
  const u32 missed = one_vs_all
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       decoding_strategy,
                       classifier_type,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the One-Versus-All Coding.


• Associated Function: cvonevsall().
  
  
• Interface:

  double
  cvonesvsall
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = onevsallcv
                              (
                              samples,
                              labels,
                              n_folds,
                              decoding_strategy,
                              classifier_type,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

Discriminant Error Correcting Output Codes Coding

---

Training - Testing Mode:

• Description: In the Discriminant Error Correcting Output Codes Coding each binary problem is created by applying the Sequential Forward Floating Search Algorithm, which is used to maximize a certain optimization criterion. The result is a discriminant tree structure where each node contains a binary problem to be learned and according to this structure the coding matrix is constructed. Each created binary classifier has +1 and -1 in the rows that correspond to the classes which discriminates and 0 in the rows that belong to classes not taken into account.


• Associated Function: decoc().
  
  
• Interface:

  u32
  decoc
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const int decoding_strategy,
    const int classifier_type,
    const int criterion_option,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • bool verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion
  const int criterion_option = FLDR;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // compute misclassified test samples
  const u32 missed = decoc
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       decoding_strategy,
                       classifier_type,
                       criterion_option,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the DECOC Coding.


• Associated Function: cvdecoc().
  
  
• Interface:

  double
  cvdecoc
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const u32 criterion_option,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion selected by user
  const int criterion_option = FLDR;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = cvdecoc
                              (
                              samples,
                              labels,
                              n_folds,
                              decoding_strategy,
                              classifier_type,
                              criterion_option,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

Discriminant Error Correcting Output Codes with Subclasses Coding

---

Training - Testing Mode:

• Description: In the Discriminant Error Correcting Output Codes with Subclasses Coding each binary problem is created by applying the Sequential Forward Floating Search (SFFS) Algorithm, which is used to maximize a certain optimization criterion. The result is a discriminant tree structure where each node contains a binary problem to be learned and according to this structure the coding matrix is constructed. Each created binary classifier has +1 and -1 in the rows that correspond to the classes which discriminates and 0 in the rows that belong to classes not taken into account. Additionally in each step of the SFFS algorithm a single class bipartition can be split into subclasses resulting in more complex discriminative surfaces. At the end of the process each class has as many coding words in the coding matrix as the subclasses that it has been split into.


• Associated Function: subdecoc().
  
  
• Interface:

  u32
  subdecoc
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const Threshold& thres,
    const int decoding_strategy,
    const int classifier_type,
    const int criterion_option,
    const bool verbose,
    imat& coding_matrix,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • thres : Threshold object which defines the threshold splitting criteria values.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • bool verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• coding_matrix : The final coding matrix produced.
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // set thresholds
  // 1st argument performance, 2nd argument minimum cluster size, 3rd
  // argument minimum improvement
  Threshold thres(0,2,1);
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion
  const int criterion_option = FLDR;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // output stream -- redirect accordingly
  ofstream verbose_output;
  
  // compute misclassified test samples
  const u32 missed = subdecoc
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       thres,
                       decoding_strategy,
                       classifier_type,
                       criterion_option,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the subDECOC Coding.


• Associated Function: cvsubdecoc().
  
  
• Interface:

  double
  cvsubdecoc
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const u32 criterion_option,
    const Threshold& thres,
    const bool verbose,
    double& mr,
    double& mc,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • thres : Threshold object which defines the threshold splitting criteria values.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• mr : Mean number of rows of coding matrices formed.
• mc : Mean number of columns of coding matrices formed.
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // set thresholds
  // 1st argument performance, 2nd argument minimum cluster size, 3rd
  // argument minimum improvement
  Threshold thres(0,2,1);
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion selected by user
  const int criterion_option = FLDR;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // mean number of rows
  double mr = 0.0;
  
  // mean number of columns
  double mc = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = cvsubdecoc
                              (
                              samples,
                              labels,
                              n_folds,
                              decoding_strategy,
                              classifier_type,
                              criterion_option,
                              thres,
                              verbose,
                              mr,
                              mc,
                              verbose_output,
                              elapsed_time
                              );
  

---

Dense Random Coding

---

Training - Testing Mode:

• Description: In the Dense Random Coding the coding matrix contains only +1 and -1 and it is constructed by choosing from a large user-specified number of random matrices with number of columns that are also specified by the user. The resulting matrix is checked for its validity, that is, each column of the coding matrix should not contain only +1 or -1 and no two columns of the matrix should be identical with each other.


• Associated Function: dense_random_ecoc().
  
  
• Interface:

  u32
  dense_random_ecoc()
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const int decoding_strategy,
    const int classifier_type,
    const u32 n_matrices,
    const u32 n_desired_classifiers,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • n_matrices : Number of random matrices to be formed.
  • n_desired_classifiers : Number of classifiers to be formed.
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // number of matrices to be formed
  const int n_matrices = 100;
  
  // number of classifiers to be formed
  const int n_desired_classifiers = 3;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // compute misclassified test samples
  const u32 missed = dense_random_ecoc
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       decoding_strategy,
                       classifier_type,
                       n_matrices,
                       n_desired_classifiers,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the Dense Random ECOC Coding.


• Associated Function: cvdenserand().
  
  
• Interface:

  double
  cvdenserand
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const u32 n_matrices,
    const u32 n_desired_classifers,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • n_matrices : Number of random matrices to be formed.
  • n_desired_classifier : Number of classifiers to be formed.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // number of matrices to be formed
  const u32 n_matrices = 100;
  
  // number of classifiers to be formed
  const u32 n_desired_classifiers = 3;
  
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = cvdenserand
                              (
                              samples,
                              labels,
                              n_folds,
                              classifier_type,
                              decoding_strategy,
                              n_matrices,
                              n_desired_classifiers,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

Sparse Random Coding

---

Training - Testing Mode:

• Description: In the Sparse Random Coding the coding matrix contains +1, -1 and 0 and it is constructed by choosing from a large user-specified number of random matrices with number of columns that are also specified by the user. The resulting matrix is checked for its validity, that is, each column of the coding matrix should not contain only +1, -1 or 0 and no two columns of the matrix should be identical with each other.


• Associated Function: sparse_random_ecoc().
  
  
• Interface:

  u32
  sparse_random_ecoc()
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const int decoding_strategy,
    const int classifier_type,
    const u32 n_matrices,
    const u32 n_desired_classifiers,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • n_matrices : Number of random matrices to be formed.
  • n_desired_classifiers : Number of classifiers to be formed.
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // number of matrices to be formed
  const int n_matrices = 100;
  
  // number of classifiers to be formed
  const int n_desired_classifiers = 3;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // compute misclassified test samples
  const u32 missed = sparse_random_ecoc
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       decoding_strategy,
                       classifier_type,
                       n_matrices,
                       n_desired_classifiers,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the Sparse Random ECOC Coding.


• Associated Function: cvsparserand().
  
  
• Interface:

  double
  cvsparserand
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const u32 n_matrices,
    const u32 n_desired_classifers,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • n_matrices : Number of random matrices to be formed.
  • n_desired_classifiers : Number of classifiers to be formed.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // number of matrices to be formed
  const u32 n_matrices = 100;
  
  // number of classifiers to be formed
  const u32 n_desired_classifiers = 3;
  
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = cvsparserand
                              (
                              samples,
                              labels,
                              n_folds,
                              classifier_type,
                              decoding_strategy,
                              n_matrices,
                              n_desired_classifiers,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

ECOC One Coding

---

Training - Testing Mode:

• Description: Problem-dependent ECOC coding design. A validation sub-set is used to extend any initial coding matrix and to increase its generalization by including new binary classifiers that are trained on difficult to learn binary problems - classes.


• Associated Function: ecoc_one().
  
  
• Interface:

  u32
  ecoc_one
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const Threshold& thres,
    const int decoding_strategy,
    const int classifier_type,
    const int criterion_option,
    const u32 n_matrices,
    const u32 n_desired_classifiers,
    const double validation,
    const int init_coding_strategy,
    const int ecocone_mode,
    const u32 max_iter,
    const double epsilon,
    const double wv,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • thres : Threshold object which defines the threshold splitting criteria values.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • n_matrices : Number of random matrices to be formed if initial coding strategy is random.
  • n_desired_classifiers : Number of classifiers to be formed if initial coding strategy is random.
  • validation : Percentage of validation set in (0,1).
  • init_coding_strategy : Initial coding strategy (see Codings Enumeration).
  • ecocone_mode : ECOC One mode (see ECOC One Modes Enumeration).
  • max_iter : Maximum number of columns added into the coding matrix.
  • epsilon : Epsilon tolerance.
  • wv : Validation error weight.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // set thresholds
  // 1st argument performance, 2nd argument minimum cluster size, 3rd
  // argument minimum improvement
  Threshold thres(0,2,1);
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion
  const int criterion_option = FLDR;
  
  // initial coding strategy
  const int init_coding_strategy = SUBDECOC;
  
  // validation percentage in [0,1]
  const double validation = 0.3;
  
  // ECOC One mode
  const u32 ecocone_mode = PAIR;
  
  // maximum number of columns
  const u32 max_iter = 10;
  
  // epsilon tolerance
  const double epsilon = 0.03;
  
  // validation error weight in [0,1]
  const double wv = 0.2;
  
  // dummies since initial coding strategy isn't random
  const double n_matrices = 0;
  const double n_desired_classifiers = 0;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // output stream -- redirect accordingly
  ofstream verbose_output;
  
  // compute misclassified test samples
  const u32 missed = ecoc_one
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       thres,
                       decoding_strategy,
                       classifier_type,
                       criterion_option,
                       n_matrices,
                       n_desired_classifiers,
                       validation,
                       init_coding_strategy,
                       ecocone_mode,
                       max_iter,
                       epsilon,
                       wv,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the ECOC One Coding.


• Associated Function: cvecocone().
  
  
• Interface:

  double
  cvecocone
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const u32 criterion_option,
    const Threshold& thres,
    const u32 n_matrices,
    const u32 n_desired_classifiers,
    const double validation,
    const int init_coding_strategy,
    const int ecocone_mode,
    const u32 max_iter,
    const double epsilon,
    const double wv,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • thres : Threshold object which defines the threshold splitting criteria values.
  • n_matrices : Number of random matrices to be formed if initial coding strategy is random.
  • n_desired_classifiers : Number of classifiers to be formed if initial coding strategy is random.
  • validation : Percentage of validation set in (0,1).
  • init_coding_strategy : Initial coding strategy (see Codings Enumeration).
  • ecocone_mode : ECOC One mode (see ECOC One Modes Enumeration).
  • max_iter : Maximum number of columns added into the coding matrix.
  • epsilon : Epsilon tolerance.
  • wv : Validation error weight.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // set thresholds
  // 1st argument performance, 2nd argument minimum cluster size, 3rd
  // argument minimum improvement
  Threshold thres(0,2,1);
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion selected by user
  const int criterion_option = FLDR;
  
  // initial coding strategy
  const int init_coding_strategy = SUBDECOC;
  
  // validation percentage in [0,1]
  const double validation = 0.3;
  
  // ECOC One mode
  const u32 ecocone_mode = PAIR;
  
  // maximum number of columns
  const u32 max_iter = 10;
  
  // epsilon tolerance
  const double epsilon = 0.03;
  
  // validation error weight in [0,1]
  const double wv = 0.2;
  
  // dummies since init coding strategy isn't random
  const double n_matrices = 0;
  const double n_desired_classifiers = 0;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = cvsubdecoc
                              (
                              samples,
                              labels,
                              n_folds,
                              decoding_strategy,
                              classifier_type,
                              criterion_option,
                              thres,
                              n_matrices,
                              n_desired_classifiers,
                              validation,
                              init_coding_strategy,
                              ecocone_mode,
                              max_iter,
                              epsilon,
                              wv,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

Forest ECOC Coding

---

Training - Testing Mode:

• Description: Problem-dependent ECOC coding design, where T binary tree structures are embedded in the ECOC matrix. This approach extends the variability of the classifiers of the DECOC design by including extra dichotomizers and avoiding to repeat previously learnt classifiers.


• Associated Function: forest_ecoc().
  
  
• Interface:

  u32
  forest_ecoc
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const int decoding_strategy,
    const int classifier_type,
    const int criterion_option,
    const u32 n_forests,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • n_forests : Maximum number of created trees.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• coding_matrix : The final coding matrix produced.
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion
  const int criterion_option = FLDR;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // output stream -- redirect accordingly
  ofstream verbose_output;
  
  // maximum number of trees
  const u32 n_forests = 3;
  
  // compute misclassified test samples
  const u32 missed = forest_ecoc
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       decoding_strategy,
                       classifier_type,
                       criterion_option,
                       n_forests,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the Forest ECOC Coding.


• Associated Function: cvforest().
  
  
• Interface:

  double
  cvforest
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const u32 criterion_option,
    const u32 n_forests,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • criterion_option : Enumeration type that denotes the user selected decomposition criterion (see Criteria Enumeration).
  • n_forests : Maximum number of trees.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion selected by user
  const int criterion_option = FLDR;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // maximum number of trees
  const u32 n_forests = 3;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = cvforest
                              (
                              samples,
                              labels,
                              n_folds,
                              decoding_strategy,
                              classifier_type,
                              criterion_option,
                              n_forests,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

Custom Coding

---

Training - Testing Mode:

• Description: By suppling a custom coding matrix as an argument the user can define a user custom defined coding strategy.


• Associated Function: custom_coding().
  
  
• Interface:

  custom_coding
    (
    const mat& training_samples,
    const icolvec& training_labels,
    const mat& testing_samples,
    const icolvec& testing_labels,
    const imat& coding_matrix,
    const int decoding_strategy,
    const int classifier_type,
    const bool verbose,
    ofstream& verbose_output,
    double& elapsed_time
    )
  

• Input Arguments:
  • training_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the training set.
  • training_labels : 1D integral valued Armadillo column vector containing the respective class labels of the training set feature vectors.
  • testing_samples : 2D real valued Armadillo matrix where its rows denote the feature vectors of the testing set.
  • testing_labels : 1D integral valued Armadillo column vector containing the respective class labels of the testing set feature vectors.
  • coding_matrix : User custom defined coding matrix.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: An unsigned integer that denotes the number of misclassified testing set feature vectors.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled
  icolvec testing_labels;
  
  // custom coding matrix -- should be filled
  imat coding_matrix;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // output stream -- redirect accordingly
  ofstream verbose_output;
  
  // compute misclassified test samples
  const u32 missed = custom_coding
                       (
                       training_samples,
                       training_labels,
                       testing_samples,
                       testing_labels,
                       coding_matrix,
                       decoding_strategy,
                       classifier_type,
                       verbose,
                       verbose_output,
                       elapsed_time
                       );
  

K fold cross-validation mode

• Description: K-fold cross-validation with the Custom Coding.


• Associated Function: cvcustom().
  
  
• Interface:

  double
  cvcustom
    (
    const mat& samples,
    const icolvec& labels,
    const u32 n_folds,
    const u32 classifier_type,
    const int decoding_strategy,
    const imat& coding_matrix,
    const bool verbose,
    ofstream& output,
    double& elapsed_time
    )
  

• Input Arguments:
  • samples : 2D real valued Armadillo matrix where its rows denote the feature vectors.
  • labels : 1D integral valued Armadillo column vector containing the respective class labels of the feature vectors.
  • n_folds : Unsigned integer set to the Number of folds.
  • decoding_strategy : Enumeration type that denotes the user selected decoding strategy (see Decodings Enumeration).
  • classifier_type : Enumeration type that denotes the user selected classifier (see Classifiers Enumeration).
  • coding_matrix : User defined coding matrix.
  • verbose : Boolean set true if user selected verbose output, false otherwise.
  
  
• Output Arguments
• verbose_output : C++ output file stream used to redirect verbose output.
• elapsed_time : Execution time in seconds.


• Returns: A real in [0,1] denoting the K-Fold cross-validation misclassification rate.


• Example:

  // matrix of training samples -- should be filled
  mat training_samples;
  
  // column vector of training labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec training_labels;
  
  // matrix of test samples -- should be filled
  mat testing_samples;
  
  // column vector of test labels -- should be filled and start from 1
  // check auxiliary function for this
  icolvec testing_labels;
  
  // user defined coding matrix -- should be filled
  imat coding_matrix;
  
  // type of decoding selected by user
  const int decoding_strategy = HAMMING;
  
  // type of classifier selected by user
  const int classifier_type = NCC;
  
  // type of decomposition criterion selected by user
  const int criterion_option = FLDR;
  
  // execution time in seconds
  double elapsed_time = 0.0;
  
  // set number of folds
  const u32 n_folds = 10;
  
  // to be filled with the execution time
  double elapsed_time = 0.0;
  
  // compute 10 fold cross-validation error rate
  const double error_rate = cvcustom
                              (
                              samples,
                              labels,
                              n_folds,
                              decoding_strategy,
                              classifier_type,
                              criterion_option,
                              codin_matrix,
                              verbose,
                              verbose_output,
                              elapsed_time
                              );
  

---

---

Setting Classifier Arguments

---

---

Setting LIBSVM Arguments

LIBSVM uses a structure that has as adjustable fields the support vector machine parameters. In the LIBSVM code a global parameter object called "param" is declared. Below you can see the param's available fields:


    param.svm_type    -> can take the value C_SVC for classification SVM or NU_SVC for nuSVM.
    param.kernel_type -> defines the SVM's kernel type, can take the values LINEAR, POLY, RBF,
                         SIGMOID, PRECOMPUTED.
    param.degree      -> Integral value defines the kernel degree in case kernel type is POLY.
    param.coef0       -> Real valued coefficient in case kernel type is SIGMOID.
    param.nu          -> nu in case NU_SVC.
    param.C           -> Cost parameter.
    param.gamma       -> Real valued gamma parameter in case kernel type RBF.

Example:

    param.svm_type = C_SVC;
    param.kernel_type = RBF;
    param.degree = 3; // no need to set it since kernel type is RBF
    param.coef0 = 0;  // no need to set it since kernel type is RBF
    param.nu = 0.5    // no need to set it since SVM type is C_SVC
    param.C = 100.0;
    param.gamma = 0.3;

For more options see LIBSVM code.

Setting Discrete AdaBoost Classifier Arguments

In the case where Discrete AdaBoost Classifier is in use the user can set the maximum number of weak classifiers to be created. We use the degree field from LIBSVM param object in order to set the maximum number of weak classifier when AdaBoost classifier is in use.


param.degree -> Sets the maximum number of weak classifiers when AdaBoost classifier is
                in use.

Example:

param.degree = 10;

Setting Sum of Error Square Classifier Arguments

In the case where Sum of Error Squares classifier is in use the user can set the regularization parameter. We use the cost field from LIBSVM param object in order to set the regularization parameter of the Sum of Error Squares classifier.


param.C -> Sets the regularization parameter when Sum of Error Squares classifier is in use.

Example:

param.C = 0.1;

---

---

Armadillo Basics

---

---

---

2D Matrices (mat, imat, umat)

---

• Armadillo's root template matrix class is Mat, where type can be one of: char, int, float, double, std::complex, etc.


• In the ECOC PAK the following Armadillo typedefs are used:

imat	=	Mat<s32>
umat	=	Mat<u32>
mat	=	Mat<double>



• Examples:

  // create a random 5 x 5 matrix
  // (uniform distribution in [0,1])
  mat A = randu<mat>(5,5);
  
  // access element at second row
  // and third column of A
  double x = A(1,2);
  
  // matrix addition
  mat B = A + A;
  
  // matrix multiplication
  mat C = A * B;
  
  // matrix element wise multiplication
  mat D = A % B;
  
  // fill B with zeros
  B.zeros();
  
  // allocate size for B
  B.set_size(10,10);
  
  // set B to 5 rows 6 columns matrix
  // set with zeros
  B.zeros(5,6);
  
  // fixed size matrices:
  
  mat::fixed<5,6> F;
  F.ones(); // fill F with ones
  




• Read here Armadillo documentation for mat, imat, umat

---

1D Vectors (rowvec, colvec, irowvec, icolvec, urowvec, ucolvec)

---

• The Col<type> class is derived from the Mat<type> class and inherits most of the member functions.


• In the ECOC PAK the following Armadillo typedefs are used:

irowvec	=	Row<s32>
urowvec	=	Row<u32>
rowvec	=	Row<double>
icolvec	=	Col<s32>
ucolvec	=	Col<u32>
colvec	=	Col<double>



• Examples:

  colvec x(10);
  colvec y = zeros<colvec>(10,1);
  
  mat    A = randu<mat>(10,10);
  
  // extract a column vector
  colvec z = A.col(5);
  
  rowvec x(10);
  rowvec y = zeros<mat>(1,10);
  
  mat A = randu<mat>(10,10);
  
  // extract a row vector
  rowvec z = A.row(5);
  



• Read here Armadillo documentation for row vectors


• Read here Armadillo documentation for column vectors