adaline_learning.cpp File Reference
Machine Learning Algorithms

Adaptive Linear Neuron (ADALINE) implementation More...

`#include <array>`
`#include <cassert>`
`#include <climits>`
`#include <cmath>`
`#include <cstdlib>`
`#include <ctime>`
`#include <iostream>`
`#include <numeric>`
`#include <vector>`
Include dependency graph for adaline_learning.cpp:

## Classes

class | machine_learning::adaline

## Namespaces

| machine_learning
| Machine learning algorithms.

## Functions

void | test1 (double eta=0.01)
void | test2 (double eta=0.01)
void | test3 (double eta=0.01)
int | main (int argc, char **argv)

## Variables

constexpr int | MAX_ITER = 500

## Detailed Description

Adaptive Linear Neuron (ADALINE) implementation

Author
  Krishna Vedala

ADALINE is one of the first and simplest single layer artificial neural network. The algorithm essentially implements a linear function

\[ f\left(x_0,x_1,x_2,\ldots\right) = \sum_j x_jw_j+\theta \]

where \(x_j\) are the input features of a sample, \(w_j\) are the coefficients of the linear function and \(\theta\) is a constant. If we know the \(w_j\), then for any given set of features, \(y\) can be computed. Computing the \(w_j\) is a supervised learning algorithm wherein a set of features and their corresponding outputs are given and weights are computed using stochastic gradient descent method.

## Function Documentation

## main()

int main | ( | int | argc,
|  | char ** | argv
| ) |  |

Main function

{
std::srand(std::time(nullptr)); // initialize random number generator
double eta = 0.1; // default value of eta
if (argc == 2) { // read eta value from commandline argument if present
eta = strtof(argv[1], nullptr);
}
test1(eta);
std::cout << "Press ENTER to continue..." << std::endl;
std::cin.get();
test2(eta);
std::cout << "Press ENTER to continue..." << std::endl;
std::cin.get();
test3(eta);
return 0;
}
test2
void test2(double eta=0.01)
Definition: adaline_learning.cpp:262
test1
void test1(double eta=0.01)
Definition: adaline_learning.cpp:224
test3
void test3(double eta=0.01)
Definition: adaline_learning.cpp:313
std::cin
std::cout
std::endl
T endl(T... args)
std::srand
T srand(T... args)
std::strtof
T strtof(T... args)
std::time
T time(T... args)
Here is the call graph for this function:

## test1()

void test1 | ( | double | eta = `0.01` | ) |

test function to predict points in a 2D coordinate system above the line \(x=y\) as +1 and others as -1. Note that each point is defined by 2 values or 2 features.

Parameters

  [in] | eta | learning rate (optional, default=0.01)

{
adaline ada(2, eta); // 2 features
const int N = 10; // number of sample points
std::array<std::vector<double>, N> X = {
std::vector<double>({0, 1}), std::vector<double>({1, -2}),
std::vector<double>({2, 3}), std::vector<double>({3, -1}),
std::vector<double>({4, 1}), std::vector<double>({6, -5}),
std::vector<double>({-7, -3}), std::vector<double>({-8, 5}),
std::vector<double>({-9, 2}), std::vector<double>({-10, -15})};
std::array<int, N> y = {1, -1, 1, -1, -1,
-1, 1, 1, 1, -1}; // corresponding y-values
std::cout << "------- Test 1 -------" << std::endl;
std::cout << "Model before fit: " << ada << std::endl;
ada.fit<N>(X, y);
std::cout << "Model after fit: " << ada << std::endl;
int predict = ada.predict({5, -3});
std::cout << "Predict for x=(5,-3): " << predict;
assert(predict == -1);
std::cout << " ...passed" << std::endl;
predict = ada.predict({5, 8});
std::cout << "Predict for x=(5,8): " << predict;
assert(predict == 1);
std::cout << " ...passed" << std::endl;
}
std::array
machine_learning::adaline
Definition: adaline_learning.cpp:46
std::vector< double >
Here is the call graph for this function:

## test2()

void test2 | ( | double | eta = `0.01` | ) |

test function to predict points in a 2D coordinate system above the line \(x+3y=-1\) as +1 and others as -1. Note that each point is defined by 2 values or 2 features. The function will create random sample points for training and test purposes.

Parameters

  [in] | eta | learning rate (optional, default=0.01)

{
adaline ada(2, eta); // 2 features
const int N = 50; // number of sample points
std::array<std::vector<double>, N> X;
std::array<int, N> Y{}; // corresponding y-values
// generate sample points in the interval
// [-range2/100 , (range2-1)/100]
int range = 500; // sample points full-range
int range2 = range >> 1; // sample points half-range
for (int i = 0; i < N; i++) {
double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
X[i] = std::vector<double>({x0, x1});
Y[i] = (x0 + 3. * x1) > -1 ? 1 : -1;
}
std::cout << "------- Test 2 -------" << std::endl;
std::cout << "Model before fit: " << ada << std::endl;
ada.fit(X, Y);
std::cout << "Model after fit: " << ada << std::endl;
int N_test_cases = 5;
for (int i = 0; i < N_test_cases; i++) {
double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
int predict = ada.predict({x0, x1});
std::cout << "Predict for x=(" << x0 << "," << x1 << "): " << predict;
int expected_val = (x0 + 3. * x1) > -1 ? 1 : -1;
assert(predict == expected_val);
std::cout << " ...passed" << std::endl;
}
}
std::rand
T rand(T... args)
Here is the call graph for this function:

## test3()

void test3 | ( | double | eta = `0.01` | ) |

test function to predict points in a 3D coordinate system lying within the sphere of radius 1 and centre at origin as +1 and others as -1. Note that each point is defined by 3 values but we use 6 features. The function will create random sample points for training and test purposes. The sphere centred at origin and radius 1 is defined as: \(x^2+y^2+z^2=r^2=1\) and if the \(r^2<1\), point lies within the sphere else, outside.

Parameters

  [in] | eta | learning rate (optional, default=0.01)

{
adaline ada(6, eta); // 2 features
const int N = 100; // number of sample points
std::array<std::vector<double>, N> X;
std::array<int, N> Y{}; // corresponding y-values
// generate sample points in the interval
// [-range2/100 , (range2-1)/100]
int range = 200; // sample points full-range
int range2 = range >> 1; // sample points half-range
for (int i = 0; i < N; i++) {
double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
double x2 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
X[i] = std::vector<double>({x0, x1, x2, x0 * x0, x1 * x1, x2 * x2});
Y[i] = ((x0 * x0) + (x1 * x1) + (x2 * x2)) <= 1.f ? 1 : -1;
}
std::cout << "------- Test 3 -------" << std::endl;
std::cout << "Model before fit: " << ada << std::endl;
ada.fit(X, Y);
std::cout << "Model after fit: " << ada << std::endl;
int N_test_cases = 5;
for (int i = 0; i < N_test_cases; i++) {
double x0 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
double x1 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
double x2 = (static_cast<double>(std::rand() % range) - range2) / 100.f;
int predict = ada.predict({x0, x1, x2, x0 * x0, x1 * x1, x2 * x2});
std::cout << "Predict for x=(" << x0 << "," << x1 << "," << x2
<< "): " << predict;
int expected_val = ((x0 * x0) + (x1 * x1) + (x2 * x2)) <= 1.f ? 1 : -1;
assert(predict == expected_val);
std::cout << " ...passed" << std::endl;
}
}
Here is the call graph for this function: