neural_network.cpp File Reference

Implementation of Multilayer Perceptron. More...

`#include <algorithm>`
`#include <cassert>`
`#include <chrono>`
`#include <cmath>`
`#include <fstream>`
`#include <iostream>`
`#include <sstream>`
`#include <string>`
`#include <valarray>`
`#include <vector>`
`#include "vector_ops.hpp"`
Include dependency graph for neural_network.cpp:

## Classes

class | machine_learning::neural_network::layers::DenseLayer
class | machine_learning::neural_network::NeuralNetwork

## Namespaces

| machine_learning
| Machine learning algorithms.
| neural_network
| Neural Network or Multilayer Perceptron.
| activations
| Various activation functions used in Neural network.
| util_functions
| Various utility functions used in Neural network.
| layers
| This namespace contains layers used in MLP.

## Functions

double | machine_learning::neural_network::activations::sigmoid (const double &x)
double | machine_learning::neural_network::activations::dsigmoid (const double &x)
double | machine_learning::neural_network::activations::relu (const double &x)
double | machine_learning::neural_network::activations::drelu (const double &x)
double | machine_learning::neural_network::activations::tanh (const double &x)
double | machine_learning::neural_network::activations::dtanh (const double &x)
double | machine_learning::neural_network::util_functions::square (const double &x)
double | machine_learning::neural_network::util_functions::identity_function (const double &x)
static void | test ()
int | main ()
| Main function. More...

## Detailed Description

Implementation of Multilayer Perceptron.

Author
  Deep Raval

A multilayer perceptron (MLP) is a class of feedforward artificial neural network (ANN). The term MLP is used ambiguously, sometimes loosely to any feedforward ANN, sometimes strictly to refer to networks composed of multiple layers of perceptrons (with threshold activation). Multilayer perceptrons are sometimes colloquially referred to as "vanilla" neural networks, especially when they have a single hidden layer.

An MLP consists of at least three layers of nodes: an input layer, a hidden layer and an output layer. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called backpropagation for training. Its multiple layers and non-linear activation distinguish MLP from a linear perceptron. It can distinguish data that is not linearly separable.

See Backpropagation for training algorithm.

Note
  This implementation uses mini-batch gradient descent as optimizer and MSE as loss function. Bias is also not included.

## Function Documentation

## drelu()

double machine_learning::neural_network::activations::drelu | ( | const double & | x | ) |

Derivative of relu function

Parameters

  X | Value

Returns
  derivative of relu(x)

{ return x >= 0.0 ? 1.0 : 0.0; }
Here is the call graph for this function:

## dsigmoid()

double machine_learning::neural_network::activations::dsigmoid | ( | const double & | x | ) |

Derivative of sigmoid function

Parameters

  X | Value

Returns
  Returns derivative of sigmoid(x)

{ return x * (1 - x); }
Here is the call graph for this function:

## dtanh()

double machine_learning::neural_network::activations::dtanh | ( | const double & | x | ) |

Derivative of Sigmoid function

Parameters

  X | Value

Returns
  Returns derivative of tanh(x)

{ return 1 - x * x; }
Here is the call graph for this function:

## identity_function()

double machine_learning::neural_network::util_functions::identity_function | ( | const double & | x | ) |

Identity function

Parameters

  X | Value

Returns
  Returns x

{ return x; }
Here is the call graph for this function:

## main()

int main | ( | void |  | ) |

Main function.

Returns
  0 on exit

{
// Testing
test();
return 0;
}
test
static void test()
Definition: neural_network.cpp:805
Here is the call graph for this function:

## relu()

double machine_learning::neural_network::activations::relu | ( | const double & | x | ) |

Relu function

Parameters

  X | Value

Returns
  relu(x)

{ return std::max(0.0, x); }
std::max
T max(T... args)
Here is the call graph for this function:

## sigmoid()

double machine_learning::neural_network::activations::sigmoid | ( | const double & | x | ) |

Sigmoid function

Parameters

  X | Value

Returns
  Returns sigmoid(x)

{ return 1.0 / (1.0 + std::exp(-x)); }
std::exp
T exp(T... args)
Here is the call graph for this function:

## square()

double machine_learning::neural_network::util_functions::square | ( | const double & | x | ) |

Square function

Parameters

  X | Value

Returns
  Returns x * x

{ return x * x; }
Here is the call graph for this function:

## tanh()

double machine_learning::neural_network::activations::tanh | ( | const double & | x | ) |

Tanh function

Parameters

  X | Value

Returns
  Returns tanh(x)

{ return 2 / (1 + std::exp(-2 * x)) - 1; }
Here is the call graph for this function:

## test()

static void test | ( |  | ) |

static

Function to test neural network

Returns
  none

{
// Creating network with 3 layers for "iris.csv"
machine_learning::neural_network::NeuralNetwork myNN =
machine_learning::neural_network::NeuralNetwork({
{4, "none"}, // First layer with 3 neurons and "none" as activation
{6,
"relu"}, // Second layer with 6 neurons and "relu" as activation
{3, "sigmoid"} // Third layer with 3 neurons and "sigmoid" as
// activation
});
// Printing summary of model
myNN.summary();
// Training Model
myNN.fit_from_csv("iris.csv", true, 100, 0.3, false, 2, 32, true);
// Testing predictions of model
assert(machine_learning::argmax(
myNN.single_predict({{5, 3.4, 1.6, 0.4}})) == 0);
assert(machine_learning::argmax(
myNN.single_predict({{6.4, 2.9, 4.3, 1.3}})) == 1);
assert(machine_learning::argmax(
myNN.single_predict({{6.2, 3.4, 5.4, 2.3}})) == 2);
return;
}
machine_learning::neural_network::NeuralNetwork
Definition: neural_network.cpp:247
machine_learning::neural_network::NeuralNetwork::fit_from_csv
void fit_from_csv(const std::string &file_name, const bool &last_label, const int &epochs, const double &learning_rate, const bool &normalize, const int &slip_lines=1, const size_t &batch_size=32, const bool &shuffle=true)
Definition: neural_network.cpp:587
machine_learning::neural_network::NeuralNetwork::summary
void summary()
Definition: neural_network.cpp:773
machine_learning::neural_network::NeuralNetwork::single_predict
std::vector< std::valarray< double > > single_predict(const std::vector< std::valarray< double >> &X)
Definition: neural_network.cpp:451
machine_learning::argmax
size_t argmax(const std::vector< std::valarray< T >> &A)
Definition: vector_ops.hpp:307
Here is the call graph for this function: