Perceptron

raise SystemExit("Stop right there!");

An exception has occurred, use %tb to see the full traceback.

SystemExit: Stop right there!

Importing libraries and packages

import os
import tensorflow as tf

# Statistics
from sklearn.metrics import accuracy_score

# Warnings
import warnings

warnings.filterwarnings("ignore")

os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"

Loading dataset

# Input data and labels of the OR table data in TensorFlow
X = tf.Variable(
    [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]], dtype=tf.float32
)
print(X)

<tf.Variable 'Variable:0' shape=(4, 2) dtype=float32, numpy=
array([[0., 0.],
       [0., 1.],
       [1., 0.],
       [1., 1.]], dtype=float32)>

Forward propagation

The data representation of the input data and the true labels in TensorFlow. The layers are a linear layer and activation functions, in the form of a net input function and a sigmoid function respectively. For the neural network representation, a function called perceptron() is used, with the linear and a sigmoid layer to create predictions. Using input data and initial weights and biases is called forward propagation.

# Setting the actual labels in TensorFlow and use the reshape()
# function to reshape the y vector into a 4 × 1 matrix
y = tf.Variable([0, 1, 1, 1], dtype=tf.float32)
y = tf.reshape(y, [4, 1])
print(y)

tf.Tensor(
[[0.]
 [1.]
 [1.]
 [1.]], shape=(4, 1), dtype=float32)

# Number of neurons (units) = 1
# X.shape[1] will equal 2 (since the indices start with zero)
# Number of features (inputs) = 2 (number of examples × number of features)
Number_of_features = X.shape[1]
Number_of_units = 1

# Defining the connections weight matrix in TensorFlow
weight = tf.Variable(
    tf.zeros([Number_of_features, Number_of_units]), dtype=tf.float32
)
print(weight)

<tf.Variable 'Variable:0' shape=(2, 1) dtype=float32, numpy=
array([[0.],
       [0.]], dtype=float32)>

# Variable for the bias
bias = tf.Variable(tf.zeros([Number_of_units, 1]), dtype=tf.float32)
print(bias)

<tf.Variable 'Variable:0' shape=(1, 1) dtype=float32, numpy=array([[0.]], dtype=float32)>

def perceptron(x):
    z = tf.add(tf.matmul(x, weight), bias)
    output = tf.sigmoid(z)
    return output

print(perceptron(X))

tf.Tensor(
[[0.5]
 [0.5]
 [0.5]
 [0.5]], shape=(4, 1), dtype=float32)

Backward propagation

The predictions are not quite accurate, yet. Backward propagation is next.

For the backpropagation of the error, use an optimizer to minimize the loss: The Stochastic Gradient Descent (SGD) optimizer will update the parameters of the networks (weights and biases) on each instance from the input data. The learning rate is the magnitude by which SGD takes a step in order to reach the global optimum of the loss function.

learning_rate = 0.01
optimizer = tf.optimizers.SGD(learning_rate)

# Defining the epochs, training loop, and loss function
no_of_epochs = 1000
for n in range(no_of_epochs):
    loss = lambda: abs(
        tf.reduce_mean(
            tf.nn.sigmoid_cross_entropy_with_logits(
                labels=y, logits=perceptron(X)
            )
        )
    )
    optimizer.minimize(loss, [weight, bias])

Statistics

tf.print(weight)

[[0.412449151]
 [0.412449151]]

tf.print(bias)

[[0.236065909]]

ypred = perceptron(X)
ypred = tf.round(ypred)

acc = accuracy_score(y, ypred)
print(acc)

0.75