Neural network technologies – InformTFB

Neural network technologies

Neural network technologies

The topic is dedicated to neural network technologies.

Previously, this topic was published for the Institute, in order to gain the first scientific research experience, and in my opinion, this is one of the most relevant topics along with the quantum network, that is, these technologies will radically change the scope of application and data processing in both situational and predictable situations.

The peculiarity of using neural network technologies is that you can create different decision-making systems that require analytical calculations based on the collected knowledge base created with the help of a specialist in a particular field and a programmer who plays the role of an intermediary between the executing machine and a specialist in a particular field.

In neural network technologies, there is a method for solving exclusive “or” (XOR) problems.

The exclusive “or” table looks like this:

· for binary addition modulo 2 (used in binary half-adders):


Rule: the result is 0 if both operands are equal; otherwise, the result is 1.

* for ternary addition modulo 2 (used in binary full adders):


Rule: the result is 0 if there are no operands equal to 1, or an even number of them.

Therefore, the neural network takes two numbers (parameters) as input and must output another equivalent number — the answer.

A neuron is a unit (input data) that receives information and then transmits information to hidden layers n, where arithmetic and logical calculations occur, followed by an output layer that outputs the result (output data).

A synapse is a connection between two neurons, synapses have 1 parameter-weight.
Thanks to it, input information changes when it is transmitted from one neuron to another.

Neurons operate with numbers in the range [0,1] or [-1,1]. If the numbers fall out of this range, then “1” must be divided by the resulting number.

The “input” field contains the total information of all neurons from the previous layer, after which it is normalized using the activation function.

The activation function is used to normalize the input data (optimization), that is, if the input receives a large number, the activation function allows you to get this number in the desired range for us.

The most basic activation functions are Linear, Sigmoid, and Hyperbolic tangent. Their main difference is the range of values:

The linear function

It is most often used for testing a neural network.


Common normalized sigmoid activation function, its range of values is [0,1].

Hyperbolic tangent

Used when numbers can be either negative or positive, since the range of the function is [-1,1].

There are methods for preprocessing data for preliminary analysis of the neural network for a sign of learning ability in a given hyperbolic range. one example is the Lipschitz method.
Where with the use of mathematical formulae with strict limitations on module, you can determine the speed of learning:

where x min is the minimum sample value of the feature.
x max – the maximum sample value of the feature.
[a, b] is the sampling interval.
x i – value of the attribute.

Or, if the neural network processing method of training does not have strict restrictions, the scaling formula is applied, which gives a non-zero average and a unit variance for the processed value:

where M(x) is the original sample mean.
σ(x) is the mean square deviation.
x i – value of the attribute.

At the stages of preprocessing data, selecting subsets of examples, searching for atypical observations, and performing exploratory data analysis, increasing the speed of algorithms will allow for faster, deeper, and more comprehensive analysis of data properties).

The neural network is trained in iterations – this is the total number of training sets completed by the neural network.
A training set is a sequence of data that a neural network operates on.
An epoch is a step that is considered completed when the entire set of training sets is completed.
The error is a percentage value, it is formed every epoch and should go down. The error is calculated using various mathematical formulas: “MSE”, “Root MSE”, “Arctan”, etc.

An example is the popular image-based object recognition system developed by Google.
Here is an example of image processing in order to get the result, what is this action/object?:

In the image, two images, a neural network is the first step of training, where it shows a picture of an animal, in this case “Cat” -the cat, and after some time of calculations after a process of classification, recognition, tells the operator that the recognition process is completed with the result: “Cat” – a cat.
As you can see in the photo, when a neural network consists of many number of neurons, the term layer is introduced.


Neural networks will allow you to update all current decision-making and support systems, automate the process, and then create an independent system after completing the first iterations of neural network training. In the future, the full globalization of automated processes, up to the support of users in solving household and other tasks on personal computers: starting with capturing the error area on the screen at the WORKSTATION station/PC and search for the similarity of points/pixels of a similar record in the database and issue a solution.

Valery Radokhleb
Valery Radokhleb
Web developer, designer

Leave a Reply

Your email address will not be published. Required fields are marked *