Biological Aspects that Inspired the Development of Artificial Neural Networks
It can be said that the study of Artificial Neural Networks began with the work
of McCulloch and Pitts in 1943. Their work was based on a desire to understand
the function of the biological brain, and they proposed a model of "computing
elements" called McCulloch-Pitts neurons, which performed a weighted sum of
the inputs to these elements followed by a threshold logic operation. The features of the biological neuron which inspired the artificial neuron are summarized as follows:
Cell body (soma): performs the central functions of the neuron. The equivalent in the artificial neuron is the summation and thresholding part of the model.
Dendrites: branch-like structures that receive input signals to be processed by
the soma. These are represented by the weighted inputs of the artificial neuron.
Axon: carries the output signals from the cell. In the artificial neuron this is
the output of the thresholding function.
When compared with electronic digital computers, the biological brain is
by far the best "computer" known to us. Nevertheless, both have their pros and cons.
The brain, for example, is not very good at carrying out arithmetic calculations,
whereas the digital computer excels at this task. On the other hand, the brain
is very good at tasks such as face recognition, a task that the digital computer
achieved successfully only as recently as 1993.
Digital computers are increasingly being called on to solve complex problems
that their biological counterparts solve with ease. This trend has been the
driving force behind the development of neural networks.
It is important to understand that, unlike its biological counterpart, an artificial neural network is not a physical entity, but one or more algorithms implemented in a computer program. In order to visualize an artificial neural network, some graphical conventions have been adopted. The picture below shows a single artificial neuron with the essential elements described above, i.e., the weighted inputs, the summation and thresholding part and the output.Credit: Chris de VilliersCredit: Chris de Villiers
Training of Artificial Neural Networks
In contrast with so-called expert systems that incorporate a knowledge (data)
base, artificial neural networks do not have a database. They need to be trained
with data appropriate to a given problem in order for the weights to be adjusted
optimally and for the desired performance to be achieved. One type of training
is known as supervised training. In supervised training a data set from the
problem under consideration is used to train the network. These data contain
certain parameters or variables (inputs) that give rise to certain conditions or
A subset of the available data, typically 70% is used to train the network.
This is an iterative process in which each input vecor and its associated output,
or target (usually a single number, but could be a vector), are presented to
the network. The input weights are then adjusted according to an appropriate
algorithm (e.g. the perceptron learning rule) in order to minimize the error,
i.e., the difference between the network’s output and the target. The process is
repeated until an acceptable error level is obtained. Thereafter, the remaining
30% of the data are used to validate the network. If validation is successful, the network is applied to the problem; if not, it is retrained using a different data set. In this case it may also be necessary to choose a different network structure, such as more or fewer nodes, hidden layers, etc.
Examples of the Practical Application of Artificial Neural Networks
Artificial neural networks have been used in a wide variety of problem solving
applications, and the field is widening as new applications are found. The
following examples serve to illustrate the variety of actual or potential neural
In the context of stereo vision it has been shown that, despite the considerable
differences in the operational principles of biological vision vs. neural network
‘vision’, the two approaches behave similarly. In this study the neural network comprised several subnetworks, each consisting of a three by three array of
input neurons connected to two successive layers of nine hidden nodes each, and
finally to one output neuron. Of significance is the fact that the neural network
was trained to detect features for stereopsis, and not to mimic its biological
counterpart. The results showed that both the neural network and its biological
counterpart respond strongly to vertical bars and edges of light.
Power line fault detection
In power transmission lines, so-called distance protection entails protecting the
power system from transmission line faults by isolating the faulty lines. Fault
conditions are line-to-line and line-to-ground faults. A neural network solution
has proven to be superior to the traditional methods of using sensing relays to
isolate the faulty lines, but long training times were required and the software
implementation was computationally inefficient, resulting in slow response. An
efficient fault detection system requires that a decision be made within one power cycle, which, in the case of a 50 Hz system, is 20 milliseconds. An alternative,
hardware implementation of a neural network allows fast, real-time distance
Among the different inspection techniques used for quality inspection of finished
goods, is the method of capturing an image of the item, enlarging the image
and inspecting it visually. An embedded automated inspection system, based
on an artificial neural network, has been found to be superior to the traditional
approaches in a printed circuit board assembly plant.
Water quality prediction
In 1974 the Department of Environment of Malaysia adopted a water quality index (WQI) to grade the level of pollution of rivers in Malaysia. The WQI was based on weighted values for dissolved oxygen, biochemical oxygen demand, chemical oxygen demand, pH value, ammoniacal nitrogen and suspended solids. Obtaining the WQI required that the raw data be converted to a sub-pollutant index before manual calculation of the WQI. A neural network solution was later implemented, which used past and present raw data to predict the WQI. Accuracy of this method was found to be between 92.4 percent and 99.96 percent.
Artificial neural networks are increasingly being applied to solving a broad spectrum of real-life problems. With increasingly powerful computers becoming readily available on the one hand, and powerful microcontroller devices finding their way into embedded applications on the other, artificial neural network applications are likely to become commonplace without us being aware of it.