effectiveness, if found OK then immediately return the same to the environment through effectors.
Step 3. The Critics provide the feedback to the learning element, if any new feedback occurs then it should be updated in the performance of the element. Then, the updated knowledge comes back to the learning element and send it to the problem generator as a learning goal for evaluating the same through experiments. The updates are sent to the performance of the element for future reference.
Figure 1.3 Learning behavior of a machine.
The learning process of ML is done in three different ways. These are supervised learning, unsupervised learning, and reinforcement learning. These three learning types have their importance in the different fields of bioinformatics research. Hence, they are explained with suitable examples in the next sections.
1.2.1 Supervised Learning
This is a very common learning mechanism in ML and used by most of the newcomer researchers in their respective fields. This learning mechanism trains the machine by using a labeled dataset in the form of compressed input–output pair as depicted in Refs. [13–15]. These datasets are available in continuous or discrete form. But the important thing is, it needs supervision with an appropriate training model. As supervised learning predicts accurate results [16], hence it is mostly used for Regression analysis and classification purposes. Figure 1.4 shows the execution model of supervised learning.
The figure shows that in supervised learning, a given set of input attributes (i. e. A1, A2, A3, A4 … … Ak) along with their output attributes (i.e. B1, B2, B3, B4 … … … Bk) are kept in a knowledge dataset. The Learning Algorithm takes an input Ai and executes with its model and produces the result Bi as the desired output. Supervised Learning has its importance in the field of Bioinformatics as concerning the heart disease scenario where inputs can be a lot of symptoms of heart diseases such as High Cholesterol, Chest Pain, and Blood Pressure, etc. and the output could be a person suffering from heart disease or not. Now all these inputs are passed on to the learning algorithm where it gets trained and if a new input is passed through the model then the machine gives an expected output. If the expected output’s accuracy is not up to the mark then there is a need for modification or up-gradation in the model.
Figure 1.4 Block diagram of supervised learning.
An example of supervised learning could be of a person who felt that he has a high cholesterol level and a chest pain and went to the doctor for a check-up. The Doctor fed the inputs given by the patient to the machine. The Machine predicted and told the doctor that the patient is suffering from a cardiac issue in his heart. It acts as an analogy to the supervised learning as the inputs given by the patient are the independent variables and their corresponding output from the machine acts as the dependent attribute. The Machine acted as a model that predicted and gave a relevant output as it is trained by similar inputs. Supervised Learning is itself a huge subfield of ML and useful for a variety of techniques used in research work. These techniques include Regression Analysis, Artificial Neural Networks (ANN), Support Vector Machines (SVM), etc.
1.2.2 Unsupervised Learning
In Unsupervised Learning, the user doesn’t have to supervise the model. Here, the model is allowed to work on its own to find the information. Here clusters are made [17–21]. A block diagram of unsupervised learning is shown in Figure 1.5.
The figure says that in unsupervised learning the inputs are collected as a set of features that are described as A1, A2, A3, A4 … … Ak. But, the output features are not available. The input parameters are passed to a learning algorithm module and diverse groups are formed that are called clusters [22–26].
Figure 1.5 Block diagram of unsupervised learning.
Unsupervised Learning has its role in Bioinformatics as concerning the heart disease scenario where inputs can be a lot of symptoms of heart diseases such as High Cholesterol, Chest Pain, and Blood Pressure, etc. These symptoms are passed onto the learning algorithm as input where clusters are made by the model and help the patient for identifying a disease (variables/values of similar types in one cluster) that may occur in near future.
1.2.3 Reinforcement Learning
In the field of ML, Reinforcement Learning was developed by John Andreae in the year 1963 when he invented a system called Stella [27]. It is a dynamic approach that works on the concept of feedbacks [28–31]. Reinforcement for a machine is the reward that it receives upon acting in the environment. When the machine acts on its environment, it receives some evaluation on its actions which is called reinforcement but is not told of which action is the correct one for achieving the goal. In this, the machine’s utility is defined by the feedback function [32]. The objective is to maximize the expected feedback. The block diagram of reinforcement learning is shown below in Figure 1.6.
The above figure tries to present that, a machine at first performs some actions in the environment. Once the actions are performed, then the machine starts to receive the feedbacks. The collected feedbacks may be positive or negative type. The positive feedbacks are kept inside the machines as knowledge. The machine tries to learn from the negative feedback so that in future such an incident may not happen again. Another important aspect of reinforcement learning is the state. The state also provides the input based on the situation to the machine for learning purposes.
Figure 1.6 Block diagram of reinforcement learning.
A few points of reinforcement learning are as follows:
The Input of the Reinforcement Learning Process: Initial state
The Output of the Reinforcement Learning Process: Diversified solutions can be present, depending on the feedbacks obtained
The training process is purely based on input.
This Reinforcement Learning model is a continuous process.
The best solution for this reinforcement learning is the maximum positive feedback.
An example of reinforcement learning could of a person who is suffering from high cholesterol and high blood pressure. He visits his family doctor and requests a medication regarding the same. After analyzing the symptoms, the doctor prescribed a diet chart and a set of medicines to minimize the cholesterol level and blood pressure. He took the medicines and felt better. Here, the patient gets positive feedback in the form of the results of the medication provided by the doctor. Now, the patient will be motivated and will consume only low-fat
