are defined randomly and will be shifted in the state space iteratively until the specified distance function is minimized.
K-Means Clustering is a simple and most common centroid-based method. The objective is to partition data points into K clusters, where each data point should belong to the cluster with the nearest mean. Initially, K mean points are randomly picked (the centroids). Then, the algorithm iterates on each data point and computes the distance to the centroids. The data point is judged to belong to the point to which the computed Euclidean distance is minimum. Thus, the method minimizes the distance between points and their corresponding centroids. Centroids are updated based on their assigned data points. The process continues until the centroids do not change. Figure 2.5 illustrates the concept of K-means clustering.
Hierarchical Clustering: In this type, the number of clusters is not defined a priori; rather, it is iteratively increases or decreases. In the beginning, all observations are included in one cluster. Then, the cluster is split according to the largest distance between the data points. Once a sufficient number of clusters is reached, the process is stopped.
Figure 2.5 Illustration of K-means clustering.
Density-Based Clustering: In this type of clustering, the algorithm tries to find the areas with high and low density of observations. Data points that are within a specified distance become centers of a cluster. Other data points either belong to a cluster border or considered as noise.
2.4.3.3 Semi-Supervised Learning
This learning approach combines both supervised and unsupervised ML techniques. Thus, the machine learns from both labeled and unlabeled data. This approach is more realistic for many applications, wherein small amount of labeled data is available, but the collection of large set of labeled data is not easy or impractical.
2.4.3.4 Reinforcement Learning
Similar to unsupervised learning in the sense that the machine has to learn by itself. However, a reward mechanism is applied to tune the algorithm based on observation of performance, enabling continuous self-update of the machine. Reinforcement learning algorithms try to define a model of the environment by determining the dynamics of the environment. The algorithm uses an agent which interacts with a dynamic environment in a trial-and-error manner. It provides feedback to the algorithm. The agent makes decisions on what actions to be performed to optimize the reward. A policy determines how the agent should behave at a given time. Thus, the algorithm learns by exploring the environment and exploiting the knowledge. The feedback from the environment is used to learn the best policy to optimize the cumulative reward.
The most commonly known reinforcement algorithm is the Q-Learning. The RL algorithm interacts with the environment to learn Q values. The Q value is initialized. The machine observes the current state, chooses an action from a set of possible actions, and performs the action. The algorithm observes the reward and the new state. The Q-value is updated based on the new state and the reward. Then, the state is set to the new state and the process repeats until a terminal state is reached.
2.4.4 ML and Wireless Networks
It is expected that future wireless networks will be highly integrated and a qualitative change will occur regarding the use of high frequencies and wide channels. In addition, the networks are expected to run a large number of base stations and serve high density of users. Future communication networks are dynamic and may also be without cells and massive-MIMO. They will be intelligent, flexible, and highly resilient [8]. ML is a promising tool for efficient management of future wireless networks.
2.5 Software-Defined Networks
Current networks are characterized by their distributed nature, as each node (router/ switch) has the ability to view and act on the system partially and locally. Thus, global learning from network nodes that have a holistic view on the system will be very complicated. Further, current network designs impose significant limitations on network performance, especially under high traffic conditions. Consequently, the increasing demand for reliable, fast, scalable, and secure networks can adversely affect the performance of existing network devices due to the need to deliver a large volume of data both in the network infrastructure and devices. Current network devices lack the flexibility to handle different types of packets that may carry different contents due to the basic implementation of hard-wired routing rules. In addition, the networks that form the backbone of the Internet must be able to adapt to the changing conditions without needing much effort for hardware and software adjustments.
In order to reach a solution to the above discussed limitation issues, the rules for data processing must be implemented through software modules and not embedded in the hardware. This approach enables network administrators to have more control over the network traffic, and thus can greatly improve the network performance and effectively use the network resources. This innovative approach is called Software-Defined Network (SDN) [9].
SDN was released as open source software in 2008 with the OpenFlow project at Stanford University. It decouples the control and data planes in routers and switches, allowing the underlying infrastructure to be separated from application and network services. Thus, SDN separates the decision-making modules about where traffic is sent [the control plane (CP)] from the underlying systems responsible for forwarding the actual traffic (the data plane). Network resources are managed by a centralized controller which performs as the network operating system. The controller can dynamically program the network in real time. It collects information about network status and operation details. Therefore, the controller can globally detect available network resources and requirements. This paradigm creates a global view of the entire network, enabling global automatic management and control without needing to configure devices individually. The SND technology has several advantages:
Efficient utilization of network resources.
Enables development of programming-based solutions for network configuration and management.
Provides a perfect ecosystem for ML paradigm and intelligent applications.
Simple and improved network management, control, and data manipulation, since network administrators can remotely alter the network configuration and operation as response to dynamical changes in the network.
High speed, through optimal handling of the traffic load.
Adopts the virtualization technology, which allows running multiple applications over the same shared hardware.
Combining AI and SDN has been attracting researchers in recent years to develop network management and operation mechanisms. The SND architecture provides centralized control of network policies and enables administrators to effectively overcome problems with ML methods.
2.5.1 SDN Architecture
The architecture of SDN is comprised of three planes:
Data Plane: comprised of the forwarding devices, i.e., switches and virtual switches. Unlike distributed network architectures, in which switching and routing devices listen to events from neighboring network elements and make decisions based on a local view, switches and routers are responsible
