another state. A transition matrix that defines the possibilities of transition from one state to another is used to tally the transitions [24]. The Markov chain is one of the critical stochastic processes. The present state grasps all the information and uses it for the process’s future evolution [25]. The Markov chain model comes under the stochastic offloading process [25]. Among many processes, the Markov chain model is chosen where dynamic decision making is the requirement considering environmental parameters like wireless channel conditions and computation load [24].
2.3.1.2 Schemes Based on Semi-Markov Chain
This section aims at the Semi-Markov chain based offloading. The Semi-Markov chain is developed using the Markov renewal process. It is defined by the renewal kernel and the initial distribution using other features that equal to the renewal kernel. Semi-Markov chain model differs from the Markov chain model by having a realization of the process of the defined state by a random process that evolves over time [26]. For the verification of security for an offloading scheme, a state transition model is used.
2.3.1.3 Schemes Based on the Markov Decision Process
The Markov decision process is a mathematical framework of a discrete-time stochastic process. It assists in decision making for models that are partly random and partly under the control of the decision-maker. By using dynamic programming, optimization challenges are worked out [27]. The Markov decision process contains a list of items, e.g., probabilities of transition, states, decision epochs, costs, and actions. The computation in the Markov decision process is very high concerning the increase in the number of states. These issues can be solved by different algorithms such as linear programming, value iteration algorithms.
2.3.1.4 Schemes Based on Hidden Markov Model
Hidden Markov is a partially observable statistical Markov model where the agent partially observes the states. These models involve “hidden” generative processes with “noisy” observations correlated to the system [28]. The Hidden Markov model-based schemes allow the system or device to accompany the processing latency, power usage, and diagnostics accuracy.
2.3.2 Computation Offloading Schemes Based on Game Theory
To model allocation problems for wireless resources, the Game theory is practiced. Game theory helps in reducing the resource allocation problem by dividing it into distributed decision-making problems. The main advantage of game theory is that it focuses on strategic interactions by eliminating the use central controller.
Game theory models are getting more attention as a source to address wireless communication problems day by day. A game theory model consists of a group of decision-making blocks. The users plan a group of strategies and after using the strategy and corresponding pay off produced.
Offloading mobile data can be expressed in 3 tuples, <T=A, B,C> where A is represented as a group of users, B={B1, B2,….. Bn} is the strategy space of the user and C={C1, C2,…..Cn} is the utilization of the user after an action. If Bi is the strategy chosen by single user i, then the remaining users chosen strategies can be represented as B-i then B = {Bi,B-i} is the strategy formed by the user. At an equilibrium level, a strategy formed by the user needs to be chosen. No user will rationally choose to switch from his selected approach, which leads to a decrease in utility.
Game theory models can be divided into two groups: (i) Cooperative game model, (ii) Non-Cooperative game model.
In the cooperative game model, all the users will cooperate to attain an equilibrium state, which provides many benefits and will maximize the utilization factor through all user cooperative decision-making. This method is called Pareto optimality. In this method, a user cannot raise his pay off without reducing another user’s pay.
In the non-cooperative game model, different users select their own strategy without coordinating with other users. Each user is more concerned about their own payoff. All the decisions taken by a single user will make them more competitive with other users.
The computational offloading schemes are based on game theory that improvises the system’s design and data offloading optimization. There are different ways it can be optimized.
(i) Data offloading is always based on the multi-user decision-making problem. The multi-users are offloading scheme service providers and those who are beneficial from the offloading schemes to improvise their benefits. The advantage of users, i.e., service providers and service users, can be taken for maximum output [29]. The solution will be a game theory that provides solutions for different problem scenarios and resources appropriately shared between the other users.
(ii) Each block in the data offloading game theory completes each system’s advantages and disadvantages. Game theory gives a very efficient way to save the nodes from acting greedily through various software [30].
2.4 QoS and Optimization in Edge Computing
Quality of Service (QoS) defines a system’s performance and quality. To improve QoS, edge computing plays a vital role in any application using network resources at the local network. The applications include IoT devices, e.g., vehicular devices [31]. To guarantee a delay bounded QoS when performing task offloading is the challenge. When a large number of users compete for communication and limited computing resources, delay bounded QoS becomes the challenge. Another reason for delay bounded QoS is delay and energy consumption due to extra communication overhead while offloading computational tasks to edge servers [32].
There are two approaches to optimize QoS. The deterministic method of task offloading technique has a condition of completing the task by 100% before the deadline, which is quite impossible in practical situations due to transient noise, etc. Hence a statistical approach can be applied to complete the task before the deadline [32]. The section below discusses the statistical delay bounded Qos.
2.4.1 Statistical Delay Bounded QoS
In statistical delay bounded Qos with a probability of threshold, the tasks are planned to be completed before a deadline. While implementing a statistical approach, it is first required to consider the challenges.
1 i) Correlation between statistical delay and task offloading: The first challenge is defining a correlation between the statistical delay requirement and task offloading. Under a constrained communication and computation resource, quantifying the correlation is achieved.
2 ii) The low complexity - Second challenge for implementing statistical delay bounded QoS is to provide a holistic solution with low time complexity. Complexity is due to heterogeneous users in terms of computing capabilities and task requests [32]. Holistic task offloading algorithm considerations are discussed below.
2.4.2 Holistic Task Offloading Algorithm Considerations
By leveraging convex optimization theory and the Gibbs sampling method, the statistical approach is illustrated. A statistical computation model and a statistical transmission model is proposed. In the statistical computation model, the CPU clock is configured to save more energy, thus providing a statistical QoS guarantee. The traditional transmission rate is provided with a statistical delay exponent for QoS guarantee in the statistical transmission model. MINLP (Mixed Integer Non-Linear Program) is then formulated with statistical delay constraint
