send appropriate follow‐up emails to recipients on the optimal time already set by the machine.
How amazing it would be if all these repetitive tasks are performed by a machine so that you can channel your energy and time on a more complex and productive task.
WHAT IS MACHINE LEARNING: HOW HAS IT EVOLVED?
“The early bird gets the first worm, but the wisest bird gets the fastest one.”
— Matshona Dhliwayo
Hard work can take you wherever you want, but smart work can make you reach there faster. Let's make our system smart enough to carry out repetitive/mundane tasks. Now, let's understand what machine learning is all about and how it can make a difference.
Machine learning is like a person learning from experience. For example, as the owner of a grocery store, you need to optimize your inventory. The question is, how can ML help you in doing so? ML can predict inventory needs based on the weekday, season, events in nearby locations, customers' behavior, and so on. But for precise predictions, you need to feed your machine with lots of data, so that machine learning algorithms find patterns in the data. Using this data, the ML algorithm can predict sales in advance. Also, if you are using computer vision technology to monitor customer behavior or if you are using a robot assistant like LoweBot in your store, both technologies help ML to keep track of inventory and notify managers if any unexpected pattern of inventory data is found.
DEFINITION OF MACHINE LEARNING
Machine learning unlocks the hidden insight of data by allowing machines to learn from examples and experiences. Instead of writing the code explicitly, what you do is feed the data to the generic algorithms in the machines. The machines then analyze this data, change the data patterns, and build the logic to serve predictions on previously unseen data.
Evolution of Machine Learning
Ever imagine what business was like 50 years ago when there were no computing machines? We can thank those genius philosophers, mathematicians, and computer scientists who made what was once fiction a reality. Today, the technology that helps us do everything from housecleaning to driving cars is no longer science fiction. Machine learning helps all businesses and individuals to improve decision making, detect diseases, increase productivity, navigate vehicles and suggest the fastest route after analyzing traffic patterns, forecast weather, detect fraud, and much more.
Figure 1.2 shows the journey of how technology came together and how machine learning evolved.
The constant evolution of ML from robotic process automation to technical expertise has made its mark in many sectors. All businesses, ranging from start‐ups to global multinationals, want to develop a robust ML strategy in an increasingly ambitious and technical market.
Currently, businesses are working to achieve the following: advancement in cybersecurity, regulation of digital data, and faster computing power. In the following chapters, this book will teach you to utilize advanced ML solutions in Dynamics 365 for executing complex tasks and sustaining accuracy for the success of businesses.
FIGURE 1.2 Evolution of machine learning.
Lifecycle of Machine Learning
Data‐driven organizations face different challenges in developing ML models, from prototyping to production. To derive practical business values, data scientists and data engineers serve the model with a huge amount of data and train it to take advantage of ML algorithms. To create a desirable ML system, businesses need to comprehend the ML lifecycle process. Now let's understand why ML lifecycle is so important for businesses.
According to sas.com, 50 percent of models never make it to production due to the following reasons:
Insufficient data. Insufficient data, when fed to the model, result in an increase in variance. Variance is a value that is equal to the difference between the prediction accuracy of training data and test data in the ML model. If the prediction accuracy between training data and test data is high, the model will produce accurate results with training data but will stop working as soon as test data is fed into it.
Nonrepresentative training data. It is the training set of data that doesn't reflect the cases of the deployment environment. This problem is also called sampling bias. It is necessary to make sure that the sample you are feeding to the model matches the environment it's going to be deployed in.
Poor quality data. It refers to the data that has missing observations, errors, outliers (values that deviate from other observations on data), and noise (spurious and unnecessary data).
Overfitting the data. It is a situation when the model learns the detail and noise in the training data so well that it produces negative results when fed with new data.
Underfitting the data. This situation occurs when you want to build an accurate model with fewer data. Due to a lack of data, the model is unable to capture the underlying trend of the data.
So, to build a model, it is crucial to have the right data, at the right time, in the right location. The ML lifecycles play a key role in building custom ML algorithms to support learning models. The main purpose of the lifecycle is to create a model with a good workflow that can be reproduced, revisited, and deployed to production easily.
Now let's understand what the machine learning lifecycle is and how it works:
The machine learning lifecycle is a repetitive process to build an efficient machine learning system called a “model.”
The machine learning lifecycle consists of four phases: (1) data preparation, (2) machine learning model, (3) validation, and (4) deployment. This lifecycle is all about gaining deeper insights from data. It is leveraged by data engineers, data scientists, and those working with data to develop, train, validate, and serve machine learning models. Figure 1.3 depicts a typical ML lifecycle and its phases.
Let's jump into these phases one by one.
Data Preparation
Machine learning algorithms need the right data to solve the problem. So first, you need to make sure that the data you have collected is on a useful scale and format. Data preparation is a process that involves converting raw data into a clean dataset before applying machine learning algorithms to the data.
