2.2 Distribution Models
The main reason behind distributing data over a large cluster is to overcome the difficulty and to cut the cost of buying expensive servers. There are several distribution models with which an increase in data volume and large volumes of read or write requests can be handled, and the network can be made highly available. The downside of this type of architecture is the complexity it introduces with the increase in the number of computers added to the cluster. Replication and sharding are the two major techniques of data distribution. Figure 2.5 shows the distribution models.
Replication—Replication is the process of placing the same set of data over multiple nodes. Replication can be performed using a peer‐to‐peer model or a master‐slave model.
Sharding—Sharding is the process of placing different sets of data on different nodes.
Sharding and Replication—Sharding and replication can either be used alone or together.
2.2.1 Sharding
Sharding is the process of partitioning very large data sets into smaller and easily manageable chunks called shards. The partitioned shards are stored by distributing them across multiple machines called nodes. No two shards of the same file are stored in the same node, each shard occupies separate nodes, and the shards spread across multiple nodes collectively constitute the data set.
Figure 2.6a shows that a 1 GB data block is split up into four chunks each of 256 MB. When the size of the data increases, a single node may be insufficient to store the data. With sharding more nodes are added to meet the demands of the massive data growth. Sharding reduces the number of transaction each node handles and increases throughput. It reduces the data each node needs to store.
Figure 2.5 Distribution model.
Figure 2.6 (a) Sharding. (b) Sharding example.
Figure 2.6b shows an example as how a data block is split up into shards across multiple nodes. A data set with employee details is split up into four small blocks: shard A, shard B, shard C, shard D and stored across four different nodes: node A, node B, node C, and node D. Sharding improves the fault tolerance of the system as the failure of a node affects only the block of the data stored in that particular node.
2.2.2 Data Replication
Replication is the process of creating copies of the same set of data across multiple servers. When a node crashes, the data stored in that node will be lost. Also, when a node is down for maintenance, the node will not be available until the maintenance process is over. To overcome these issues, the data block is copied across multiple nodes. This process is called data replication, and the copy of a block is called replica. Figure 2.7 shows data replication.
Replication makes the system fault tolerant since the data is not lost when an individual node fails as the data is redundant across the nodes. Replication increases the data availability as the same copy of data is available across multiple nodes. Figure 2.8 illustrates that the same data is replicated across node A, node B, and node C. Data replication is achieved through the master‐slave and peer‐peer models.
Figure 2.7 Replication.
Figure 2.8 Data replication.
2.2.2.1 Master‐Slave Model
Master‐slave configuration is a model where one centralized device known as the master controls one or more devices known as slaves. In a master‐slave configuration a replica set constitutes a master node and several slave nodes. Once the relationship between master and slave is established, the flow of control is only from master to the slaves. In master‐slave replication, all the incoming data are written on the master node, and the same data is replicated over several slave nodes. All the write requests are handled by the master node, and the data update, insert, or delete occurs in the master node, while the read requests are handled by slave nodes. This architecture supports intensive read requests as the increasing demands can be handled by appending additional slave nodes. If a master node fails, write requests cannot be fulfilled until the master node is resumed or a new master node is created from one of the slave nodes. Figure 2.9 shows data replication in a master‐slave configuration.
2.2.2.2 Peer‐to‐Peer Model
In the master‐slave model only the slaves are guaranteed against single point of failure. The cluster still suffers from single point of failure, if the master fails. Also, the writes are limited to the maximum capacity that a master can handle; hence, it provides only read scalability. These drawbacks in the master‐slave model are overcome in the peer‐to‐peer model. In a peer‐to‐peer configuration there is no master‐slave concept, all the nodes have the same responsibility and are at the same level. The nodes in a peer‐to‐peer configuration act both as client and the server. In the master‐slave model, communication is always initiated by the master, whereas in a peer‐to‐peer configuration, either of the devices involved in the process can initiate communication. Figure 2.10 shows replication in the peer‐to‐peer model.
Figure 2.9 Master‐Slave model.
In the peer‐to‐peer model the workload or the task is partitioned among the nodes. The nodes consume as well as donate the resources. Resources such as disk storage space, memory, bandwidth, processing power, and so forth, are shared among the nodes.
Reliability of this type of configuration is improved through replication. Replication is the process of sharing the same data across multiple nodes to avoid single point of failure. Also, the nodes connected in a peer‐to‐peer configuration are geographically distributed across the globe.
2.2.3 Sharding and Replication
In sharding when a node goes down, the data stored in the node will be lost. So it provides only a limited fault tolerance to the system. Sharding and replication can be combined to make the system fault tolerant and highly
