As a rule, large projects (like as Amazon website) have a strong back-end part. It is entrusted with processing complex calculations. Very often the back-end is the reason for the slow operation of the web application. In the process of developing complex projects, you should pay attention to query optimization and rational use of server performance. If your project has a well-developed back-end part, but the load continues to grow and this leads to unstable work, we recommend starting scalable. If the load is temporary, you can increase the server's power. Otherwise, you should divide the back-end into several servers. It's enough to transfer some of the scripts and modules to the second server. After that, you need to distribute queries evenly between them. This will make it possible to take off the load from the main server and redirect it to other hardware. Redirects mainly are done on the front-end side.
Scaling the database
If the database has a huge amount of data and loads a separate server, you can scale this cluster. There are two approaches to database scaling:
-Distribution of computing processes.
-Dividing the database.
Distribution of computing processes in the database
If you have not a large database, but a lot of complex calculations that load the server, we recommend that you separate the calculation process. Thus, each server will have a complete copy of the database and only perform its own limited work area. This is a more complex task, which requires more effort to implement. First of all, this is due to the synchronization of data between the servers, in order to keep every copy of the database up-to-date.
There are several ways to solve this problem:
- Application-level synchronization. In this case, the scripts automatically record changes to each copy of the database. This approach has many risks since any failure can disrupt the synchronization of data.
- Replication. Allows making changes on one "main" server, after that there is an automatic change on the other servers.
- Multi-master replication. Very similar to replication, only in this case the script can access any server. Replication will be extended to other servers.
Dividing the database
If you do not have many calculations, but the database consists of many items (often such problems have large e-commerce websites), in this case you need to divide the database into several servers. Each server will have some data and interact with other servers. There are several data distribution schemes:
-Vertical partitioning. The essence of this scheme is that some tables are moved to another server. In this case, the flexibility is lost, since there is no way to make complex SQL queries. If we make a correct and logical division of the base, this problem can be minimized.
-Horizontal partitioning. In this scheme, the database store one and the same table, only with different parts. For example, you have a table with 100 million user items. We divided it into 2 servers, on one server only men's items will be stored, and on the second - only women. In this case, you need to make changes to the back-end in order to correctly distribute the requests between the servers.
Division of requests between servers
Once your system has multiple servers, you will be faced with the question of the correct division of requests between servers. There are two methods for dividing requests (balancing):
-The balancing node. This is a simple method of the division the load between requests. The client interacts with the main server, which independently determines which server will be contacted to process the request. The main problem is the reliability of this method, since all logic is located at one point. If this node will be broken, it will stop the whole system running.
-Balance on the client side. A safer and more flexible method for dividing requests between servers. The idea is to get the user parameters. After this, based on this data, the requests are automatically divided. Very often, IP and user countries are used as main values. Knowing this, the user automatically redirects to the necessary servers. This approach is used by large projects, for example, Facebook and Google.
Large projects with high load require certain knowledge and experience. Scaling a project is a complex process that must be considered before starting the design. This will save money and time in the future. In general, there are many schemes and methods of scaling: front-end, back-end, database. They provide an opportunity to develop a structure that will work with hundreds of millions of users per month. To get a high-quality result, you need to develop the right architecture and use reliable servers. For scalable projects, we recommend using the Amazon Elastic Compute Cloud (Amazon EC2). Trust only professional web development companies. Our company, Merehead, has experience developing complex scalable projects with high load. If you have any questions, you can contact us and we will be glad to help you.