The so-called high concurrency refers to: at the same time or within a very short period of time, there are a large number of requests to reach the server, each request requires the server to spend resources to process, and make the corresponding feedback.

Commonly used ideas and tools for high concurrency handling

High Concurrency from a Server-Side Perspective

Server-side processing requests need to consume server-side resources, such as the number of processes that can be opened at the same time, the number of threads that can run at the same time, the number of network connections, cpu, I/O, memory and so on, because the server-side resources are limited, then the server can handle the request at the same time is also limited. The essence of the problem of high concurrency is: the limited nature of resources

Problems caused by high concurrency

The server’s processing and response will become slower and slower, and even discard some requests without processing, or worse, cause the server to crash.

Basic Ideas for High Concurrency Handling

1) From the client side
  • Minimize the number of requests, e.g., rely on the client’s own cache or processing power
  • Minimize unnecessary consumption of server-side resources, such as: reuse of certain resources, such as connection pooling client-side processing of the basic principle is: can not access the server do not access
2) From the server side
  • Increase the supply of resources, such as: larger network bandwidth, the use of more configurable servers, the use of high-performance Web servers, the use of high-performance databases
  • Request diversion, such as: the use of clusters, distributed system architecture
  • Application optimization, e.g., using more efficient programming languages, optimizing algorithms for processing business logic, optimizing SQL for accessing databases.

Basic principle: divide and conquer, and improve the processing speed of individual requests.

Basic means of high concurrency handling

1) The client sends a request level, the common means are:
  • Try to use the browser’s cache function to reduce access to the server side, such as: js, css, images, etc.
  • Consider using the function of compressed transmission to reduce network traffic, which will also improve the transmission speed
  • Consider using asynchronous requests to get data in batches
2) Front-end to receive client request level, common means are:
  • Separation of static and dynamic, some static resources can be returned directly from Nginx
  • According to the different requests, distributed to different back-end processing, such as: load balancing, business split access, etc.
  • In front of another layer to do multiple Nginx load balancing, such as: LVS, F5, etc.
  • CDN services can also be used on the front end
  • Dynamic content can also be cached to minimize access to back-end services
3) Web server level, common means are:
  • Use the latest JVM with configuration optimization
  • Configuration optimization of web servers, e.g., adjusting the amount of memory, number of threads, etc.
  • Providing multiple web servers that can provide the same service to achieve load balancing
  • Carefully plan the size of applications to be deployed on the web server
  • Cluster web servers
4) At the Web application level, common means are:
  • Staticization of dynamic content
  • Java development optimization
  • Optimize the algorithm for processing business logic
  • Reasonable and efficient use of caching
  • Optimize access to the database Sql, you can consider using stored procedures and other database capabilities
  • Reasonable use of multi-threaded to speed up business processing
  • Part of the business can consider in-memory database, or pure memory processing
  • Try to avoid remote calls, a large number of I / O and other time-consuming operations
  • Reasonable planning of transactions and other resource-consuming operations
  • Reasonable use of asynchronous processing
  • Consider using pre-processing or pre-calculation for some operations to reduce the amount of real-time calculations.
  • Try to directly invoke and directly process the operations between internal systems, and reduce WebService, workflow, etc.
5) Database level, common means are:
  • Reasonable choice of database engine, such as Mysql’s InnoDB and MyISAM engine
  • Perform configuration optimization
  • Consider using stored procedures to handle complex data logic
  • Database clustering for read-write separation
  • Reasonable design of the database table structure, indexes, etc.
  • Separate libraries and tables to reduce the amount of data in a single library or table.