What is a Task Scheduling Framework
In many applications, some tasks need to be performed periodically, such as generating reports, backing up data, updating caches, and so on. These tasks usually need to be executed automatically within a specified time interval, and need to run stably and reliably, and should be able to effectively manage and monitor the execution of these tasks, so the task scheduling framework was born .
Using the task scheduling framework can help us manage the execution time and frequency of tasks and the execution results of tasks. Task scheduling frameworks usually provide flexible configuration options to easily configure task execution plans and support various types of tasks, such as calling a method, executing a shell script, sending an email, and so on. At the same time, the task scheduling framework also provides management and monitoring functions such as task execution logs, execution history, and error logs, so as to debug and troubleshoot problems in task execution.
In addition, the task scheduling framework can also help us realize distributed task scheduling. By deploying the logic of task scheduling in a distributed manner, it can achieve task load balancing, high availability, elastic expansion and contraction, etc., so that the task scheduling system can cope with high Concurrent and high-load scenarios can also improve the reliability and efficiency of task scheduling.
Therefore, the task scheduling framework is designed to solve problems such as scheduled task execution, task management and monitoring, and distributed task scheduling. It can help us improve task reliability, efficiency, and management effects.
What task scheduling frameworks are there
Quartz
Quart z is a powerful open source timing task framework that supports calendar-based job scheduling, cluster scheduling, persistent and distributed tasks and other features. It is a relatively mature task scheduling framework, widely used, and the official documentation is relatively complete. Quartz provides many functions and extension points, which can extend various types of tasks through the plug-in mechanism, support distributed deployment, and also support management and monitoring through JMX. However, it requires additional configuration after integrating Spring, which may be cumbersome, and requires some additional work in a distributed environment to ensure task consistency.
#####Spring Task
Spring Task is a module of task scheduling that comes with Spring. It provides a simple way to schedule scheduled tasks and supports task scheduling based on cron expressions and fixed intervals. It is very convenient to use. It is integrated in Spring, does not require any additional configuration, and can easily use various Spring features. However, the functions of Spring Task are relatively simple. It can only implement basic scheduled task scheduling, does not support distributed deployment, and does not have some advanced features.
#####EJob
EJob is a lightweight distributed timing task framework that supports task scheduling based on cron expressions and fixed intervals. It also provides a visual task management interface and multi-node cluster scheduling features to support distributed deployment and task scheduling. It provides some advanced features, such as task sharding, task routing, etc., which are suitable for some more complex task scenarios. At the same time, it also provides some basic task monitoring and alarm functions. However, EJob has relatively few documents, and it may be more difficult to deploy and use.
XXL-Job
XXL-Job is a distributed task scheduling platform that supports task scheduling based on cron expressions and fixed intervals, and provides features such as a visual task management interface, task execution logs, and an alarm mechanism. XXL-Job is a distributed task scheduling platform that provides two parts: task scheduling center and executor. It has advanced features such as task sharding, task routing, and task retry, and is suitable for large-scale task scheduling. It also provides many practical features, such as task timeout alarm and task monitoring.
Elastic-Job
Elastic-Job is a lightweight distributed timing task framework that supports task scheduling based on cron expressions and fixed intervals, and provides features such as fragmentation mechanism, fault tolerance, and dynamic scaling. . Elastic-Job is a distributed task scheduling framework based on Zookeeper. It provides high availability, elastic scaling and other features, and is suitable for large-scale task scheduling. It provides support for various task types, such as simple tasks, data flow tasks, etc. At the same time, it also provides simple task monitoring and alarm functions. However, since it relies on Zookeeper, deployment and maintenance can be cumbersome.
Why choose XXL-Job
X XL- Job supports cluster deployment , provides an operation and maintenance interface with low maintenance costs , and comes with error prevention . Compared with elastic-job , no additional components (zookeeper) are required. It supports scheduling strategies , fragmentation , and failover , and is more suitable for distribution. formula . It mainly has the following advantages:
- XXL-Job supports task clusters, and guarantees idempotence through the Job load balancing polling mechanism
- The latest version of the job supports job compensation. If the job fails to execute, it can be automatically retried, and an early warning email can be sent after several failed retries.
- Support Job logging
- Dynamically configure timing rules. Traditional timing job trigger rules are hard-coded in the code
- The two parts of “scheduling” and “task” can be decoupled from each other to improve the overall stability and scalability of the system
- In order to improve the security of the system, the dispatch center and the executor perform security verification, and only when the Access Tokens of both parties’ match are allowed to communicate
XXL-Job multi-instance deployment repeated scheduling problem
XXL-Job also has a problem when performing distributed multi-point deployment, which may cause repeated execution of tasks. Although the probability is extremely low, I have encountered it in actual work.
Why is there repeated execution of tasks? Look at the code:
XXL-Job implements mutual exclusion of multi-node scheduling through database row locks, so the following two situations will cause repeated scheduling: 1. If a database cluster is used, it may not be guaranteed that each node accesses the same database, which will lead to locking the row of different databases, so that the purpose of scheduling mutual exclusion cannot be achieved, and the task will eventually be executed repeatedly. 2. The database uses optimistic locking to implement row locks, such as TIDB, which will also result in the inability to achieve mutual exclusion.
The problem arises, what is the solution to this problem? Two solutions: 1. The database used supports pessimistic locks as row locks, while ensuring that different nodes of XXL-Job access the same database. 2. Apply your own mechanism to implement anti-duplication scheduling, such as through Redis marking . Solution 1 is mainly to formulate specifications in the development process. Solution 2 can implement a general implementation of Java aspects. It needs to modify part of the XXL-Job code, and pass more information about the Jobinfo of XXL-Job from the Admin side to the executor. The following is a code snippet from the AOP:
Task Execution Monitoring
XXL-Job, as a reliable task scheduling framework, can guarantee the execution of a certain task at a certain time, but for tasks that require high accuracy, we need to perform double verification. The city A IDC is used as the task scheduling and execution unit. The city B IDC is used as a task execution monitoring unit to ensure that the task is executed and executed correctly. If it is not executed or not executed correctly, it will give an alarm in time. Of course, if city A&B IDC fail at the same time, problems will also occur, but the probability will be extremely low .
