Forem: stelin

Best distributed task scheduling framework — Openjob 1.0.7 released

stelin — Mon, 11 Sep 2023 09:07:39 +0000

Introduction

Openjob is a new distributed task scheduling framework based on Akka architecture. Supports multiple cronjob, delay task, workflow, lightweight distributed computing, unlimited horizontal scaling, with high scalability and fault tolerance. Also has complete management, powerful alarm monitoring, and support multiple languages

High reliability
- Distributed with stateless design, using the Master/Worker architecture, supports multiple databases (MySQL/PostgreSQL/Oracle)
High performance
- System uses a consistency sharding algorithm, lock-free design, task scheduling is accurate down to the second, supporting lightweight distributed computing and unlimited horizontal scaling
Cronjob
- Supports distributed cronjob, fixed rate tasks, high-performance second tasks, and onetime tasks
Distributed computing
- Supports multiple distributed programming models such as standalone, broadcast, Map, MapReduce, and sharding, easy to complete distributed computing for big data
Delay task
- High performance delay task based on Redis , support multi-level storage, and provides rich statistics and reports
Workflow
- Supports workflow scheduling engine, visual DAG design, and easy to complete complex task scheduling
Permission management
- User management, supports menu, button, and data permission settings, flexible management of user permissions
Alarm monitoring
- Overall monitoring metrics, rich and alarm in time, easy to locate and resolve online problem
Multiple languages
- Support multiple languages such as Java, Go, PHP, and Python, as well as build with frameworks such as Spring Boot, Gin, and Swoft If you are looking for a high-performance distributed task scheduling framework that supports cronjob, delay task, lightweight computing, workflow, and supports multiple programming languages, then Openjob is definitely the way to go.

Released

Openjob 1.0.7 newly supports H2/TiDB database and adds second delay task, fixed rate task, broadcast task, sharding task, and Map Reduce lightweight computing.

Second Delay

Second delay support scheduling at interval of 1 to 60 seconds. After each task execution is completed, scheduling will be triggered again at second intervals.

High reliability: Second delay have high reliability characteristics. If a machine goes down, it can be re-run on another machine.
Rich task types: Second delay belong to the cron type and can be applied to all task types and execution types.

Fixed Rate

Crontab must be divisible by 60, if task executed every 50 minutes, Crontab cannot support it.

Map Reduce

MapReduce is a lightweight distributed batch task. Implemented by the MapProcessor or MapReduceProcessor. Compared with big data batch task (such as Hadoop, Spark, etc.), MapReduce does not need to import data into the big data platform, and has no additional storage and computing costs. It can realize second data processing and the advantages of low cost, fast speed and programming.

Example

Define executor by annotation as follows:

/**
 * @author stelin swoft@qq.com
 * @since 1.0.7
 */
@Component("mapReduceTestProcessor")
public class MapReduceTestProcessor implements MapReduceProcessor {
    private static final Logger logger = LoggerFactory.getLogger("openjob");

    private static final String TWO_NAME = "TASK_TWO";

    private static final String THREE_NAME = "TASK_THREE";

    @Override
    public ProcessResult process(JobContext context) {
        if (context.isRoot()) {
            List<MapChildTaskTest> tasks = new ArrayList<>();
            for (int i = 1; i < 5; i++) {
                tasks.add(new MapChildTaskTest(i));
            }

            logger.info("Map Reduce root task mapList={}", tasks);
            return this.map(tasks, TWO_NAME);
        }

        if (context.isTask(TWO_NAME)) {
            MapChildTaskTest task = (MapChildTaskTest) context.getTask();
            List<MapChildTaskTest> tasks = new ArrayList<>();
            for (int i = 1; i < task.getId()*2; i++) {
                tasks.add(new MapChildTaskTest(i));
            }

            logger.info("Map Reduce task two mapList={}", tasks);
            return this.map(tasks, THREE_NAME);
        }

        if (context.isTask(THREE_NAME)) {
            MapChildTaskTest task = (MapChildTaskTest) context.getTask();
            logger.info("Map Reduce task three mapTask={}", task);
            return new ProcessResult(true, String.valueOf(task.getId() * 2));
        }

        return ProcessResult.success();
    }

    @Override
    public ProcessResult reduce(JobContext jobContext) {
        List<String> resultList = jobContext.getTaskResultList().stream().map(TaskResult::getResult)
                .collect(Collectors.toList());
        logger.info("Map Reduce resultList={}", resultList);
        return ProcessResult.success();
    }

    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    public static class MapChildTaskTest {
        private Integer id;
    }
}

Sharding

Sharding model includes static sharding and dynamic sharding:

Static sharding: The main scenario is to process a fixed number of sharding. For example, there are 256 databases in sub-databases, which requires many worker to be processed in a distributed.
Dynamic sharding: The main scenario is distributed processing of unknown data. For example, a large table is constantly changing and requires distributed batch running.

Feature

Compatible with elastic-job’s static sharding.
Supports languages: Java, PHP, Python, Shell, and Go.
High availability: When the worker that executes a sharding task becomes abnormal, it will be dynamically assigned to other normal worker to execute the task.
Flow control: You can set the number of concurrent subtasks for a single worker. For example, if there are 100 sharding and a total of 3 worker, up to 5 sharding can be controlled to execute concurrently, and the others are waiting in the queue.
Retry on failure: Automatically retry when subtask execution fails.

Example

Define executor by annotation as follows:

/**
 * @author stelin swoft@qq.com
 * @since 1.0.7
 */
@Component
public class ShardingAnnotationProcessor {
    private static final Logger logger = LoggerFactory.getLogger("openjob");

    @Openjob("annotationShardingProcessor")
    public ProcessResult shardingProcessor(JobContext jobContext) {
        logger.info("Sharding annotation processor execute success! shardingId={} shardingNum={} shardingParams={}",
                jobContext.getShardingId(), jobContext.getShardingNum(), jobContext.getShardingParam());
        logger.info("jobContext={}", jobContext);
        return ProcessResult.success();
    }
}

Broadcast

Broadcast task instance will be broadcast to all workers to the application for execution. Task will be completed when all workers have completed execution. If any worker execute fail, the task will fail.

Application scenarios

Batch operation

Broadcast to all workers to run a script.
Broadcast to all workers to cleaning data regularly.
Dynamically start a service on each worker

Data aggregation

Initialize using JavaProcessor->preProcess
When each worker executes the process, it returns the result according to the business.
Execute postProcess to obtain the execution results of all workers.

Feature

There are many types of broadcast tasks, such as script or Java Processor. If Java processor, it can supports preProcess and postProcess advanced features.

preProcess will be executed before each worker execute process, and will only be executed once.
process is the actual task execution.
postProcess will be executed once after the process execution on each worker is completed and successfully executed, and the results can be returned as data transmission.

Example

Define executor by annotation as follows:

/**
 * @author stelin swoft@qq.com
 * @since 1.0.7
 */
@Component("broadcastPostProcessor")
public class BroadcastProcessor implements JavaProcessor {
    private static final Logger logger = LoggerFactory.getLogger("openjob");

    @Override
    public void preProcess(JobContext context) {
        logger.info("Broadcast pre process!");
    }

    @Override
    public ProcessResult process(JobContext context) throws Exception {
        logger.info("Broadcast process!");
        return new ProcessResult(true, "{\"data\":\"result data\"}");
    }

    @Override
    public ProcessResult postProcess(JobContext context) {
        logger.info("Broadcast post process taskList={}", context.getTaskResultList());
        System.out.println(context.getTaskResultList());
        return ProcessResult.success();
    }
}

Official website: https://openjob.io
Github: https://github.com/open-job/openjob
Live demo: https://demo.openjob.io
- openjob
- openjob.io

Openjob: best distributed task scheduling framework

stelin — Sun, 18 Jun 2023 14:28:00 +0000

What is Openjob?

Openjob is a new distributed task scheduling framework based on Akka. Supports a variety of cronjob, delayed jobs, and workflow. Uses a consistency sharding algorithm and supports unlimited horizontal scaling.

Feature

Openjob not only supports basic cronjob, but also provides delayed jobs, distributed computing, and workflow

Cronjob

Cronjob，support Unix Crontab expression
Second，execution cycle less than 60 seconds
Fixed rate，execute tasks at a fixed frequency with minute unit

Delay Task

Distributed, high-performance delay task based on Redis, and providing rich reports and statistics

Distributed Computing

Standalone， execute on a worker client
Broadcast， execute on all worker clients
Map，a map function can distribute big data to multiple machines for execution, like Hadoop map
MapReduce， MapReduce is an extension of the Map.After all map sub-tasks are completed, the Reduce method is executed, which can process the results and data of the task execution in the Reduce method.
Sharding， like Elastic-Job model, configure sharding numbers on the management, which can be scheduled to different client by sharding, and supports multiple languages.

Processor

Processor, execute by function or class(support Java/Golang/PH)
HTTP, http request, used to periodically request an HTTP
Shell,shell script

Visual operations

Dashboard，rich task statistics and reports
Task history，task execution history records
Task log，complete task log, and suppport storage (H2/Mysql/Elasticsearch).
Task running stack，detailed recording of task execution stack information

Alarms and permissions

Provides task event monitoring alarms, detailed alarm histories, and support notifications with WeChat, Feishu, and webhook triggers.
Designed with namespace, support button-level access and easy to manage complex project.

Multiple languages

Java java and its frameworks, with native support.
Go golang support use go mod install
PHP PHP support use Golang agent to execute task by shell mode 。Swoole frameworks support composer install.
Python python support use Golang agent to execute task by shell mode

Use cases

Openjob is well-suited for business scenarios that have task schedule and delay task. such as every day to clean data and report generation. It is also suitable for lightweight computing, and Map/MapReduce can process big data computing. For complex task flows or workflow, it can design workflow with UI

Open source

Item	Quartz	Elastic-Job	XXL-JOB	Openjob
Cronjob	Cron	Cron	Cron	* Cronjob * second * Onetime * Fixed rate
Delay task	No	No	No	Distributed, high-performance delay task based on Redis
Workflow	No	No	No	Workflow design with UI
Distributed Computing	No	Sharding	Sharding	* Broadcast * Map/MapReduce * Sharding
Multiple languages	Java	* Java * Shell	* Java * Shell	* Java * Go(Gin、beego) * PHP(Swoft) * Python(Agent) * Shell * HTTP * Kettle
Visualization	No	Weak	* Task history * Task log（Not support storage） * Dashboard	* Task history * Task log（support H2/Mysql/Elasticsearch） * Dashboard * Full permissions * Task log stack
Manageable	No	enable、disable task	* enable、disable task * execute once * stop	* enable、disable task * execute once * kill * stop
Alarms	No	email	email	* custom event * email * webhook
Performance	Every task scheduling try to acquire a lock through the database, causes a high pressure on the database	ZooKeeper is performance bottleneck	Task scheduling is only by master, causes a high pressure on master	Uses sharding algorithm, each node can be scheduled without lock, supports unlimited horizontal scaling, and supports big task scheduling

Official website: https://openjob.io
Github: https://github.com/open-job/openjob
Live demo: https://demo.openjob.io

PHP microservices framework — Swoft 2.0.5 published

stelin — Thu, 08 Aug 2019 13:16:28 +0000

What is Swoft?

Swoft is a PHP microservices coroutine framework based on the Swoole extension. Like Go, Swoft has a built-in coroutine web server and a common coroutine client and is resident in memory, independent of traditional PHP-FPM. There are similar Go language operations, similar to the Spring Cloud framework flexible annotations, powerful global dependency injection container, comprehensive service governance, flexible and powerful AOP, standard PSR specification implementation and so on.

Through three years of accumulation and direction exploration, Swoft has made Swoft the Spring Cloud in the PHP world, which is the best choice for PHP's high-performance framework and microservices management.

Efficient seconds crontab

The following simple lines of code define a timed task that is executed every second, which can completely replace the system crontab.

<?php declare(strict_types=1);

namespace App\Crontab;

use Swoft\Crontab\Annotaion\Mapping\Cron;
use Swoft\Crontab\Annotaion\Mapping\Scheduled;

/**
 * Class CronTask
 *
 * @since 2.0
 *
 * @Scheduled()
 */
class CronTask
{
    /**
     * @Cron("* * * * * *")
     */
    public function secondTask()
    {
        printf("second task run: %s ", date('Y-m-d H:i:s', time()));
    }
}

The crontab starts with the service and will see the following display:

Exception management

During the development process, if the error message is a string of characters, it is not convenient for the business to solve the problem. If the component is used, all error messages will be displayed as follows:

Update

Fixed：

Fixing early interrupt request logic may result in incorrect formatting of the corresponding Content-Type f031398
Fixed the use of sgo to create child coroutines, at the end did not clean up the mapping with the top coroutine de11ae5b
Fix Xml format parsing method 24d0038
Fix class proxy Unique string problem 444ddeb
Fixed context()->get() problem and discard Context::mustGet() method 6acc1b5
Fixed interface injection, class name access error 2d9a31f
Repair model uses the paginate method, the result set has no mapping field 3027287
Fixed websocket server error when there are multiple workers and actively closes non-current worker connections 7666969

Update:

The default worker num setting is obtained by the swoole_cpu_num function 553f6500
The validator is adjusted to, the default is not required, the user opens e5b258d
No longer throws containerException inside the swowt to the user-used layer 1e74893
console output Add interactive methods such as confirm select 0181138a

Enhancement：

The console command option sets CommandOption, which now allows options to be described as multiple lines of information e5914983
All messaging methods for websocket server, support for incoming opcode parameter dc164ffe9
The websocket module allows setting the default opcode of the current module for automatic processing of return data settings opcode 9e3e9672
Enhanced websocket related classes, now the message processing method allows to inject more data object types Request Response 3d6c60b4
When the worker exits, the websocket server will automatically close all connections 5ea10ecc
Add Timer timer package eb92ee6
Add Coroutine Method Package d8c2b88

Resource

GitHub: https://github.com/swoft-cloud/swoft

How to use PHP to implement microservices?

stelin — Mon, 22 Jul 2019 03:53:06 +0000

How to use PHP to implement microservices?

Why do you say service governance?

   With the increasing popularity of Internet browsing, the traditional MVC single architecture has become more and more bloated and more difficult to maintain as the scale of applications continues to expand.

   We must take measures to split the application, which is to split the original application into multiple applications according to business characteristics. For example, a large e-commerce system may include user systems, commodity systems, order systems, evaluation systems, etc., and we can separate them into separate applications. The multi-application architecture is characterized by independent applications and not calling each other.

  Although multiple applications solve the bloated application problem, the applications are independent of each other, and some common services or codes cannot be reused.

Single application solution

For a large Internet system, there are usually multiple applications, and there are often common services between applications, and there are call relationships between applications. In addition, there are other challenges for large-scale Internet systems, such as how to deal with rapidly growing users, how to manage R&D teams to quickly iterate product development, how to keep product upgrades more stable, and so on.

Therefore, in order to make the service be well reused, the module is easier to expand and maintain. We want the service to be separated from the application. A service is no longer an application, but is maintained as a separate service. The application itself is no longer a bloated module stack, but a modular service component.

Service

Features

So what is the feature of using "services"?

Application split into services by business
Individual services can be deployed independently
Service can be shared by multiple apps
Communication between services
The system is more clear on the architecture
Core modules are stable and upgraded in units of service components, avoiding the risk of frequent releases
Easy development and management
Individual team maintenance, clear work, clear responsibilities
Service reuse, code reuse
Very easy to expand

The challenge of service

After the system is serviced, the dependency is complicated, and the number of interactions between the service and the service is increased. In the development mode of fpm, because the resident memory cannot be brought to us, every request must be from zero. Starting to load to the exit process, adding a lot of useless overhead, the database connection can not be reused and can not be protected, because fpm is the process-based fpm process number also determines the number of concurrent, this is also Fpm development is simple to bring us problems. So why is the Internet platform Java more popular now, .NET and PHP will not work in this regard. Needless to say PHP non-memory resident. In addition, there are many other issues that need to be addressed.

More and more services, complex configuration management
Complex service dependencies
Load balancing between services
Service expansion
Service monitoring
Service downgrade
Service authentication
Service online and offline
Service documentation ......

You can imagine the benefits that resident memory brings to us.

only start frame initialization If resident memory we just initialize the memory in the memory at startup, concentrate on processing the request
Connection multiplexing, some engineers can not specifically understand, if you do not need to connect to the pool, how about making a connection to a request? This will result in too many backend resource connections. For some basic services, such as Redis, databases, connections are an expensive drain.

So is there a good plan? The answer is yes, and many people are using this framework, it is - Swoft. Swoft is a RPC framework with the Service Governance feature. Swoft is the first PHP resident memory coroutine full stack framework, based on the core concept of the Spring Boot convention is greater than the configuration

Swoft provides a more elegant way to use RPC services like Dubbo, Swoft performance is great with similar Golang performance, here is my PC vs. Swoft performance. Happening.

ab stress test processing speed is very amazing, in the i78 generation CPU, 16GB memory 100000 million requests only use 5s time in the fpm development mode is basically impossible to achieve. Sufficient to demonstrate the high performance and stability of Swoft,

Elegant service governance

Service Registration and Discovery

In the microservices governance process, registration of services initiated to third-party clusters, such as consul / etcd, is often involved. This chapter uses the swoft-consul component in the Swoft framework to implement service registration and discovery.

Implementation logic

<?php declare(strict_types=1);


namespace App\Common;


use ReflectionException;
use Swoft\Bean\Annotation\Mapping\Bean;
use Swoft\Bean\Annotation\Mapping\Inject;
use Swoft\Bean\Exception\ContainerException;
use Swoft\Consul\Agent;
use Swoft\Consul\Exception\ClientException;
use Swoft\Consul\Exception\ServerException;
use Swoft\Rpc\Client\Client;
use Swoft\Rpc\Client\Contract\ProviderInterface;

/**
 * Class RpcProvider
 *
 * @since 2.0
 *        
 * @Bean()
 */
class RpcProvider implements ProviderInterface
{
    /**
     * @Inject()
     *
     * @var Agent
     */
    private $agent;

    /**
     * @param Client $client
     *
     * @return array
     * @throws ReflectionException
     * @throws ContainerException
     * @throws ClientException
     * @throws ServerException
     * @example
     * [
     *     'host:port',
     *     'host:port',
     *     'host:port',
     * ]
     */
    public function getList(Client $client): array
    {
        // Get health service from consul
        $services = $this->agent->services();

        $services = [

        ];

        return $services;
    }
}

Service Blow

In a distributed environment, especially a distributed system of microservice architectures, it is very common for one software system to call another remote system. The callee of such a remote call may be another process, or another host across the network. The biggest difference between this remote call and the internal call of the process is that the remote call may fail or hang. No response until timeout. Worse, if there are multiple callers calling the same suspended service, it is very likely that a service's timeout waits quickly spread to the entire distributed system, causing a chain reaction that consumes the entire A large amount of resources in distributed systems. Eventually it can lead to system paralysis.

The Circuit Breaker mode is designed to prevent disasters caused by such waterfall-like chain reactions in distributed systems.

In the basic circuit breaker mode, the protection supplier is not called when the circuit breaker is in the open state, but we also need additional measures to reset the circuit breaker after the supplier resumes service. One possible solution is that the circuit breaker periodically detects whether the service of the supplier is restored. Once restored, the status is set to close. The state when the circuit breaker is retried is a half-open state.

The use of the fuse is simple and powerful. It can be annotated with a @Breaker. The fuse of the Swoft can be used in any scenario, such as when the service is called. It can be blown when requesting a third party. Downgrade

<?php declare(strict_types=1);


namespace App\Model\Logic;

use Exception;
use Swoft\Bean\Annotation\Mapping\Bean;
use Swoft\Breaker\Annotation\Mapping\Breaker;

/**
 * Class BreakerLogic
 *
 * @since 2.0
 *
 * @Bean()
 */
class BreakerLogic
{
    /**
     * @Breaker(fallback="loopFallback")
     *
     * @return string
     * @throws Exception
     */
    public function loop(): string
    {
        // Do something
        throw new Exception('Breaker exception');
    }

    /**
     * @return string
     * @throws Exception
     */
    public function loopFallback(): string
    {
        // Do something
    }
}

Service Restriction

Restriction, Fuse, Degradation This emphasis cannot be overstated because it is really important. When the service is not working, it must be blown. Current limiting is a tool to protect itself. If there is no self-protection mechanism, no matter how many connections are received, if the back-end processing is not enough, the front-end traffic will definitely hang when it is very large.

The current limit is to limit the number of concurrent and requested when accessing scarce resources, such as spikes and snapped goods, so as to effectively cut the peak and smooth the flow curve. The purpose of current limiting is to limit the rate of concurrent access and concurrent requests, or to request a speed limit within a time window to protect the system. Once the rate limit is reached or exceeded, the service can be denied, or queued.

The bottom layer of the Swoft current limiter uses the token bucket algorithm, and the underlying layer relies on Redis to implement distributed current limiting.

The Swoft speed limiter not only limits the flow controller, it also limits the methods in any bean and controls the access rate of the method. Here is a detailed explanation of the following examples.

<?php declare(strict_types=1);

namespace App\Model\Logic;

use Swoft\Bean\Annotation\Mapping\Bean;
use Swoft\Limiter\Annotation\Mapping\RateLimiter;

/**
 * Class LimiterLogic
 *
 * @since 2.0
 *
 * @Bean()
 */
class LimiterLogic
{
    /**
     * @RequestMapping()
     * @RateLimiter(rate=20, fallback="limiterFallback")
     *
     * @param Request $request
     *
     * @return array
     */
    public function requestLimiter2(Request $request): array
    {
        $uri = $request->getUriPath();
        return ['requestLimiter2', $uri];
    }

    /**
     * @param Request $request
     *
     * @return array
     */
    public function limiterFallback(Request $request): array
    {
        $uri = $request->getUriPath();
        return ['limiterFallback', $uri];
    }
}

Key This supports the symfony/expression-language expression. If the speed limit is called, the limiterFallback method defined in fallback will be called.

Configuration Center

Before we talk about the configuration center, let's talk about the configuration file. We are no stranger to it. It provides us with the ability to dynamically modify the program. A quote from someone else is:

Dynamic adjustment of the flight attitude of the system runtime!

I can call our work to repair parts on fast-flying airplanes. We humans are always unable to control and predict everything. For our system, we always need to reserve some control lines to make adjustments when we need them, to control the system direction (such as gray control, current limit adjustment), which is especially important for the Internet industry that embraces change.

For the stand-alone version, we call it the configuration (file); for the distributed cluster system, we call it the configuration center (system);

What exactly is a distributed configuration center?

With the development of the business and the upgrade of the micro-service architecture, the number of services and the configuration of programs are increasing (various micro-services, various server addresses, various parameters), and the traditional configuration file method and database method cannot meet the development. Personnel requirements for configuration management:

Security: The configuration follows the source code stored in the code base, which is easy to cause configuration leaks;
Timeliness: Modify the configuration and restart the service to take effect.
Limitations: Dynamic adjustments cannot be supported: for example, log switches, function switches;

Therefore, we need to configure the center to manage the configuration! Freeing business developers from complex and cumbersome configurations, they only need to focus on the business code itself, which can significantly improve development and operational efficiency. At the same time, the configuration and release of the package will further enhance the success rate of the release, and provide strong support for the fine control and emergency handling of operation and maintenance.

About distributed configuration centers, there are many open source solutions on the web, such as:

Apollo is a distributed configuration center developed by Ctrip's framework department. It can centrally manage the configuration of different environments and different clusters of applications. It can be pushed to the application end in real time after configuration modification. It has the features of standardized authority and process management, and is suitable for microservices. Configure management scenarios.

This chapter uses Apollo as an example to pull configuration and secure restart services from the remote configuration center. If you are unfamiliar with Apollo, you can first look at the Swoft extension Apollo component and read Apollo Official documentation.

This chapter uses Apollo in Swoft as an example. When the Apollo configuration changes, restart the service (http-server / rpc-server/ ws-server). The following is an example of an agent:

<?php declare(strict_types=1);


namespace App\Model\Logic;

use Swoft\Apollo\Config;
use Swoft\Apollo\Exception\ApolloException;
use Swoft\Bean\Annotation\Mapping\Bean;
use Swoft\Bean\Annotation\Mapping\Inject;

/**
 * Class ApolloLogic
 *
 * @since 2.0
 *
 * @Bean()
 */
class ApolloLogic
{
    /**
     * @Inject()
     *
     * @var Config
     */
    private $config;

    /**
     * @throws ApolloException
     */
    public function pull(): void
    {
        $data = $this->config->pull('application');

        // Print data
        var_dump($data);
    }
}

The above is a simple Apollo configuration pull, in addition to this method, Swoft-Apollo provides more ways to use.

Official link

Swoft 2.0.3 major update, launching elegant microservices governance

stelin — Tue, 09 Jul 2019 03:56:28 +0000

What is Swoft?

Update

Remove

Remove request->json() method(c9e8f04)

Enhancement

Add interface dependency injection(6169f84)
Add getFile to obtain information after file upload and save(fe7e3a6)
Add restart() to restart service(2ffec37)
Add call 1.x RPC service support(30d73c3)
Add AOP class name matching supports regular expressions(bc5e479)
Add RPC Server /Http server Middleware namespace use error prompt(b1cec04)
Add validator exclusion attribute field withunfields(b1bf44f)
Add auto write timestamp(dc58011)
Add model action event(dc58011)
Add database migration script(26bb464)
Add entity automatically interact with json and arrays(dc58011)
Add model batch update method withbatchUpdateByIds(dc58011)

Fixed：

Fix cookies some problems when setting up, add some related methods with withCookie(b05afbb01)
Fix no capture processing error when the console uses the coroutine mode to run the command.(8a5418bf)
Fix restart websocket server command does not stop the old server problem first(db2d935)
Fix task return value is null(a69347c)
Fix RPC Server only class middleware can't be used(204bc7f)
Fix RPC Server return value is null(4d091be)
Fix Logger and CLog log level cannot be overwritten and invalid(8eb8aba)
Fix attributes in the model do not support custom expressions(dc58011)

Update:

Validator optimization, support for custom validation rules(d959a4f)
Rename the error handling management class ErrorHanlders to ErrorManager (f3a8f04b)
The exception handling of the console component is changed to the unified processing style provided by the error component. (4f47204)
Console component allows to set the disable command group(c5a0269)
In the default error handling, the error capture level is allowed to be set. The default level is E_ALL | E_STRICT (afff9029)
Optimization When the ws server is started, the http processing function is enabled at the same time, and the information panel adds prompts.(83a81170)
Optimize Start the ws server and add the rpc server to start. The information panel does not display the rpc server information.(3d1d0d848)

Extra:

Document add support search by google
New apollo Component
New consul Component
New breaker Component
New limter component

Elegant service governance

Swoft officially recommends that developers use service mesh patterns, such as the Istio/Envoy framework, to separate business and service governance, but Swoft also provides a set of microservices components for small and medium-sized businesses to quickly build microservices.

Service registration and discovery

For service registration and discovery, the swoft-consul component provided by Swoft is required, if other third parties are similar.

Registration and cancellation services

Listen for the SwooleEvent::START event, register the service

/**
 * Class RegisterServiceListener
 *
 * @since 2.0
 *
 * @Listener(event=SwooleEvent::START)
 */
class RegisterServiceListener implements EventHandlerInterface
{
    /**
     * @Inject()
     *
     * @var Agent
     */
    private $agent;

    /**
     * @param EventInterface $event
     */
    public function handle(EventInterface $event): void
    {
        /* @var HttpServer $httpServer */
        $httpServer = $event->getTarget();

        $service = [
            // ....
        ];

        $scheduler = Swoole\Coroutine\Scheduler();
        $scheduler->add(function () use ($service) {
            // Register
            $this->agent->registerService($service);
            CLog::info('Swoft http register service success by consul!');
        });
        $scheduler->start();
    }
}

Listen for the SwooleEvent::SHUTDOWN event, cancel the service

/**
 * Class DeregisterServiceListener
 *
 * @since 2.0
 *
 * @Listener(SwooleEvent::SHUTDOWN)
 */
class DeregisterServiceListener implements EventHandlerInterface
{
    /**
     * @Inject()
     *
     * @var Agent
     */
    private $agent;

    /**
     * @param EventInterface $event
     */
    public function handle(EventInterface $event): void
    {
        /* @var HttpServer $httpServer */
        $httpServer = $event->getTarget();

        $scheduler = Swoole\Coroutine\Scheduler();
        $scheduler->add(function () use ($httpServer) {
            $this->agent->deregisterService('swoft');
        });
        $scheduler->start();
    }
}

Service discovery

Defining a service provider

/**
 * Class RpcProvider
 *
 * @since 2.0
 *        
 * @Bean()
 */
class RpcProvider implements ProviderInterface
{
    /**
     * @Inject()
     *
     * @var Agent
     */
    private $agent;

    /**
     * @param Client $client
     *
     * @return array
     * @example
     * [
     *     'host:port'
     * ]
     */
    public function getList(Client $client): array
    {
        // Get health service from consul
        $services = $this->agent->services();

        $services = [

        ];

        return $services;
    }
}

Configuration service provider

return [
    'user'           => [
      'class'   => ServiceClient::class,
      'provider' => bean(RpcProvider::class)
      // ...
    ]
];

Service breaker

Swoft uses the @Breaker annotation to achieve a blow, which can be blown on any method.

/**
 * Class BreakerLogic
 *
 * @since 2.0
 *
 * @Bean()
 */
class BreakerLogic
{
    /**
     * @Breaker(fallback="funcFallback")
     *
     * @return string
     * @throws Exception
     */
    public function func(): string
    {
        // Do something

        throw new Exception('Breaker exception');
    }

    /**
     * @return string
     */
    public function funcFallback(): string
    {
        return 'funcFallback';
    }
}

Service limit

Swoft uses the @RateLimiter annotation to implement service throttling, which can be throttled on any method, not just the controller, and KEY also supports symfony/expression-language expression language.

/**
 * Class LimiterController
 *
 * @since 2.0
 *
 * @Controller(prefix="limiter")
 */
class LimiterController
{
    /**
     * @RequestMapping()
     * @RateLimiter(key="request.getUriPath()", fallback="limiterFallback")
     *
     * @param Request $request
     *
     * @return array
     */
    public function requestLimiter(Request $request): array
    {
        $uri = $request->getUriPath();
        return ['requestLimiter', $uri];
    }

    /**
     * @param Request $request
     *
     * @return array
     */
    public function limiterFallback(Request $request): array
    {
        $uri = $request->getUriPath();
        return ['limiterFallback', $uri];
    }
}

Configuration center

The configuration center needs to use the Swoft-apollo component provided by Swoft officially, if other third parties are similar.

Declare Agent

/**
 * Class AgentCommand
 *
 * @since 2.0
 *
 * @Command("agent")
 */
class AgentCommand
{
    /**
     * @Inject()
     *
     * @var Config
     */
    private $config;

    /**
     * @CommandMapping(name="index")
     */
    public function index(): void
    {
        $namespaces = [
            'application'
        ];

        while (true) {
            try {
                $this->config->listen($namespaces, [$this, 'updateConfigFile']);
            } catch (Throwable $e) {
                CLog::error('Config agent fail(%s %s %d)!', $e->getMessage(), $e->getFile(), $e->getLine());
            }
        }
    }

    /**
     * @param array $data
     *
     * @throws ContainerException
     * @throws ReflectionException
     */
    public function updateConfigFile(array $data): void
    {
        foreach ($data as $namespace => $namespaceData) {
            $configFile = sprintf('@config/%s.php', $namespace);

            $configKVs = $namespaceData['configurations'] ?? '';
            $content   = '<?php return ' . var_export($configKVs, true) . ';';
            Co::writeFile(alias($configFile), $content, FILE_NO_DEFAULT_CONTEXT);

            CLog::info('Apollo update success！');

            /** @var HttpServer $server */
            $server = bean('httpServer');
            $server->restart();
        }
    }
}

Start Agent

The Agent only needs to run before the service (Http/RPC/Websocket) is started.

php bin/swoft agent:index

Resource

GitHub: https://github.com/swoft-cloud/swoft
Website：https://en.swoft.org
Document：https://en.swoft.org/docs
Discuss：swoft-cloud/community

Forem: stelin

Best distributed task scheduling framework — Openjob 1.0.7 released

Introduction

Released

Second Delay

Fixed Rate

Map Reduce

Example

Sharding

Feature

Example

Broadcast

Application scenarios​

Feature​

Example

More

Openjob: best distributed task scheduling framework

What is Openjob?

Feature

Cronjob

Delay Task

Distributed Computing

Processor

Visual operations

Alarms and permissions

Multiple languages

Use cases

Open source

More

PHP microservices framework — Swoft 2.0.5 published

What is Swoft?

Efficient seconds crontab

Exception management

Update

Resource

How to use PHP to implement microservices?

How to use PHP to implement microservices?

Why do you say service governance?

Single application solution

Service

Features

The challenge of service

Elegant service governance

Service Registration and Discovery

Service Blow

Service Restriction

Configuration Center

What exactly is a distributed configuration center?

Official link

Swoft 2.0.3 major update, launching elegant microservices governance

What is Swoft?

Update

Elegant service governance

Service registration and discovery

Registration and cancellation services

Service discovery

Service breaker

Service limit

Configuration center

Declare Agent

Start Agent

Resource

Application scenarios

Feature