Servicebus-JobScheduler

This repo provides an implemented concept of task scheduling using azure service bus delayed messages. this project includes also a client simulator that initiated new Job Definitions Upserts that trigger the flows.

how it works?

using Publish Subscribe, the console application when starting, registers subscribers (handlers) to service bus topic subscriptions.
Each handler, processes the message and publishes to next topic if needed. To initiate the process, the Tester Client simulates JobDefinition Upserts. see TopicsFlow.vsdx visio file for logical flow

Job Scheduler capabilties:

• enables scheduling by time interval moreinfo
• enables scheduling by cron expressions moreinfo
• immediate retries on errors
• exponential backoff retries by config
• dead letter queueing for zero data loss + transparency
• enables azure service bus partitioning for high scale by config
• enables ad-hoc job runs for manual/recovery scenarios moreinfo

Simulator capabilties:

• flexabilty to run as single process, or process that regsieters one or more handlers
• simuate errors (for the POC)
• local database for JobDefinition (for the POC)
• local database for JobOutputs (for the POC)

prerequisites:

• dotnet 5 sdk
• optional: azure account
• optional: azure service bus namespace
• optional: docker on windows

setup:

    git clone https://github.com/AmirSasson/Servicebus-JobScheduler.git
    cd Servicebus-JobScheduler   
    code .

• You can choose 2 modes for the service bus:

  • Local - used for development, when you less care about persistency and stabilty. see run option --local.

  • Cloud - production use, using azure ServiceBus, to use this option add appsettings.overrides.json under src/Servicebus.JobScheduler.ExampleApp next to existingappsettings.json with content:

    {
      "ServiceBus": {
        "ConnectionString": "<<YOUR AZURE SERVICE BUS CONNECTION STRING>>"
      }
    }

Where to find the ConnectionString in the portal

• from within the root folder dotnet build

run

• cd ./src/Servicebus.JobScheduler.ExampleApp
• from within the ExampleApp folder dotnet run - first run, by default, will configure Azure ServiceBus Entities for you.
• (Non-Cloud) - you can also run a local In Memory version of ServiceBus dotnet run -l true
• you can also use --modes <<mode[]>> argument to run a specific/multiple modes of process i.e.
  dotnet run --run-setup true --all-modes false --tester-simulator false
  or
  dotnet run --modes JobDefinitionChange_ImmediateScheduleRule --tester-simulator false
  or
  dotnet run --modes JobWindowValid_ScheduleNextRun JobWindowValid_RuleTimeWindowExecution --tester-simulator false
  see dotnet run -- --help to view more running modes

test

• from within the root folder dotnet test (tests tbd)

deploy (cloud)

currently this app is running as a linux container on AKS.

Scheduling jobs

jobs can be scheduled in 3 different methods:

1. Time Interval sliding window - occurs every X seconds from the actual schduling time.
   in the below example, lets assume it was called on time 10:30:10 so the first job window will start immediately (10:30:10) and its time range would be 10:27:10 <-> 10:30:10 (range of 180 seconds)
   second job would start on 10:32:10 and its time range would be 10:29:10 <-> 10:32:10 (after 120 from previouse window)

    new JobDefinition
    {
      ....                            
        Schedule = new JobSchedule { PeriodicJob = true, RunIntervalSeconds = 120 },                   
      ....
    }

2. Cron Job scheduling .in the below example, lets assume it was called on time 10:30:10, the cron represents At every 2nd minute so the first job window will start immediately (10:30:10) and its time range would be 10:28:10 <-> 10:30:00 (up to range of 120 seconds)
   second job would start on 10:32:00 and its time range would be 10:30:00 <-> 10:32:00 (exatly 120 seconds and At every 2nd minute)

    new JobDefinition
    {
      ....
        Schedule = new JobSchedule { PeriodicJob = true, CronSchedulingExpression = "*/2 * * * *" },                  
      ....      
    }

OR you can schedule a cron job that will look on a specific lookback time window, in this example it would run every 2 minutes, and look back on a 1 hour (3600sec) time range, (overlapped time ranges)

  new JobDefinition
  {
    ....
      Schedule = new JobSchedule { PeriodicJob = true, CronSchedulingExpression = "*/2 * * * *" ,RunIntervalSeconds = 3600},                  
    ....      
  }

3. adhoc runs of a predfeined time window
   in this example lets assume you would like to re-run jobs from 12/28/2020 1:00:00 PM till 12/28/2020 4:00:00 PM, we cancel the window validation phase in this case.

  new JobDefinition
  {
    ....                          
      Schedule = new JobSchedule { PeriodicJob = true, ScheduleEndTime = DateTime.Parse("12/28/2020 4:00:00 PM"), ForceSuppressWindowValidation = true, RunIntervalSeconds = 120 },
      LastRunWindowUpperBound = DateTime.Parse("12/28/2020 1:00:00 PM")        
    ....

dockerize:

from root folder run:
create an azure container registry, with admin credentials you can provide to the login command

docker login <<registryname>>.azurecr.io
docker build -t <<registryname>>.azurecr.io/<<my-repository-name>>:latest .
docker push --all-tags <<registryname>>.azurecr.io/<<my-repository-name>>

to run the container locally: docker run <<registryname>>.azurecr.io/<<my-repository-name>>:latest

• currently as docker container on AKS (linux), aks deployment yml included.

troubleshooting:

• make sure docker deamon is runnning linux containers
• restart docker
• Azure/azure-service-bus#182