Add snap cronjobs #651

FrederikBolding · 2022-07-21T10:59:57Z

Fixes: #353

Cronjob feature will add new possibility to periodically run specific Snap RPC methods. Cronjob feature is implemented as a new permission with caveats that are used to specify job methods, parameters and schedule using CRON syntax.

For interpretation of a CRON syntax, cron-parser library is added: https://www.npmjs.com/package/cron-parser
cron-parser is added as the dependency to the snap-utils and snap-controllers packages.

New caveat specification is added under the snap-utils package inside cronjob.ts.
Cronjob feature's core is implemented as the Controller inside CronjobController.ts which is part of snap-controllers package.

CronjobController is supposed to be registered in the extension within MetaMask Controller.

Example Cronjob Specification caveat:

{
      expression: '* * * * *',
      request: {
        method: 'exampleMethodOne',
        params: ['p1'],
      },
    }

Cronjob Controller will on its start subscribe to certain Snap events and check for possible cronjobs in order to register or remove them from schedule.
CronjobController will listen for the following events:

SnapController:snapInstalled - will add cronjobs and schedule for snap
SnapController:snapRemoved - will remove all added/scheduled cronjobs for snap
SnapController:snapUpdated - will remove registered jobs and scheduled jobs and add them again according to the newest specification

david0xd · 2022-10-20T10:01:01Z

Here is the PR that will follow up this one (when merged/released) in order to complete the integration with MetaMask (Flask) Extension: MetaMask/metamask-extension#16239

david0xd · 2022-10-24T09:05:34Z

After some research I found out that using of a CRON syntax might still be the best (in my opinion).

For many scheduling requirements and cases it's highly possible for a developer to come across the CRON standard.
I found out that the CRON syntax can be very extensive with possibilities for scheduling and since we're using the cron-parser which is quite popular library, we're getting most of it in a reliable way.
It seems to be easy to start working with it once you get some documentation reference. We can put some explanation and few examples of the CRON schedule. I find this tool very useful as well https://crontab.guru/every-2-minutes

Here are some examples of the CRON expressions and their meaning:
*/2 * * * * -> “At every 2nd minute.”
0 0 1 */6 * -> “At 00:00 on day-of-month 1 in every 6th month.”
0 9-17 * * * -> “At minute 0 past every hour from 9 through 17.”
1-59/2 * * * * -> “At every 2nd minute from 1 through 59.”
*/15 * * 1 6 -> “At every 15th minute on Saturday in January.”

There were some ideas for splitting parameters into the simpler format by its meaning, but it seems if the original format is kept, it's more powerful and meaningful for the one who already knows to use it. Since users of this feature would be developers, I do not think they're going to have much of a trouble with this.

If we continue to use pure CRON way of doing the schedule we will have less to maintain and test in the future (avoid potential bugs as well). That way we have a feature out of the box that works well, is widely used and specified which means less concern about its functionality now and later.

packages/controllers/src/cronjob/CronjobController.test.ts

packages/controllers/src/snaps/endowments/cronjob.test.ts

packages/controllers/src/test-utils/controller.ts

packages/utils/src/cronjob.ts

packages/controllers/src/cronjob/CronjobController.test.ts

packages/controllers/src/cronjob/CronjobController.ts

packages/controllers/src/snaps/endowments/cronjob.test.ts

packages/utils/src/cronjob.test.ts

Mrtenz · 2022-10-27T08:27:31Z

packages/utils/src/cronjob.test.ts

+  it('successfully parses cronjob expression that is provided as an object', () => {
+    const cronjobExpression = {
+      minute: '*',
+      hour: '*',
+      dayOfMonth: '*',
+      month: '*',
+      dayOfWeek: '*',
+    };
+    expect(() => parseCronExpression(cronjobExpression)).not.toThrow();
+  });
+
+  it('successfully parses cronjob expression that is provided as a string', () => {
+    const cronjobExpression = '* * * * *';
+    expect(() => parseCronExpression(cronjobExpression)).not.toThrow();
+  });


Rather than asserting that these don't throw, maybe check that the return value is correct.

Maybe add a few different tests to ensure that the object properties are put in the right place too.

I've removed throw assertion and added couple of things that check the properties of the object returned which is from an external library cron-parser. These methods will only work if the parsing is successful, otherwise it will throw an error which would break the test. I hope that's ok now.

packages/controllers/src/cronjob/CronjobController.ts

Mrtenz · 2022-10-31T11:11:28Z

packages/controllers/src/cronjob/CronjobController.ts

+    this.updateJobLastRunState(job.id, Date.now());
+    this.#messenger.call('SnapController:handleRequest', {
+      snapId: job.snapId,
+      origin: '',


Can we remove this? Or should it maybe be something like 'cron'?

We currently use empty string for transaction insights as well - It is currently required to have a value by the snap executor. We should perhaps figure out a way to only require it for certain handlers? Out of scope for this PR imo.

Fix build error Fix test Add part of cronjob caveat implementation Register cronjob caveat Add more caveat validations and tests Fix imports after rebase Fix conflict-merge issue after rebase Fix validation functions build issue Add configuration for caveat mapper Update CronjobService and add tests Add some more unit tests for CronjobService Update CronjobService to become CronjobController Add some refactoring and new features for the CronjobController Refactor and clean up More refactor and tests Add missing docstring Improve test coverage for CronjobController Improve test coverage for utils Improve test coverage for CronjobController Fix execution-environments commands Fix jest threshold after rebase Refactor directory structure Add support for object-like cron expressions Fix and rework a couple of things for structural expressions Add some refactoring & test improvements Code refactoring Add way to unregister all snaps on controller destroy method call Change timers and snapIds to be private with getters Remove eslint ignore instructions where they are not needed anymore Redesign the test with daily timeout check (schedule skip) Refactor unit tests

FrederikBolding force-pushed the fb/cronjob-feature branch from 72aebf0 to 93729fe Compare August 24, 2022 08:08

david0xd force-pushed the fb/cronjob-feature branch 2 times, most recently from c1335f9 to a6ef418 Compare September 20, 2022 11:10

ritave assigned david0xd Sep 28, 2022

david0xd force-pushed the fb/cronjob-feature branch 3 times, most recently from 3a1903c to 03d308d Compare October 3, 2022 14:20

FrederikBolding force-pushed the fb/cronjob-feature branch from 6296ecb to c1425b1 Compare October 13, 2022 10:07

david0xd force-pushed the fb/cronjob-feature branch from b813338 to 2bef716 Compare October 19, 2022 17:19

david0xd mentioned this pull request Oct 20, 2022

[FLASK] Add snap cronjobs MetaMask/metamask-extension#16239

Merged

5 tasks

david0xd marked this pull request as ready for review October 24, 2022 09:05

david0xd requested a review from a team as a code owner October 24, 2022 09:05

FrederikBolding requested a review from Mrtenz October 25, 2022 09:51

Mrtenz requested changes Oct 25, 2022

View reviewed changes

Mrtenz requested changes Oct 27, 2022

View reviewed changes

FrederikBolding commented Oct 27, 2022

View reviewed changes

packages/controllers/src/cronjob/CronjobController.ts Outdated Show resolved Hide resolved

david0xd force-pushed the fb/cronjob-feature branch from b7b33ce to cbf0e5b Compare October 27, 2022 16:52

FrederikBolding changed the title ~~Cronjob feature~~ Add snap cronjobs Oct 31, 2022

FrederikBolding force-pushed the fb/cronjob-feature branch 2 times, most recently from 2697cd5 to e462748 Compare October 31, 2022 10:35

Mrtenz requested changes Oct 31, 2022

View reviewed changes

FrederikBolding and others added 6 commits October 31, 2022 12:47

Jest configuration updates after rebase & test run

a93f74b

Remove problematic test snapshot

a1b7d72

Change way of calling private methods in tests

a3ccb10

Improve tests

f5f0f85

Test onCronjob handler and fix type

b5ab1ba

FrederikBolding force-pushed the fb/cronjob-feature branch from e462748 to b5ab1ba Compare October 31, 2022 11:48

Mrtenz approved these changes Oct 31, 2022

View reviewed changes

FrederikBolding merged commit da0e3ab into main Oct 31, 2022

FrederikBolding deleted the fb/cronjob-feature branch October 31, 2022 12:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add snap cronjobs #651

Add snap cronjobs #651

FrederikBolding commented Jul 21, 2022 •

edited by david0xd

david0xd commented Oct 20, 2022

david0xd commented Oct 24, 2022

Mrtenz Oct 27, 2022

Mrtenz Oct 27, 2022

david0xd Oct 27, 2022

Mrtenz Oct 31, 2022

FrederikBolding Oct 31, 2022

Add snap cronjobs #651

Add snap cronjobs #651

Conversation

FrederikBolding commented Jul 21, 2022 • edited by david0xd

david0xd commented Oct 20, 2022

david0xd commented Oct 24, 2022

Mrtenz Oct 27, 2022

Choose a reason for hiding this comment

Mrtenz Oct 27, 2022

Choose a reason for hiding this comment

david0xd Oct 27, 2022

Choose a reason for hiding this comment

Mrtenz Oct 31, 2022

Choose a reason for hiding this comment

FrederikBolding Oct 31, 2022

Choose a reason for hiding this comment

FrederikBolding commented Jul 21, 2022 •

edited by david0xd