From 3e7406b857da0a7db132424d0628c9b6c4108c98 Mon Sep 17 00:00:00 2001 From: Alex Rosengarten Date: Mon, 9 Mar 2020 11:06:23 -0700 Subject: [PATCH] recipe2plan 3/n: Plan code-generation (#4865) ## Summary First iteration of generating Plan objects. ## Changelog * creating structure of ts r2p script * Created fast CLI rapper for r2p script * WIP figuring out ways to resolve a recipe * fix sp * Passing lint, still WIP * Build macros for recipe2plan work * Lightweight iteration on recipe2plan, stubbed out tests * Revised method to find corresponding create handles * Added type info * Added method to get all handles to manifest + get all handles by Id * fix: no flatMap * - currently, fails since it cannot find associated stores for handles. * uncleaned, but working recipe resolution * Cleaned up for r2p, pt 1 * Fixed test, added TODO * Fixed build rule, simplified runtime * tools/sigh lint * Impl suggestsions for r2p * Improved build rules * Removed flatMap, added TODOs and link to GH issue * fix, bad comparison op * Stub plan generator works * updated macro to take a package * Testing import is conditional on package * Incremental improvement * tidying * Fixed tests * added particle creation method * flushing out more of plan gen * Squashed commit of the following: commit 17a2ab1277abfbb4af6499e78f41aaf6c3ea59a6 Author: Alex Rosengarten Date: Wed Mar 4 16:30:45 2020 -0800 Fixed indentation commit 9094055ef7e774d50cc62cb4b8fa4a11241e19ae Author: Alex Rosengarten Date: Wed Mar 4 16:03:51 2020 -0800 revising from feedback commit d3508219e0f344e8b8ff0c926674c37d7b9c20e2 Merge: 9054c231 3acb7e7e Author: Alex Rosengarten Date: Wed Mar 4 15:56:55 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas commit 3acb7e7e493d5f76fa93c0a32a503df7e08a0c8d Author: Alex Rosengarten Date: Wed Mar 4 15:56:42 2020 -0800 XXS CL to add a hash field to the Schema proto (#4752) commit 9054c2319334fc4a7d93e5a713bba6e4ea03ad7a Merge: d15008eb 337b7b26 Author: Alex Rosengarten Date: Wed Mar 4 15:55:27 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas commit 337b7b26500e977e069979c66480e82a38a7e7fb Author: Alex Rosengarten Date: Wed Mar 4 15:55:06 2020 -0800 recipe2plan 2/n: Validate Handles, Tests (#4834) * creating structure of ts r2p script * Created fast CLI rapper for r2p script * WIP figuring out ways to resolve a recipe * fix sp * Passing lint, still WIP * Build macros for recipe2plan work * Lightweight iteration on recipe2plan, stubbed out tests * Revised method to find corresponding create handles * Added type info * Added method to get all handles to manifest + get all handles by Id * fix: no flatMap * - currently, fails since it cannot find associated stores for handles. * uncleaned, but working recipe resolution * Cleaned up for r2p, pt 1 * stubbed out second test (short, short) * Squashed commit of the following: commit 62dcc57fac16e26dd9c724d54be4a6c8ee87bc98 Author: Alex Rosengarten Date: Mon Mar 2 14:38:14 2020 -0800 Fixed test, added TODO * updates to test, can't get two to fail * Squashed commit of the following: commit bede8d95ea4c1d7c5115f9d34bd2476e0deb7fef Author: Alex Rosengarten Date: Mon Mar 2 14:52:02 2020 -0800 Fixed build rule, simplified runtime commit 62dcc57fac16e26dd9c724d54be4a6c8ee87bc98 Author: Alex Rosengarten Date: Mon Mar 2 14:38:14 2020 -0800 Fixed test, added TODO * can't get test to fail... hmm... * Squashed commit of the following: commit f6df54ae7d999c724800c969af59a3c29442995a Author: Alex Rosengarten Date: Mon Mar 2 15:44:32 2020 -0800 tools/sigh lint commit bede8d95ea4c1d7c5115f9d34bd2476e0deb7fef Author: Alex Rosengarten Date: Mon Mar 2 14:52:02 2020 -0800 Fixed build rule, simplified runtime commit 62dcc57fac16e26dd9c724d54be4a6c8ee87bc98 Author: Alex Rosengarten Date: Mon Mar 2 14:38:14 2020 -0800 Fixed test, added TODO * adjusting recipe copying to include triggers * Revert "adjusting recipe copying to include triggers" This reverts commit 9bc3734f * four tests complete * Added test, fixed minor async issue * Squashed commit of the following: commit 4b05a287434dcd3a3622ea9c9ebb499fe1d5bc46 Merge: f4c1d26a 99000c25 Author: Alex Rosengarten Date: Tue Mar 3 11:18:14 2020 -0800 Merge branch 'r2p' of github.com:alxrsngrtn/arcs into r2p commit f4c1d26a893e2d6635642567b6eec8d681bdc252 Merge: 140111f5 5c6310da Author: Alex Rosengarten Date: Tue Mar 3 11:14:35 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into r2p Fixed lint errors commit 99000c25e79b55be6c4a09df73a3d1a34227e630 Merge: 140111f5 5c6310da Author: Alex Rosengarten Date: Tue Mar 3 11:14:35 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into r2p Fixed lint errors commit 140111f580adf02c3fe0d2c3a4560dc13c9b2510 Author: Alex Rosengarten Date: Tue Mar 3 11:08:12 2020 -0800 Removed flatMap, added TODOs and link to GH issue commit 5c6310da9d1e372544e816c1dd595ef25e5099bf Author: jblebrun Date: Tue Mar 3 11:02:34 2020 -0800 Simplify service test pattern (#4817) * Make a simple test lifecycle registry, instead of creating empty testactivity * Remove use of `runBlockingTest`: according to https://github.com/Kotlin/kotlinx.coroutines/issues/1222, if the test results in coroutines being finished on other threads, it's possible to receive "This job has not yet completed" exceptions, even though the jobs were properly terminated. Since we don't need the delay-skipping properties of `runBlockingTest`, I think it's OK to use `runBlocking`. commit 472bc841296987e01feee21061781bba0f315921 Author: Alex Rosengarten Date: Tue Mar 3 10:27:56 2020 -0800 Improved build rules commit ca1ebf89390202da9e777f08f7e69f03020d3cf1 Author: Alex Rosengarten Date: Tue Mar 3 10:17:13 2020 -0800 Impl suggestsions for r2p commit b5578ea297e4d3033fdfdefd5f3a325e2cbe503f Author: Jason Feinstein Date: Tue Mar 3 09:54:17 2020 -0800 Add tests for dereferencing references to the HandleManagerTest(s) (#4816) * Add dereferencing tests to the android handle manager test. * Add dereferencing tests to core HandleManager. commit a6957979be506e3b7afdbf077aae4b06942b1bea Author: Jason Feinstein Date: Mon Mar 2 18:14:07 2020 -0800 Create RawEntityDereferencerTest, storage Reference-> CrdtEntity.Reference (#4812) * Create RawEntityDereferencerTest, make storage Reference implement CrdtEntity.Reference. Also: Create ParcelableReference. * Add dep. Also, apparently read/writeBoolean is Q-only. * Just write null if there is no version map. * Just write null if there is no version map. commit ba7a1074ffb487da00625dcfce536763da0b9395 Author: Gogul Balakrishnan Date: Mon Mar 2 17:51:35 2020 -0800 Add a decoder for PrimitiveTypeProto and an option to disable android constraints in BUILD. (#4793) commit f6df54ae7d999c724800c969af59a3c29442995a Author: Alex Rosengarten Date: Mon Mar 2 15:44:32 2020 -0800 tools/sigh lint commit bede8d95ea4c1d7c5115f9d34bd2476e0deb7fef Author: Alex Rosengarten Date: Mon Mar 2 14:52:02 2020 -0800 Fixed build rule, simplified runtime commit 62dcc57fac16e26dd9c724d54be4a6c8ee87bc98 Author: Alex Rosengarten Date: Mon Mar 2 14:38:14 2020 -0800 Fixed test, added TODO * fix ineq * existing tests passing * Updating invalid type test * Squashed commit of the following: commit b3d92d0f69f98571237f06e91ff480209f7aab9d Author: Alex Rosengarten Date: Wed Mar 4 12:17:17 2020 -0800 updated test name commit ba7884b62ccc7af616a9f7f1013475946f3cfb0b Author: Alex Rosengarten Date: Wed Mar 4 12:16:09 2020 -0800 nested unit tests commit ef36fdf11762e2a742f9d1dbd2542d63a2170d5a Author: Alex Rosengarten Date: Wed Mar 4 12:15:00 2020 -0800 renamed to tryResolve commit 5703f0278610f836a0b07f1c577cbcbe107f55ca Author: Alex Rosengarten Date: Wed Mar 4 12:07:18 2020 -0800 rm generator commit 658cb1ad9dd354f97115d8e8cd28a66dc6b032ab Author: Alex Rosengarten Date: Wed Mar 4 12:01:54 2020 -0800 implemented more review suggestions commit 0056ee308c6482aeed05051b9e83ef8954af6e72 Merge: cae89c3c 0f0f82a4 Author: Alex Rosengarten Date: Wed Mar 4 11:54:23 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into r2p commit 0f0f82a439cd00b2ea451f487bc3f96ff63635f1 Author: Maria Mandlis Date: Wed Mar 4 09:33:29 2020 -0800 add queryable capability (#4794) commit d032c75e6996b213e13e5fe6fb42620c6678d71e Author: Maria Mandlis Date: Wed Mar 4 09:23:07 2020 -0800 add creatimeTimestamp to entities (kt) (#4823) commit 30267fc33bc9f74d12f2d7af6564a3f254fc6eaa Author: jblebrun Date: Wed Mar 4 09:09:39 2020 -0800 Remove some unused deps (#4831) commit c51918146dcced836a4a506edfeb3766bdb01a4a Author: Cameron Silvestrini Date: Wed Mar 4 17:14:07 2020 +1100 Increase size of DatabaseImplTest to medium (#4827) * Increase size of DatabaseImplTest to medium Was a bit flaky. * Check flakiness * Revert presubmit tweak commit fa621c63ee680cb91cace2bf2c310b7534eb4dc5 Author: jblebrun Date: Tue Mar 3 22:05:07 2020 -0800 Fix a race condition when setting up `ServiceStore` message channel (#4826) This seems to be the cause of the flakiness in `AndroidAllocatorTest` and `AndroidHandleManagerTest` (#4781) commit 78bf4f05edf7ee8e3162cc8b307be6057f6abedc Author: Gogul Balakrishnan Date: Tue Mar 3 18:00:49 2020 -0800 Utility to convert type proto to field type (if possible). (#4814) commit cae89c3c18f1bea2d9be8861271ab6de0ac0c0a6 Merge: 0360752f e5875390 Author: Alex Rosengarten Date: Tue Mar 3 15:50:34 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into r2p commit e587539077b35ef73bbbc46d67118231262a3b65 Author: jblebrun Date: Tue Mar 3 15:07:16 2020 -0800 Apply simplified test pattern tests in `arcs.android.host` (#4821) As in 5c6310da commit 1af1aac716359bda5bdee4b4f4ab62abfe3ad5e9 Author: Joshua Pratt Date: Wed Mar 4 09:48:15 2020 +1100 Multinomials (#4804) * multinomial * added tests * making constant readable again * reindent * added multivariate rearrangement tests * updated comment * pr comments: Co-authored-by: Ragav Sachdeva commit 0360752fdd3e597c750fbaf3e035fa47c705289a Author: Alex Rosengarten Date: Tue Mar 3 11:42:31 2020 -0800 fix, bad comparison op commit 4b05a287434dcd3a3622ea9c9ebb499fe1d5bc46 Merge: f4c1d26a 99000c25 Author: Alex Rosengarten Date: Tue Mar 3 11:18:14 2020 -0800 Merge branch 'r2p' of github.com:alxrsngrtn/arcs into r2p commit f4c1d26a893e2d6635642567b6eec8d681bdc252 Merge: 140111f5 5c6310da Author: Alex Rosengarten Date: Tue Mar 3 11:14:35 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into r2p Fixed lint errors commit 99000c25e79b55be6c4a09df73a3d1a34227e630 Merge: 140111f5 5c6310da Author: Alex Rosengarten Date: Tue Mar 3 11:14:35 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into r2p Fixed lint errors commit 140111f580adf02c3fe0d2c3a4560dc13c9b2510 Author: Alex Rosengarten Date: Tue Mar 3 11:08:12 2020 -0800 Removed flatMap, added TODOs and link to GH issue commit 5c6310da9d1e372544e816c1dd595ef25e5099bf Author: jblebrun Date: Tue Mar 3 11:02:34 2020 -0800 Simplify service test pattern (#4817) * Make a simple test lifecycle registry, instead of creating empty testactivity * Remove use of `runBlockingTest`: according to https://github.com/Kotlin/kotlinx.coroutines/issues/1222, if the test results in coroutines being finished on other threads, it's possible to receive "This job has not yet completed" exceptions, even though the jobs were properly terminated. Since we don't need the delay-skipping properties of `runBlockingTest`, I think it's OK to use `runBlocking`. commit 472bc841296987e01feee21061781bba0f315921 Author: Alex Rosengarten Date: Tue Mar 3 10:27:56 2020 -0800 Improved build rules commit ca1ebf89390202da9e777f08f7e69f03020d3cf1 Author: Alex Rosengarten Date: Tue Mar 3 10:17:13 2020 -0800 Impl suggestsions for r2p commit b5578ea297e4d3033fdfdefd5f3a325e2cbe503f Author: Jason Feinstein Date: Tue Mar 3 09:54:17 2020 -0800 Add tests for dereferencing references to the HandleManagerTest(s) (#4816) * Add dereferencing tests to the android handle manager test. * Add dereferencing tests to core HandleManager. commit a6957979be506e3b7afdbf077aae4b06942b1bea Author: Jason Feinstein Date: Mon Mar 2 18:14:07 2020 -0800 Create RawEntityDereferencerTest, storage Reference-> CrdtEntity.Reference (#4812) * Create RawEntityDereferencerTest, make storage Reference implement CrdtEntity.Reference. Also: Create ParcelableReference. * Add dep. Also, apparently read/writeBoolean is Q-only. * Just write null if there is no version map. * Just write null if there is no version map. commit ba7a1074ffb487da00625dcfce536763da0b9395 Author: Gogul Balakrishnan Date: Mon Mar 2 17:51:35 2020 -0800 Add a decoder for PrimitiveTypeProto and an option to disable android constraints in BUILD. (#4793) commit f6df54ae7d999c724800c969af59a3c29442995a Author: Alex Rosengarten Date: Mon Mar 2 15:44:32 2020 -0800 tools/sigh lint commit bede8d95ea4c1d7c5115f9d34bd2476e0deb7fef Author: Alex Rosengarten Date: Mon Mar 2 14:52:02 2020 -0800 Fixed build rule, simplified runtime commit 62dcc57fac16e26dd9c724d54be4a6c8ee87bc98 Author: Alex Rosengarten Date: Mon Mar 2 14:38:14 2020 -0800 Fixed test, added TODO * fixed nested tests * rewording, reorganizing * assert more ergonomic commit 298fdccfac759798efef358bf840eef8f010e03a Author: Maria Mandlis Date: Wed Mar 4 15:29:37 2020 -0800 update wasm readme (#4832) commit d15008eb184f62ec559a3f4309e0fb490694078e Author: Alex Rosengarten Date: Wed Mar 4 15:11:14 2020 -0800 fixed build commit bbfc3d2ece4f53f07c60d68efab779c2946baf2e Author: Alex Rosengarten Date: Wed Mar 4 15:04:11 2020 -0800 EntitySpec Interface can return a schema commit 3b75c3411042448f9eff919da2f6066b40e1779a Author: Alex Rosengarten Date: Wed Mar 4 14:09:36 2020 -0800 ktlint fix commit 506457e1d3a091af956ef551432352d5b23ffbc1 Author: Alex Rosengarten Date: Wed Mar 4 13:59:46 2020 -0800 addField arguments are more ergonomic commit 579b2e620c806efb083d19b38b8906631e17d604 Author: Alex Rosengarten Date: Wed Mar 4 13:45:28 2020 -0800 grooming schema registry commit 3cb8023529ef4dcd9478d8159062c17bd9c0d126 Author: Alex Rosengarten Date: Wed Mar 4 11:39:30 2020 -0800 rm unused field commit 1cff99f726f69b49a11b2939e118d446371b20a8 Author: Alex Rosengarten Date: Wed Mar 4 11:38:32 2020 -0800 rm import commit 855401ce852d46bb0287f511740d2c8b4a088906 Author: Alex Rosengarten Date: Wed Mar 4 11:31:14 2020 -0800 no difference w/ wasm golden commit 552e9ad9fc7488f37f26f9825037a3109331e36a Author: Alex Rosengarten Date: Wed Mar 4 11:26:43 2020 -0800 Added test for mapOf commit 14309076e2a5ec45eb3433f486ea84135af57322 Author: Alex Rosengarten Date: Wed Mar 4 11:07:15 2020 -0800 sigh lint commit 9c22195065e71ab0cbadbf981bbc28d9e90f47f4 Author: Alex Rosengarten Date: Wed Mar 4 11:06:50 2020 -0800 minor adjustments commit 74ae7707dcf0615f2ae212802c5cf50498cfa61f Author: Alex Rosengarten Date: Wed Mar 4 10:58:20 2020 -0800 Improved shape of registry commit f69f99b37f791c0173106aab5410b71b3157032d Author: Alex Rosengarten Date: Wed Mar 4 10:32:31 2020 -0800 excluding wasm from generation commit dcefb1658667d596a29b5002e1228366bfe9a388 Author: Alex Rosengarten Date: Wed Mar 4 10:13:46 2020 -0800 Generation adds to schema registry commit f3b9ae5a3706278818c2a712da83d74677c5512c Author: Alex Rosengarten Date: Wed Mar 4 07:57:15 2020 -0800 packaging in a schema registry commit 2ce6da20abf56c5d3d195af3bb885476898ffdb6 Author: Alex Rosengarten Date: Tue Mar 3 22:30:39 2020 -0800 First iteration of schema gen on entities * Squashed commit of the following: commit b26f13e4d01285bd889b0a9f6a52411659b490f8 Author: Alex Rosengarten Date: Thu Mar 5 17:14:33 2020 -0800 updated goldens commit f66fdd85a1348711747da45de15cca8ccb1ca3a0 Author: Alex Rosengarten Date: Thu Mar 5 16:56:26 2020 -0800 further cleanup commit 46b4f17ae3fc0f184dcaa7cae6c08f528431fbe7 Merge: cfcf317b5 8621c58f4 Author: Alex Rosengarten Date: Thu Mar 5 16:54:11 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into kt-gen-utils commit cfcf317b5be48b34db0f552ed68665b8cf510653 Merge: 60582ed3a 623daf73a Author: Alex Rosengarten Date: Thu Mar 5 16:28:56 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into kt-gen-utils commit 60582ed3aedb290f473c1c560283744625208102 Author: Alex Rosengarten Date: Thu Mar 5 15:27:30 2020 -0800 lint fix, added a test commit 196a4a9e1b8d9580f483e909615bd6ab60eaa962 Author: Alex Rosengarten Date: Thu Mar 5 15:16:35 2020 -0800 cleaned up ktUtils, passing tests commit 142fa2b50622fbda8479890f61161b5469998f5a Author: Alex Rosengarten Date: Wed Mar 4 17:42:04 2020 -0800 first iteration for kt utils commit e55cb0b54e4f5c0c1e9990a1aa9d792c8c111960 Author: Alex Rosengarten Date: Wed Mar 4 17:36:00 2020 -0800 Squashed commit of the following: commit 17a2ab1277abfbb4af6499e78f41aaf6c3ea59a6 Author: Alex Rosengarten Date: Wed Mar 4 16:30:45 2020 -0800 Fixed indentation commit 9094055ef7e774d50cc62cb4b8fa4a11241e19ae Author: Alex Rosengarten Date: Wed Mar 4 16:03:51 2020 -0800 revising from feedback commit d3508219e0f344e8b8ff0c926674c37d7b9c20e2 Merge: 9054c231 3acb7e7e Author: Alex Rosengarten Date: Wed Mar 4 15:56:55 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas commit 9054c2319334fc4a7d93e5a713bba6e4ea03ad7a Merge: d15008eb 337b7b26 Author: Alex Rosengarten Date: Wed Mar 4 15:55:27 2020 -0800 Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas commit d15008eb184f62ec559a3f4309e0fb490694078e Author: Alex Rosengarten Date: Wed Mar 4 15:11:14 2020 -0800 fixed build commit bbfc3d2ece4f53f07c60d68efab779c2946baf2e Author: Alex Rosengarten Date: Wed Mar 4 15:04:11 2020 -0800 EntitySpec Interface can return a schema commit 3b75c3411042448f9eff919da2f6066b40e1779a Author: Alex Rosengarten Date: Wed Mar 4 14:09:36 2020 -0800 ktlint fix commit 506457e1d3a091af956ef551432352d5b23ffbc1 Author: Alex Rosengarten Date: Wed Mar 4 13:59:46 2020 -0800 addField arguments are more ergonomic commit 579b2e620c806efb083d19b38b8906631e17d604 Author: Alex Rosengarten Date: Wed Mar 4 13:45:28 2020 -0800 grooming schema registry commit 3cb8023529ef4dcd9478d8159062c17bd9c0d126 Author: Alex Rosengarten Date: Wed Mar 4 11:39:30 2020 -0800 rm unused field commit 1cff99f726f69b49a11b2939e118d446371b20a8 Author: Alex Rosengarten Date: Wed Mar 4 11:38:32 2020 -0800 rm import commit 855401ce852d46bb0287f511740d2c8b4a088906 Author: Alex Rosengarten Date: Wed Mar 4 11:31:14 2020 -0800 no difference w/ wasm golden commit 552e9ad9fc7488f37f26f9825037a3109331e36a Author: Alex Rosengarten Date: Wed Mar 4 11:26:43 2020 -0800 Added test for mapOf commit 14309076e2a5ec45eb3433f486ea84135af57322 Author: Alex Rosengarten Date: Wed Mar 4 11:07:15 2020 -0800 sigh lint commit 9c22195065e71ab0cbadbf981bbc28d9e90f47f4 Author: Alex Rosengarten Date: Wed Mar 4 11:06:50 2020 -0800 minor adjustments commit 74ae7707dcf0615f2ae212802c5cf50498cfa61f Author: Alex Rosengarten Date: Wed Mar 4 10:58:20 2020 -0800 Improved shape of registry commit f69f99b37f791c0173106aab5410b71b3157032d Author: Alex Rosengarten Date: Wed Mar 4 10:32:31 2020 -0800 excluding wasm from generation commit dcefb1658667d596a29b5002e1228366bfe9a388 Author: Alex Rosengarten Date: Wed Mar 4 10:13:46 2020 -0800 Generation adds to schema registry commit f3b9ae5a3706278818c2a712da83d74677c5512c Author: Alex Rosengarten Date: Wed Mar 4 07:57:15 2020 -0800 packaging in a schema registry commit 2ce6da20abf56c5d3d195af3bb885476898ffdb6 Author: Alex Rosengarten Date: Tue Mar 3 22:30:39 2020 -0800 First iteration of schema gen on entities * using kt gen utils * fixed merge issue * Added new kt util * creating handle connections * handle connection map, lots of progress * incomplete, but valid plan generator * Groomed plan gen * fix jvm golden * 1 missing semicolon * buildifier fix * Code quality improvements. * Code quality improvements. * Added golden in testdata, revealed codegen issue from refactoring! * Added golden test for r2p * mv tests to goldens dir * updating whitespace * name change * rm default package * guard imports Co-authored-by: Alex Rosengarten --- java/arcs/core/data/testdata/BUILD | 3 +- .../data/testdata/WriterReaderExample.arcs | 21 +++ src/tools/kotlin-generation-utils.ts | 27 +++- src/tools/plan-generator.ts | 143 ++++++++++++++++++ src/tools/recipe2plan-cli.ts | 13 +- src/tools/recipe2plan.ts | 22 +-- .../tests/goldens/WriterReaderExample.kt | 39 +++++ src/tools/tests/plan-generator-tests.ts | 31 ++++ src/tools/tests/recipe2plan-test.ts | 21 +++ .../java/arcs/build_defs/internal/kotlin.bzl | 7 +- 10 files changed, 296 insertions(+), 31 deletions(-) create mode 100644 java/arcs/core/data/testdata/WriterReaderExample.arcs create mode 100644 src/tools/plan-generator.ts create mode 100755 src/tools/tests/goldens/WriterReaderExample.kt create mode 100644 src/tools/tests/plan-generator-tests.ts create mode 100644 src/tools/tests/recipe2plan-test.ts diff --git a/java/arcs/core/data/testdata/BUILD b/java/arcs/core/data/testdata/BUILD index 06d43663208..b7222ed3fd8 100644 --- a/java/arcs/core/data/testdata/BUILD +++ b/java/arcs/core/data/testdata/BUILD @@ -23,6 +23,7 @@ arcs_manifest_proto( arcs_kt_plan( name = "example_plan", - src = "Example.arcs", + src = "WriterReaderExample.arcs", + package = "arcs.core.data.testdata", visibility = ["//visibility:public"], ) diff --git a/java/arcs/core/data/testdata/WriterReaderExample.arcs b/java/arcs/core/data/testdata/WriterReaderExample.arcs new file mode 100644 index 00000000000..2be6bd2dd52 --- /dev/null +++ b/java/arcs/core/data/testdata/WriterReaderExample.arcs @@ -0,0 +1,21 @@ +particle Reader + data: reads Thing {name: Text} + +particle Writer + data: writes Thing {name: Text} + +@trigger + launch startup + arcId writingArcId +recipe Ingestion + thing: create persistent 'my-handle-id' + Writer + data: writes thing + +@trigger + launch startup + arcId readingArcId +recipe Consumption + data: map 'my-handle-id' + Reader + data: reads data diff --git a/src/tools/kotlin-generation-utils.ts b/src/tools/kotlin-generation-utils.ts index 5c43036098d..9def446ad2f 100644 --- a/src/tools/kotlin-generation-utils.ts +++ b/src/tools/kotlin-generation-utils.ts @@ -33,32 +33,32 @@ export class KotlinGenerationUtils { * * @param name name of the function * @param args list of arguments to the function - * @param emptyName alternative name for the function with empty arguments. * @param startIndent (optional) starting indentation level. + * @param emptyName alternative name for the function with empty arguments. */ - applyFun(name: string, args: string[], emptyName: string = name, startIndent: number = 0): string { + applyFun(name: string, args: string[], startIndent: number = 0, emptyName: string = name): string { if (args.length === 0) return `${emptyName}()`; return `${name}(${this.joinWithIndents(args, startIndent + name.length + 2)})`; } /** Formats `mapOf` with correct indentation and defaults. */ mapOf(args: string[], startIndent: number = 0): string { - return this.applyFun('mapOf', args, 'emptyMap', startIndent); + return this.applyFun('mapOf', args, startIndent, 'emptyMap'); } /** Formats `mutableMapOf` with correct indentation and defaults. */ mutableMapOf(args: string[], startIndent: number = 0): string { - return this.applyFun('mutableMapOf', args, 'mutableMapOf', startIndent); + return this.applyFun('mutableMapOf', args, startIndent, 'mutableMapOf'); } /** Formats `listOf` with correct indentation and defaults. */ listOf(args: string[], startIndent: number = 0): string { - return this.applyFun('listOf', args, 'emptyList', startIndent); + return this.applyFun('listOf', args, startIndent, 'emptyList'); } /** Formats `setOf` with correct indentation and defaults. */ setOf(args: string[], startIndent: number = 0): string { - return this.applyFun('setOf', args, 'emptySet', startIndent); + return this.applyFun('setOf', args, startIndent, 'emptySet'); } /** @@ -70,7 +70,12 @@ export class KotlinGenerationUtils { joinWithIndents(items: string[], extraIndent: number = 0): string { const candidate = items.join(', '); if (extraIndent + candidate.length <= this.pref.lineLength) return candidate; - return `\n${leftPad(items.join(',\n'), this.pref.indent)}\n`; + return `\n${this.indent(items.join(',\n'))}\n`; + } + + /** Indent a codeblock with the preferred indentation. */ + indent(block: string): string { + return leftPad(block, this.pref.indent); } } @@ -82,3 +87,11 @@ export function leftPad(input: string, indent: number, skipFirst: boolean = fals .join('\n'); } +/** Format a Kotlin string. */ +export function quote(s: string) { return `"${s}"`; } + +/** Produces import statement if target is not within the same package. */ +export function tryImport(importName: string, packageName: string): string { + const nonWild = importName.replace('.*', ''); + return packageName === nonWild ? '' : `import ${importName}`; +} diff --git a/src/tools/plan-generator.ts b/src/tools/plan-generator.ts new file mode 100644 index 00000000000..bee59230fe1 --- /dev/null +++ b/src/tools/plan-generator.ts @@ -0,0 +1,143 @@ +/** + * @license + * Copyright 2020 Google LLC. + * This code may only be used under the BSD style license found at + * http://polymer.github.io/LICENSE.txt + * Code distributed by Google as part of this project is also + * subject to an additional IP rights grant found at + * http://polymer.github.io/PATENTS.txt + */ +import {Recipe} from '../runtime/recipe/recipe.js'; +import {Type} from '../runtime/type.js'; +import {Particle} from '../runtime/recipe/particle.js'; +import {KotlinGenerationUtils, quote, tryImport} from './kotlin-generation-utils.js'; +import {HandleConnection} from '../runtime/recipe/handle-connection.js'; +import {StorageKey} from '../runtime/storageNG/storage-key.js'; +import {Direction} from '../runtime/manifest-ast-nodes.js'; + +const ktUtils = new KotlinGenerationUtils(); + +export class PlanGeneratorError extends Error { + constructor(message: string) { + super(message); + this.name = 'PlanGeneratorError'; + } +} + +/** Generates plan objects from resolved recipes. */ +export class PlanGenerator { + constructor(private resolvedRecipes: Recipe[], private scope: string) { + } + + /** Generates a Kotlin file with plan classes derived from resolved recipes. */ + generate(): string { + const planOutline = [ + this.fileHeader(), + ...this.createPlans(), + this.fileFooter() + ]; + + return planOutline.join('\n'); + } + + /** Converts a resolved recipe into a `Plan` object. */ + createPlans(): string[] { + return this.resolvedRecipes.map(recipe => { + const planName = `${recipe.name}Plan`; + + const particles = recipe.particles.map(p => this.createParticle(p)); + + const start = `object ${planName} : `; + return `${start}${ktUtils.applyFun('Plan', [ktUtils.listOf(particles)], start.length)}`; + }); + } + + /** Generates a Kotlin `Plan.Particle` instantiation from a Particle. */ + createParticle(particle: Particle): string { + const spec = particle.spec; + const location = (spec && (spec.implBlobUrl || (spec.implFile && spec.implFile.replace('/', '.')))) || ''; + + const connectionMappings = Object.entries(particle.connections) + .map(([key, conn]) => `"${key}" to ${this.createHandleConnection(conn)}`); + + return ktUtils.applyFun('Particle', [ + quote(particle.name), + quote(location), + ktUtils.mapOf(connectionMappings, 12) + ]); + } + + /** Generates a Kotlin `Plan.HandleConnection` from a HandleConnection. */ + createHandleConnection(connection: HandleConnection): string { + const storageKey = this.createStorageKey(connection.handle.storageKey); + const mode = this.createDirection(connection.direction); + const type = this.createType(connection.type); + const ttl = 'null'; + + return ktUtils.applyFun('HandleConnection', [storageKey, mode, type, ttl], 24); + } + + /** Generates a Kotlin `HandleMode` from a Direction. */ + createDirection(direction: Direction): string { + switch (direction) { + case 'reads': return 'HandleMode.Read'; + case 'writes': return 'HandleMode.Write'; + case 'reads writes': return 'HandleMode.ReadWrite'; + default: throw new PlanGeneratorError( + `HandleConnection direction '${direction}' is not supported.`); + } + } + + /** Generates a Kotlin `StorageKey` from a StorageKey. */ + createStorageKey(storageKey: StorageKey | undefined): string { + return `StorageKeyParser.parse("${(storageKey || '').toString()}")`; + } + + /** Generates a Kotlin `core.arc.type.Type` from a Type. */ + // TODO(alxr): Implement + createType(type: Type): string { + switch (type.tag) { + case 'Collection': + break; + case 'Entity': + break; + case 'Handle': + break; + case 'Reference': + break; + case 'Singleton': + break; + case 'TypeVariable': + break; + case 'Arc': + case 'BigCollection': + case 'Count': + case 'Interface': + case 'Slot': + case 'Tuple': + default: + throw Error(`Type of ${type.tag} is not supported.`); + } + return 'null'; + } + + fileHeader(): string { + return `\ +/* ktlint-disable */ +@file:Suppress("PackageName", "TopLevelName") + +package ${this.scope} + +// +// GENERATED CODE -- DO NOT EDIT +// + +${tryImport('arcs.core.data.*', this.scope)} +${tryImport('arcs.core.storage.*', this.scope)} +`; + } + + fileFooter(): string { + return ``; + } +} diff --git a/src/tools/recipe2plan-cli.ts b/src/tools/recipe2plan-cli.ts index 7e061794df7..ff608655418 100644 --- a/src/tools/recipe2plan-cli.ts +++ b/src/tools/recipe2plan-cli.ts @@ -14,8 +14,8 @@ import {Runtime} from '../runtime/runtime.js'; import {recipe2plan} from './recipe2plan.js'; const opts = minimist(process.argv.slice(2), { - string: ['outdir', 'outfile'], - alias: {d: 'outdir', f: 'outfile'}, + string: ['outdir', 'outfile', 'package'], + alias: {d: 'outdir', f: 'outfile', p: 'package'}, default: {outdir: '.'} }); @@ -30,6 +30,7 @@ Description Options --outfile, -f output filename; required --outdir, -d output directory; defaults to '.' + --package, -p kotlin package. --help usage info `); process.exit(0); @@ -40,6 +41,12 @@ if (!opts.outfile) { process.exit(1); } + +if (!opts.package) { + console.error(`Parameter --package is required.`); + process.exit(1); +} + // TODO(alxr): Support generation from multiple manifests if (opts._.length > 1) { console.error(`Only a single manifest is allowed`); @@ -56,7 +63,7 @@ async function main() { Runtime.init('../..'); fs.mkdirSync(opts.outdir, {recursive: true}); - const plans = await recipe2plan(opts._[0]); + const plans = await recipe2plan(opts._[0], opts.package); const outPath = path.join(opts.outdir, opts.outfile); console.log(outPath); diff --git a/src/tools/recipe2plan.ts b/src/tools/recipe2plan.ts index b03977723df..6aac1da6531 100644 --- a/src/tools/recipe2plan.ts +++ b/src/tools/recipe2plan.ts @@ -8,35 +8,23 @@ * http://polymer.github.io/PATENTS.txt */ import {Runtime} from '../runtime/runtime.js'; -import {Recipe} from '../runtime/recipe/recipe.js'; import {StorageKeyRecipeResolver} from './storage-key-recipe-resolver.js'; +import {PlanGenerator} from './plan-generator.js'; /** * Generates Kotlin Plans from recipes in an arcs manifest. * * @param path path/to/manifest.arcs + * @param scope kotlin package name * @return Generated Kotlin code. */ -export async function recipe2plan(path: string): Promise { +export async function recipe2plan(path: string, scope: string): Promise { const manifest = await Runtime.parseFile(path); const recipes = await (new StorageKeyRecipeResolver(manifest)).resolve(); - const plans = await generatePlans(recipes); + const generator = new PlanGenerator(recipes, scope); - return plans.join('\n'); + return generator.generate(); } - - -/** - * Converts each resolved recipes into a Kotlin Plan class. - * - * @param resolutions A series of resolved recipes. - * @return List of generated Kotlin plans - */ -async function generatePlans(resolutions: Recipe[]): Promise { - // TODO Implement - return ['']; -} - diff --git a/src/tools/tests/goldens/WriterReaderExample.kt b/src/tools/tests/goldens/WriterReaderExample.kt new file mode 100755 index 00000000000..fb8adcf0504 --- /dev/null +++ b/src/tools/tests/goldens/WriterReaderExample.kt @@ -0,0 +1,39 @@ +/* ktlint-disable */ +@file:Suppress("PackageName", "TopLevelName") + +package arcs.core.data.testdata + +// +// GENERATED CODE -- DO NOT EDIT +// + +import arcs.core.data.* +import arcs.core.storage.* + +object IngestionPlan : Plan( + listOf( + Particle( + "Writer", + "", + mapOf( + "data" to HandleConnection(StorageKeyParser.parse(""), HandleMode.Write, null, null) + ) + ) + ) +) +object ConsumptionPlan : Plan( + listOf( + Particle( + "Reader", + "", + mapOf( + "data" to HandleConnection( + StorageKeyParser.parse("ramdisk://"), + HandleMode.Read, + null, + null + ) + ) + ) + ) +) diff --git a/src/tools/tests/plan-generator-tests.ts b/src/tools/tests/plan-generator-tests.ts new file mode 100644 index 00000000000..9f5092ef136 --- /dev/null +++ b/src/tools/tests/plan-generator-tests.ts @@ -0,0 +1,31 @@ +/** + * @license + * Copyright 2020 Google LLC. + * This code may only be used under the BSD style license found at + * http://polymer.github.io/LICENSE.txt + * Code distributed by Google as part of this project is also + * subject to an additional IP rights grant found at + * http://polymer.github.io/PATENTS.txt + */ + +import {PlanGenerator} from '../plan-generator.js'; +import {assert} from '../../platform/chai-node.js'; + +describe('recipe2plan', () => { + describe('plan-generator', () => { + it('imports arcs.core.data when the package is different', () => { + const generator = new PlanGenerator([], 'some.package'); + + const actual = generator.fileHeader(); + + assert.include(actual, 'import arcs.core.data.*'); + }); + it('does not import arcs.core.data when the package is the same', () => { + const generator = new PlanGenerator([], 'arcs.core.data'); + + const actual = generator.fileHeader(); + + assert.notInclude(actual, 'import arcs.core.data.*'); + }); + }); +}); diff --git a/src/tools/tests/recipe2plan-test.ts b/src/tools/tests/recipe2plan-test.ts new file mode 100644 index 00000000000..3f46d3ca9d6 --- /dev/null +++ b/src/tools/tests/recipe2plan-test.ts @@ -0,0 +1,21 @@ +/** + * @license + * Copyright 2020 Google LLC. + * This code may only be used under the BSD style license found at + * http://polymer.github.io/LICENSE.txt + * Code distributed by Google as part of this project is also + * subject to an additional IP rights grant found at + * http://polymer.github.io/PATENTS.txt + */ +import {assert} from '../../platform/chai-web.js'; +import {fs} from '../../platform/fs-web.js'; +import {recipe2plan} from '../recipe2plan.js'; + +describe('recipe2plan', () => { + it('generates plans from recipes in a manifest', async () => { + assert.deepStrictEqual( + await recipe2plan('java/arcs/core/data/testdata/WriterReaderExample.arcs', 'arcs.core.data.testdata'), + fs.readFileSync('src/tools/tests/goldens/WriterReaderExample.kt', 'utf8') + ); + }); +}); diff --git a/third_party/java/arcs/build_defs/internal/kotlin.bzl b/third_party/java/arcs/build_defs/internal/kotlin.bzl index 5e5656f0f2c..4f55f015c07 100644 --- a/third_party/java/arcs/build_defs/internal/kotlin.bzl +++ b/third_party/java/arcs/build_defs/internal/kotlin.bzl @@ -322,12 +322,13 @@ def arcs_kt_android_test_suite(name, manifest, package, srcs = None, tags = [], data = data, ) -def arcs_kt_plan(name, src, deps = [], out = None, visibility = None): +def arcs_kt_plan(name, src, package, deps = [], out = None, visibility = None): """Converts recipes in manifests into Kotlin Plans. Args: name: the name of the target to create src: an Arcs manifest file + package: name of kotlin package for generated file deps: list of dependencies (other manifests) out: the name of the output artifact (a Kotlin file). visibility: list of visibilities @@ -339,8 +340,8 @@ def arcs_kt_plan(name, src, deps = [], out = None, visibility = None): srcs = [src], outs = outs, deps = deps, - progress_message = "Producing Plans", - sigh_cmd = "recipe2plan --outdir $(dirname {OUT}) --outfile $(basename {OUT}) {SRC}", + progress_message = "Generating Plans", + sigh_cmd = "recipe2plan --outdir $(dirname {OUT}) --outfile $(basename {OUT}) --package " + package + " {SRC}", ) def arcs_kt_jvm_test_suite(name, package, srcs = None, tags = [], deps = [], data = []):