Skip to content

Commit

Permalink
recipe2plan 4/n: Generating Kotlin Types (#4869)
Browse files Browse the repository at this point in the history 
## Summary
Enhances #4865, generates Types within Kotlin Plan classes.

For an example, see Golden's modification.

## Changelog
* creating structure of ts r2p script

* Created fast CLI rapper for r2p script

* WIP figuring out ways to resolve a recipe

* fix sp

* Passing lint, still WIP

* Build macros for recipe2plan work

* Lightweight iteration on recipe2plan, stubbed out tests

* Revised method to find corresponding create handles

* Added type info

* Added method to get all handles to manifest + get all handles by Id

* fix: no flatMap

* - currently, fails since it cannot find associated stores for handles.

* uncleaned, but working recipe resolution

* Cleaned up for r2p, pt 1

* Fixed test, added TODO

* Fixed build rule, simplified runtime

* tools/sigh lint

* Impl suggestsions for r2p

* Improved build rules

* Removed flatMap, added TODOs and link to GH issue

* fix, bad comparison op

* Stub plan generator works

* updated macro to take a package

* Testing import is conditional on package

* Incremental improvement

* tidying

* Fixed tests

* added particle creation method

* flushing out more of plan gen

* Squashed commit of the following:

commit 17a2ab1
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 16:30:45 2020 -0800

    Fixed indentation

commit 9094055
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 16:03:51 2020 -0800

    revising from feedback

commit d350821
Merge: 9054c23 3acb7e7
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:56:55 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas

commit 3acb7e7
Author: Alex Rosengarten <alxr@google.com>
Date:   Wed Mar 4 15:56:42 2020 -0800

    XXS CL to add a hash field to the Schema proto (#4752)

commit 9054c23
Merge: d15008e 337b7b2
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:55:27 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas

commit 337b7b2
Author: Alex Rosengarten <alxr@google.com>
Date:   Wed Mar 4 15:55:06 2020 -0800

    recipe2plan 2/n: Validate Handles, Tests (#4834)

    * creating structure of ts r2p script

    * Created fast CLI rapper for r2p script

    * WIP figuring out ways to resolve a recipe

    * fix sp

    * Passing lint, still WIP

    * Build macros for recipe2plan work

    * Lightweight iteration on recipe2plan, stubbed out tests

    * Revised method to find corresponding create handles

    * Added type info

    * Added method to get all handles to manifest + get all handles by Id

    * fix: no flatMap

    * - currently, fails since it cannot find associated stores for handles.

    * uncleaned, but working recipe resolution

    * Cleaned up for r2p, pt 1

    * stubbed out second test (short, short)

    * Squashed commit of the following:

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * updates to test, can't get two to fail

    * Squashed commit of the following:

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * can't get test to fail... hmm...

    * Squashed commit of the following:

    commit f6df54a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 15:44:32 2020 -0800

        tools/sigh lint

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * adjusting recipe copying to include triggers

    * Revert "adjusting recipe copying to include triggers"

    This reverts commit 9bc3734

    * four tests complete

    * Added test, fixed minor async issue

    * Squashed commit of the following:

    commit 4b05a28
    Merge: f4c1d26 99000c2
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:18:14 2020 -0800

        Merge branch 'r2p' of github.com:alxrsngrtn/arcs into r2p

    commit f4c1d26
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 99000c2
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 140111f
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:08:12 2020 -0800

        Removed flatMap, added TODOs and link to GH issue

    commit 5c6310d
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 11:02:34 2020 -0800

        Simplify service test pattern (#4817)

        * Make a simple test lifecycle registry, instead of creating empty
        testactivity

        * Remove use of `runBlockingTest`: according to
        Kotlin/kotlinx.coroutines#1222, if the test
        results in coroutines being finished on other threads, it's possible to
        receive "This job has not yet completed" exceptions, even though the
        jobs were properly terminated. Since we don't need the delay-skipping
        properties of `runBlockingTest`, I think it's OK to use `runBlocking`.

    commit 472bc84
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:27:56 2020 -0800

        Improved build rules

    commit ca1ebf8
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:17:13 2020 -0800

        Impl suggestsions for r2p

    commit b5578ea
    Author: Jason Feinstein <jwf@google.com>
    Date:   Tue Mar 3 09:54:17 2020 -0800

        Add tests for dereferencing references to the HandleManagerTest(s) (#4816)

        * Add dereferencing tests to the android handle manager test.

        * Add dereferencing tests to core HandleManager.

    commit a695797
    Author: Jason Feinstein <jwf@google.com>
    Date:   Mon Mar 2 18:14:07 2020 -0800

        Create RawEntityDereferencerTest, storage Reference-> CrdtEntity.Reference (#4812)

        * Create RawEntityDereferencerTest, make storage Reference implement CrdtEntity.Reference.

        Also: Create ParcelableReference.

        * Add dep. Also, apparently read/writeBoolean is Q-only.

        * Just write null if there is no version map.

        * Just write null if there is no version map.

    commit ba7a107
    Author: Gogul Balakrishnan <bgogul@google.com>
    Date:   Mon Mar 2 17:51:35 2020 -0800

        Add a decoder for PrimitiveTypeProto and an option to disable android constraints in BUILD. (#4793)

    commit f6df54a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 15:44:32 2020 -0800

        tools/sigh lint

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * fix ineq

    * existing tests passing

    * Updating invalid type test

    * Squashed commit of the following:

    commit b3d92d0
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:17:17 2020 -0800

        updated test name

    commit ba7884b
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:16:09 2020 -0800

        nested unit tests

    commit ef36fdf
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:15:00 2020 -0800

        renamed to tryResolve

    commit 5703f02
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:07:18 2020 -0800

        rm generator

    commit 658cb1a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:01:54 2020 -0800

        implemented more review suggestions

    commit 0056ee3
    Merge: cae89c3 0f0f82a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:54:23 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p

    commit 0f0f82a
    Author: Maria Mandlis <mmandlis@chromium.org>
    Date:   Wed Mar 4 09:33:29 2020 -0800

        add queryable capability (#4794)

    commit d032c75
    Author: Maria Mandlis <mmandlis@chromium.org>
    Date:   Wed Mar 4 09:23:07 2020 -0800

        add creatimeTimestamp to entities (kt) (#4823)

    commit 30267fc
    Author: jblebrun <jibbl@google.com>
    Date:   Wed Mar 4 09:09:39 2020 -0800

        Remove some unused deps (#4831)

    commit c519181
    Author: Cameron Silvestrini <csilvestrini@users.noreply.github.com>
    Date:   Wed Mar 4 17:14:07 2020 +1100

        Increase size of DatabaseImplTest to medium (#4827)

        * Increase size of DatabaseImplTest to medium

        Was a bit flaky.

        * Check flakiness

        * Revert presubmit tweak

    commit fa621c6
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 22:05:07 2020 -0800

        Fix a race condition when setting up `ServiceStore` message channel (#4826)

        This seems to be the cause of the flakiness in `AndroidAllocatorTest`
        and `AndroidHandleManagerTest` (#4781)

    commit 78bf4f0
    Author: Gogul Balakrishnan <bgogul@google.com>
    Date:   Tue Mar 3 18:00:49 2020 -0800

        Utility to convert type proto to field type (if possible). (#4814)

    commit cae89c3
    Merge: 0360752 e587539
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 15:50:34 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p

    commit e587539
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 15:07:16 2020 -0800

        Apply simplified test pattern tests in `arcs.android.host` (#4821)

        As in 5c6310d

    commit 1af1aac
    Author: Joshua Pratt <jp10010101010000@gmail.com>
    Date:   Wed Mar 4 09:48:15 2020 +1100

        Multinomials (#4804)

        * multinomial

        * added tests

        * making constant readable again

        * reindent

        * added multivariate rearrangement tests

        * updated comment

        * pr comments:

        Co-authored-by: Ragav Sachdeva <ragavsachdeva007@gmail.com>

    commit 0360752
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:42:31 2020 -0800

        fix, bad comparison op

    commit 4b05a28
    Merge: f4c1d26 99000c2
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:18:14 2020 -0800

        Merge branch 'r2p' of github.com:alxrsngrtn/arcs into r2p

    commit f4c1d26
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 99000c2
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 140111f
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:08:12 2020 -0800

        Removed flatMap, added TODOs and link to GH issue

    commit 5c6310d
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 11:02:34 2020 -0800

        Simplify service test pattern (#4817)

        * Make a simple test lifecycle registry, instead of creating empty
        testactivity

        * Remove use of `runBlockingTest`: according to
        Kotlin/kotlinx.coroutines#1222, if the test
        results in coroutines being finished on other threads, it's possible to
        receive "This job has not yet completed" exceptions, even though the
        jobs were properly terminated. Since we don't need the delay-skipping
        properties of `runBlockingTest`, I think it's OK to use `runBlocking`.

    commit 472bc84
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:27:56 2020 -0800

        Improved build rules

    commit ca1ebf8
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:17:13 2020 -0800

        Impl suggestsions for r2p

    commit b5578ea
    Author: Jason Feinstein <jwf@google.com>
    Date:   Tue Mar 3 09:54:17 2020 -0800

        Add tests for dereferencing references to the HandleManagerTest(s) (#4816)

        * Add dereferencing tests to the android handle manager test.

        * Add dereferencing tests to core HandleManager.

    commit a695797
    Author: Jason Feinstein <jwf@google.com>
    Date:   Mon Mar 2 18:14:07 2020 -0800

        Create RawEntityDereferencerTest, storage Reference-> CrdtEntity.Reference (#4812)

        * Create RawEntityDereferencerTest, make storage Reference implement CrdtEntity.Reference.

        Also: Create ParcelableReference.

        * Add dep. Also, apparently read/writeBoolean is Q-only.

        * Just write null if there is no version map.

        * Just write null if there is no version map.

    commit ba7a107
    Author: Gogul Balakrishnan <bgogul@google.com>
    Date:   Mon Mar 2 17:51:35 2020 -0800

        Add a decoder for PrimitiveTypeProto and an option to disable android constraints in BUILD. (#4793)

    commit f6df54a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 15:44:32 2020 -0800

        tools/sigh lint

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * fixed nested tests

    * rewording, reorganizing

    * assert more ergonomic

commit 298fdcc
Author: Maria Mandlis <mmandlis@chromium.org>
Date:   Wed Mar 4 15:29:37 2020 -0800

    update wasm readme (#4832)

commit d15008e
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:11:14 2020 -0800

    fixed build

commit bbfc3d2
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:04:11 2020 -0800

    EntitySpec Interface can return a schema

commit 3b75c34
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 14:09:36 2020 -0800

    ktlint fix

commit 506457e
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 13:59:46 2020 -0800

    addField arguments are more ergonomic

commit 579b2e6
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 13:45:28 2020 -0800

    grooming schema registry

commit 3cb8023
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:39:30 2020 -0800

    rm unused field

commit 1cff99f
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:38:32 2020 -0800

    rm import

commit 855401c
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:31:14 2020 -0800

    no difference w/ wasm golden

commit 552e9ad
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:26:43 2020 -0800

    Added test for mapOf

commit 1430907
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:07:15 2020 -0800

    sigh lint

commit 9c22195
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:06:50 2020 -0800

    minor adjustments

commit 74ae770
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 10:58:20 2020 -0800

    Improved shape of registry

commit f69f99b
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 10:32:31 2020 -0800

    excluding wasm from generation

commit dcefb16
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 10:13:46 2020 -0800

    Generation adds to schema registry

commit f3b9ae5
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 07:57:15 2020 -0800

    packaging in a schema registry

commit 2ce6da2
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Tue Mar 3 22:30:39 2020 -0800

    First iteration of schema gen on entities

* Squashed commit of the following:

commit b26f13e
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Thu Mar 5 17:14:33 2020 -0800

    updated goldens

commit f66fdd8
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Thu Mar 5 16:56:26 2020 -0800

    further cleanup

commit 46b4f17
Merge: cfcf317 8621c58
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Thu Mar 5 16:54:11 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into kt-gen-utils

commit cfcf317
Merge: 60582ed 623daf7
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Thu Mar 5 16:28:56 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into kt-gen-utils

commit 60582ed
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Thu Mar 5 15:27:30 2020 -0800

    lint fix, added a test

commit 196a4a9
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Thu Mar 5 15:16:35 2020 -0800

    cleaned up ktUtils, passing tests

commit 142fa2b
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 17:42:04 2020 -0800

    first iteration for kt utils

commit e55cb0b
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 17:36:00 2020 -0800

    Squashed commit of the following:

    commit 17a2ab1
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 16:30:45 2020 -0800

        Fixed indentation

    commit 9094055
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 16:03:51 2020 -0800

        revising from feedback

    commit d350821
    Merge: 9054c23 3acb7e7
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 15:56:55 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas

    commit 9054c23
    Merge: d15008e 337b7b2
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 15:55:27 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas

    commit d15008e
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 15:11:14 2020 -0800

        fixed build

    commit bbfc3d2
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 15:04:11 2020 -0800

        EntitySpec Interface can return a schema

    commit 3b75c34
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 14:09:36 2020 -0800

        ktlint fix

    commit 506457e
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 13:59:46 2020 -0800

        addField arguments are more ergonomic

    commit 579b2e6
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 13:45:28 2020 -0800

        grooming schema registry

    commit 3cb8023
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:39:30 2020 -0800

        rm unused field

    commit 1cff99f
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:38:32 2020 -0800

        rm import

    commit 855401c
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:31:14 2020 -0800

        no difference w/ wasm golden

    commit 552e9ad
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:26:43 2020 -0800

        Added test for mapOf

    commit 1430907
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:07:15 2020 -0800

        sigh lint

    commit 9c22195
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:06:50 2020 -0800

        minor adjustments

    commit 74ae770
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 10:58:20 2020 -0800

        Improved shape of registry

    commit f69f99b
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 10:32:31 2020 -0800

        excluding wasm from generation

    commit dcefb16
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 10:13:46 2020 -0800

        Generation adds to schema registry

    commit f3b9ae5
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 07:57:15 2020 -0800

        packaging in a schema registry

    commit 2ce6da2
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 22:30:39 2020 -0800

        First iteration of schema gen on entities

* using kt gen utils

* fixed merge issue

* Added new kt util

* creating handle connections

* handle connection map, lots of progress

* incomplete, but valid plan generator

* Groomed plan gen

* fix jvm golden

* 1 missing semicolon

* Impl createType w/ tests

* buildifier fix

* Code quality improvements.

* Code quality improvements.

* Added golden in testdata, revealed codegen issue from refactoring!

* Added golden test for r2p

* mv tests to goldens dir

* updating whitespace

* name change

* rm default package

* guard imports

* fixes: now passing tests after merge

* fix: need to wrap a list

* extracted generic function to kt utils

* using schemas on specs

* using schemas on specs

* Fixed tests

* clean up

* impl suggestions from feedback

* fixed lint / functionality issue

* schema --> SCHEMA

* rm unused imports

* Fixed the reference key issue; added Flags function

* better plance for `Flags` call

Co-authored-by: Alex Rosengarten <alxrsngrtn@google.com>
  • Loading branch information
@alxmrs
alxmrs and alxmrs committed Mar 10, 2020
1 parent 5bf3565 commit 141f598
Show file tree
Hide file tree
Showing 8 changed files with 122 additions and 40 deletions.
5 changes: 5 additions & 0 deletions src/tools/kotlin-generation-utils.ts
View file
Expand Up @@ -95,3 +95,8 @@ export function tryImport(importName: string, packageName: string): string {
const nonWild = importName.replace('.*', '');
return packageName === nonWild ? '' : `import ${importName}`;
}

/** Coalesces input string such that the first character is upper case. */
export function upperFirst(s: string): string {
return s[0].toUpperCase() + s.slice(1);
}
66 changes: 42 additions & 24 deletions src/tools/plan-generator.ts
View file
Expand Up @@ -10,10 +10,11 @@
import {Recipe} from '../runtime/recipe/recipe.js';
import {Type} from '../runtime/type.js';
import {Particle} from '../runtime/recipe/particle.js';
import {KotlinGenerationUtils, quote, tryImport} from './kotlin-generation-utils.js';
import {KotlinGenerationUtils, quote, tryImport, upperFirst} from './kotlin-generation-utils.js';
import {HandleConnection} from '../runtime/recipe/handle-connection.js';
import {StorageKey} from '../runtime/storageNG/storage-key.js';
import {Direction} from '../runtime/manifest-ast-nodes.js';
import {Dictionary} from '../runtime/hot.js';

const ktUtils = new KotlinGenerationUtils();

Expand All @@ -26,39 +27,50 @@ export class PlanGeneratorError extends Error {

/** Generates plan objects from resolved recipes. */
export class PlanGenerator {
private specRegistry: Dictionary<string> = {};

constructor(private resolvedRecipes: Recipe[], private scope: string) {
}

/** Generates a Kotlin file with plan classes derived from resolved recipes. */
generate(): string {
async generate(): Promise<string> {
const planOutline = [
this.fileHeader(),
...this.createPlans(),
...(await this.createPlans()),
this.fileFooter()
];

return planOutline.join('\n');
}

/** Converts a resolved recipe into a `Plan` object. */
createPlans(): string[] {
return this.resolvedRecipes.map(recipe => {
async createPlans(): Promise<string[]> {
const plans: string[] = [];
for (const recipe of this.resolvedRecipes) {
const planName = `${recipe.name}Plan`;

const particles = recipe.particles.map(p => this.createParticle(p));
const particles: string[] = [];
for (const particle of recipe.particles) {
await this.collectParticleConnectionSpecs(particle);
particles.push(await this.createParticle(particle));
}

const start = `object ${planName} : `;
return `${start}${ktUtils.applyFun('Plan', [ktUtils.listOf(particles)], start.length)}`;
});
const plan = `${start}${ktUtils.applyFun('Plan', [ktUtils.listOf(particles)], start.length)}`;
plans.push(plan);
}
return plans;
}

/** Generates a Kotlin `Plan.Particle` instantiation from a Particle. */
createParticle(particle: Particle): string {
async createParticle(particle: Particle): Promise<string> {
const spec = particle.spec;
const location = (spec && (spec.implBlobUrl || (spec.implFile && spec.implFile.replace('/', '.')))) || '';

const connectionMappings = Object.entries(particle.connections)
.map(([key, conn]) => `"${key}" to ${this.createHandleConnection(conn)}`);
const connectionMappings: string[] = [];
for (const [key, conn] of Object.entries(particle.connections)) {
connectionMappings.push(`"${key}" to ${await this.createHandleConnection(conn)}`);
}

return ktUtils.applyFun('Particle', [
quote(particle.name),
Expand All @@ -67,11 +79,20 @@ export class PlanGenerator {
]);
}

/** Aggregate mapping of schema hashes and schema properties from particle connections */
async collectParticleConnectionSpecs(particle: Particle): Promise<void> {
for (const connection of particle.spec.connections) {
const specName = [particle.spec.name, upperFirst(connection.name), 'Spec'].join('_');
const schemaHash = await connection.type.getEntitySchema().hash();
this.specRegistry[schemaHash] = specName;
}
}

/** Generates a Kotlin `Plan.HandleConnection` from a HandleConnection. */
createHandleConnection(connection: HandleConnection): string {
async createHandleConnection(connection: HandleConnection): Promise<string> {
const storageKey = this.createStorageKey(connection.handle.storageKey);
const mode = this.createDirection(connection.direction);
const type = this.createType(connection.type);
const type = await this.createType(connection.type);
const ttl = 'null';

return ktUtils.applyFun('HandleConnection', [storageKey, mode, type, ttl], 24);
Expand All @@ -94,31 +115,28 @@ export class PlanGenerator {
}

/** Generates a Kotlin `core.arc.type.Type` from a Type. */
// TODO(alxr): Implement
createType(type: Type): string {
async createType(type: Type): Promise<string> {
switch (type.tag) {
case 'Collection':
break;
return ktUtils.applyFun('CollectionType', [await this.createType(type.getContainedType())]);
case 'Count':
return ktUtils.applyFun('CountType', [await this.createType(type.getContainedType())]);
case 'Entity':
break;
case 'Handle':
break;
return ktUtils.applyFun('EntityType', [`${this.specRegistry[await type.getEntitySchema().hash()]}.SCHEMA`]);
case 'Reference':
break;
return ktUtils.applyFun('ReferenceType', [await this.createType(type.getContainedType())]);
case 'Singleton':
break;
return ktUtils.applyFun('SingletonType', [await this.createType(type.getContainedType())]);
case 'TypeVariable':
break;
case 'Arc':
case 'BigCollection':
case 'Count':
case 'Handle':
case 'Interface':
case 'Slot':
case 'Tuple':
default:
throw Error(`Type of ${type.tag} is not supported.`);
}
return 'null';
}

fileHeader(): string {
Expand Down
3 changes: 2 additions & 1 deletion src/tools/recipe2plan-cli.ts
View file
Expand Up @@ -12,6 +12,7 @@ import fs from 'fs';
import path from 'path';
import {Runtime} from '../runtime/runtime.js';
import {recipe2plan} from './recipe2plan.js';
import {Flags} from '../runtime/flags.js';

const opts = minimist(process.argv.slice(2), {
string: ['outdir', 'outfile', 'package'],
Expand Down Expand Up @@ -63,7 +64,7 @@ async function main() {
Runtime.init('../..');
fs.mkdirSync(opts.outdir, {recursive: true});

const plans = await recipe2plan(opts._[0], opts.package);
const plans: string = await recipe2plan(opts._[0], opts.package);

const outPath = path.join(opts.outdir, opts.outfile);
console.log(outPath);
Expand Down
11 changes: 7 additions & 4 deletions src/tools/recipe2plan.ts
View file
Expand Up @@ -10,6 +10,7 @@
import {Runtime} from '../runtime/runtime.js';
import {StorageKeyRecipeResolver} from './storage-key-recipe-resolver.js';
import {PlanGenerator} from './plan-generator.js';
import {Flags} from '../runtime/flags.js';


/**
Expand All @@ -20,11 +21,13 @@ import {PlanGenerator} from './plan-generator.js';
* @return Generated Kotlin code.
*/
export async function recipe2plan(path: string, scope: string): Promise<string> {
const manifest = await Runtime.parseFile(path);
return await Flags.withDefaultReferenceMode(async () => {
const manifest = await Runtime.parseFile(path);

const recipes = await (new StorageKeyRecipeResolver(manifest)).resolve();
const recipes = await (new StorageKeyRecipeResolver(manifest)).resolve();

const generator = new PlanGenerator(recipes, scope);
const generator = new PlanGenerator(recipes, scope);

return generator.generate();
return generator.generate();
})();
}
9 changes: 3 additions & 6 deletions src/tools/schema2graph.ts
View file
Expand Up @@ -9,6 +9,7 @@
*/
import {Schema} from '../runtime/schema.js';
import {ParticleSpec} from '../runtime/particle-spec.js';
import {upperFirst} from './kotlin-generation-utils.js';

export class SchemaNode {
schema: Schema;
Expand Down Expand Up @@ -55,7 +56,7 @@ export class SchemaGraph {
for (const connection of this.particleSpec.connections) {
const schema = connection.type.getEntitySchema();
if (schema) {
this.createNodes(schema, `${this.particleSpec.name}_${this.upperFirst(connection.name)}`);
this.createNodes(schema, [this.particleSpec.name, upperFirst(connection.name)].join('_'));
}
}

Expand Down Expand Up @@ -101,7 +102,7 @@ export class SchemaGraph {
if (nestedSchema) {
// We have a reference field. Generate a node for its nested schema and connect it into the
// refs map to indicate that this node requires nestedNode's class to be generated first.
const nestedNode = this.createNodes(nestedSchema, name + '_' + this.upperFirst(field));
const nestedNode = this.createNodes(nestedSchema, [name, upperFirst(field)].join('_'));
node.refs.set(field, nestedNode);
}
}
Expand Down Expand Up @@ -137,10 +138,6 @@ export class SchemaGraph {
}
}

private upperFirst(s: string): string {
return s[0].toUpperCase() + s.slice(1);
}

// Traverses the graph to yield schemas in the order in which they should be generated.
// The traversal is primarily breadth-first, but some nodes may be pushed back due to
// unsatisfied constraints.
Expand Down
9 changes: 7 additions & 2 deletions src/tools/tests/goldens/WriterReaderExample.kt
View file
Expand Up @@ -16,7 +16,12 @@ object IngestionPlan : Plan(
"Writer",
"",
mapOf(
"data" to HandleConnection(StorageKeyParser.parse(""), HandleMode.Write, null, null)
"data" to HandleConnection(
StorageKeyParser.parse(""),
HandleMode.Write,
EntityType(Writer_Data_Spec.SCHEMA),
null
)
)
)
)
Expand All @@ -30,7 +35,7 @@ object ConsumptionPlan : Plan(
"data" to HandleConnection(
StorageKeyParser.parse("reference-mode://{db://25e71af4e9fc8b6958fc46a8f4b7cdf6b5f31516@arcs/Thing}{db://25e71af4e9fc8b6958fc46a8f4b7cdf6b5f31516@arcs/!:writingArcId/handle/my-handle-id}"),
HandleMode.Read,
null,
EntityType(Reader_Data_Spec.SCHEMA),
null
)
)
Expand Down
56 changes: 56 additions & 0 deletions src/tools/tests/plan-generator-tests.ts
View file
Expand Up @@ -10,9 +10,14 @@

import {PlanGenerator} from '../plan-generator.js';
import {assert} from '../../platform/chai-node.js';
import {Manifest} from '../../runtime/manifest.js';

describe('recipe2plan', () => {
describe('plan-generator', () => {
let emptyGenerator: PlanGenerator;
beforeEach(() => {
emptyGenerator = new PlanGenerator([], '');
});
it('imports arcs.core.data when the package is different', () => {
const generator = new PlanGenerator([], 'some.package');

Expand All @@ -27,5 +32,56 @@ describe('recipe2plan', () => {

assert.notInclude(actual, 'import arcs.core.data.*');
});
it('creates valid types that refer to registered Schemas', async () => {
const manifest = await Manifest.parse(`\
particle A
data: writes Thing {num: Number}
recipe R
h: create persistent 'some-id'
A
data: writes h`);

await emptyGenerator.collectParticleConnectionSpecs(manifest.recipes[0].particles[0]);
const actual = await emptyGenerator.createType(manifest.particles[0].handleConnections[0].type);

assert.include(actual, 'EntityType(A_Data_Spec.SCHEMA)');
assert.notInclude(actual, 'SingletonType');
});
it('creates valid types that are derived from other types (via nesting)', async () => {
const manifest = await Manifest.parse(`\
particle A
data: writes [Thing {num: Number}]
recipe R
h: create persistent 'some-id'
A
data: writes h`);

await emptyGenerator.collectParticleConnectionSpecs(manifest.recipes[0].particles[0]);
const actual = await emptyGenerator.createType(manifest.particles[0].handleConnections[0].type);

assert.include(actual, 'EntityType(A_Data_Spec.SCHEMA)');
assert.notInclude(actual, 'SingletonType');
assert.include(actual, 'CollectionType');
});
it('creates valid types that are derived from a few other types (via lots of nesting)', async () => {
const manifest = await Manifest.parse(`\
particle A
data: writes [&Thing {num: Number}]
recipe R
h: create persistent 'some-id'
A
data: writes h`);

await emptyGenerator.collectParticleConnectionSpecs(manifest.recipes[0].particles[0]);
const actual = await emptyGenerator.createType(manifest.particles[0].handleConnections[0].type);

assert.include(actual, 'EntityType(A_Data_Spec.SCHEMA)');
assert.notInclude(actual, 'SingletonType');
assert.include(actual, 'CollectionType');
assert.include(actual, 'ReferenceType');
});
});
});
3 changes: 0 additions & 3 deletions src/tools/tests/recipe2plan-test.ts
View file
Expand Up @@ -10,9 +10,6 @@
import {assert} from '../../platform/chai-web.js';
import {fs} from '../../platform/fs-web.js';
import {recipe2plan} from '../recipe2plan.js';
import {CapabilitiesResolver, StorageKeyOptions} from '../../runtime/capabilities-resolver.js';
import {RamDiskStorageKey} from '../../runtime/storageNG/drivers/ramdisk.js';
import {Capabilities} from '../../runtime/capabilities.js';
import {Flags} from '../../runtime/flags.js';

describe('recipe2plan', () => {
Expand Down

0 comments on commit 141f598

Please sign in to comment.