Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history 
commit 17a2ab1
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 16:30:45 2020 -0800

    Fixed indentation

commit 9094055
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 16:03:51 2020 -0800

    revising from feedback

commit d350821
Merge: 9054c23 3acb7e7
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:56:55 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas

commit 3acb7e7
Author: Alex Rosengarten <alxr@google.com>
Date:   Wed Mar 4 15:56:42 2020 -0800

    XXS CL to add a hash field to the Schema proto (PolymerLabs#4752)

commit 9054c23
Merge: d15008e 337b7b2
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:55:27 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into s2k-schemas

commit 337b7b2
Author: Alex Rosengarten <alxr@google.com>
Date:   Wed Mar 4 15:55:06 2020 -0800

    recipe2plan 2/n: Validate Handles, Tests (PolymerLabs#4834)

    * creating structure of ts r2p script

    * Created fast CLI rapper for r2p script

    * WIP figuring out ways to resolve a recipe

    * fix sp

    * Passing lint, still WIP

    * Build macros for recipe2plan work

    * Lightweight iteration on recipe2plan, stubbed out tests

    * Revised method to find corresponding create handles

    * Added type info

    * Added method to get all handles to manifest + get all handles by Id

    * fix: no flatMap

    * - currently, fails since it cannot find associated stores for handles.

    * uncleaned, but working recipe resolution

    * Cleaned up for r2p, pt 1

    * stubbed out second test (short, short)

    * Squashed commit of the following:

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * updates to test, can't get two to fail

    * Squashed commit of the following:

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * can't get test to fail... hmm...

    * Squashed commit of the following:

    commit f6df54a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 15:44:32 2020 -0800

        tools/sigh lint

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * adjusting recipe copying to include triggers

    * Revert "adjusting recipe copying to include triggers"

    This reverts commit 9bc3734

    * four tests complete

    * Added test, fixed minor async issue

    * Squashed commit of the following:

    commit 4b05a28
    Merge: f4c1d26 99000c2
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:18:14 2020 -0800

        Merge branch 'r2p' of github.com:alxrsngrtn/arcs into r2p

    commit f4c1d26
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 99000c2
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 140111f
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:08:12 2020 -0800

        Removed flatMap, added TODOs and link to GH issue

    commit 5c6310d
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 11:02:34 2020 -0800

        Simplify service test pattern (PolymerLabs#4817)

        * Make a simple test lifecycle registry, instead of creating empty
        testactivity

        * Remove use of `runBlockingTest`: according to
        Kotlin/kotlinx.coroutines#1222, if the test
        results in coroutines being finished on other threads, it's possible to
        receive "This job has not yet completed" exceptions, even though the
        jobs were properly terminated. Since we don't need the delay-skipping
        properties of `runBlockingTest`, I think it's OK to use `runBlocking`.

    commit 472bc84
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:27:56 2020 -0800

        Improved build rules

    commit ca1ebf8
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:17:13 2020 -0800

        Impl suggestsions for r2p

    commit b5578ea
    Author: Jason Feinstein <jwf@google.com>
    Date:   Tue Mar 3 09:54:17 2020 -0800

        Add tests for dereferencing references to the HandleManagerTest(s) (PolymerLabs#4816)

        * Add dereferencing tests to the android handle manager test.

        * Add dereferencing tests to core HandleManager.

    commit a695797
    Author: Jason Feinstein <jwf@google.com>
    Date:   Mon Mar 2 18:14:07 2020 -0800

        Create RawEntityDereferencerTest, storage Reference-> CrdtEntity.Reference (PolymerLabs#4812)

        * Create RawEntityDereferencerTest, make storage Reference implement CrdtEntity.Reference.

        Also: Create ParcelableReference.

        * Add dep. Also, apparently read/writeBoolean is Q-only.

        * Just write null if there is no version map.

        * Just write null if there is no version map.

    commit ba7a107
    Author: Gogul Balakrishnan <bgogul@google.com>
    Date:   Mon Mar 2 17:51:35 2020 -0800

        Add a decoder for PrimitiveTypeProto and an option to disable android constraints in BUILD. (PolymerLabs#4793)

    commit f6df54a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 15:44:32 2020 -0800

        tools/sigh lint

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * fix ineq

    * existing tests passing

    * Updating invalid type test

    * Squashed commit of the following:

    commit b3d92d0
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:17:17 2020 -0800

        updated test name

    commit ba7884b
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:16:09 2020 -0800

        nested unit tests

    commit ef36fdf
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:15:00 2020 -0800

        renamed to tryResolve

    commit 5703f02
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:07:18 2020 -0800

        rm generator

    commit 658cb1a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 12:01:54 2020 -0800

        implemented more review suggestions

    commit 0056ee3
    Merge: cae89c3 0f0f82a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Wed Mar 4 11:54:23 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p

    commit 0f0f82a
    Author: Maria Mandlis <mmandlis@chromium.org>
    Date:   Wed Mar 4 09:33:29 2020 -0800

        add queryable capability (PolymerLabs#4794)

    commit d032c75
    Author: Maria Mandlis <mmandlis@chromium.org>
    Date:   Wed Mar 4 09:23:07 2020 -0800

        add creatimeTimestamp to entities (kt) (PolymerLabs#4823)

    commit 30267fc
    Author: jblebrun <jibbl@google.com>
    Date:   Wed Mar 4 09:09:39 2020 -0800

        Remove some unused deps (PolymerLabs#4831)

    commit c519181
    Author: Cameron Silvestrini <csilvestrini@users.noreply.github.com>
    Date:   Wed Mar 4 17:14:07 2020 +1100

        Increase size of DatabaseImplTest to medium (PolymerLabs#4827)

        * Increase size of DatabaseImplTest to medium

        Was a bit flaky.

        * Check flakiness

        * Revert presubmit tweak

    commit fa621c6
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 22:05:07 2020 -0800

        Fix a race condition when setting up `ServiceStore` message channel (PolymerLabs#4826)

        This seems to be the cause of the flakiness in `AndroidAllocatorTest`
        and `AndroidHandleManagerTest` (PolymerLabs#4781)

    commit 78bf4f0
    Author: Gogul Balakrishnan <bgogul@google.com>
    Date:   Tue Mar 3 18:00:49 2020 -0800

        Utility to convert type proto to field type (if possible). (PolymerLabs#4814)

    commit cae89c3
    Merge: 0360752 e587539
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 15:50:34 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p

    commit e587539
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 15:07:16 2020 -0800

        Apply simplified test pattern tests in `arcs.android.host` (PolymerLabs#4821)

        As in 5c6310d

    commit 1af1aac
    Author: Joshua Pratt <jp10010101010000@gmail.com>
    Date:   Wed Mar 4 09:48:15 2020 +1100

        Multinomials (PolymerLabs#4804)

        * multinomial

        * added tests

        * making constant readable again

        * reindent

        * added multivariate rearrangement tests

        * updated comment

        * pr comments:

        Co-authored-by: Ragav Sachdeva <ragavsachdeva007@gmail.com>

    commit 0360752
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:42:31 2020 -0800

        fix, bad comparison op

    commit 4b05a28
    Merge: f4c1d26 99000c2
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:18:14 2020 -0800

        Merge branch 'r2p' of github.com:alxrsngrtn/arcs into r2p

    commit f4c1d26
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 99000c2
    Merge: 140111f 5c6310d
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:14:35 2020 -0800

        Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
        Fixed lint errors

    commit 140111f
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 11:08:12 2020 -0800

        Removed flatMap, added TODOs and link to GH issue

    commit 5c6310d
    Author: jblebrun <jibbl@google.com>
    Date:   Tue Mar 3 11:02:34 2020 -0800

        Simplify service test pattern (PolymerLabs#4817)

        * Make a simple test lifecycle registry, instead of creating empty
        testactivity

        * Remove use of `runBlockingTest`: according to
        Kotlin/kotlinx.coroutines#1222, if the test
        results in coroutines being finished on other threads, it's possible to
        receive "This job has not yet completed" exceptions, even though the
        jobs were properly terminated. Since we don't need the delay-skipping
        properties of `runBlockingTest`, I think it's OK to use `runBlocking`.

    commit 472bc84
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:27:56 2020 -0800

        Improved build rules

    commit ca1ebf8
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Tue Mar 3 10:17:13 2020 -0800

        Impl suggestsions for r2p

    commit b5578ea
    Author: Jason Feinstein <jwf@google.com>
    Date:   Tue Mar 3 09:54:17 2020 -0800

        Add tests for dereferencing references to the HandleManagerTest(s) (PolymerLabs#4816)

        * Add dereferencing tests to the android handle manager test.

        * Add dereferencing tests to core HandleManager.

    commit a695797
    Author: Jason Feinstein <jwf@google.com>
    Date:   Mon Mar 2 18:14:07 2020 -0800

        Create RawEntityDereferencerTest, storage Reference-> CrdtEntity.Reference (PolymerLabs#4812)

        * Create RawEntityDereferencerTest, make storage Reference implement CrdtEntity.Reference.

        Also: Create ParcelableReference.

        * Add dep. Also, apparently read/writeBoolean is Q-only.

        * Just write null if there is no version map.

        * Just write null if there is no version map.

    commit ba7a107
    Author: Gogul Balakrishnan <bgogul@google.com>
    Date:   Mon Mar 2 17:51:35 2020 -0800

        Add a decoder for PrimitiveTypeProto and an option to disable android constraints in BUILD. (PolymerLabs#4793)

    commit f6df54a
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 15:44:32 2020 -0800

        tools/sigh lint

    commit bede8d9
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:52:02 2020 -0800

        Fixed build rule, simplified runtime

    commit 62dcc57
    Author: Alex Rosengarten <alxrsngrtn@google.com>
    Date:   Mon Mar 2 14:38:14 2020 -0800

        Fixed test, added TODO

    * fixed nested tests

    * rewording, reorganizing

    * assert more ergonomic

commit 298fdcc
Author: Maria Mandlis <mmandlis@chromium.org>
Date:   Wed Mar 4 15:29:37 2020 -0800

    update wasm readme (PolymerLabs#4832)

commit d15008e
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:11:14 2020 -0800

    fixed build

commit bbfc3d2
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 15:04:11 2020 -0800

    EntitySpec Interface can return a schema

commit 3b75c34
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 14:09:36 2020 -0800

    ktlint fix

commit 506457e
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 13:59:46 2020 -0800

    addField arguments are more ergonomic

commit 579b2e6
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 13:45:28 2020 -0800

    grooming schema registry

commit 3cb8023
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:39:30 2020 -0800

    rm unused field

commit 1cff99f
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:38:32 2020 -0800

    rm import

commit 855401c
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:31:14 2020 -0800

    no difference w/ wasm golden

commit 552e9ad
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:26:43 2020 -0800

    Added test for mapOf

commit 1430907
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:07:15 2020 -0800

    sigh lint

commit 9c22195
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 11:06:50 2020 -0800

    minor adjustments

commit 74ae770
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 10:58:20 2020 -0800

    Improved shape of registry

commit f69f99b
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 10:32:31 2020 -0800

    excluding wasm from generation

commit dcefb16
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 10:13:46 2020 -0800

    Generation adds to schema registry

commit f3b9ae5
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Wed Mar 4 07:57:15 2020 -0800

    packaging in a schema registry

commit 2ce6da2
Author: Alex Rosengarten <alxrsngrtn@google.com>
Date:   Tue Mar 3 22:30:39 2020 -0800

    First iteration of schema gen on entities
  • Loading branch information
@alxmrs
alxmrs committed Mar 5, 2020
1 parent aff420c commit c461ac4
Show file tree
Hide file tree
Showing 13 changed files with 330 additions and 41 deletions.
28 changes: 28 additions & 0 deletions java/arcs/core/data/SchemaRegistry.kt
View file
@@ -0,0 +1,28 @@
/*
* Copyright 2020 Google LLC.
*
* This code may only be used under the BSD style license found at
* http://polymer.github.io/LICENSE.txt
*
* Code distributed by Google as part of this project is also subject to an additional IP rights
* grant found at
* http://polymer.github.io/PATENTS.txt
*/
package arcs.core.data

typealias SchemaHash = String

/**
* A registry for generated [Schema]s.
*/
object SchemaRegistry {
private val schemas = mutableMapOf<SchemaHash, Schema>()

/** Store a [Schema] in the registry. */
fun register(schema: Schema) {
schemas[schema.hash] = schema
}

/** Given a [SchemaHash], return the [Schema] for that hash, if it exists. */
operator fun get(hash: SchemaHash) = schemas[hash]
}
1 change: 1 addition & 0 deletions java/arcs/core/data/proto/recipe.proto
View file
Expand Up @@ -121,6 +121,7 @@ message TypeVariableProto {
message SchemaProto {
repeated string names = 1;
map<string, TypeProto> fields = 2;
string hash = 3;
}

enum OPERATOR {
Expand Down
2 changes: 1 addition & 1 deletion java/arcs/core/storage/api/BUILD
View file
Expand Up @@ -12,7 +12,7 @@ arcs_kt_library(
srcs = glob(["*.kt"]),
deps = [
"//java/arcs/core/common",
"//java/arcs/core/data:rawentity",
"//java/arcs/core/data",
"//java/arcs/core/data/util:data-util",
"//third_party/kotlin/kotlinx_coroutines",
],
Expand Down
4 changes: 4 additions & 0 deletions java/arcs/core/storage/api/Entity.kt
View file
Expand Up @@ -13,6 +13,7 @@ package arcs.core.storage.api

import arcs.core.common.Referencable
import arcs.core.data.RawEntity
import arcs.core.data.Schema
import arcs.core.data.util.ReferencablePrimitive
import kotlin.IllegalArgumentException
import kotlin.reflect.KClass
Expand All @@ -38,6 +39,9 @@ interface EntitySpec<T : Entity> {
* TODO: replace this with kotlinx.serialization
*/
fun deserialize(data: RawEntity): T

/** Returns the corresponding [Schema] for the specified [Entity]. */
fun schema(): Schema
}

/**
Expand Down
20 changes: 17 additions & 3 deletions src/tools/schema2base.ts
View file
Expand Up @@ -15,8 +15,16 @@ import {Runtime} from '../runtime/runtime.js';
import {SchemaGraph, SchemaNode} from './schema2graph.js';
import {ParticleSpec} from '../runtime/particle-spec.js';

export type AddFieldOptions = Readonly<{
field: string;
typeChar: string;
isOptional?: boolean;
refClassName?: string;
isCollection?: boolean;
}>;

export interface ClassGenerator {
addField(field: string, typeChar: string, isOptional: boolean, refClassName: string|null): void;
addField(opts: AddFieldOptions): void;
generate(schemaHash: string, fieldCount: number): string;
}

Expand Down Expand Up @@ -78,14 +86,20 @@ export abstract class Schema2Base {
for (const [field, descriptor] of fields) {
if (descriptor.kind === 'schema-primitive') {
if (['Text', 'URL', 'Number', 'Boolean'].includes(descriptor.type)) {
generator.addField(field, descriptor.type[0], false, null);
generator.addField({field, typeChar: descriptor.type[0]});
} else {
throw new Error(`Schema type '${descriptor.type}' for field '${field}' is not supported`);
}
} else if (descriptor.kind === 'schema-reference') {
generator.addField(field, 'R', false, node.refs.get(field).name);
generator.addField({field, typeChar: 'R', refClassName: node.refs.get(field).name});
} else if (descriptor.kind === 'schema-collection' && descriptor.schema.kind === 'schema-reference') {
// TODO: support collections of references
} else if (descriptor.kind === 'schema-collection') {
const schema = descriptor.schema;
if (!['Text', 'URL', 'Number', 'Boolean'].includes(schema.type)) {
throw new Error(`Schema type '${schema.type}' for field '${field}' is not supported`);
}
generator.addField({field, typeChar: schema.type[0], isCollection: true});
} else {
throw new Error(`Schema kind '${descriptor.kind}' for field '${field}' is not supported`);
}
Expand Down
4 changes: 2 additions & 2 deletions src/tools/schema2cpp.ts
View file
Expand Up @@ -7,7 +7,7 @@
* subject to an additional IP rights grant found at
* http://polymer.github.io/PATENTS.txt
*/
import {Schema2Base, ClassGenerator} from './schema2base.js';
import {Schema2Base, ClassGenerator, AddFieldOptions} from './schema2base.js';
import {SchemaNode} from './schema2graph.js';
import {ParticleSpec} from '../runtime/particle-spec.js';
import {Type} from '../runtime/type.js';
Expand Down Expand Up @@ -113,7 +113,7 @@ class CppGenerator implements ClassGenerator {

constructor(readonly node: SchemaNode, readonly namespace: string) {}

addField(field: string, typeChar: string, isOptional: boolean, refClassName: string|null) {
addField({field, typeChar, refClassName, isOptional = false, isCollection = false}: AddFieldOptions) {
const fixed = fixName(field);
const valid = `${field}_valid_`;
let {type, defaultVal, isString} = typeMap[typeChar];
Expand Down
76 changes: 68 additions & 8 deletions src/tools/schema2kotlin.ts
View file
Expand Up @@ -7,7 +7,7 @@
* subject to an additional IP rights grant found at
* http://polymer.github.io/PATENTS.txt
*/
import {Schema2Base, ClassGenerator} from './schema2base.js';
import {Schema2Base, ClassGenerator, AddFieldOptions} from './schema2base.js';
import {SchemaNode} from './schema2graph.js';
import {ParticleSpec} from '../runtime/particle-spec.js';
import minimist from 'minimist';
Expand All @@ -28,10 +28,10 @@ const keywords = [
];

const typeMap = {
'T': {type: 'String', decodeFn: 'decodeText()', defaultVal: `""`},
'U': {type: 'String', decodeFn: 'decodeText()', defaultVal: `""`},
'N': {type: 'Double', decodeFn: 'decodeNum()', defaultVal: '0.0'},
'B': {type: 'Boolean', decodeFn: 'decodeBool()', defaultVal: 'false'},
'T': {type: 'String', decodeFn: 'decodeText()', defaultVal: `""`, schemaType: 'FieldType.Text'},
'U': {type: 'String', decodeFn: 'decodeText()', defaultVal: `""`, schemaType: 'FieldType.Text'},
'N': {type: 'Double', decodeFn: 'decodeNum()', defaultVal: '0.0', schemaType: 'FieldType.Number'},
'B': {type: 'Boolean', decodeFn: 'decodeBool()', defaultVal: 'false', schemaType: 'FieldType.Boolean'},
};

export class Schema2Kotlin extends Schema2Base {
Expand All @@ -54,7 +54,14 @@ package ${this.scope}
// Current implementation doesn't support references or optional field detection
import arcs.sdk.*
${this.opts.wasm ? 'import arcs.sdk.wasm.*' : 'import arcs.core.storage.api.toPrimitiveValue\nimport arcs.core.data.RawEntity\nimport arcs.core.data.util.toReferencable\nimport arcs.core.data.util.ReferencablePrimitive'}
${this.opts.wasm ?
`import arcs.sdk.wasm.*` :
`\
import arcs.sdk.Entity
import arcs.core.data.*
import arcs.core.data.util.toReferencable
import arcs.core.data.util.ReferencablePrimitive
import arcs.core.storage.api.toPrimitiveValue`}
`;
}

Expand Down Expand Up @@ -130,7 +137,7 @@ abstract class Abstract${particleName} : ${this.opts.wasm ? 'WasmParticleImpl' :
}
}

class KotlinGenerator implements ClassGenerator {
export class KotlinGenerator implements ClassGenerator {
fields: string[] = [];
fieldVals: string[] = [];
setFields: string[] = [];
Expand All @@ -144,11 +151,13 @@ class KotlinGenerator implements ClassGenerator {
fieldSerializes: string[] = [];
fieldDeserializes: string[] = [];
fieldsForToString: string[] = [];
singletonSchemaFields: string[] = [];
collectionSchemaFields: string[] = [];

constructor(readonly node: SchemaNode, private readonly opts: minimist.ParsedArgs) {}

// TODO: allow optional fields in kotlin
addField(field: string, typeChar: string, isOptional: boolean, refClassName: string|null) {
addField({field, typeChar, refClassName, isOptional = false, isCollection = false}: AddFieldOptions) {
// TODO: support reference types in kotlin
if (typeChar === 'R') return;

Expand Down Expand Up @@ -181,6 +190,46 @@ class KotlinGenerator implements ClassGenerator {
this.fieldSerializes.push(`"${field}" to ${fixed}.toReferencable()`);
this.fieldDeserializes.push(`${fixed} = data.singletons["${fixed}"].toPrimitiveValue(${type}::class, ${defaultVal})`);
this.fieldsForToString.push(`${fixed} = $${fixed}`);
if (isCollection) {
this.collectionSchemaFields.push(`"${field}" to ${typeMap[typeChar].schemaType}`);
} else {
this.singletonSchemaFields.push(`"${field}" to ${typeMap[typeChar].schemaType}`);
}
}

mapOf(items: string[]): string {
switch (items.length) {
case 0:
return `emptyMap()`;
case 1:
return `mapOf(${items[0]})`;
default:
return `\
mapOf(
${this.leftPad(items.join(',\n'), 4)}
)`;
}

}

createSchema(schemaHash: string): string {
const schemaNames = this.node.schema.names.map(n => `SchemaName("${n}")`);
return `\
Schema(
listOf(${schemaNames.join(',\n' + ' '.repeat(8))}),
SchemaFields(
singletons = ${this.leftPad(this.mapOf(this.singletonSchemaFields), 8, true)},
collections = ${this.leftPad(this.mapOf(this.collectionSchemaFields), 8, true)}
),
"${schemaHash}"
)`;
}

leftPad(input: string, indent: number, skipFirst: boolean = false) {
return input
.split('\n')
.map((line: string, idx: number) => (idx === 0 && skipFirst) ? line : ' '.repeat(indent) + line)
.join('\n');
}

generate(schemaHash: string, fieldCount: number): string {
Expand Down Expand Up @@ -256,7 +305,18 @@ ${this.opts.wasm ? `
}
class ${name}_Spec() : ${this.getType('EntitySpec')}<${name}> {
${this.opts.wasm ? '' : `\
companion object {
val schema = ${this.leftPad(this.createSchema(schemaHash), 8, true)}
init {
SchemaRegistry.register(schema)
}
}
override fun schema() = schema
`}
override fun create() = ${name}()
${!this.opts.wasm ? `
override fun deserialize(data: RawEntity): ${name} {
Expand Down
18 changes: 9 additions & 9 deletions src/tools/storage-key-recipe-resolver.ts
View file
Expand Up @@ -32,13 +32,14 @@ export class StorageKeyRecipeResolver {
/**
* Produces resolved recipes with storage keys.
*
* TODO(alxr): Apply to long-running recipes appropriately.
* TODO(#4818) Add passes to assign storage keys.
* @throws Error if recipe fails to resolve on first or second pass.
* @yields Resolved recipes with storage keys
*/
async resolve(): Promise<Recipe[]> {
const recipes = [];
for (const recipe of this.runtime.context.allRecipes) {
this.validateHandles(recipe);
const arc = this.runtime.newArc(this.getArcId(recipe), ramDiskStorageKeyPrefixForTest());
const opts = {errors: new Map<Recipe | RecipeComponent, string>()};
const resolved = await this.tryResolve(recipe, arc, opts);
Expand All @@ -63,7 +64,8 @@ export class StorageKeyRecipeResolver {
*/
async tryResolve(recipe: Recipe, arc: Arc, opts?: IsValidOptions): Promise<Recipe | null> {
const normalized = recipe.clone();
normalized.normalize();
const successful = normalized.normalize(opts);
if (!successful) return null;
if (normalized.isResolved()) return normalized;

return await (new RecipeResolver(arc).resolve(recipe, opts));
Expand Down Expand Up @@ -97,14 +99,14 @@ export class StorageKeyRecipeResolver {
}

/**
* TODO(#4818) method to match `map` and `copy` fated handles with storage keys from `create` handles.
* Checks that handles are existent, disambiguous, and initiated by a long-running arc.
*
* @throws when a mapped handle is associated with too many stores (ambiguous mapping).
* @throws when a mapped handle isn't associated with any store (no matches found).
* @throws when handle is mapped to a handle from an ephemeral recipe.
* @throws when a map or copy handle is associated with too many stores (ambiguous mapping).
* @throws when a map or copy handle isn't associated with any store (no matches found).
* @throws when a map or copy handle is associated with a handle from an ephemeral recipe.
* @param recipe long-running or ephemeral recipe
*/
matchKeysToHandles(recipe: Recipe) {
validateHandles(recipe: Recipe) {
recipe.handles
.filter(h => h.fate === 'map' || h.fate === 'copy')
.forEach(handle => {
Expand All @@ -121,8 +123,6 @@ export class StorageKeyRecipeResolver {
if (!match.recipe.isLongRunning) {
throw Error(`Handle ${handle.localName} mapped to ephemeral handle ${match.localName}.`);
}

handle.storageKey = match.storageKey;
});
}
}
44 changes: 42 additions & 2 deletions src/tools/tests/goldens/generated-schemas.jvm.kt
View file
Expand Up @@ -9,10 +9,11 @@ package arcs.sdk
// Current implementation doesn't support references or optional field detection

import arcs.sdk.*
import arcs.core.storage.api.toPrimitiveValue
import arcs.core.data.RawEntity
import arcs.sdk.Entity
import arcs.core.data.*
import arcs.core.data.util.toReferencable
import arcs.core.data.util.ReferencablePrimitive
import arcs.core.storage.api.toPrimitiveValue

class GoldInternal1() : Entity {

Expand Down Expand Up @@ -71,6 +72,23 @@ class GoldInternal1() : Entity {

class GoldInternal1_Spec() : EntitySpec<GoldInternal1> {

companion object {
val schema = Schema(
listOf(),
SchemaFields(
singletons = mapOf("val" to FieldType.Text),
collections = emptyMap()
),
"485712110d89359a3e539dac987329cd2649d889"
)

init {
SchemaRegistry.register(schema)
}
}

override fun schema() = schema

override fun create() = GoldInternal1()

override fun deserialize(data: RawEntity): GoldInternal1 {
Expand Down Expand Up @@ -179,6 +197,28 @@ class Gold_Data() : Entity {

class Gold_Data_Spec() : EntitySpec<Gold_Data> {

companion object {
val schema = Schema(
listOf(),
SchemaFields(
singletons = mapOf(
"num" to FieldType.Number,
"txt" to FieldType.Text,
"lnk" to FieldType.Text,
"flg" to FieldType.Boolean
),
collections = emptyMap()
),
"d8058d336e472da47b289eafb39733f77eadb111"
)

init {
SchemaRegistry.register(schema)
}
}

override fun schema() = schema

override fun create() = Gold_Data()

override fun deserialize(data: RawEntity): Gold_Data {
Expand Down

0 comments on commit c461ac4

Please sign in to comment.