-
Notifications
You must be signed in to change notification settings - Fork 117
Add append
, create
and rename enrich
, additive
to upsert
, update
destination sync options
#2998
base: main
Are you sure you want to change the base?
Conversation
19acdbf
to
612c39c
Compare
…ync modes Add destination configLoader test
9e53bd4
to
0e45c91
Compare
core/src/models/Destination.ts
Outdated
displayName: "Additive", | ||
upsert: { | ||
key: "upsert", | ||
displayName: "Upsert", | ||
description: "Sync all records, but do not delete (create, update)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking we should also update these descriptions away from the word "sync". Any suggestions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Export
perhaps?
@@ -860,6 +868,28 @@ describe("models/destination - with custom exportRecord plugin", () => { | |||
await record.destroy(); | |||
}); | |||
|
|||
test("Append syncMode only allows creating records", async () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
await record.destroy(); | ||
}); | ||
|
||
test("Create syncMode only allows creating records", async () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
core/src/models/Destination.ts
Outdated
append: { | ||
key: "append", | ||
displayName: "Append", | ||
description: "Create new records (create)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
description: "Create new records (create)", | |
description: "Always create new records (create)", |
log( | ||
`[ config ] Found syncMode "${syncMode}" in Destination config "${configId}". "${syncMode}" is still supported but value should be updated to new name: "${newSyncMode}".`, | ||
"warning" | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a DeprecationHelper to use for this kind of thing!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reminder about the Deprecation
utility
create: true, | ||
update: false, | ||
delete: false, | ||
description: "Create new records, but only when they don't exist (create)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can these descriptions be retrieved from core? The boolean flags seem to be the same as well...
Fix invalid syncModes in snowflake connection
823c3c0
to
1e61b2d
Compare
Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use the Deprecation
utility in the config loader, and this is good-to-go!
description: values.description, | ||
}); | ||
|
||
export const BatchSyncModeData: Record<BatchSyncMode, BatchSyncModeDataValues> = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
much nicer!
log( | ||
`[ config ] Found syncMode "${syncMode}" in Destination config "${configId}". "${syncMode}" is still supported but value should be updated to new name: "${newSyncMode}".`, | ||
"warning" | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reminder about the Deprecation
utility
(re-ran a flaky test) |
Change description
This PR adds the ability to support
append
orcreate
(create: true, update: false, delete: false
) in destination apps. It also migrates theenrich
andadditive
syncMode names to the more commonly known termsupdate
andupsert
.The db will migrate the syncModes to the new values. In config files,
enrich
andadditive
are still supported but a warning is shown, and will be updated to the new value if the user updates the config via the UI.All plugins have been updated to use the new terms.
Checklists
Development
Impact
Please explain any security, performance, migration, or other impacts if relevant:
Migrations considered:
grouparoo validate
) and will continue to support the legacy values. However, if the user saves the config via the UI, the values are replaced with the new values.Code review