Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connect: intentions are now managed as a new config entry kind "service-intentions" #8834

Merged
merged 3 commits into from
Oct 6, 2020

Conversation

rboyer
Copy link
Member

@rboyer rboyer commented Oct 6, 2020

  • Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
    copies of consul will not see new config entries it doesn't understand
    replicate down.

  • Add shim conversion code so that the old API/CLI method of interacting
    with intentions will continue to work so long as none of these are
    edited via config entry endpoints. Almost all of the read-only APIs will
    continue to function indefinitely.

  • Add new APIs that operate on individual intentions without IDs so that
    the UI doesn't need to implement CAS operations.

  • Add a new serf feature flag indicating support for
    intentions-as-config-entries.

  • The old line-item intentions way of interacting with the state store
    will transparently flip between the legacy memdb table and the config
    entry representations so that readers will never see a hiccup during
    migration where the results are incomplete. It uses a piece of system
    metadata to control the flip.

  • The primary datacenter will begin migrating intentions into config
    entries on startup once all servers in the datacenter are on a version
    of Consul with the intentions-as-config-entries feature flag. When it is
    complete the old state store representations will be cleared. We also
    record a piece of system metadata indicating this has occurred. We use
    this metadata to skip ALL of this code the next time the leader starts
    up.

  • The secondary datacenters continue to run the old intentions
    replicator until all servers in the secondary DC and primary DC support
    intentions-as-config-entries (via serf flag). Once this condition it met
    the old intentions replicator ceases.

  • The secondary datacenters replicate the new config entries as they are
    migrated in the primary. When they detect that the primary has zeroed
    it's old state store table it waits until all config entries up to that
    point are replicated and then zeroes its own copy of the old state store
    table. We also record a piece of system metadata indicating this has
    occurred. We use this metadata to skip ALL of this code the next time
    the leader starts up.

…ce-intentions"

- Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
copies of consul will not see new config entries it doesn't understand
replicate down.

- Add shim conversion code so that the old API/CLI method of interacting
with intentions will continue to work so long as none of these are
edited via config entry endpoints. Almost all of the read-only APIs will
continue to function indefinitely.

- Add new APIs that operate on individual intentions without IDs so that
the UI doesn't need to implement CAS operations.

- Add a new serf feature flag indicating support for
intentions-as-config-entries.

- The old line-item intentions way of interacting with the state store
will transparently flip between the legacy memdb table and the config
entry representations so that readers will never see a hiccup during
migration where the results are incomplete. It uses a piece of system
metadata to control the flip.

- The primary datacenter will begin migrating intentions into config
entries on startup once all servers in the datacenter are on a version
of Consul with the intentions-as-config-entries feature flag. When it is
complete the old state store representations will be cleared. We also
record a piece of system metadata indicating this has occurred. We use
this metadata to skip ALL of this code the next time the leader starts
up.

- The secondary datacenters continue to run the old intentions
replicator until all servers in the secondary DC and primary DC support
intentions-as-config-entries (via serf flag). Once this condition it met
the old intentions replicator ceases.

- The secondary datacenters replicate the new config entries as they are
migrated in the primary. When they detect that the primary has zeroed
it's old state store table it waits until all config entries up to that
point are replicated and then zeroes its own copy of the old state store
table. We also record a piece of system metadata indicating this has
occurred. We use this metadata to skip ALL of this code the next time
the leader starts up.
@rboyer rboyer added the theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies label Oct 6, 2020
@rboyer rboyer added this to the 1.9 milestone Oct 6, 2020
@rboyer rboyer requested a review from a team October 6, 2020 16:36
@rboyer rboyer self-assigned this Oct 6, 2020
@github-actions github-actions bot added theme/api Relating to the HTTP API interface theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/envoy/xds Related to Envoy support labels Oct 6, 2020
Copy link
Contributor

@freddygv freddygv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: assuming nothing comes up in the integration tests run

@rboyer rboyer merged commit a2a8e9c into master Oct 6, 2020
@rboyer rboyer deleted the intention-configs branch October 6, 2020 18:24
kaxcode pushed a commit that referenced this pull request Oct 7, 2020
…ce-intentions" (#8834)

- Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
copies of consul will not see new config entries it doesn't understand
replicate down.

- Add shim conversion code so that the old API/CLI method of interacting
with intentions will continue to work so long as none of these are
edited via config entry endpoints. Almost all of the read-only APIs will
continue to function indefinitely.

- Add new APIs that operate on individual intentions without IDs so that
the UI doesn't need to implement CAS operations.

- Add a new serf feature flag indicating support for
intentions-as-config-entries.

- The old line-item intentions way of interacting with the state store
will transparently flip between the legacy memdb table and the config
entry representations so that readers will never see a hiccup during
migration where the results are incomplete. It uses a piece of system
metadata to control the flip.

- The primary datacenter will begin migrating intentions into config
entries on startup once all servers in the datacenter are on a version
of Consul with the intentions-as-config-entries feature flag. When it is
complete the old state store representations will be cleared. We also
record a piece of system metadata indicating this has occurred. We use
this metadata to skip ALL of this code the next time the leader starts
up.

- The secondary datacenters continue to run the old intentions
replicator until all servers in the secondary DC and primary DC support
intentions-as-config-entries (via serf flag). Once this condition it met
the old intentions replicator ceases.

- The secondary datacenters replicate the new config entries as they are
migrated in the primary. When they detect that the primary has zeroed
it's old state store table it waits until all config entries up to that
point are replicated and then zeroes its own copy of the old state store
table. We also record a piece of system metadata indicating this has
occurred. We use this metadata to skip ALL of this code the next time
the leader starts up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme/api Relating to the HTTP API interface theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/envoy/xds Related to Envoy support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants