Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Peering Mesh Gateway Updates for GA #15344

Merged
merged 6 commits into from Nov 14, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
24,286 changes: 12,145 additions & 12,141 deletions .github/go_test_coverage.txt

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion website/content/api-docs/agent/service.mdx
Expand Up @@ -635,7 +635,7 @@ service definition keys for compatibility with the config file format.
- `Kind` `(string: "")` - The kind of service. Defaults to "" which is a
typical Consul service. This value may also be "connect-proxy" for
[Connect](/docs/connect) proxies representing another service,
"mesh-gateway" for instances of a [mesh gateway](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters),
"mesh-gateway" for instances of a [mesh gateway](/docs/connect/gateways/mesh-gateway#connect-proxy-configuration),
"terminating-gateway" for instances of a [terminating gateway](/docs/connect/gateways/terminating-gateway),
or "ingress-gateway" for instances of a [ingress gateway](/docs/connect/gateways/ingress-gateway).

Expand Down
2 changes: 1 addition & 1 deletion website/content/api-docs/discovery-chain.mdx
Expand Up @@ -93,7 +93,7 @@ The table below shows this endpoint's support for
parameter.

- `OverrideMeshGateway` `(MeshGatewayConfig: <optional>)` - Overrides the final
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#connect-proxy-configuration)
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway#connect-proxy-configuration)
for this any service resolved in the compiled chain.

This value comes from either the [proxy
Expand Down
6 changes: 2 additions & 4 deletions website/content/docs/architecture/index.mdx
Expand Up @@ -105,10 +105,8 @@ In the following diagram, the servers in each data center participate in a WAN g
</Tab>
</Tabs>

### Cluster peering (beta)
### Cluster peering

You can create peering connections between two or more independent clusters so that services deployed to different datacenters or admin partitions can communicate. An [admin partition](/docs/enterprise/admin-partitions) is a feature in Consul Enterprise that enables you to define isolated network regions that use the same Consul servers. In the cluster peering model, you create a token in one of the datacenters or partitions and configure another datacenter or partition to present the token to establish the connection.

-> **Cluster peering is currently in beta:** Functionality associated with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support.

Refer to [What is Cluster Peering?](/docs/connect/cluster-peering) for additional information.
Refer to [What is Cluster Peering?](/docs/connect/cluster-peering) for additional information.
Expand Up @@ -8,22 +8,20 @@ description: >-

# Create and Manage Cluster Peering Connections

~> **Cluster peering is currently in beta:** Functionality associated with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support.<br /><br />Cluster peering is not currently available in the HCP Consul offering.

A peering token enables cluster peering between different datacenters. Once you generate a peering token, you can use it to establish a connection between clusters. Then you can export services and create intentions so that peered clusters can call those services.

## Create a peering connection

Cluster peering is not enabled by default on Consul servers. To peer clusters, you must first configure all Consul servers so that `peering` is `enabled` and the gRPC port(8502) accepts traffic from the peering cluster (e.g., `client_addr="0.0.0.0"`). For additional information, refer to [Configuration Files](/docs/agent/config/config-files).
Cluster peering is enabled by default on Consul servers as of 1.14. For additional information, like disabling peering, refer to [Configuration Files](/docs/agent/config/config-files).

After enabling peering for all Consul servers, complete the following steps in order:
Use the following steps to create a peering connection:

1. Create a peering token
1. Establish a connection between clusters
1. Export services between clusters
1. Authorize services for peers

You can generate peering tokens and initiate connections on any available agent using either the API or the Consul UI. If you use the API, we recommend performing these operations through a client agent in the partition you want to connect.
You can generate peering tokens and initiate connections on any available agent using either the API, CLI, or the Consul UI. If you use the API or CLI, we recommend performing these operations through a client agent in the partition you want to connect.

The UI does not currently support exporting services between clusters or authorizing services for peers.

Expand Down
20 changes: 3 additions & 17 deletions website/content/docs/connect/cluster-peering/index.mdx
Expand Up @@ -7,8 +7,6 @@ description: >-

# What is Cluster Peering?

~> **Cluster peering is currently in beta**: Functionality associated with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support. <br/><br/>Cluster peering is not currently available in the HCP Consul offering.

You can create peering connections between two or more independent clusters so that services deployed to different partitions or datacenters can communicate.

## Overview
Expand Down Expand Up @@ -43,23 +41,11 @@ Regardless of whether you connect your clusters through WAN federation or cluste
| Forwards service requests for service discovery | &#9989; | &#10060; |
| Shares key/value stores | &#9989; | &#10060; |

## Beta release features and constraints

The cluster peering beta release includes the following features and functionality:

- **Consul v1.14 beta only**: Dynamic traffic control with a service resolver config entry can target failover and redirects to service instances in a peered cluster.
- Consul datacenters that are already federated stay federated. You do not need to migrate WAN federated clusters to cluster peering.
- Mesh gateways for _service to service traffic_ between clusters are available. For more information on configuring mesh gateways across peers, refer to [Service-to-service Traffic Across Peered Clusters](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-peers).
- You can generate peering tokens, establish, list, read, and delete peerings, and manage intentions for peering connections with both the API and the UI.
- You can configure [transparent proxies](/docs/connect/transparent-proxy) for peered services.
- You can use the [`peering` rule for ACL enforcement](/docs/security/acl/acl-rules#peering) of peering APIs.
## Important Cluster Peering Constraints

Not all features and functionality are available in the beta release. In particular, consider the following technical constraints:
Consider the following technical constraints:

- Mesh gateways for _server to server traffic_ are not available.
- Services with node, instance, and check definitions totaling more than 50MB cannot be exported to a peer.
- The `service-splitter` and `service-router` configuration entry kinds cannot directly target a peer. To split or route traffic to a service instance on a peer, you must supplement your desired dynamic routing definition with a `service-resolver` config entry on the dialing cluster. Refer to [L7 traffic management between peers](/docs/connect/cluster-peering/create-manage-peering#L7-traffic) for more information.
- Two admin partitions in the same datacenter cannot be peered. Use [`exported-services`](/docs/connect/config-entries/exported-services#exporting-services-to-peered-clusters) directly.
- The `consul intention` CLI command is not supported. To manage intentions that specify services in peered clusters, use [configuration entries](/docs/connect/config-entries/service-intentions).
- Accessing key/value stores across peers is not supported.
- Because non-Enterprise Consul instances are restricted to the `default` namespace, Consul Enterprise instances cannot export services from outside of the `default` namespace to non-Enterprise peers.
- Cross-cluster mesh gateways are supported in `remote` mode only.
14 changes: 5 additions & 9 deletions website/content/docs/connect/cluster-peering/k8s.mdx
Expand Up @@ -7,10 +7,6 @@ description: >-

# Cluster Peering on Kubernetes

~> **Cluster peering is currently in beta:** Functionality associated
with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in
beta may have performance issues, scaling issues, and limited support.<br/><br/>Cluster peering is not currently available in the HCP Consul offering.

To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions (CRDs) for each side of the peering.

The following CRDs are used to create and manage a peering connection:
Expand All @@ -26,9 +22,9 @@ As of Consul v1.14, you can also [implement service failovers and redirects to c

You must implement the following requirements to create and use cluster peering connections with Kubernetes:

- Consul v1.13.1 or later
- Consul v1.14.0 or later
- At least two Kubernetes clusters
- The installation must be running on Consul on Kubernetes version 0.47.1 or later
- The installation must be running on Consul on Kubernetes version 1.0.0 or later

### Prepare for installation

Expand All @@ -53,7 +49,7 @@ Complete the following procedure after you have provisioned a Kubernetes cluster
```yaml
global:
name: consul
image: "hashicorp/consul:1.13.1"
image: "hashicorp/consul:1.14.0"
peering:
enabled: true
connectInject:
Expand Down Expand Up @@ -88,7 +84,7 @@ Install Consul on Kubernetes by using the CLI to apply `values.yaml` to each clu
```

```shell-session
$ helm install ${HELM_RELEASE_NAME} hashicorp/consul --create-namespace --namespace consul --version "0.47.1" --values values.yaml --kube-context $CLUSTER1_CONTEXT
$ helm install ${HELM_RELEASE_NAME} hashicorp/consul --create-namespace --namespace consul --version "1.0.0" --values values.yaml --kube-context $CLUSTER1_CONTEXT
```

1. In `cluster-02`, run the following commands:
Expand All @@ -98,7 +94,7 @@ Install Consul on Kubernetes by using the CLI to apply `values.yaml` to each clu
```

```shell-session
$ helm install ${HELM_RELEASE_NAME} hashicorp/consul --create-namespace --namespace consul --version "0.47.1" --values values.yaml --kube-context $CLUSTER2_CONTEXT
$ helm install ${HELM_RELEASE_NAME} hashicorp/consul --create-namespace --namespace consul --version "1.0.0" --values values.yaml --kube-context $CLUSTER2_CONTEXT
```

## Create a peering connection for Consul on Kubernetes
Expand Down
Expand Up @@ -344,7 +344,7 @@ spec:
name: 'MeshGateway',
type: 'MeshGatewayConfig: <optional>',
description: `Controls the default
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#connect-proxy-configuration)
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway#connect-proxy-configuration)
for all proxies. Added in v1.6.0.`,
children: [
{
Expand Down
Expand Up @@ -444,7 +444,7 @@ represents a location outside the Consul cluster. They can be dialed directly wh
name: 'MeshGateway',
type: 'MeshGatewayConfig: <optional>',
description: `Controls the default
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#connect-proxy-configuration)
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway#connect-proxy-configuration)
for this upstream.`,
children: [
{
Expand Down Expand Up @@ -595,7 +595,7 @@ represents a location outside the Consul cluster. They can be dialed directly wh
name: 'MeshGateway',
type: 'MeshGatewayConfig: <optional>',
description: `Controls the default
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#connect-proxy-configuration)
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-wan-datacenters#connect-proxy-configuration)
for this upstream.`,
children: [
{
Expand Down Expand Up @@ -753,7 +753,7 @@ represents a location outside the Consul cluster. They can be dialed directly wh
name: 'MeshGateway',
type: 'MeshGatewayConfig: <optional>',
description: `Controls the default
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#connect-proxy-configuration)
[mesh gateway configuration](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-wan-datacenters#connect-proxy-configuration)
for this service. Added in v1.6.0.`,
children: [
{
Expand Down
8 changes: 5 additions & 3 deletions website/content/docs/connect/connectivity-tasks.mdx
Expand Up @@ -19,13 +19,15 @@ in different clouds or runtime environments where general interconnectivity betw
isn't feasible. One scenario where this is useful is when connecting networks with overlapping IP address space.

These gateways operate by sniffing the SNI header out of the mTLS connection and then routing the connection to the
appropriate destination based on the server name requested. The data within the mTLS session is not decrypted by
the Gateway.
Comment on lines -22 to -23
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not true for peering.

appropriate destination based on the server name requested.

As of Consul 1.8.0, mesh gateways can also forward gossip and RPC traffic between Consul servers.
This is enabled by [WAN federation via mesh gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways).

For more information about mesh gateways, review the [complete documentation](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters)
As of Consul 1.14.0, mesh gateways can route both data-plane (service-to-service) and control-plane (consul-to-consul) traffic for peered clusters.
See [Mesh Gateways for Peering Control Plane Traffic](/docs/connect/gateways/mesh-gateway/peering-via-mesh-gateways)

For more information about mesh gateways, review the [complete documentation](/docs/connect/gateways/mesh-gateway)
and the [mesh gateway tutorial](https://learn.hashicorp.com/tutorials/consul/service-mesh-gateways).

![Mesh Gateway Architecture](/img/mesh-gateways.png)
Expand Down
6 changes: 4 additions & 2 deletions website/content/docs/connect/gateways/index.mdx
Expand Up @@ -23,13 +23,15 @@ Mesh gateways enable service mesh traffic to be routed between different Consul
in different clouds or runtime environments where general interconnectivity between all services in all datacenters
isn't feasible.

They operate by sniffing and extracting the server name indication (SNI) header from the service mesh session and routing the connection to the appropriate destination based on the server name requested. The gateway does not decrypt the data within the mTLS session.
They operate by sniffing and extracting the server name indication (SNI) header from the service mesh session and routing the connection to the appropriate destination based on the server name requested.

Mesh gateways enable the following scenarios:

* **Federate multiple datacenters across a WAN**. Since Consul 1.8.0, mesh gateways can forward gossip and RPC traffic between Consul servers. See [WAN federation via mesh gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways) for additional information.
- **Service-to-service communication across datacenters**. Refer to [Enabling Service-to-service Traffic Across Datacenters](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters) for additional information.
- **Service-to-service communication across WAN-federated datacenters**. Refer to [Enabling Service-to-service Traffic Across Datacenters](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-wan-datacenters) for additional information.
- **Service-to-service communication across admin partitions**. Since Consul 1.11.0, you can create administrative boundaries for single Consul deployments called "admin partitions". You can use mesh gateways to facilitate cross-partition communication. Refer to [Enabling Service-to-service Traffic Across Admin Partitions](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-partitions) for additional information.
- **Bridge multiple datacenters using Cluster Peering**. Since Consul 1.14.0, mesh gateways can be used to route peering control-plane traffic between peered Consul Servers. See [Mesh Gateways for Peering Control Plane Traffic](/docs/connect/gateways/mesh-gateway/peering-via-mesh-gateways) for more information.
- **Service-to-service communication across peered datacenters**. Refer to [Mesh Gateways between Peered Clusters](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-peers) for more information.

-> **Mesh gateway tutorial**: Follow the [mesh gateway tutorial](https://learn.hashicorp.com/tutorials/consul/service-mesh-gateways) to learn concepts associated with mesh gateways.

Expand Down