Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xds/federation: support federation in LRS #5128

Merged
merged 4 commits into from Jan 26, 2022
Merged

Conversation

menghanl
Copy link
Contributor

  • update xdsclient's LoadReport method to take a ServerConfig instead of just a string
  • update clusterresolver and clusterimpl balancer config to take a full ServerConfig as LRS server
  • update CDS resource update LRS server field from a string to an enum (can be make an interface to support more types later)
  • update CDS balancer to read ServerConfig from xdsclient bootstrap and populate child policy config
  • cleanup bootstrap config json handling so the struct can be reused
    • change json unmarshal from taking a list (this is what used in bootstrap json) to taking an item

RELEASE NOTES:

  • xds: support federation

var servers []*xdsServer
if err := json.Unmarshal(data, &servers); err != nil {
return fmt.Errorf("xds: json.Unmarshal(data) for field xds_servers failed during bootstrap: %v", err)
// MarshalJSON marshal the ServerConfig to json.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/marshal/marshals/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 99 to 101
if sc == nil {
return ""
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When can this be nil? Would a comment for the same be useful?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deleted this.

In an Equal(), I was comparing String() of this config.
I changed that Equal() to check for nil first.

Comment on lines 342 to 352
var servers []*ServerConfig
if err := json.Unmarshal(v, &servers); err != nil {
return nil, fmt.Errorf("xds: json.Unmarshal(data) for field xds_servers failed during bootstrap: %v", err)
}
if len(servers) < 1 {
return nil, fmt.Errorf("xds: bootstrap file parsing failed during bootstrap: file doesn't contain any management server to connect to")
}
config.XDSServer = servers[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about having a type like this:

type xdsServers []*ServerConfig

func (xs *xdsServers) UnmarshalJSON(data []byte) error {
	if err := json.Unmarshal(v, &xs); err != nil {
		return nil, fmt.Errorf("xds: json.Unmarshal(data) for field xds_servers failed during bootstrap: %v", err)
	}
	if len(xds) < 1 {
		return nil, fmt.Errorf("xds: bootstrap file parsing failed during bootstrap: file doesn't contain any management server to connect to")
	}
	return nil
}

This can be used here and in Authority.UnmarshalJSON, thereby reducing some repeated code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My attempt to make it a type failed, because of infinite recursion.
I ended up making it a func.

//
// The same options used for creating the Client will be used (including
// NodeProto, and dial options if necessary).
// ReportLoad starts an load reporting stream to the given server. All load
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/starts an/starts a/

s/All load report/All load reports/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

xds/internal/xdsclient/loadreport.go Show resolved Hide resolved
SecurityCfg: sc,
MaxRequests: circuitBreakersFromCluster(cluster),
LBPolicy: lbPolicy,
}

// Note that this is different from the gRFC (gRFC says to include the full
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Add gFRC number in the comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -35,6 +37,18 @@ const (
ClusterTypeAggregate
)

// ClusterLRSServerConfigType is the type of LRS server config.
type ClusterLRSServerConfigType int
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand what this type offers over the existing EnableLRS bool field. Is this a forward looking change to support something in the future?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is a forward looking change.

The spec actually says this field should be the authority service config itself. But that adds complexity, because this function doesn't have access to the xdsclient or the bootstrap file.

I think making it as close to the proto message as possible is future proof.

// from.
LoadReportingServerName *string `json:"lrsLoadReportingServerName,omitempty"`

// LoadReportingServerName *string `json:"lrsLoadReportingServerName,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this line?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

MaxConcurrentRequests: newUint32(testMaxRequests),
Type: DiscoveryMechanismTypeEDS,
EDSServiceName: testEDSServcie,
Cluster: testClusterName,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need any negative tests for this field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you meant here.
A test with an invalid JSON?

@easwars easwars assigned menghanl and unassigned easwars Jan 20, 2022
@menghanl menghanl assigned easwars and unassigned menghanl Jan 21, 2022
// full ServerConfig{URL,creds,server feature} here). This information is
// not available here, because this function doesn't have access to the
// xdsclient bootstrap information now (can be added if necessary). The
// ServerConfig will be read and populate by the CDS balancer when
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/populate/populated/

@easwars easwars assigned menghanl and unassigned easwars Jan 25, 2022
- update xdsclient's LoadReport method to take a ServerConfig instead of just a string
- update clusterresolver and clusterimpl balancer config to take a full ServerConfig as LRS server
- update CDS resource update LRS server field from a string to an enum (can be make an interface to support more types later)
- update CDS balancer to read ServerConfig from xdsclient bootstrap and populate child policy config
- cleanup bootstrap config json handling so the struct can be reused
  - change json unmarshal from taking a list (this is what used in bootstrap json) to taking an item
@menghanl menghanl merged commit 0a68f8a into grpc:master Jan 26, 2022
@menghanl menghanl deleted the xds_fed_lrs branch January 26, 2022 19:39
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants