Viz Chart design: Affinity vs Tolerations

[philomory]

What problem are you trying to solve?

In trying to use the linkerd-viz chart, I'm trying to assign node affinity, as well as tolerations, to the pods of the deployments created by the chart. As with the linkerd-control-plane chart, you can set node affinity by setting the top-level nodeAffinity value (indeed this is the only way you can set node affinity), which sets node affinity for the pods of every deployment created by the chart. And, just like the linkerd-control-plane chart, there's a top-level tolerations key.

Unfortunately, unlike the linkerd-control-plane chart, the top-level tolerations key does not set the tolerations for the pods of every deployment created by the chart. It sets the tolerations for 60% of them (three out of five). For the other two deployments (metrics-api and prometheus), the tolerations have to be set using dedicated properties, e.g. metricsApi.tolerations and prometheus.tolerations.

This isn't some egregious, show-stopping issue, by any stretch, but it's a confusing design choice that's prone to lead to errors, and isn't particularly obvious from the documentation (I mean you can see that the metricsAPI.tolerations and prometheus.tolerations properties exist, but there's nothing explicitly pointing out that this chart behaves differently than linkerd-control-plane, nor that nodeAffinity behaves differently than nodeSelector and tolerations).

How should the problem be solved?

Either the metrics-api and prometheus deployments should have dedicated nodeAffinity settings to go with their dedicated nodeSelector and tolerations settings, or they should use the top-level nodeSelector and tolerations settings along with the top-level nodeAffinity setting.

I don't have a strong opinion as to which; in principal I suppose the ideal case would be for every deployment to have its own ${thatDeployment}.nodeAffinity, ${thatDeployment}.nodeSelector, and ${thatDeployment}.tolerations values, which if left unspecified result in defaulting to the equivalent top-level keys. But if that seems more trouble than it's worth, simply picking one way of handling these and sticking to it would suffice.

Any alternatives you've considered?

Simply updating the documentation to explicitly call out the current behavior would also work. Although, speaking of updating the documentation, the auto-generated value documentation states that the various tolerations values are supposed to be strings when in actual fact they need to be arrays of maps.

How would users interact with this feature?

No response

Would you like to work on this feature?

maybe

[philomory]

I'm not sure if this should be a separate issue or if it can be rolled into this one, but I've just realized that, in addition to the other issues already noted, the web deployment of the linkerd-viz chart can't actually have any affinity set whatsoever.