Unable to configure tolerations for kcp, etcd and front proxy

[martivo]

It is not possible to create toleration configurations for front-proxy, kcp and etcd Pods. It is possible to configure affinity but this is usually combined with a toleration in case there are dedicated nodes for some pods.

Propose to add

etcd:
    tolerations: []
kcp:
    tolerations: []
kcpFrontProxy:
    tolerations: []

Usage example

kcp:
    tolerations:
      - key: "kcp"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
    affinity:
      nodeAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 90
            preference:
              matchExpressions:
                - key: kcp
                  operator: In
                  values:
                    - "true"

Would result in a pod that has both affinity and toleration for the taint. I can create needed PR if this proposal is acceptable.

[embik]

Hey @martivo, that sounds like a great addition to the Helm chart. Please feel free to submit a PR!

[martivo]

Please see #137

[martivo]

Will close once it is released and can confirmed that it solves issue.