Multi-Tenant Policies op OpenShift met Kyverno

2023‑02-03 Sudo Friday OpenShift Kubernetes Kyverno Multi-Tenant Policy

  Wander Boessenkool

In een Multi-Tenant OpenShift (of standaard Kubernetes) omgeving wil je als beheerder je eindgebruikers soms de vrijheid geven om zelf Project(Requests)s of Namespaces aan te maken, maar wwel binnen vastomlijnde naamgevings protocollen. In het verleden hebben we al eens gekeken hoe je dit met OPA Gatekeeper kunt doen, en hoe je zelf een Validating of Mutating Admission Webhook kunt maken. Dit keer kijken we hoe je dit met Kyverno kunt doen.

In deze video laat Wander ons zien hoe met Kyverno dit soort policies kunt schrijven, en hoe je automatisch namespaces kunt laten labelen voor het gebruik van ClusterResourceQuotas.

Links:

• Kyverno website
• Multi-Tenant Kyverno Policy repo
• Sudo Friday over Gatekeeper
• Multi-Tenant Gatekeeper Policy repo

Voorbeeld Code

De twee Policies zoals op het moment van publiceren van de video:

• label-namespaces.yaml

   1apiVersion: kyverno.io/v1
   2kind: ClusterPolicy
   3metadata:
   4  name: label-namespaces
   5  annotations:
   6    policies.kyverno.io/title: Label namespaces with quotagroup
   7    policies.kyverno.io/description: |
   8      This policy sets the `quotagroup` label on namespaces that did not
   9      already have that label to the value of their namespace prefix, splitting
  10      on the `-` character. This can be useful when combined with
  11      ClusterResourceQuotas and restrictions on waht namespace names users and
  12      serviceaccounts can create.      
  13spec:
  14  validationFailureAction: enforce
  15  background: false
  16  rules:
  17  - name: add-quotagroup-label
  18    context:
  19    - name: namespaceprefix
  20      variable:
  21        jmesPath: split(request.object.metadata.name, '-')[0] 
  22    match:
  23      any:
  24      - resources:
  25          kinds:
  26            - Namespace
  27    preconditions:
  28      all:
  29      - key: "{{ namespaceprefix }}"
  30        operator: NotIn
  31        value:
  32        - openshift
  33        - kube
  34    mutate:
  35      patchStrategicMerge:
  36        metadata:
  37          labels:
  38            +(quotagroup): "{{ namespaceprefix }}"
  

• restrict-namespace-names.yaml

    1apiVersion: kyverno.io/v1
    2kind: ClusterPolicy
    3metadata:
    4  name: restrict-namespace-names
    5  annotations:
    6    policies.kyverno.io/title: Allowed Namespace Names for Regular Users
    7    policies.kyverno.io/description: |
    8      This policy limits the creation of namespaces projectRequests to the following:
    9      - Regular users can create Projects and ProjectRequest that:
   10        - Start with their own username  followed by a dash and something else
   11        - Start with the the same prefix as one of their groups. For exanmple
   12          a user in the groups ateam-devs and bteam-something-something can create
   13          namesapces that start with `ateam-*` and `bteam-*`.
   14        - Namespaces starting with `openshift-` are also allowed by this policy,
   15          but they are blocked by the default OpenShift controller for any user that
   16          is not a Cluster Admin.
   17      - Regular users with the permissions to create namespaces directly (most likely
   18        cluster admins) can do whatever they please directly with namespaces.
   19      - ServiceAccounts can create new namespaces if:
   20        - They come from a namespace with the `-cicd` postfix
   21        - The new namespace has the same prefix as the namespace the serviceAccount
   22          belongs to
   23        - ServiceAccounts from `openshift-*` and `kube-*` can do whatever they please.      
   24spec:
   25  validationFailureAction: enforce
   26  background: false
   27  rules:
   28  - name: user-validate-ns-name
   29    context:
   30    - name: allowedgroupnamespaceprefixes
   31      variable:
   32        jmesPath: request.userInfo.groups[?contains(@, ':') == `false`][].[split(@, '-')[0]][].join('-', [@, '*'])
   33    - name: allowedusernamespaceprefixes
   34      variable:
   35        value: "{{ request.userInfo.username }}-*"
   36    match:
   37      any:
   38      - resources:
   39          kinds:
   40            - ProjectRequest
   41            - Project
   42    preconditions:
   43      all:
   44      - key: "{{ request.operation }}"
   45        operator: Equals
   46        value: CREATE
   47      - key: "{{ serviceAccountName }}"
   48        operator: Equals
   49        value: ""
   50    validate:
   51      message: |
   52        The only names approved for your namespaces are the ones starting with:
   53        {{ allowedgroupnamespaceprefixes | join(', ', @) }}, and {{ allowedusernamespaceprefixes }}        
   54      deny:
   55        conditions:
   56          all:
   57          - key: "{{request.object.metadata.name}}"
   58            operator: AnyNotIn
   59            value: "{{ allowedgroupnamespaceprefixes }}"
   60          - key: "{{request.object.metadata.name}}"
   61            operator: AnyNotIn
   62            value: "{{ allowedusernamespaceprefixes }}"
   63          - key: "{{request.object.metadata.name}}"
   64            operator: AnyNotIn
   65            value: openshift-*
   66  - name: sa-validate-ns-name
   67    context:
   68    - name: namespacepostfix
   69      variable:
   70        value: "{{ split(serviceAccountNamespace, '-')[-1] }}"
   71    - name: namespaceprefix
   72      variable:
   73        value: "{{ split(serviceAccountNamespace, '-')[0] }}"
   74    - name: allowedsanamespaceprefixes
   75      variable:
   76        value: "{{ split(serviceAccountNamespace, '-')[0] }}-*"
   77    match:
   78      any:
   79      - resources:
   80          kinds:
   81            - Namespace
   82            - ProjectRequest
   83            - Project
   84    preconditions:
   85      all:
   86      - key: "{{ request.operation }}"
   87        operator: Equals
   88        value: CREATE
   89      - key: "{{ serviceAccountName }}"
   90        operator: NotEquals
   91        value: ""
   92      - key: "{{ namespaceprefix }}"
   93        operator: NotIn
   94        value:
   95        - openshift
   96        - kube
   97    validate:
   98      message: |
   99        Only serviceAccounts from `*-cicd` namespaces are allowed to create new
  100        namespaces, and then only when they share the namespace-prefix, in this case:
  101        {{ allowedsanamespaceprefixes }}        
  102      deny:
  103        conditions:
  104          any:
  105          - key: "{{ namespacepostfix }}"
  106            operator: NotIn
  107            value:
  108            - cicd
  109          - key: "{{request.object.metadata.name}}"
  110            operator: AnyNotIn
  111            value: "{{ allowedsanamespaceprefixes }}"