Accurate documentation

Accurate is a Kubernetes controller for soft multi-tenancy environments. It is currently developed and maintained by Cybozu.

The repository is at https://github.com/cybozu-go/accurate .

Overview

Accurate is a Kubernetes controller to help operations in large soft multi-tenancy environments.

Soft multi-tenancy in Kubernetes

Kubernetes does not provide multi-tenancy functions on its own. It merely provides Namespaces along with Role-Based Access Control (RBAC) to isolate resources such as Pods.

Soft multi-tenancy is a kind of technique to implement Namespace-based multi-tenancy on Kubernetes. On the other hand, hard multi-tenancy provides a virtual kube-apiserver for each tenant to isolate privileges completely.

In a soft multi-tenancy environment, a cluster admin grants privileges in a Namespace to a group of tenant users by creating RoleBinding object like this:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: tenant
  name: admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: admin
subjects:
- kind: Group
  name: group-for-tenant
  apiGroup: rbac.authorization.k8s.io

admin ClusterRole is a built-in role to give admin privileges on any kind of namespace-scope resources. With this RoleBinding, users in group-for-tenant can freely create/edit/delete namespace-scope resources in tenant Namespace.

In many cases, a tenant needs to have multiple Namespaces to run multiple independent applications. However, tenant users are not allowed to create or delete Namespaces because Namespace is a cluster-scope resource. Otherwise, they would be able to delete other tenants' Namespaces!

What is Accurate?

Accurate introduces a namespace-scope custom resource called SubNamespace. With SubNamespace, tenant users can create a Namespace by creating a SubNamespace, and delete the created Namespace by deleting the SubNamespace.

The created Namespace is considered a child of the Namespace where the SubNamespace is created. The child Namespace may inherit labels and annotations from its parent Namespace.

Accurate also propagates resources such as Role, RoleBinding, or Secret from a parent Namespace to its children Namespaces. Without propagating Role/RoleBinding, the tenant user would be able to do nothing in newly created Namespaces.

Features

• Resource propagation between namespaces

  Accurate can propagate any namespace-scope resource including custom resources between Namespaces.

• Inheriting labels and annotations creation/update from parent namespaces

  Namespace labels often play important roles. For example, Pod Security Admission uses Namespace labels to control security policies.

• SubNamespace custom resource for tenant users

  This is the feature to allow tenant users to create and delete Namespaces by themselves. SubNamespaces can be created in either a root Namespace or a Namespace created by SubNamespace. A root Namespace is a Namespace labeled with accurate.cybozu.com/type=root.

• Template namespaces

  A template Namespace is a Namespace labeled with accurate.cybozu.com/type=template. Labels, annotations, and resources can be propagated from a template Namespace to other Namespaces referencing the template with accurate.cybozu.com/template=<name> label.

  This feature is implemented to allow resource propagation between normal Namespaces.

• kubectl plugin

  kubectl-accurate is a kubectl plugin to make the operations for Accurate easy.

Concepts

Namespace types

Accurate defines the following types of Namespaces:

• Template: Namespace labeled with accurate.cybozu.com/type=template
• Root: Namespace labeled with accurate.cybozu.com/type=root
• Sub-namespace: Namespace labeled with accurate.cybozu.com/parent=<name>

Any Namespace other than sub-namespaces can reference a template Namespace with accurate.cybozu.com/template=<name> label.

Sub-namespace can reference a root or another sub-namespace as its parent.

When configured to do so, Accurate propagates the Namespace labels, annotations, and namespace-scope resources from a referenced Namespace to referencing Namespaces.

Circular references are prohibited by an admission webhook.

Resource propagation

Accurate propagates any namespace-scope resource that are annotated with accurate.cybozu.com/propagate=<mode>.

Mode is one of the following:

• create: the resource will be created in referencing Namespaces if missing.
• update: the resource will be created in referencing Namespaces if missing, or will be updated if not identical, or will be deleted when the resource in the referenced Namespace is deleted.

Propagating generated resources (DEPRECATED)

If a resource annotated with accurate.cybozu.com/propagate-generated=<mode> creates a resource and set an owner reference in the created resource, Accurate automatically adds accurate.cybozu.com/propagate=<mode> to the created resource.

This can be used, for example, to propagate Secret created from SealedSecret.

Getting started

Follow these steps to install Accurate on your cluster and start using it.

1. Adjust configurations for your environment
2. Install cert-manager and apply Accurate manifests
3. Install kubectl-accurate to local machines

Configurations

Helm Chart values

Read Helm Chart for details.

Configuration file

accurate-controller reads its configurations from a configuration file.

The repository includes an example as follows:

# Labels to be propagated to sub-namespaces.
# It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
# https://pkg.go.dev/path#Match
labelKeys:
- team

# Annotations to be propagated to sub-namespaces.
# It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
# https://pkg.go.dev/path#Match
annotationKeys:
# An example to propagate an annotation for MetalLB
# https://metallb.universe.tf/usage/#requesting-specific-ips
- metallb.universe.tf/address-pool

# Labels to be propagated to sub-namespaces from SubNamespace resource.
# It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
# https://pkg.go.dev/path#Match
subNamespaceLabelKeys:
- app

# Annotations to be propagated to sub-namespaces from SubNamespace resource.
# It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
# https://pkg.go.dev/path#Match
subNamespaceAnnotationKeys:
- foo.bar/baz

# List of GVK for namespace-scoped resources that can be propagated.
# Any namespace-scoped resource is allowed.
watches:
- group: rbac.authorization.k8s.io
  version: v1
  kind: Role
- group: rbac.authorization.k8s.io
  version: v1
  kind: RoleBinding
- version: v1
  kind: Secret
- version: v1
  kind: ResourceQuota

# List of nameing policy for SubNamespaces.
# root and match are both regular expressions.
# When a SubNamespace is created in a tree starting from a root namespace and the root namespace's name matches the "root" regular expression, the SubNamespace name is validated with the "match" regular expression.
#
# "match" namingPolicies can use variables of regexp capture group naming of "root" namingPolicies.
# example:
#   root: ^app-(?P<team>.*)
#   match: ^app-${team}-.*
#   root namespace: app-team1
#   compiled match naming policy: ^app-team1-.*
# This feature is provided using https://pkg.go.dev/regexp#Regexp.Expand
namingPolicies: []

Only labels and annotations specified in the configuration file will be inherited.
Be careful that some labels or annotations affect security configurations or the system. For example, pod-security.kubernetes.io/* labels control the security capabilities of Pods in a Namespace.

Likewise, Accurate watches only namespace-scope resources specified in the configuration file.

You can edit the Helm Chart values as needed.

<snip>
controller:
  config:
    # controller.config.labelKeys -- Labels to be propagated to sub-namespaces.
    # It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
    ## https://pkg.go.dev/path#Match
    labelKeys: []
    # - team

    # controller.config.annotationKeys -- Annotations to be propagated to sub-namespaces.
    # It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
    ## https://pkg.go.dev/path#Match
    annotationKeys: []
    # An example to propagate an annotation for MetalLB
    # https://metallb.universe.tf/usage/#requesting-specific-ips
    # - metallb.universe.tf/address-pool

    # Labels to be propagated to sub-namespaces from SubNamespace resource.
    # It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
    # https://pkg.go.dev/path#Match
    subNamespaceLabelKeys:
    - app

    # Annotations to be propagated to sub-namespaces from SubNamespace resource.
    # It is also possible to specify a glob pattern that can be interpreted by Go's "path.Match" func.
    # https://pkg.go.dev/path#Match
    subNamespaceAnnotationKeys:
    - foo.bar/baz

    # controller.config.watches -- List of GVK for namespace-scoped resources that can be propagated.
    # Any namespace-scoped resource is allowed.
    watches:
      - group: rbac.authorization.k8s.io
        version: v1
        kind: Role
      - group: rbac.authorization.k8s.io
        version: v1
        kind: RoleBinding
      - version: v1
        kind: Secret

    # controller.config.namingPolicies -- List of nameing policy for SubNamespaces.
    # root and match are both regular expressions.
    # When a SubNamespace is created in a tree starting from a root namespace and the root namespace's name matches the "root" regular expression, the SubNamespace name is validated with the "match" regular expression.
    #
    # "match" namingPolicies can use variables of regexp capture group naming of "root" namingPolicies.
    # example:
    #   root: ^app-(?P<team>.*)
    #   match: ^app-${team}-.*
    #   root namespace: app-team1
    #   compiled match naming policy: ^app-team1-.*
    # This feature is provided using https://pkg.go.dev/regexp#Regexp.Expand
    namingPolicies:
      - root:  foo
        match: foo_.*
      - root:  bar
        match: bar_.*
      - root:  ^app-(?P<team>.*)
        match: ^app-${team}-.*
<snip>

ClusterRoleBindings

A built-in ClusterRole admin is bound by default to allow accurate-controller to watch and propagate namespace-scope resources. However, admin does not contain verbs for ResourceQuota and may not contain custom resources.

If you need to watch and propagate resources not included in admin ClusterRole, add additional ClusterRole/ClusterRoleBinding to accurate-controller-manager ServiceAccount. Set the controller.additionalRBAC.rules in the Helm Chart values.

The following example Helm chart values is to watch and propagate ResourceQuotas.

<snip>
controller:
  additionalRBAC:
    # controller.additionalRBAC.rules -- Specify the RBAC rules to be added to the controller.
    # ClusterRole and ClusterRoleBinding are created with the names `{{ release name }}-additional-resources`.
    # The rules defined here will be used for the ClusterRole rules.
    rules:
      - apiGroups:
          - ""
        resources:
          - resourcequotas
        verbs:
          - get
          - list
          - watch
          - create
          - patch
          - delete
<snip>

Feature Gates

Feature gates are a set of key=value pairs that describe operator features. You can turn these features on or off using the --feature-gates command line flag. Use -h flag to see a full set of feature gates.

To set feature gates, use the --feature-gates flag assigned to a list of feature pairs:

--feature-gates=...,DisablePropagateGenerated=false

The following table is a summary of the feature gates that you can set.

• The "Since" column contains the Accurate release when a feature is introduced or its release stage is changed.
• The "Until" column, if not empty, contains the last Accurate release in which you can still use a feature gate.

Each feature gate is designed for enabling/disabling a specific feature:

• DisablePropagateGenerated: Disable propagating generated resources, which is a feature subject for removal soon.

Deploying Accurate

1. (Optional) Prepare cert-manager

   Accurate depends on cert-manager to issue TLS certificate for admission webhooks. If cert-manager is not installed on your cluster, install it as follows:

   curl -fsLO https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml
   kubectl apply -f cert-manager.yaml
   

2. Setup Accurate Helm repository

   helm repo add accurate https://cybozu-go.github.io/accurate/
   helm repo update
   

3. Configuration Helm chart values

   Read Configurations for details.

4. Install the Accurate Helm chart

   helm install --create-namespace --namespace accurate accurate accurate/accurate -f values.yaml
   

Accurate Helm Chart

How to use Accurate Helm repository

You need to add this repository to your Helm repositories:

helm repo add accurate https://cybozu-go.github.io/accurate/
helm repo update

Quick start

Installing cert-manager

curl -fsL https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml | kubectl apply -f -

Installing CustomResourceDefinitions (optional)

Accurate does not use the official helm method of installing CRD resources. This is because it makes upgrading CRDs impossible with helm CLI alone. The helm team explain the limitations of their approach here.

The Accurate Helm chart default is to install and manage CRDs with Helm and add annotations preventing Helm from uninstalling the CRD when the Helm release is uninstalled.

The recommended approach is to let helm manage CRDs, but if you want to manage CRDs yourself, now is the time.

kubectl apply -k https://github.com/cybozu-go/accurate//config/crd-only/

[!NOTE] Since the CRDs contain configuration of conversion webhooks, you may have to tweak the webhook settings if installing the chart using non-standard values.

If you decided to manage CRDs outside of Helm, make sure you set the crds.enabled Helm value to false.

Installing the Chart

[!NOTE] This installation method requires cert-manager to be installed beforehand.

To install the chart with the release name accurate using a dedicated namespace(recommended):

helm install --create-namespace --namespace accurate accurate accurate/accurate

Specify parameters using --set key=value[,key=value] argument to helm install.

Alternatively a YAML file that specifies the values for the parameters can be provided like this:

helm install --create-namespace --namespace accurate accurate -f values.yaml accurate/accurate

Values

Generate Manifests

You can use the helm template command to render manifests.

helm template --namespace accurate accurate accurate/accurate

Installing kubectl plugin

kubectl-accurate is a plugin for kubectl to make operations of Accurate easy.

It is strongly recommended to install kubectl-accurate though Accurate can be used without the plugin.

Installing using Krew

Krew is the plugin manager for kubectl command-line tool.

See the documentation for how to install Krew.

kubectl krew update
kubectl krew install accurate

Installing manually

1. Set OS to the operating system name

   OS is one of linux, windows, or darwin (MacOS).

   If Go is available, OS can be set automatically as follows:

   OS=$(go env GOOS)
   

2. Set ARCH to the operating system name

   ARCH is one of amd64 or arm64.

   If Go is available, ARCH can be set automatically as follows:

   ARCH=$(go env GOARCH)
   

3. Set VERSION to the accurate version

   See the Accurate release page: https://github.com/cybozu-go/accurate/releases

   VERSION=< The version you want to install >
   

4. Download the binary and put it in a directory of your PATH.

   The following is an example to install the plugin in /usr/local/bin.

   curl -L -sS https://github.com/cybozu-go/accurate/releases/download/$(VERSION)/kubectl-accurate_$(VERSION)_$(OS)_$(ARCH).tar.gz \
     | tar xz -C /usr/local/bin kubectl-accurate
   

5. Check the installation

   Run kubectl accurate -h and see the output looks like:

   $ kubectl accurate -h
   accurate is a subcommand of kubectl to manage Accurate features.
   
   Usage:
     accurate [command]
   
   Available Commands:
     completion  generate the autocompletion script for the specified shell
     help        Help about any command
     list        List namespace trees hierarchically
     namespace   namespace subcommand
     sub         sub-namespace command
     template    template subcommand
   ...
   

Usage

Accurate can be used imperatively using kubectl or declaratively with YAML manifests.

The following sections will show you the usage in both ways.

Showing information

Show the hierarchical list of sub-namespaces

Use kubectl accurate list:

$ kubectl accurate list
├── root1
├── root2
│   └── sub1
├── root3
├── subroot1
│   └── sn1
└── subroot2

Show all template Namespaces

Use kubectl accurate template list:

$ kubectl accurate template list
├── template1
├── template2
│   ├── reference1
│   └── reference2
└── template3

Show the properties of a Namespace

Use kubectl accurate ns describe:

$ kubectl accurate ns describe root2
Name: root2
Type: root
# of children: 1

Resources:
Kind     Name     From     Mode
-------- -------- -------- --------
Role     role1    tmpl3    create
Secret   mysecret          create

Setting up templates

Template is a feature of Accurate to propagate labels, annotations, and resources between normal Namespaces.

Any Namespace except for sub-namespaces can reference a template Namespace. So, a template Namespace can reference another template Namespace.

In the following examples, <name> represents a Namespace name to be changed. Likewise, <template> represents a template Namespace name.

Setting a Namespace as a template

Using kubectl accurate:

kubectl accurate ns set-type <name> template

Applying YAML manifests:

apiVersion: v1
kind: Namespace
metadata:
  name: <name>
  labels:
    accurate.cybozu.com/type: template

Reverting a template Namespace to a normal one

Using kubectl accurate:

kubectl accurate ns set-type <name> none

Applying YAML manifests:

Remove accurate.cybozu.com/type label.

Setting a reference to a template Namespace

Using kubectl accurate:

kubectl accurate template set <name> <template>

Applying YAML manifests:

apiVersion: v1
kind: Namespace
metadata:
  name: <name>
  labels:
    accurate.cybozu.com/template: <template>

Unsetting a reference to a template Namespace

Using kubectl accurate:

kubectl accurate template unset <name>

Applying YAML manifests:

Remove accurate.cybozu.com/template label.

Propagating resources

Accurate propagates only resources annotated with accurate.cybozu.com/propagate=<mode>.

The Group/Version/Kind of the resource must be listed in the configuration file.

In the following examples, <mode> represents either create or update. Read Concepts about the propagation modes.

Annotating a resource for propagation

The following is an example to propagate Secrets.

Using kubectl:

kubectl annotate secrets <name> accurate.cybozu.com/propagate=<mode>

Applying YAML manifests:

apiVersion: v1
kind: Secret
metadata:
  namespace: default
  name: <name>
  annotations:
    accurate.cybozu.com/propagate: <mode>

Annotating a resource to propagate resources created from it (DEPRECATED)

For example, a Secret created from cert-manager's Certificate can automatically be propagated.

To do this, Certificate should be annotated with accurate.cybozu.com/propagate-generated=<mode> at the time of creation.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  namespace: default
  name: example-cert
  annotations:
    accurate.cybozu.com/propagate-generated: <mode>
spec:
  ...

accurate-controller needs to be able to get Certificate objects.

Sub-namespace operations

Sub-namespaces is a feature of Accurate to allow tenant users to create Namespaces and delete the created Namespaces.

Sub-namespaces can be created under either a root Namespace or a sub-namespace.

In the following examples, <name> represents a Namespace name to be changed. Likewise, <parent> represents a root or another sub-namespace.

Setting a Namespace as a root Namespace

Suppose that Accurate is configured to propagate team label.

Using kubectl accurate:

kubectl accurate ns set-type <name> root
kubectl label ns <name> team=foo

Applying YAML manifests:

apiVersion: v1
kind: Namespace
metadata:
  name: <name>
  labels:
    accurate.cybozu.com/type: root
    team: foo

Accurate only propagates labels/annotations that have been configured in that respect via the labelKeys and annotationKeys parameters in config.yaml. This prevents the propagation of labels/annotations that were not meant to do so.

Preparing resources for tenant users

In almost all cases, a root Namespace should have RoleBinding for a group of tenant users. The RoleBinding should be annotated with accurate.cybozu.com/propagate=update.

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: <name>
  name: admin
  annotations:
    accurate.cybozu.com/propagate: update
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: admin
subjects:
- kind: Group
  name: foo
  apiGroup: rbac.authorization.k8s.io

You may want to prepare more objects such as ResourceQuotas.

Reverting a root Namespace to a normal one

Using kubectl accurate:

kubectl accurate ns set-type <name> none

Applying YAML manifests:

Remove accurate.cybozu.com/type label.

Creating a sub-namespace

Using kubectl accurate:

kubectl accurate sub create <name> <parent>

Applying YAML manifests:

apiVersion: accurate.cybozu.com/v2
kind: SubNamespace
metadata:
  namespace: <parent>
  name: <name>

Creating a sub-namespace with additional labels/annotations

Using kubectl accurate:

kubectl accurate sub create <name> <parent> --labels=foo=bar --annotations=baz=zot

Applying YAML manifests:

apiVersion: accurate.cybozu.com/v2
kind: SubNamespace
metadata:
  namespace: <parent>
  name: <name>
spec:
  labels:
    foo: bar
  annotations:
    baz: zot

You can edit these spec.labels/spec.annotations with kubectl edit:

kubectl edit SubNamespace -n=<parent> <name>

The spec.labels/spec.annotations that can be propagated to sub-namespaces can be set with the subNamespaceLabelKeys/subNamespaceAnnotationKeys parameters in config.yaml.

Deleting a created sub-namespace

Using kubectl accurate:

kubectl accurate sub delete <name>

Applying YAML manifests:

Delete the created SubNamespace object.

Changing the parent of a sub-namespace

Only cluster admins can do this.

Using kubectl accurate:

kubectl accurate sub move <name> <new-parent>

Applying YAML manifests:

apiVersion: v1
kind: Namespace
metadata:
  name: <name>
  labels:
    accurate.cybozu.com/parent: <new-parent>

Converting a normal Namespace to a sub-namespace

Only cluster admins can do this.

Using kubectl accurate:

kubectl accurate sub graft <name> <parent>

Applying YAML manifests:

apiVersion: v1
kind: Namespace
metadata:
  name: <name>
  labels:
    accurate.cybozu.com/parent: <parent>

Converting a sub-namespace to a root Namespace

Only cluster admins can do this.

Using kubectl accurate:

kubectl accurate sub cut <name>

Applying YAML manifests:

apiVersion: v1
kind: Namespace
metadata:
  name: <name>
  labels:
    accurate.cybozu.com/type: root
    # and remove accurate.cybozu.com/parent label

Custom Resources

• SubNamespace

Sub Resources

• SubNamespaceList
• SubNamespaceSpec

SubNamespace

SubNamespace is the Schema for the subnamespaces API Deprecated: This type will be removed in one of the next releases.

Back to Custom Resources

SubNamespaceList

SubNamespaceList contains a list of SubNamespace Deprecated: This type will be removed in one of the next releases.

Back to Custom Resources

SubNamespaceSpec

SubNamespaceSpec defines the desired state of SubNamespace

Back to Custom Resources

Commands

kubectl-accurate

kubectl-accurate is a kubectl plugin for Accurate.

Features

• Hierarchical view of namespace trees
• Show the information about a namespace.
  • List of propagating/propagated resources in the namespace.
  • Root or not.
  • Template or not.
  • The parent namespace, if it is a sub-namespace.
  • The template namespace, if set.
• Operations for root namespaces
  • Make an independent namespace to a root namespace.
  • Make a root namespace back to an independent namespace, if it has no child sub-namespaces.
• Operations for setting a template namespace
• Operations for sub-namespaces
  • Create a sub-namespace under a root namespace or another sub-namespace.
  • Deleting a sub-namespace.
  • Move a sub-namespace under a different root or sub-namespace.
  • Convert an independent namespace to a sub-namespace.
  • Convert a sub-namespace to a root namespace.

Generic options

kubectl-accurate takes the same generic options as kubectl including:

Flags:
      --as string                      Username to impersonate for the operation
      --as-group stringArray           Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
      --cache-dir string               Default cache directory (default "$HOME/.kube/cache")
      --certificate-authority string   Path to a cert file for the certificate authority
      --client-certificate string      Path to a client certificate file for TLS
      --client-key string              Path to a client key file for TLS
      --cluster string                 The name of the kubeconfig cluster to use
      --context string                 The name of the kubeconfig context to use
      --insecure-skip-tls-verify       If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
      --kubeconfig string              Path to the kubeconfig file to use for CLI requests.
  -n, --namespace string               If present, the namespace scope for this CLI request
      --request-timeout string         The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
  -s, --server string                  The address and port of the Kubernetes API server
      --tls-server-name string         Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used
      --token string                   Bearer token for authentication to the API server
      --user string                    The name of the kubeconfig user to use

Note that kubectl-accurate does not use the namespace given by -n / --namespace flag. It always take namespace names as positional arguments.

Commands

There is an alias for namespace sub-command that is ns.

list [ROOT]

List namespace trees hierarchically. If ROOT is given, only the tree starting from ROOT namespace is shown.

namespace describe NS

Describe the information about a namespace NS related to Accurate.

namespace set-type NS TYPE

Set the type of a namespace NS to TYPE.

Valid types are root or template.

To unset the type, specify none as the type.

template list [TEMPLATE]

List template namespace trees hierarchically. If TEMPLATE is not given, all template namespaces are shown hierarchically. If TEMPLATE is given, only the tree under the TEMPLATE namespace will be shown.

template set NS TEMPLATE

Set TEMPLATE namespace as the template of NS namespace.

template unset NS

Unset the template of NS namespace.

sub create NAME NS

Create a SubNamespace named NAME in NS namespace.

After that, Accurate will create a namespace NAME as a sub-namespace of NS.

sub delete NAME

Delete a SubNamespace named NAME in the parent namespace of NAME namespace.

After that, Accurate will delete NAME namespace.

sub move NS PARENT

Move a sub-namespace NS to a different root or sub-namespace.

After that, Accurate will create SubNamespace object in the new parent namespace.

sub graft NS PARENT

Like sub move, but this converts a non-sub-namespace NS to a sub-namespace of PARENT.

NS must not be a root namespace or have a template.

sub cut NS

Make a sub-namespace NS a new root namespace. The child sub-namespaces under NS will be moved along with it.

Propagated resources with mode update in NS will be deleted.

sub list [ROOT]

Alias for kubectl-accurate list command.

accurate-controller

accurate-controller is a Kubernetes controller to manage sub-namespaces and to propagate resources from parents to their children namespaces.

Configuration file

accurate-controller reads a configuration file on startup. The default location is /etc/accurate/config.yaml. The location can be changed with --config-file flag.

The configuration file should be a JSON or YAML file having the following keys:

Example:

labelKeys:
- team

annotationKeys:
- foo.bar/baz

subNamespaceLabelKeys:
- app

subNamespaceAnnotationKeys:
- foo.bar/baz

watches:
- group: rbac.authorization.k8s.io
  version: v1
  kind: Role
- group: rbac.authorization.k8s.io
  version: v1
  kind: RoleBinding
- version: v1
  kind: Secret

Environment variables

Command-line flags

Flags:
      --add_dir_header                   If true, adds the file directory to the header
      --alsologtostderr                  log to standard error as well as files
      --apiserver-qps-throttle int       The maximum QPS to the API server. (default 50)
      --cert-dir string                  webhook certificate directory
      --config-file string               Configuration file path (default "/etc/accurate/config.yaml")
      --health-probe-addr string         Listen address for health probes (default ":8081")
  -h, --help                             help for accurate-controller
      --leader-election-id string        ID for leader election by controller-runtime (default "accurate")
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --log_file string                  If non-empty, use this log file
      --log_file_max_size uint           Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --logtostderr                      log to standard error instead of files (default true)
      --metrics-addr string              The address the metric endpoint binds to (default ":8080")
      --skip_headers                     If true, avoid header prefixes in the log messages
      --skip_log_headers                 If true, avoid headers when opening log files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          number for the log level verbosity
      --version                          version for accurate-controller
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging
      --webhook-addr string              Listen address for the webhook endpoint (default ":9443")
      --zap-devel                        Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error)
      --zap-encoder encoder              Zap log encoding (one of 'json' or 'console')
      --zap-log-level level              Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
      --zap-stacktrace-level level       Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').

Labels used by Accurate

The table below is a list of labels used by Accurate.

Annotations used by Accurate

The table below is a list of annotations used by Accurate.

Design notes

• Overview
• Why do we need another namespace controller in the first place?
• Goals
• Things to be avoided
  • No webhooks for propagated resources
• SubNamespaces are not related to parent-child relationships

Overview

Accurate aims to implement functionalities found in Hierarchical Namespace Controller (HNC), but in a different way.

Accurate provides 1) resource-propagation between namespaces, and 2) sub-namespace concept for multi-tenancy. Since we consider resource-propagation alone is highly useful, the feature is available between any namespaces.

Why do we need another namespace controller in the first place?

Some of the HNC designs and specifications contradict our use cases.

• HNC opt-outs resources when propagating them.
  • For safety and accuracy, we need opt-in propagation.
• HNC opt-outs root namespaces.
  • For easier-maintenance, we need opt-in root namespaces.
• HNC does not propagate namespace labels and annotations.
  • We need to propagate some namespace labels/annotations.

Since these are fundamentally different requirements, we decided to develop our own solution.

Goals

• Any namespace-scoped resource can be copied or propagated
  • The kinds of resources are given by the configuration file of Accurate.
  • Only resources annotated with accurate.cybozu.com/propagate: <mode> will be propagated.
  • Of course, Accurate controller needs to be privileged to manage them.
• Support the following propagation modes:
  • create: if the resource does not exist, copy the resource from the parent namespace.
  • update: if the resource is missing or different from the parent namespace, create or update it. If the parent resource is deleted, the copy will also be deleted.
• ⚠️ Propagate generated resources (DEPRECATED)
  • Resources created and controlled by another resource can be automatically propagated.
  • The generator resource should be annotated with accurate.cybozu.com/propagate-generated: <mode>.
• Propagate labels and annotations of parent or template namespaces
  • The label/annotation keys are given through the configuration file of Accurate.
  • Only labels/annotations specified in the configuration file of Accurate will be propagated.
  • Label/annotation deletions from parent or template namespaces will not be propagated.
• Opt-in root namespaces
  • Only namespaces labeled with accurate.cybozu.com/type: root can be the root of a namespace tree.
• Tenant users can create and delete sub-namespaces by creating and deleting a custom resource in a root or a sub-namespace.
  • If a namespace has one or more sub-namespaces, Accurate prevents the deletion of the namespace - unless allow cascading deletion of namespaces is enabled.
• Template namespace
  • Namespaces that are not a sub-namespace can specify a template from which labels, annotations, and resources can be propagated.
• Admins can change the parent namespace of a sub-namespace.

Things to be avoided

Accurate prevents the following problems by a validating admission webhook for Namespace.

• Circular references among namespaces.
• Allowing a sub-namespace to set a template (sub-namespaces should inherit things only from the parent).
• Marking a sub-namespace as a root namespace.
• Deleting accurate.cybozu.com/type=root label from root namespaces having one or more sub-namespaces.
• Deleting accurate.cybozu.com/type=template label from template namespaces having one or more instance namespaces.
• Dangling sub-namespaces (sub-namespaces whose parent namespace is missing).
• Dangling instance namespaces (namespaces whose template namespace is missing).
• Changing a sub-namespace to a non-root namespace when it has child sub-namespaces.

Accurate prevents the following problem by a validating admission webhook for SubNamespace.

• Creating a SubNamespace object in a non-root and non-sub- namespace.

No webhooks for propagated resources

Accurate does not use admission webhooks for resources propagated from a parent namespace to its sub-namespaces. The decision was made from the following points:

• Since Accurate can propagate any namespace-scoped resource, adding an admission webhook for them might cause troubles.

  For instance, admission webhooks for Pods may cause chicken-and-egg problem upon bootstrapping.

• Avoid interrupting users who do not expect limitations from Accurate.

SubNamespaces are not related to parent-child relationships

Accurate does not rely on SubNamespace resources to look up sub-namespaces of a namespace or to find the parent of a sub-namespace.

By doing so, Accurate can easily restructure existing namespaces.

How Accurate reconciles resources

Accurate primarily watches Namespaces and SubNamespaces. In addition, it needs to watch any kind of resources specified in its config file.

SubNamespace (custom resource)

• For a new SubNamespace, Accurate creates a sub-namespace.
• For a deleting SubNamespace, Accurate deletes the sub-namespace if the sub-namespace exists and its accurate.cybozu.com/parent is the same as metadata.namespace of SubNamespace.

Namespaces

Namespaces that are labeled with accurate.cybozu.com/template

These namespaces reference a template namespace and propagate the labels, annotations, and watched resources from the template namespace.

• Accurate should propagate labels and/or annotations from the template namespace.
• Accurate should create copies of resources in the template namespace whose accurate.cybozu.com/propagate annotation is create if they are missing.
• Accurate should create or update copies of resources in the template namespace whose accurate.cybozu.com/propagate annotation is update if they are missing or different.
• Accurate should delete resources in the reconciling namespace that are annotated with accurate.cybozu.com/propagate=update provided that:
  • the value of accurate.cybozu.com/from annotation is not the template namespace name, or
  • there is not a resource of the same kind and the same name in the template namespace.

Namespaces w/o accurate.cybozu.com/type and accurate.cybozu.com/template labels

If these labels are removed from the Namespace, Accurate should delete propagated resources with mode == update.

Template namespace

Template namespaces are namespaces labeled with accurate.cybozu.com/type=template.

• Accurate should propagate labels and/or annotations to namespaces that references the template namespace with accurate.cybozu.com/template label.

Root namespace

Root namespaces are namespaces labeled with accurate.cybozu.com/type=root.

• Accurate should propagate labels and/or annotations to its sub-namespaces.

Sub-namespace

Sub-namespaces are namespaces created by Accurate. Sub-namespaces have accurate.cybozu.com/parent label.

• Accurate should propagate labels and/or annotations from the parent namespace.
• Accurate should propagate labels and/or annotations of the reconciling namespace to its sub-namespaces, if any.
• Accurate should create copies of resources in the parent namespace whose accurate.cybozu.com/propagate annotation is create if they are missing.
• Accurate should create or update copies of resources in the parent namespace whose accurate.cybozu.com/propagate annotation is update if they are missing or different.
• Accurate should delete resources in the reconciling namespace that are annotated with accurate.cybozu.com/propagate=update provided that:
  • the value of accurate.cybozu.com/from annotation is not the parent namespace name, or
  • there is not a resource of the same kind and the same name in the parent namespace.

Watched namespace-scoped resources

Any namespace-scoped resource can be propagated from a template or from a parent namespace.

Resources annotated with accurate.cybozu.com/from

These resources are propagated from a parent or a template namespace. The annotation value is the parent namespace name.

• If the parent resource exists and is annotated with accurate.cybozu.com/propagate=update, Accurate compares the resource with the parent resource, and if they differ, updates the resource.
• If the resource is annotated with accurate.cybozu.com/propagate=update and there isn't a resource of the same kind and the same name in the parent namespace, Accurate deletes the resource.

The last rule is for cases where the parent resource is deleted while the controller is stopped. With this rule, Accurate can delete such orphaned resources when the controller starts.

Resources annotated with accurate.cybozu.com/propagate

These resources can be propagated to other namespaces.

• If the resource exists and the annotation value is create, Accurate creates a copy in all sub-namespaces if missing.
• If the resource exists and the annotation value is update, Accurate creates or updates a copy in all sub-namespaces if missing or different.
• When a resource is deleted, Accurate checks sub-namespaces and delete the resource of the same kind and the same name if the resource is annotated with accurate.cybozu.com/propagate=update.

Resources owned by another resource that is annotated with accurate.cybozu.com/propagate-generated (DEPRECATED)

Accurate annotates the resource with accurate.cybozu.com/propagate. The annotation value is the same as accurate.cybozu.com/propagate-generated annotation.

Release procedure

This document describes how to release a new version.

Labeling

Release notes are automatically generated based on PRs included in the release. Those PRs are categorized based on the label assigned to them. Please refer to .github/release.yml for the kind of labels.

Versioning

Follow semantic versioning 2.0.0 to choose the new version number.

Bump version

1. Determine a new version number. Then set VERSION variable.

   # Set VERSION and confirm it. It should not have "v" prefix.
   VERSION=x.y.z
   echo $VERSION
   

2. Add a git tag to the main HEAD, then push it.

   git checkout main
   git pull
   git tag -a -m "Release v$VERSION" "v$VERSION"
   git tag -ln | grep $VERSION
   git push origin v$VERSION
   

Maintenance

How to update supported Kubernetes

Accurate supports the three latest Kubernetes versions. If a new Kubernetes is released, please update the following files.

• Update Kubernetes version in e2e/Makefile and .github/workflows/ci.yaml.
• Update kubectl version in aqua.yaml.
• Update k8s.io/* and sigs.k8s.io/controller-runtime packages version in go.mod.

If Kubernetes or controller-runtime API has changed, please fix the relevant source code.

How to update dependencies

Renovate will create PRs that update dependencies once a week. However, Kubernetes is only updated with patched versions.