Tenet

Tenet is a Kubernetes controller that aims to facilitate setting-up Network Policies on tenant namespaces.

It is designed to work in conjunction with Cilium Network Policies and Accurate.

Overview

Tenet is a Kubernetes controller that aims to facilitate setting-up Network Policies on tenant namespaces.

It is designed to work in conjunction with Cilium Network Policies and Accurate.

Motivation

To enhance the security without sacrificing convenience, we want to provide default network policies for tenants to consume. For instance, we want egress communications to be disabled by default in tenant namespaces. Tenants should be able to add exceptions to fit the needs of their workflows while being prevented from adding exceptions that are too broad, i.e. grant access to node resources.

Features

Allow cluster administrators to provide network policy templates tenants can opt into
- currently CiliumNetworkPolicy and CiliumClusterwideNetworkPolicy templates are supported
Automatically generate network policies on namespaces that opt into them
- when used in conjunction with Accurate, resource generation is also performed on SubNamespaces
Allow cluster administrators to place restrictions on the expressivity of network policies
- currently only IP address restrictions are supported

Usage

Tenet provides two custom resources to cluster administrators: NetworkPolicyTemplate and NetworkPolicyAdmissionRule.

Tenet provides the following annotation to users and tenants: tenet.cybozu.io/network-policy-template.

NetworkPolicyTemplate

NetworkPolicyTemplate enables administrators to write CiliumNetworkPolicy or CiliumClusterwideNetworkPolicy templates that tenants can opt-into via the tenet.cybozu.io/network-policy-template annotation in their Namespace resources. Templates can be supplied with values sources from the .metadata field of the Namespace resource that reference them. When annotations are placed on a root namespace managed by Accurate the annotations, and thus the templated CiliumNetworkPolicies, can be propagated to child namespaces. For instance, given the following NetworkPolicyTemplate,

# network-policy-template.yaml
apiVersion: tenet.cybozu.io/v1beta1
kind: NetworkPolicyTemplate
metadata:
    name: allow-intra-namespace-egress
spec:
    policyTemplate: |
      apiVersion: cilium.io/v2
      kind: CiliumNetworkPolicy
      metadata:
        name: allow-intra-namespace-egress
      spec:
        endpointSelector: {}
        egress:
        - toEndpoints:
          - matchLabels:
              "k8s:io.kubernetes.pod.namespace": {{.Name}}

When a tenant namespace is annotated like below,

# namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
  annotations:
      tenet.cybozu.io/network-policy-template: allow-intra-namespace-egress
  labels:
      accurate.cybozu.com/type: root

The following CiliumNetworkPolicy gets created in the my-namespace namespace:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-intra-namespace-egress
  namespace: my-namespace
spec:
  endpointSelector: {}
  egress:
  - toEndpoints:
    - matchLabels:
         "k8s:io.kubernetes.pod.namespace": my-namespace

If my-namespace is an Accurate root namespace, any of its child namespace will inherit the tenet.cybozu.io/network-policy-template annotation and CiliumNetworkPolicies will be created with the templates filled-in.

To write CiliumClusterwideNetworkPolicy templates, set .spec.clusterwide: true on NetworkPolicyTemplate.

NetworkPolicyAdmissionRule

To restrict the scope of whitelist permissions tenants can write via CiliumNetworkPolicies or NetworkPolicies, cluster administrators can write NetworkPolicyAdmissionRule resources. This allows administrators to forbid the use of specific CIDR ranges as endpoint selectors for network policies. For instance, the following NetworkPolicyAdmissionRule will reject network policies in namespaces that do not hold the team: neco label, i.e. all tenant namespaces, from specifing IP addresses within the 10.72.16.0/20 range for egress rules.

# admission-rule.yaml
apiVersion: tenet.cybozu.io/v1beta1
kind: NetworkPolicyAdmissionRule
metadata:
    name: forbid-bmc
spec:
    namespaceSelector:
      excludeLabels:
        team: neco
    forbiddenIPRanges:
      - cidr: 10.72.16.0/20
        type: egress

IP address restrictions can be applied on ingress or egress type network policies. When type: all is specified, the restrictions apply to both ingress and egress.

Specifications

namespaceSelector

This selects namespaces for which the admission rules apply.

forbiddenIPRanges

This defines IP ranges, in CIDR form, against which users cannot define network policies.

forbiddenEntities

This defines Cilium entities that users are not allowed to refer to in their network policies.

Template Opt-in

Tenants and users can opt into network policy templates via the following annotation placed on Namespace resources:

tenet.cybozu.io/network-policy-template - a comma-separated list of NetworkPolicyTemplate names

In a cluster where cluster administrators have control over Namespace definitions, for instance in a situation where Accurate is deployed and cluster administrators manage root namespaces, users will not be able to remove inherited annotations to bypass restrictions.

Example

# namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
  annotations:
      tenet.cybozu.io/network-policy-template: allow-intra-namespace-egress,forbid-bmc
  labels:
      accurate.cybozu.com/type: root

This will create the appropriate CiliumNetworkPolicies as defined in the relevant NetworkPolicyTemplates.

Release procedure

This document describes how to release a new version.

Versioning

Follow semantic versioning 2.0.0 to choose the new version number.

Prepare change log entries

Add notable changes since the last release to CHANGELOG.md. It should look like:

(snip)
## [Unreleased]

### Added
- Implement ... (#35)

### Changed
- Fix a bug in ... (#33)

### Removed
- Deprecated `-option` is removed ... (#39)

(snip)

Bump version

Determine a new version number. Then set VERSION variable.

# Set VERSION and confirm it. It should not have "v" prefix.
$ VERSION=x.y.z
$ echo $VERSION

Make a branch to release
```
$ git neco dev "bump-$VERSION"
```
Edit CHANGELOG.md for the new version (example).

Commit the change and push it.

$ git commit -a -m "Bump version to $VERSION"
$ git neco review

Merge this branch.

Add a git tag to the main HEAD, then push it.

# Set VERSION again.
$ VERSION=x.y.z
$ echo $VERSION

$ git checkout main
$ git pull
$ git tag -a -m "Release v$VERSION" "v$VERSION"

# Make sure the release tag exists.
$ git tag -ln | grep $VERSION

$ git push origin "v$VERSION"

GitHub actions will build and push artifacts such as container images and create a new GitHub release.

Tenet Documentation