Joining Site Clusters

A site cluster is a Kubernetes cluster that runs protected applications or receives them after a failover. Each site cluster joins the hub as an Open Cluster Management (OCM) managed cluster. After the join, the hub installs the DR addons and the site stack on it without further manual steps.

The hub must be installed first, as described in Installing the Hub.

Site Prerequisites

Before a site joins, the following must be in place on the site cluster:

• Kubernetes: A supported Kubernetes or OpenShift version, as listed in Disaster Recovery Requirements.
• Simplyblock storage: A simplyblock storage cluster with the Simplyblock Operator and the simplyblock CSI driver (see Install Simplyblock CSI). The CSI driver must support csi-addons VolumeReplication, and VolumeGroupReplication when consistency groups are enabled in the protection plan.
• Snapshot support: The snapshot controller and the VolumeSnapshot and VolumeGroupSnapshot CRDs. They are installed together with simplyblock by the Simplyblock Operator Helm chart (snapshotcontroller.create, enabled by default).
• csi-addons: The csi-addons controller and CRDs. They are installed together with simplyblock storage.
• Protected StorageClasses: The StorageClasses and VolumeSnapshotClasses to be protected carry a selectable label, for example, simplyblock.io/replicated: "true". The protection plan selects the classes by this label (see Labeling the StorageClasses).
• Hub connectivity: The site cluster can reach the hub API server. The hub never connects to the site.
• Optional components: KubeVirt for virtual machines, and Multus with an isolated NetworkAttachmentDefinition for test failovers of VMs with secondary networks.

The Ramen Recipe CRD and Velero do not have to be installed in advance. They are part of the site stack.

Creating a Join Token

A join token is created on the hub for each site. The token is valid for 24 hours by default (--validity):

Creating a join token for site-a

TOKEN=$(kubectl --context hub -n dr-simplyblock exec deploy/dr-hub -- \
  dr-bootstrap token --cluster site-a)

The spoke chart also needs the hub API server URL and its CA certificate:

Reading the hub CA certificate

HUB_CA=$(kubectl --context hub config view --raw --minify \
  -o jsonpath='{.clusters[0].cluster.certificate-authority-data}')

Installing the Spoke Chart

The dr-simplyblock-spoke chart installs the OCM klusterlet and registers the site with the hub:

Joining site-a to the hub

helm --kube-context site-a install dr-simplyblock-spoke simplyblock/dr-simplyblock-spoke \
  --namespace dr-simplyblock --create-namespace \
  --set clusterName=site-a \
  --set hub.apiserver=https://hub.example.com:6443 \
  --set hub.token="$TOKEN" \
  --set hub.caData="$HUB_CA" \
  --set 'stackOmit={snapshotter,csi-addons-crds,csi-addons-controller}'

Clusters that join with a bootstrap token are approved automatically. No clusteradm accept step is needed.

Value	Default	Description
clusterName	Required	Name the cluster registers under. It must be equal to spec.sites[].cluster in the protection plans.
hub.apiserver	Required	URL of the hub API server.
hub.token	Required	Join token created with dr-bootstrap token.
hub.caData	Required	Base64-encoded PEM CA certificate of the hub API server, as in a kubeconfig's certificate-authority-data.
hub.publiclyTrustedCA	false	Allows an empty hub.caData when the hub API server certificate is publicly trusted.
images.registry	quay.io/open-cluster-management	Registry of the klusterlet images. Points to a mirror for air-gapped sites.
images.tag	v1.3.1	Tag of the klusterlet images.
stackOmit	[]	Site stack components the hub should not deliver to this cluster.

Warning

The cluster name is permanent. It is the name of the managed cluster on the hub and of the site in every protection plan. After a hub loss, sites must rejoin under their old names.

What the Hub Installs on a Site

When a cluster joins, dr-hub enables the following OCM addons on it:

• Policy addons: governance-policy-framework and config-policy-controller.
• Work manager: work-manager.
• DR agent: dr-agent in the simplyblock-dr-agent namespace. It reports the site status to the hub every 15 seconds and runs hooks, probes, tests, and cleanups on the site.

It then installs the site stack, which consists of the following components:

Component	Content
snapshotter	external-snapshotter v8.6.0 with volume group snapshots enabled. Not needed on a simplyblock site, which already runs the snapshot controller of the Simplyblock Operator chart. Always omitted with stackOmit.
csi-addons-crds	csi-addons v0.14.0 CRDs. Not needed on a simplyblock site, which already runs csi-addons. Always omitted with stackOmit.
csi-addons-controller	csi-addons v0.14.0 controller. Not needed on a simplyblock site. Always omitted with stackOmit.
recipe-crd	The Ramen Recipe CRD.
velero	Velero v1.16.1 with velero-plugin-for-aws v1.12.0, kubevirt-velero-plugin v0.8.0, and Kopia. No default backup storage location is configured.
olm-stubs	Stub CRDs for the OLM kinds that the Ramen DR cluster operator expects.
ramen-dr-cluster	The Ramen DR cluster operator.
agent-velero-rbac	Permissions for dr-agent in the Velero namespace.

The component list of a release is printed with dr-bootstrap render spoke:

Listing the site stack components

kubectl --context hub -n dr-simplyblock exec deploy/dr-hub -- dr-bootstrap render spoke

Removing a site from DR does not delete the components from the site cluster.

Note

The site stack reports the OLM Subscription of the Ramen DR cluster operator as not applied. This is expected, because the site does not run OLM.

Opting Out of Site Stack Components

On a site that already runs some of the components, for example, Velero from OpenShift API for Data Protection (OADP), the hub can be told not to deliver them:

• Single components: The spoke chart value stackOmit lists the components to skip.
• Entire stack: The label dr.simplyblock.io/stack=false on the ManagedCluster stops the hub from delivering any stack component to the site.

Joining a site without the Velero component

helm --kube-context site-b install dr-simplyblock-spoke simplyblock/dr-simplyblock-spoke \
  --namespace dr-simplyblock --create-namespace \
  --set clusterName=site-b \
  --set hub.apiserver=https://hub.example.com:6443 \
  --set hub.token="$TOKEN" \
  --set hub.caData="$HUB_CA" \
  --set 'stackOmit={snapshotter,csi-addons-crds,csi-addons-controller,velero}'

Turning off the site stack for a managed cluster

kubectl --context hub label managedcluster site-b dr.simplyblock.io/stack=false

Omitted components must be provided in compatible versions by other means.

Labeling the StorageClasses

A protection plan picks the protected StorageClasses on every site with a label selector. Only the selection label is set by the administrator.

Labeling a simplyblock StorageClass and VolumeSnapshotClass for protection

kubectl --context site-a label storageclass simplyblock-csi-sc simplyblock.io/replicated=true
kubectl --context site-a label volumesnapshotclass simplyblock-snap simplyblock.io/replicated=true

StorageClasses and VolumeSnapshotClasses with the same names must exist on every site of a plan, because the site mapper does not translate class names yet (see Site Profiles and Mappings). A VolumeSnapshotClass is only protected if its driver matches the provisioner of a selected StorageClass. On simplyblock storage, the class names the storage cluster of the site in its cluster_id parameter, which is how DR finds the backend to pair.

Zones

For asynchronous plans, a site is a whole cluster: the protection plan names the cluster (spec.sites[].cluster), and every node of it belongs to the site. No node has to be labeled for DR. Within one plan, a cluster is exactly one site, while the same cluster can be a site in several plans.

For synchronous plans, a site is a zone of one stretch cluster: every site of the plan names the same cluster and its own spec.sites[].zone, matched against the node label topology.kubernetes.io/zone. See Replication Types.

Verifying the Join

On the hub, the managed cluster must be joined and available:

Checking the managed clusters

kubectl --context hub get managedclusters

Example output of the managed cluster listing

NAME          HUB ACCEPTED   MANAGED CLUSTER URLS   JOINED   AVAILABLE   AGE
site-a        true                                  True     True        12m
site-b        true                                  True     True        9m

The dr-agent addon must be available in the cluster's namespace:

Checking the dr-agent addon of site-a

kubectl --context hub -n site-a get managedclusteraddon dr-agent

Finally, the DRConfig status lists every site agent and the state of every site stack:

Checking the site agents and site stacks

kubectl --context hub get drconfig default -o jsonpath='{.status.agents}'
kubectl --context hub get drconfig default -o jsonpath='{.status.stack}'

Each entry of status.agents reports the cluster, whether the agent is available, its version, the Velero namespace, and when it last reported. Each entry of status.stack reports whether all stack components are applied at the current version, with a message if they are not.

Next Steps

• Configuration
• Protection Plans