#Creating Clusters on Huawei Cloud Stack

This document provides comprehensive instructions for creating Kubernetes clusters on the Huawei Cloud Stack platform using Cluster API.

#Prerequisites

Before creating clusters, ensure all of the following prerequisites are met:

#1. Required Plugin Installation

Install the following plugins on the 's global cluster:

• Alauda Container Platform Kubeadm Provider
• Alauda Container Platform HCS Infrastructure Provider

For detailed installation instructions, refer to the Installation Guide.

#2. HCS Infrastructure Input Preparation

Prepare all HCS-specific inputs before writing any YAML in this document:

• HCS credential Secret values
• Provider-recognized compute values such as imageName, flavorName, and availabilityZone
• Cluster network inventory, including the subnets and free IP ranges used by the cluster
• Control plane ELB address planning, including vipAddress, vipSubnetName, and fixed L4 and L7 IPs
• Static IP pool planning for control plane and worker nodes

See Infrastructure Resources for Huawei Cloud Stack for the complete checklist, source information, and constraints.

#Cluster Creation Overview

At a high level, you'll create the following Cluster API resources in the 's global cluster to provision infrastructure and bootstrap a functional Kubernetes cluster.

Before you write any YAML in this page, complete the preparation checklist in Infrastructure Resources for Huawei Cloud Stack. This checklist covers the values that the provider expects, where to get them, and which values must be planned before you fill the manifests.

The cluster creation process follows this order:

1. Configure HCS authentication (Secret)
2. Create machine configuration pool (HCSMachineConfigPool)
3. Configure machine template (HCSMachineTemplate)
4. Configure KubeadmControlPlane
5. Configure HCSCluster
6. Create the Cluster

#Control Plane Configuration

The control plane manages cluster state, scheduling, and the Kubernetes API. This section shows how to configure a highly available control plane.

#Configure HCS Authentication

HCS authentication information is stored in a Secret resource.

apiVersion: v1
kind: Secret
metadata:
  name: <credential-secret-name>
  namespace: cpaas-system
type: Opaque
data:
  accessKey: <base64-encoded-access-key>
  secretKey: <base64-encoded-secret-key>
  projectID: <base64-encoded-project-id>
  region: <base64-encoded-region>
  externalGlobalDomain: <base64-encoded-domain>

You can reuse an existing HCS credential Secret. Its name does not need to match the cluster name, but HCSCluster.spec.identityRef.name must reference this Secret.

#Configure Machine Configuration Pool

The HCSMachineConfigPool defines pre-configured hostnames and static IP addresses for VMs.

Use one subnet selector per networks[] entry. For new manifests, set either subnetName or subnetId, but not both. Existing manifests may keep the deprecated subenetName field; if you also add subnetName while updating that manifest, its value must exactly match subenetName. Do not supply conflicting values across subenetName, subnetName, and subnetId.

If you use subnetName in the machine configuration pool, include the same subnet name in HCSCluster.spec.network.subnets.

For the initial cluster create flow, listing an existing subnet by name is enough because the controller resolves subnet metadata before the cluster becomes Ready. If you later add another subnet to an existing Ready HCSCluster, do not append only name. Patch the parent HCSCluster.spec.network.subnets entry with the full subnet object so later machine or ELB operations can reuse the resolved subnet metadata.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSMachineConfigPool
metadata:
  name: <cluster-name>
  namespace: cpaas-system
spec:
  configs:
    - hostname: master-1
      networks:
        - subnetName: <subnet-name>
          ipAddress: 192.168.1.11
    - hostname: master-2
      networks:
        - subnetName: <subnet-name>
          ipAddress: 192.168.1.12
    - hostname: master-3
      networks:
        - subnetName: <subnet-name>
          ipAddress: 192.168.1.13

*For new manifests, set either subnetName or subnetId. Existing manifests may continue to use subenetName, and may add subnetName only if both fields use the same value. Do not provide conflicting subnet selector values.

Note: The CRD schema lists subnetName, subenetName, and subnetId as optional fields and does not express their allowed combinations. Follow the provider-level rules above when writing manifests.

Note: To attach multiple NICs to one node, add multiple networks[] entries. The provider only uses these entries to attach NICs and assign subnet selectors plus static IPs. It does not support declaring per-NIC roles, default gateways, static routes, or per-NIC DNS settings.

#Configure Machine Template

The HCSMachineTemplate defines the VM specifications for control plane nodes.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSMachineTemplate
metadata:
  name: <cluster-name>-control-plane
  namespace: cpaas-system
spec:
  template:
    spec:
      imageName: <vm-image-name>
      flavorName: <instance-flavor>
      availabilityZone: <availability-zone>
      rootVolume:
        type: SSD
        size: 100
      configPoolRef:
        name: <cluster-name>
      dataVolumes:
        - size: 10
          type: SSD
          mountPath: /var/lib/etcd
          format: xfs
        - size: 100
          type: SSD
          mountPath: /var/lib/kubelet
          format: xfs
        - size: 100
          type: SSD
          mountPath: /var/lib/containerd
          format: xfs
        - size: 40
          type: SSD
          mountPath: /var/cpaas
          format: xfs

*Required when dataVolumes is specified.

Note: Do not set runtime identity fields such as providerID or serverId in HCSMachineTemplate manifests. The provider assigns these values when it creates HCS instances.

Note: Tenant administrators cannot retrieve the provider-recognized flavorName and availabilityZone values from the HCS UI. Get the exact values from the HCS administrator before you apply the manifest.

#Configure KubeadmControlPlane

The KubeadmControlPlane defines the Kubernetes control plane configuration.

apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  name: <cluster-name>
  namespace: cpaas-system
spec:
  replicas: 3
  version: <kubernetes-version>
  rolloutStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0
  kubeadmConfigSpec:
    files:
      - path: /etc/kubernetes/admission/psa-config.yaml
        owner: root:root
        permissions: "0644"
        content: |
          apiVersion: apiserver.config.k8s.io/v1
          kind: AdmissionConfiguration
          plugins:
          - name: PodSecurity
            configuration:
              apiVersion: pod-security.admission.config.k8s.io/v1
              kind: PodSecurityConfiguration
              defaults:
                enforce: "privileged"
                enforce-version: "latest"
                audit: "baseline"
                audit-version: "latest"
                warn: "baseline"
                warn-version: "latest"
              exemptions:
                usernames: []
                runtimeClasses: []
                namespaces:
                - kube-system
                - cpaas-system
      - path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
        owner: root:root
        permissions: "0644"
        content: |
          {
            "apiVersion": "kubelet.config.k8s.io/v1beta1",
            "kind": "KubeletConfiguration",
            "protectKernelDefaults": true,
            "tlsCertFile": "/etc/kubernetes/pki/kubelet.crt",
            "tlsPrivateKeyFile": "/etc/kubernetes/pki/kubelet.key",
            "streamingConnectionIdleTimeout": "5m",
            "clientCAFile": "/etc/kubernetes/pki/ca.crt"
          }
      - path: /etc/kubernetes/encryption-provider.conf
        owner: root:root
        permissions: "0600"
        content: |
          apiVersion: apiserver.config.k8s.io/v1
          kind: EncryptionConfiguration
          resources:
          - resources:
            - secrets
            providers:
            - aescbc:
                keys:
                - name: key1
                  secret: bootstrap-placeholder
      - path: /etc/kubernetes/audit/policy.yaml
        owner: root:root
        permissions: "0644"
        content: |
          apiVersion: audit.k8s.io/v1
          kind: Policy
          rules:
          - level: Metadata
    clusterConfiguration:
      imageRepository: <image-repository>
      dns:
        imageTag: <dns-image-tag>
      etcd:
        local:
          imageTag: <etcd-image-tag>
      apiServer:
        extraArgs:
          audit-log-format: json
          audit-log-mode: batch
          audit-log-path: /etc/kubernetes/audit/audit.log
          audit-policy-file: /etc/kubernetes/audit/policy.yaml
          admission-control-config-file: /etc/kubernetes/admission/psa-config.yaml
          encryption-provider-config: /etc/kubernetes/encryption-provider.conf
          kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt
          profiling: "false"
          tls-min-version: VersionTLS12
        extraVolumes:
          - name: vol-dir-0
            hostPath: /etc/kubernetes
            mountPath: /etc/kubernetes
            pathType: Directory
      controllerManager:
        extraArgs:
          bind-address: "::"
          flex-volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
          profiling: "false"
          tls-min-version: VersionTLS12
      scheduler:
        extraArgs:
          bind-address: "::"
          profiling: "false"
          tls-min-version: VersionTLS12
    postKubeadmCommands:
      - chmod 600 /var/lib/kubelet/config.yaml
    initConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
          protect-kernel-defaults: "true"
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
    joinConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
  machineTemplate:
    nodeDrainTimeout: 1m
    nodeDeletionTimeout: 5m
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: HCSMachineTemplate
      name: <cluster-name>-control-plane

The HCS controller also injects files while resolving cloud-init data. It writes /etc/kubernetes/pki/kubelet.crt, /etc/kubernetes/pki/kubelet.key, and /etc/kubernetes/encryption-provider.conf for control plane machines. For the first control plane machine, the controller generates the encryption provider configuration. After the control plane is initialized, it tries to reuse the existing kube-apiserver encryption provider configuration. If you include a bootstrap file at /etc/kubernetes/encryption-provider.conf, treat it as a placeholder because the controller-generated or synchronized file takes precedence.

Note: Configure apiServer.extraArgs and apiServer.extraVolumes together. If the volume is not mounted, kube-apiserver cannot read the files written under /etc/kubernetes.

Note: The rolloutStrategy.rollingUpdate.maxSurge: 0 example above is for highly available static-IP control planes. Keep this setting for fixed-size control plane pools with at least three replicas so replacements happen in a scale-down-then-scale-up order. If you create a single-control-plane HCS cluster (spec.replicas: 1), do not copy the rolloutStrategy block into the create manifest. KubeadmControlPlane validation rejects that scale-in style rollout configuration for a single replica.

Note: HCS also supports creating a single-control-plane cluster by setting spec.replicas: 1 and preparing one control plane config entry in the referenced HCSMachineConfigPool. Treat this as a creation-only topology, and leave the rollout strategy unset in that create manifest. The upgrade flow in this documentation does not support single-control-plane HCS clusters.

Use the OS Support Matrix only for the component versions it explicitly lists, such as coredns and etcd image tags for supported MicroOS images. It is not a complete source for all HCS manifest values. Before you apply this YAML, also use the approved release baseline for values such as imageRepository, DNS image repository, Kube-OVN version, Kube-OVN join CIDR, Pod CIDR, and Service CIDR.

#Configure HCSCluster

The HCSCluster resource defines the HCS infrastructure configuration.

The HCS provider creates an Elastic Load Balance (ELB) on the HCS platform for the Kubernetes API server. This ELB must keep Hybrid Load Balancing enabled so cluster nodes can also reach the API server through the ELB address.

For the documented HCS workflow, provide vipAddress, elbVirsubnetL4Ips, and elbVirsubnetL7Ips. Each elbVirsubnetL4Ips[].ips and elbVirsubnetL7Ips[].ips entry must contain two IPs.

If you set vipDomainName, configure HCS Cloud DNS Private Zones so that the domain resolves to vipAddress.

List every cluster subnet in spec.network.subnets before you reference it anywhere else. vipSubnetName, elbVirsubnetL4Ips[].subnetName, elbVirsubnetL7Ips[].subnetName, and the subnetName values used by HCSMachineConfigPool must all exist in spec.network.subnets.

For the initial cluster create flow, the controller can resolve existing subnet metadata from name. For an existing Ready cluster, append a full subnet object instead of only name. Include id, and include neutronSubnetId for any subnet that the control plane ELB will use. Keep cidr, gatewayIp, primaryDNS, and secondaryDNS in the subnet inventory as well.

Do not disable Hybrid Load Balancing on the provider-created ELB after the cluster is created. The cluster depends on that ELB mode so nodes can reach the API server through the ELB address.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HCSCluster
metadata:
  name: <cluster-name>
  namespace: cpaas-system
spec:
  controlPlaneLoadBalancer:
    vipAddress: <control-plane-vip-address>
    vipSubnetName: <vip-subnet-name>
    vipDomainName: <control-plane-vip-domain-name>
    elbVirsubnetL4Ips:
      - subnetName: <subnet-name>
        ips:
          - <l4-ip-1>
          - <l4-ip-2>
    elbVirsubnetL7Ips:
      - subnetName: <subnet-name>
        ips:
          - <l7-ip-1>
          - <l7-ip-2>
  networkType: kube-ovn
  network:
    vpc:
      name: <vpc-name>
    subnets:
      - name: <subnet-name>
    securityGroup:
      name: <security-group-name>
  identityRef:
    name: <credential-secret-name>

Do not include spec.controlPlaneEndpoint in the create manifest. In the HCS create flow, the controller derives and populates this field from spec.controlPlaneLoadBalancer after the HCSCluster is created. Do not set controlPlaneEndpoint manually, and do not add an empty controlPlaneEndpoint object. If controlPlaneEndpoint is explicitly present in the manifest, it must include both host and port.

#Configure Cluster

The Cluster resource in Cluster API declares the cluster and references the control plane and infrastructure resources.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <cluster-name>
  namespace: cpaas-system
  annotations:
    cpaas.io/sentry-deploy-type: Baremetal
    cpaas.io/alb-address-type: ClusterAddress
    capi.cpaas.io/resource-group-version: infrastructure.cluster.x-k8s.io/v1beta1
    capi.cpaas.io/resource-kind: HCSCluster
    cpaas.io/kube-ovn-join-cidr: <kube-ovn-join-cidr>
    cpaas.io/kube-ovn-version: <kube-ovn-version>
  labels:
    cluster-type: HCS
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
        - <pods-cidr>
    services:
      cidrBlocks:
        - <services-cidr>
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: <cluster-name>
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: HCSCluster
    name: <cluster-name>

#Cluster Verification

After deploying all cluster resources, verify that the cluster has been created successfully.

#Using kubectl

# Check cluster status
kubectl get cluster -n cpaas-system <cluster-name>

# Verify control plane nodes
kubectl get kubeadmcontrolplane -n cpaas-system <cluster-name>

# Check machine status
kubectl get machines -n cpaas-system

# Verify cluster deployment status
kubectl get clustermodule <cluster-name> -o jsonpath='{.status.base.deployStatus}'

#Expected Results

A successfully created cluster should show:

• Cluster status: Running or Provisioned
• All control plane machines: Running
• Kubernetes nodes: Ready
• Cluster Module Status: Completed

#Adding Worker Nodes

For instructions on adding worker nodes to the cluster, refer to Managing Nodes.

#Upgrading Clusters

For instructions on upgrading cluster components, refer to Upgrading Clusters.