user guide - huawei cloud · user guide issue 01 date 2019-07-04 huawei technologies co., ltd. ......

Cloud Container Engine

User Guide

Issue 01

Date 2019-07-04

HUAWEI TECHNOLOGIES CO., LTD.

Copyright © Huawei Technologies Co., Ltd. 2019. All rights reserved.No part of this document may be reproduced or transmitted in any form or by any means without prior writtenconsent of Huawei Technologies Co., Ltd. Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.All other trademarks and trade names mentioned in this document are the property of their respectiveholders. NoticeThe purchased products, services and features are stipulated by the contract made between Huawei and thecustomer. All or part of the products, services and features described in this document may not be within thepurchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,and recommendations in this document are provided "AS IS" without warranties, guarantees orrepresentations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in thepreparation of this document to ensure accuracy of the contents, but all statements, information, andrecommendations in this document do not constitute a warranty of any kind, express or implied.

Issue 01 (2019-07-04) Copyright © Huawei Technologies Co., Ltd. i

Contents

1 Overview......................................................................................................................................... 11.1 Basic Concepts............................................................................................................................................................... 11.2 Accessing CCE............................................................................................................................................................... 21.3 Related Services............................................................................................................................................................. 31.4 User Permissions............................................................................................................................................................ 31.5 Related Concepts............................................................................................................................................................ 3

2 Getting Started............................................................................................................................... 52.1 Creating a VPC and a Key Pair...................................................................................................................................... 52.2 Creating a Container Cluster.......................................................................................................................................... 62.3 Creating a Containerized Application............................................................................................................................ 72.4 Configuring Auto Scaling Policies................................................................................................................................. 92.5 Monitoring an Application........................................................................................................................................... 102.6 Deleting Resources....................................................................................................................................................... 10

3 Cluster Management...................................................................................................................123.1 Cluster Overview..........................................................................................................................................................123.2 Creating a VM Cluster..................................................................................................................................................133.3 Creating a BMS Cluster................................................................................................................................................183.4 Connecting to a Kubernetes Cluster Using kubectl......................................................................................................213.5 Configuring kube-dns/CoreDNS HA Using kubectl.................................................................................................... 233.6 Creating a Node in a VM Cluster................................................................................................................................. 243.7 Cluster Auto Scaling.....................................................................................................................................................263.8 Changing Cluster Specifications.................................................................................................................................. 283.9 Managing Node Labels.................................................................................................................................................293.10 Deleting a Cluster....................................................................................................................................................... 303.11 Cluster Statuses...........................................................................................................................................................313.12 Monitoring a Node......................................................................................................................................................323.13 Managing Namespaces............................................................................................................................................... 33

4 Application Management.......................................................................................................... 364.1 Application Overview...................................................................................................................................................364.2 Creating a Stateless Application...................................................................................................................................374.3 Creating a Stateful Application.................................................................................................................................... 424.4 Basic Operations on Applications................................................................................................................................ 46

Cloud Container EngineUser Guide Contents

Issue 01 (2019-07-04) Copyright © Huawei Technologies Co., Ltd. ii

4.5 Setting Container Specifications.................................................................................................................................. 484.6 Affinity and Anti-Affinity Scheduling......................................................................................................................... 504.7 Application Scaling...................................................................................................................................................... 574.8 Interconnection with Prometheus (Monitoring)........................................................................................................... 604.9 APM Settings (Performance Bottleneck Analysis)...................................................................................................... 614.10 Using a Third-party Image to Create an Application................................................................................................. 61

5 Application Access Settings...................................................................................................... 645.1 Overview...................................................................................................................................................................... 645.2 Intra-Cluster Access..................................................................................................................................................... 645.3 Intra-VPC Access......................................................................................................................................................... 695.4 External Access - Elastic IP Address........................................................................................................................... 755.5 External Access - Elastic Load Balancer......................................................................................................................78

6 Job Management..........................................................................................................................836.1 Creating a One-off Jobs................................................................................................................................................ 836.2 Creating a Cron Job...................................................................................................................................................... 86

7 Configuration Center..................................................................................................................917.1 Creating a Configuration Item......................................................................................................................................917.2 Using a Configuration Item.......................................................................................................................................... 947.3 Creating a Secret...........................................................................................................................................................957.4 Using a Secret...............................................................................................................................................................98

8 Storage Management................................................................................................................ 1008.1 Overview.................................................................................................................................................................... 1008.2 Using Local Hard Disks............................................................................................................................................. 1008.3 Using EVS Disks........................................................................................................................................................ 1058.4 Using SFS File Systems..............................................................................................................................................112

9 Log Management....................................................................................................................... 1199.1 Collecting Standard Output Logs of Containers.........................................................................................................1199.2 Collecting Logs in a Specified Path of a Container....................................................................................................119

10 Container Orchestration.........................................................................................................12210.1 Basic Concepts......................................................................................................................................................... 12210.2 Preparing a Template Package..................................................................................................................................12210.3 Uploading a Template Package................................................................................................................................ 12410.4 Creating an Application Using Template................................................................................................................. 12510.5 Using an EVS Disk...................................................................................................................................................12610.6 Using Load Balancers...............................................................................................................................................12710.7 Upgrading a Template-based Application................................................................................................................ 12810.8 Rolling Back a Template-based Application............................................................................................................ 12810.9 Uninstalling Template-based Applications...............................................................................................................128

11 Image Repository.....................................................................................................................130


Issue 01 (2019-07-04) Copyright © Huawei Technologies Co., Ltd. iii

12 Application O&M....................................................................................................................131

13 Kubectl Usage Guide.............................................................................................................. 135

14 Reference...................................................................................................................................13714.1 Formula for Calculating the Reserved Resources of a Node....................................................................................13714.2 How Do I Enable ICMP Security Group Rules?......................................................................................................13814.3 Connecting to a Kubernetes Cluster Using Helm.....................................................................................................138


Issue 01 (2019-07-04) Copyright © Huawei Technologies Co., Ltd. iv

1 Overview

1.1 Basic ConceptsCloud Container Engine (CCE) is a highly reliable and high-performance service that allowsenterprises to manage containerized applications. With support for Kubernetes-nativeapplications and tools, CCE makes it simple to set up an environment for running containersin the cloud.

Compatible with Kubernetes and Docker, CCE allows you to quickly create and managecontainerized applications. CCE also provides enhanced interconnection capabilities,including interconnection with Elastic Load Balancing (ELB) and the cloud storage services:Elastic Volume Service (EVS) and Scalable File Service (SFS).

Before you use CCE, it is important that you have a basic understanding of what Docker isand how to use Docker commands. For more information about Docker, visit https://docs.docker.com/.

The following sections dive into basic CCE concepts in more detail.

Kubernetes Cluster

Kubernetes coordinates a highly available cluster of cloud resources, such as nodes andvirtual private clouds (VPCs), required for running containers.

Stateless Applications

Stateless application instances are independent from each other and provide the samefunctions. They support auto scaling and rolling upgrades. Examples of stateless applicationsinclude Nginx and WordPress.

Stateful Applications

Stateful application instances are dependent on each other and have stable persistent storageand network identifiers. They support ordered deployment, scaling in, and deletion. Examplesof stateful applications include MySQL High Availability (HA) and etcd applications.

Cloud Container EngineUser Guide 1 Overview

Issue 01 (2019-07-04) Copyright © Huawei Technologies Co., Ltd. 1

https://docs.docker.com/

https://docs.docker.com/

Job

A job is a resource object that Kubernetes uses to control tasks in batches. A job creates oneor more pods and ensures that a specified number of them successfully terminate. There arethree main types of job: non-parallel jobs that create only one pod, parallel jobs with a fixedcompletion count, and parallel jobs with a work queue.

A cron job runs periodically at a specified time. A cron job object is like one line of a Linuxcron table file. Cron jobs are useful for creating periodic and recurring tasks, like runningbackups or sending emails.

Namespace

Namespaces enable division of cluster resources and objects among multiple users. Typically,namespaces are best suited for scenarios where a large number of users work across multipleprojects. Multiple namespaces can be created in a single cluster with the data isolated fromeach other. This enables namespaces to share the services of the same cluster withoutaffecting each other.

For example, you can deploy applications in a development environment in one namespace,and deploy applications in a test environment in another namespace.

Container Image

A container image is a read-only template used to create containers. For example, a containerimage could contain a complete Ubuntu operating system with required programs and theirdependency files installed.

Docker provides a simple way to build new container images or update existing containerimages. In addition to building container images on your own, you can download containerimages that other users have created.

1.2 Accessing CCEYou can access CCE in either of the following ways:

l Using the management console

You can access CCE using a web-based management console. If you have registeredwith the management console, log in and choose Cloud Container Engine from theservice list.

l Using HTTP-based application programming interfaces (APIs)

For more information, see the Cloud Container Engine API Reference (Paris).

l Using Kubectl

For more information, see 13 Kubectl Usage Guide.

NOTE

If you use Kubectl or APIs to create an application or job from images on the My Images page of theCCE console, the imagePullSecrets parameter in yaml files must be set to default-secret. The default-secret is the default key used for pulling container images.imagePullSecrets: - name: default-secret



1.3 Related Servicesl Elastic Cloud Server (ECS)

An ECS is a computing server that provides CPUs, memory, images, and Elastic VolumeService (EVS) disks and allows on-demand allocation and elastic scaling. ECS integratesVPC, virtual firewalls, and multi-data-copy features to build an efficient, reliable, andsecure computing environment that ensures service stability and continuity.

In CCE, a node is an ECS with multiple EVS disks (one 40-GB system disk and at leastone data disk). You can specify ECS specifications when creating a node.

l Elastic Volume Service (EVS)

EVS provides persistent block storage for services such as ECS. You can attach EVSdisks to an ECS and expand the EVS disk capacity.

In CCE, a node is an ECS with multiple EVS disks (one 40-GB system disk and at leastone data disk). You can specify EVS disk capacity when creating a node.

l Virtual Private Cloud (VPC)

A VPC is an exclusive logical network. In a VPC, you can customize the security group,VPN, IP address segments, and bandwidth. You can manage and configure internalnetworks and modify network configurations, simplifying network management. Youcan also customize ECS access rules within the security group and between securitygroups to strengthen security protection.

For network security purposes, all container clusters created on CCE run in VPCs.

l Elastic Load Balance (ELB)

The ELB service automatically distributes access traffic to multiple ECSs to balancetheir service load.

CCE allows applications to work with ELB to improve fault tolerance and serviceavailability of applications.

1.4 User PermissionsThe public cloud system provides two types of user permissions by default: user managementand resource management.

l User management refers to the management of users, user groups, and user group rights.

l Resource management refers to the operations that can be performed by users on cloudservice resources.

1.5 Related Concepts

Region

A region is a geographic area in which CCE is located.

CCE services in the same region can communicate with each other over an intranet, but thosein different regions cannot.



Public cloud data centers are deployed in different places worldwide. CCE is thereforeavailable in different regions. For example, applications can be designed to meet requirementsof users in specific regions or comply with local laws or regulations.

Availability ZoneEach region consists of many availability zones (AZs), in which power and networks arephysically isolated from each other. AZs in the same region can communicate with each otherover an intranet. Each AZ provides cost-effective and low-latency network connections thatare unaffected by faults that may occur in other AZs. Using CCE deployed in an independentAZ protects your applications against failures that occur in a single place.

ProjectProjects are used to group and isolate OpenStack resources (computing, storage, and networkresources). A project can be a department or a project team. Multiple projects can be createdfor one account.



2 Getting Started

Nginx is a lightweight web server. You can quickly create an Nginx containerized applicationand set up an Nginx web server on CCE.

In this chapter, an Nginx containerized application is created on CCE. This process takesabout 15 minutes.

After the Nginx containerized application is created successfully, you can access the Nginxweb page, which is shown in the following figure.

Figure 2-1 Nginx web page

2.1 Creating a VPC and a Key PairBefore creating your first cluster, you must create a VPC and a key pair. A VPC provides anisolated, configurable, and manageable virtual network environment for CCE clusters, and akey pair is used for identity authentication during remote node login. Once you have created aVPC and a key pair, you can use them for all clusters you subsequently create.

NOTE

If you already have a VPC and a key pair, skip the tasks in this section.

Cloud Container EngineUser Guide 2 Getting Started


Table 2-1 Creating a VPC and key pair

No. Task Procedure

1 Creating aVPC

You need to create a VPC to provide an isolated, configurable,and manageable virtual network for CCE clusters.1. Log in to the management console.2. Click Service List, and choose Network > Virtual Private

Cloud.3. On the Virtual Private Cloud page, click Create VPC to

create a VPC.4. Follow the online instructions to create a VPC. Retain the

default settings for the parameters unless otherwise specified.

2 Creating a keypair

You need to create a key pair for identity authentication upon aremote node login.1. Log in to the management console.2. Click Service List, and choose Computing > Elastic Cloud

Server.3. In the navigation pane, choose Key Pair. Click Create Key

Pair.4. Enter a key pair name, and click OK.5. In the dialog box that is displayed, click OK.6. View and save the key pair. To ensure security, a key pair can

be downloaded only once. Keep it secure to avoid loginproblems.

2.2 Creating a Container ClusterA container cluster is a logical grouping of cloud servers that run applications. Each clusternode corresponds to a cloud server. When you first use CCE, you must create an initial clusterand add a node to the cluster.

Prerequisites

You have created a VPC and a key pair as described in 2.1 Creating a VPC and a Key Pair.

Procedure

Step 1 Log in to the CCE console. On the Dashboard page, click Create VM Cluster.

Step 2 Set Cluster Name to cluster-01, retain the default settings for other parameters, and clickNext.

Step 3 Set the parameters for adding a node to the cluster. Set the network and login parameters asfollows, and retain the default settings for other parameters:l Set EIP to Automatically assign to enable access to the node from the Internet, and

retain the default values for other parameters.



l In the Key Pair field, select the created key pair used for logging in to the node.

Step 4 Click Create Now, and then click Submit.

It takes 6 to 10 minutes to create a cluster. Information about the progress of the creationprocess will be displayed.

Step 5 Choose Resource Management > Nodes, and check the node status. If the node status isAvailable, an elastic IP address has been bound to the node.

----End

2.3 Creating a Containerized ApplicationA containerized application is a group of instances running on CCE. CCE provides third-partyapplication hosting to help you manage the full lifecycle from deployment to O&M. Thissection describes how to use a container image to create a containerized application.

Prerequisitesl A cluster has been created and a node has been added to the cluster.l An elastic IP address has been bound to the node and the elastic IP address of the node

has been obtained.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Application Management.

Step 2 Click Create Application, and set Select Application Type to Stateless Applications.

Step 3 Set the following parameters as specified, and retain the default settings for other parameters:l Set Application Name to nginx.l In the Cluster Name field, select the cluster created in 2.2 Creating a Container

Cluster.l Set Instances to 1.

Step 4 Click Next. Click Add Container. On the Third-party Images tab page, set AuthenticateSecret to No and Image Address to nginx:latest, indicating that the latest Nginx version isselected from the third-party images. Click OK.

Step 5 Retain the default settings for the image parameters, and click Next.

Step 6 Click Add Access Mode, and set the parameters listed in Table 2-2 as specified.

NOTE

In this example, the Nginx application will be accessible from the Internet by using the elastic IPaddress.

Table 2-2 Setting application access parameters

Parameter Description

Service Name Name of the application that can be accessed externally. Set thisparameter to nginx.




Access Mode Set this parameter to External access.

Access Type Set this parameter to EIP.

Protocol Set this parameter to TCP.

Container Port Port on which the containerized application listens. Set thisparameter to 80 for the Nginx image.

Access Port Select Automatically generated.An access port number will be automatically generated andallocated.

Step 7 Click OK and then Next. Skip the advanced settings, and click Create Now.

To view the running application in the application list, click Back to Application List.

Step 8 Access the Nginx application.

1. In the navigation pane, choose Application Management. Copy the external accessaddress of the Nginx application.

Figure 2-2 External Access Address

2. Enter the external access address in the address box of the browser to connect to theapplication. Once you have successfully accessed the Nginx application, the welcomepage shown in Figure 2-3 will be displayed.



Figure 2-3 Accessing the Nginx application

----End

2.4 Configuring Auto Scaling PoliciesCCE supports the following auto scaling policies to meet requirements of different servicescenarios: metric-based policy, scheduled policy, and periodic policy.

Currently, CCE supports only automatic cluster scaling. By configuring auto scaling policies,nodes are automatically added to a cluster when an auto scaling policy is enforced. If youneed to scale in a cluster, follow the steps in Deleting a Node.

This section uses a scheduled policy as an example to describe how to configure a scalingpolicy for an application.

Procedure


Step 2 Click the Nginx application you have created. On the Application Management > nginxpage, click the Scaling tab.

Step 3 In the Auto Scaling area, click Add Scaling Policy, and set the parameters.



Figure 2-4 Adding a scaling policy

Step 4 Click OK to enable the policy.

Step 5 After the specified time arrives, click the Instances tab. The instance list shows that aninstance has been added to the application.

----End

2.5 Monitoring an ApplicationAfter creating an application, you can view the operating status and resource usage of theapplication.

Procedure


Step 2 Click the nginx application in the application list to view details.

Step 3 On the Instances tab page, click next to the instance name. On the Monitoring tab page,you can view the CPU usage and memory usage of the application instance. The CPU usageand memory usage are available only when the instance is running.

----End

2.6 Deleting ResourcesFees are generated during node and application running. To avoid paying unnecessary fees,you are advised to delete created nodes and applications once they are no longer needed.



PrerequisitesYou have created an Nginx application.

Procedure

Step 1 Log in to the CCE console.

Step 2 Delete cluster resources.

1. In the navigation pane, choose Resource Management > VM Clusters.2. Click More > Delete for the cluster to be deleted, and follow the prompts to delete the

cluster.

----End



3 Cluster Management

3.1 Cluster OverviewKubernetes coordinates a highly available cluster of cloud resources, such as nodes and VPCs,required for running containers.

Clusters, Subnets, and VPCsl A VPC is similar to a private local area network (LAN) managed by a home gateway. It

is a private network built on the public cloud and provides a basic network environmentfor running ECSs, ELBs, and middleware. You can configure networks of differentscales as required.

l A VPC can be divided into one or more subnets. Security groups are configured todetermine whether these subnets can communicate with each other. This ensures thatsubnets can be isolated from each other, so that you can deploy different services ondifferent subnets.

l A cluster consists of one or more ECSs (also known as nodes) in the same subnet. Itprovides a computing resource pool for running containers.

As shown in Figure 3-1, multiple VPCs are configured in a region. A VPC consists ofsubnets. The subnets communicate with each other through the subnet gateway. A cluster iscreated in a subnet. Therefore, there are three scenarios:l Different clusters are created in different VPCs.l Different clusters are created in the same subnet.l Different clusters are created in different subnets.

Cloud Container EngineUser Guide 3 Cluster Management


Figure 3-1 Clusters, subnets, and VPCs

Cluster Authorization OverviewBy default, Kubernetes RBAC is enabled for clusters created by CCE. For details, see officialdocuments at https://Kubernetes.io.

Precautions for Configuring NodesSome of a node's resources are required to run the Kubernetes components and Kubernetesresources necessary to make this node function as part of your cluster. Therefore, you maynotice a disparity between your node's total resources and the allocatable ones in KubernetesEngine. Since larger nodes tend to run more containers, the amount of resources thatKubernetes Engine reserves scales up for larger nodes.

To ensure node stability, some resources on cluster nodes are reserved by CCE depending onnode capacities for running Kubernetes components, such as kubelet, kube-proxy, and Docker.

3.2 Creating a VM ClusterBefore you create a containerized application, at least one cluster must be available. Atpresent, a maximum of five clusters can be created.

Basic Resources of a ClusterTable 3-1 lists the basic resources that you need for creating a cluster.

Table 3-1 Basic resources of a cluster

Resource Description

Masters and relatedresources

Associated with CCE resource tenants, and invisible to you.



https://kubernetes.io/docs/reference/access-authn-authz/rbac/

Resource Description

ECSs (optional) An ECS corresponds to a cluster node that provides computingresources.An ECS is named in the format of Cluster name-Randomnumber. The name format is user-defined. ECSs created inbatches are named in the format of Cluster name-Randomnumber 1-Random number 2.

Security groups Two security groups are created for a cluster: one for managingcluster masters, and the other for managing cluster nodes.NOTICE

To ensure that a cluster runs properly, retain the settings of securitygroups and security group rules configured during cluster creation.

1. Security group for mastersName format: Clustername-cce-control-Random numberFunctions:l Allows outbound traffic.l Allows other nodes to access Kubernetes services of

masters.2. Security group for nodes

Name format: Clustername-cce-node-Random numberFunctions:l Allows outbound traffic.l Allows remote login to Linux or Windows operating

systems using ports 22 and 3389.l Allows communication between Kubernetes

components using ports 4789 and 10250.l Allows external nodes to access Kubernetes using ports

30000 to 32767.l Allows communication between nodes in the same

security group.

Disks (optional) Two disks are configured for each node. One is the systemdisk, and the other is the data disk used to run Docker.

Elastic IP address(optional)

An elastic IP address (EIP) must be associated with a node inorder to enable communication with the Internet.

Prerequisite

You have created a VPC and key pair as described in 2.1 Creating a VPC and a Key Pair.

Creating a Cluster

Step 1 Log in to the CCE console. On the Dashboard page, click Create VM Cluster.

Step 2 Set the parameters listed in Table 3-2. The parameters marked with * are mandatory.



Table 3-2 Parameters for creating a cluster


* Region Physical location of a cluster.

* Cluster Name Name of the cluster to be created.

* Version Cluster version, which corresponds to the Kubernetes base version.

* Size Size of the cluster to be created.

* HighAvailability

l Yes: Three masters will be created in the same AZ for the cluster. Amaster manages and controls the entire cluster. The cluster remainsavailable when two of the masters are faulty.

l No: Only one master is created for the cluster. The cluster becomesunavailable if the master is faulty, but running applications are notaffected.

* VPC VPC where the new cluster is located.If no VPC is available, click Create a VPC and create one. For details,see 2.1 Creating a VPC and a Key Pair.

* Subnet Subnet in which the nodes run.

* NetworkModel

l Tunnel network: A virtual network built on top of a VPC network,applicable to common scenarios.

l VPC network: A VPC network that delivers higher performance andapplies to high-performance and intensive interaction scenarios. Onlyone cluster using the VPC network model can be created under aVPC.

Description Description of the cluster.

Step 3 After the configuration is complete, click Next.

Step 4 Select whether to create a node in the cluster.l No: Create a cluster without nodes. Go to Step 6.l Yes: Create the first node for the cluster.

Step 5 Set the parameters listed in Table 3-3, and click Next.

Table 3-3 Parameters for creating a node


AZ Physical location where resources use independent power suppliesand networks. AZs are physically isolated but interconnectedthrough an internal network. To improve application reliability, youare advised to create cloud servers in different AZs.

Node Name Name of the node to be created.




NodeSpecifications

l General-purpose: provides general computing, storage, andnetwork configurations for the majority of application scenarios;typically used for web servers, development and testenvironments, and small database applications.

l Memory-optimized: provides instances with a larger memorysize; typically used in memory-intensive applications that processa large amount of data, such as relational databases and NoSQLdatabases.

l General computing-plus: provides dedicated resources to ensurestable performance; typically used for enterprise-gradeapplications that require high computing performance andstability.

Operating System Only EulerOS 2.2 is supported.OS upgrade is not supported in the current version.

Nodes Number of nodes.

EIP An independent public IP address. If a node needs to access theInternet, assign a new EIP or use an existing EIP.NOTE

By default, the SNAT function of VPCs is disabled on CCE. If SNAT isenabled, EIPs are not required for accessing external networks.

l Do not use: A cloud server without an EIP cannot access theInternet. It can be used only as a cloud server for deployingservices or clusters on a private network.

l Automatically assign: An EIP with exclusive bandwidth isautomatically assigned to each cloud server. When creating anECS, ensure that the EIP quota is sufficient. Set the specificationsand bandwidth as required.

l Specify: An existing EIP is assigned to the cloud server.

Disk Disk type, which can be System Disk or Data Disk.l The system disk capacity is 40–1024 GB, which is user defined.

The default value is 40 GB.l The data disk capacity is 100–32678 GB, which is user defined.

The default value is 100 GB.Data disks deliver two levels of I/O performance:l Common I/O: EVS disks of this level provide reliable block

storage and a maximum IOPS of 1000 per disk. They are suitablefor key applications.

l Ultra-high I/O: EVS disks of this level provide a maximum IOPSof 20,000 and a minimum read/write latency of 1 ms. They aresuitable for RDS, NoSQL, and data warehouse applications.

Key Pair Used for identity authentication when you remotely log in to a node.Select an existing key pair.If no key pair is available, click Create a key pair and create one.



Step 6 Click Create Now, review the details, and click Submit.

It takes 6 to 10 minutes to create a cluster. Information about the progress of the creationprocess will be displayed.

After a cluster is created, two default system applications heapster-apiserver and corednsare automatically generated under namespace kube-system.

----End

Related Operations

After creating a cluster, you can:

l Use the Kubernetes command line (CLI) tool kubectl to connect to the cluster. Fordetails, see 3.4 Connecting to a Kubernetes Cluster Using kubectl.

l Add one or more nodes to the cluster. For details, see 3.6 Creating a Node in a VMCluster.

l Log in to a node. For details, see Logging In to a VM Node.

l Change the specifications of a cluster. For details, see 3.8 Changing ClusterSpecifications.

l Create a namespace. You can create multiple namespaces in a cluster and classify theminto different logical groups to share cluster resources. The logical groups can bemanaged separately. For more information about how to create a namespace for a cluster,see 3.13 Managing Namespaces.

l Click the cluster name to view cluster details. Table 3-4 describes the cluster details tabs.

Table 3-4 Cluster details

Tab Description

Cluster Details View the details and operating status of the cluster.

Monitoring Check the CPU and memory usage of the cluster over the past 1hour, 3 hours, or 12 hours.

Events l View cluster events on the Events tab page.l Set search criteria. For example, you can set the time segment or

enter an event name to view corresponding events.

Auto Scaling Cluster auto scaling dynamically changes the number of nodes in acluster to meet your service requirements. Auto scaling is triggeredto reduce labor costs when applications cannot be scheduled due toinsufficient resources in a cluster.For details, see 3.7 Cluster Auto Scaling.



Tab Description

Kubectl To access a Kubernetes cluster from a client, you can use theKubernetes CLI tool kubectl.For details, see 3.4 Connecting to a Kubernetes Cluster Usingkubectl.

3.3 Creating a BMS ClusterPrivate Bare Metal Server (BMS) clusters are Kubernetes container clusters with highcomputing and high network performance. To use a BMS cluster, enable the BMS servicefirst.

To provide a high-speed container network, you need to add a high-speed network interfacecard (NIC) when creating a BMS.

Constraints

BMS clusters do not support native container cluster monitoring and performance analysis forKubernetes (Heapster/Metrics-server).

You need to choose physical.o2.medium flavor when you use the BMS expansion feature inCCE.

Prerequisitesl Before creating your first cluster, you must create a VPC and a key pair. For details, see

2.1 Creating a VPC and a Key Pair.

l The BMS service has been enabled. For details, see Applying for a BMS.

l A high-speed network has been created. For details, see Managing High-SpeedNetworks. A high-speed network is an internal network among BMSs and providesunlimited bandwidth for connecting BMSs in the same AZ. If you want to deployservices requiring high throughput and low latency, you can create high-speed networks.

Creating a Cluster

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > BMSClusters, and click Create BMS Cluster.

Step 2 Set the parameters listed in Table 3-5. The parameters marked with * are mandatory.

Table 3-5 Parameters for creating a cluster


* Region Physical location of a cluster.

* Name Name of the cluster to be created.

* Version Base version of this Kubernetes cluster



https://kubernetes.io/docs/reference/kubectl/overview/

https://support.huaweicloud.com/en-us/eu-west-0-usermanual-bms/en-us_topic_0053536894.html




* Size Size of the cluster to be created.

* High Availability l Yes: Three masters will be created in the same AZ for thecluster. A master manages and controls the entire cluster. Thecluster remains available when two of the masters are faulty.

l No: Only one master is created for the cluster. The clusterbecomes unavailable if the master is faulty, but runningapplications are not affected.

* VPC VPC where the new cluster is located.If no VPC is available, click Create a VPC and create one.

* Subnet Subnet environment where the BMS runs.

* High-SpeedNetwork

Select a high-speed network.The high-speed network is an internal network for BMSs thatprovides unlimited bandwidth for BMSs in the same AZ.For details, see Managing High-Speed Networks.

Description Cluster description.

Step 3 After the configuration is complete, click Create Now.

Step 4 Review the settings and then click Submit.

The request for creating a BMS cluster is submitted successfully. You can go to the cluster listand wait until the cluster becomes available. It takes 5 to 10 minutes to complete the clustercreation.

----End

Adding Existing Nodes to a BMS Cluster

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > BMSClusters.

Step 2 Click Add to Cluster for the target cluster. The BMSs that can be added to the cluster aredisplayed.

Step 3 Select the BMS that you want to add to the cluster. Click Next.

Step 4 Follow the instructions on the current page to add the BMS to the cluster.

NOTE

If the login session expires or the network connection is interrupted during the procedure of adding aBMS node to the cluster, you need to uninstall the BMS node and then add the BMS node to the clusteragain.

l Install

a. Check whether the BMS has an exclusive data disk. If not, attach a data disk to theBMS to increase system stability.The data disk can be a raw disk or a Linux LVM partition.




b. If no EIP is bound to the BMS, bind an EIP.

i. Click Service List, and choose Network > Elastic IP.ii. On the Elastic IP page, click Assign EIP to create an EIP.iii. Bind the EIP to the BMS.

c. Log in as the cloud user to the BMS.d. Create the install.yaml file.

vi install.yamle. Follow the prompts to complete steps 5 to 6 in CCE console.

l Uninstall

a. Log in to the BMS to be uninstalled as the linux user. To uninstall the BMSs inbatches, log in to either of them.

b. Create the uninstall.yaml configuration file.vi uninstall.yaml

c. Follow the prompts to complete steps 3 to 4 in CCE console.



After the operations are complete, click Next.

The system displays a message, asking you to confirm that all the required operations arecompleted on the BMS. Click OK.

Step 5 Review the configurations of the BMS to be added to the cluster. Click Finish.

The system automatically switches to the Nodes page. Wait until the BMS is successfullyadded to the cluster.

----End

3.4 Connecting to a Kubernetes Cluster Using kubectlTo access a Kubernetes cluster from a client, you can use the Kubernetes CLI tool kubectl.

Prerequisites

CCE allows you to access a cluster from a VPC network or a public network.

l Intra-VPC access: You need to apply for an ECS on the ECS console and ensure that theECS is in the same VPC as the current cluster.

l Public network access: You need to prepare an ECS that can connect to a public network.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > VMClusters. Click Kubectl for the cluster you want to connect.

Step 2 Set the access mode for the cluster.




Figure 3-2 Connecting to the Kubernetes cluster using kubectl

After kubectl is successfully configured, you can connect to a Kubernetes cluster usingkubectl.

----End



Related OperationsAfter connecting to the cluster, you can use Kubernetes to manage applications. For details,see 13 Kubectl Usage Guide.

3.5 Configuring kube-dns/CoreDNS HA Using kubectlkube-dns/CoreDNS provides the domain name service (DNS) for clusters. If only one kube-dns/CoreDNS is deployed in a cluster, the entire cluster will not run properly if the kube-dns/CoreDNS fails. Therefore, you are advised to configure kube-dns/CoreDNS HA for a cluster.

This section describes how to use kubectl to configure kube-dns/CoreDNS HA.

NOTE

By default, kube-dns is installed for CCE clusters of Kubernetes v1.9, and CoreDNS is installed forCCE clusters of Kubernetes v1.11 and later.

PrerequisitesThe cluster is accessible from the Internet, or the cluster and the client are in the same VPC.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > VMClusters. Click Kubectl for the cluster to which you want to connect.

Step 2 Set the API access mode for the cluster.

Step 3 Configure the CLI tool.

After the CLI tool is successfully configured, you can use it to manually configure kube-dns/CoreDNS HA.

Step 4 Log in to the client.

Step 5 Edit the deployment configuration file of kube-dns/CoreDNS.

The following uses the CoreDNS as an example:

kubectl edit deployment coredns -n kube-system

Change the value of replicas in the spec section in the deployment configuration file to thenumber of CoreDNS instances required.

Example:

apiVersion: extensions/v1beta1kind: Deploymentmetadata: annotations: deployment.kubernetes.io/revision: "1" creationTimestamp: 2019-02-11T09:36:04Z generation: 1 labels: app: coredns kubernetes-app: coredns kubernetes.io/cluster-service: "true" kubernetes.io/name: CoreDNS release: cceaddon-coredns name: coredns




namespace: kube-system resourceVersion: "1927" selfLink: /apis/extensions/v1beta1/namespaces/kube-system/deployments/coredns uid: 737b9296-2de0-11e9-b629-fa163e7fb882spec: progressDeadlineSeconds: 600 replicas: 2 revisionHistoryLimit: 10 selector: matchLabels: app: coredns kubernetes-app: coredns strategy: rollingUpdate: maxSurge: 10% maxUnavailable: 0 type: RollingUpdate template: metadata: annotations: checksum/config: 3095a9b4028195e7e0b8b22c550bf183d0b7a8a7eba20808b36081d0b39f8b81

----End

3.6 Creating a Node in a VM ClusterA node is a virtual or physical machine that provides computing resources. You must havesufficient node resources in your cluster to ensure that operations, such as creatingapplications can be performed.

Prerequisitesl A cluster is available. For more information about how to create a cluster, see 3.2

Creating a VM Cluster.l A key pair has been created for identity authentication during remote node login. For

more information about how to create a key pair, see 2.1 Creating a VPC and a KeyPair.

Creating a Node


Step 2 In the navigation pane, choose Resource Management > VM Clusters. Click Create Nodefor the cluster where you want to create a node.

Step 3 Set the parameters for creating a node.

Table 3-6 Parameters for creating a node


AZ Physical location where resources use independent power suppliesand networks. AZs are physically isolated but interconnectedthrough an internal network. To improve application reliability, youare advised to create cloud servers in different AZs.

Node Name Name of the node to be created.




NodeSpecifications

l General-purpose: provides general computing, storage, andnetwork configurations for the majority of application scenarios;typically used for web servers, development and testenvironments, and small database applications.

l Memory-optimized: provides instances with a larger memorysize; typically used in memory-intensive applications that processa large amount of data, such as relational databases and NoSQLdatabases.

l General computing-plus: provides dedicated resources to ensurestable performance; typically used for enterprise-gradeapplications that require high computing performance andstability.

Operating System Only EulerOS 2.2 is supported.OS upgrade is not supported in the current version.

Nodes Number of nodes.

EIP An independent public IP address. If a node needs to access theInternet, assign a new EIP or use an existing EIP.NOTE

By default, the SNAT function of VPCs is disabled on CCE. If SNAT isenabled, EIPs are not required for accessing external networks.

l Do not use: A cloud server without an EIP cannot access theInternet. It can be used only as a cloud server for deployingservices or clusters on a private network.

l Automatically assign: An EIP with exclusive bandwidth isautomatically assigned to each cloud server. When creating anECS, ensure that the EIP quota is sufficient. Set the specificationsand bandwidth as required.

l Specify: An existing EIP is assigned to the cloud server.

Disk Disk type, which can be System Disk or Data Disk.l The system disk capacity is 40–1024 GB, which is user defined.

The default value is 40 GB.l The data disk capacity is 100–32678 GB, which is user defined.

The default value is 100 GB.Data disks deliver two levels of I/O performance:l Common I/O: EVS disks of this level provide reliable block

storage and a maximum IOPS of 1000 per disk. They are suitablefor key applications.

l Ultra-high I/O: EVS disks of this level provide a maximum IOPSof 20,000 and a minimum read/write latency of 1 ms. They aresuitable for RDS, NoSQL, and data warehouse applications.

Key Pair Used for identity authentication when you remotely log in to a node.Select an existing key pair.If no key pair is available, click Create a key pair and create one.



Step 4 Click Create Now, review the details, and click Submit.

----End

Logging In to a VM NodeUse the key authentication mode to log in to a VM node created by CCE. For moreinformation, see Login Using an SSH Key.

NOTE

If you use a Windows OS to log in to a Linux node, set the image username (that is, Auto-loginusername) to root.

Deleting a NodeDeleting a node will also delete applications and services running on the node. Exercisecaution when performing this operation.

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > Nodes.

Step 2 Click Delete for the node to be deleted.

Step 3 Follow the prompts to delete the node.

----End

3.7 Cluster Auto ScalingCluster auto scaling dynamically changes the number of nodes in a cluster based on serviceloads. When applications cannot be scheduled due to insufficient resources in a cluster, thecluster will be automatically scaled out, which reduces labor costs.

NOTE

l Currently, master nodes in clusters cannot be scaled out or in.

l Auto scaling-in of worker nodes is not supported. You need to manually scale in worker nodes basedon the resource usage of the nodes.

l The autoscaler add-on automatically adjusts the size of a Kubernetes cluster so that all pods have aplace to run and there are no unnecessary nodes. For details, see autoscaler.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > VMClusters. Click Auto Scaling for the cluster that you want to scale out.

Step 2 Click Edit, and configure the parameters for configuring AS policies in Table 3-7 asspecified.



https://support.huaweicloud.com/en-us/eu-west-0-usermanual-ecs/en-us_topic_0017955380.html

Table 3-7 Parameters for configuring AS policies


Minimum Nodes Minimum number of nodes in a cluster.The value must be 1 or greater, and smaller than the maximumnumber of nodes in a cluster.

Maximum Nodes Maximum number of nodes in a cluster.The value must be greater than the minimum number of nodes in acluster, and be equal to or smaller than the node quota of thecluster.NOTE

The node quota of a cluster depends on the maximum number of nodesallowed in a single cluster or the node quota of your account. The smallerof these two values is used as the node quota of a cluster.

Cooldown Period (s) Interval (in seconds) between consecutive scaling operations. Thecooldown period ensures that a scaling operation is initiated onlywhen a previous scaling operation is finished and the system isrunning stably.Value range: 60–3600Default value: 900It takes about 2 to 10 minutes to create a node. If the cooldownperiod is less than 900 seconds, node creation may fail.

Node Configuration If capacity expansion is required after the scaling policy isexecuted, the system creates a node.1. Click Set and set the node parameters. For details about how to

set the node parameters, see Table 3-6.2. Click OK.

Step 3 Review the scaling configuration and node parameters, and click OK.

Step 4 Click the Scale-out Policies tab, and click Add Policy.l Policy Name: Enter a policy name (for example, policy01).l Set Policy Type. Currently, the following types of auto scaling policies are supported:

– Metric-based policy: scaling based on the CPU or memory settings. Relevantparameters are described in Table 3-8.

Table 3-8 Parameters for adding a metric-based policy


Policy Type Set this parameter to Metric-based policy.

Metric Select Allocated CPU or Allocated memory.

Trigger Condition Conditions for triggering a policy when the average CPUor memory allocation value is greater than or less than aspecified percentage.




Duration Metric monitoring interval.For example, if you set this parameter to 15 min, themetrics are monitored every 15 minutes.

Consecutive Times If you set this parameter to 3, the action is triggered if themetrics meet the specified threshold three consecutivetimes.

Action Action executed after all the conditions for a specifiedpolicy are met.

– Scheduled policy: scaling at a specified time. Relevant parameters are described in

Table 3-9.

Table 3-9 Parameters for adding a scheduled policy


Policy Type Set this parameter to Scheduled policy.

Trigger Time Time at which a policy is triggered.


– Periodic policy: scaling at a specified time on a daily, weekly, or monthly basis.

Relevant parameters are described in Table 3-10.

Table 3-10 Parameters for adding a periodic policy


Policy Type Set this parameter to Periodic policy.

Select Time Time at which a policy is triggered.


Step 5 Click OK.

----End

3.8 Changing Cluster SpecificationsThis section describes how to change cluster specifications.

NOTE

If you set Network Model to VPC network during cluster creation, the cluster specifications cannot bechanged after the cluster is created.



Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > VMClusters.

Step 2 Choose More > Resize.

Step 3 Change the cluster management scale as required and click Submit.

----End

3.9 Managing Node LabelsNode labels are attached to nodes to define different attributes for the nodes, facilitating nodemanagement and affinity or anti-affinity configuration.

Application Scenarios

Node labels are mainly used in the following scenarios:

l Node management: Labels are used to classify and manage nodes.

l Affinity or anti-affinity between applications and nodes:

– Memory size, I/O performance, and the number of CPU cores required forapplications vary depending on service demands. You can attach labels to definethese attributes for nodes, so that applications can be deployed on appropriate nodesbased on affinity or anti-affinity policies.

– A system can be divided into modules, where each module consists of multiplemicroservices. To ensure efficient O&M, you can attach module labels to nodes, sothat the modules can be deployed on their corresponding nodes. These moduleswork independently without affecting each other and can be easily maintained.

Fixed Labels

Table 3-11 lists the fixed labels attached to a node when it is created.

Table 3-11 Fixed labels

Key Value

failure-domain.beta.kubernetes.io/region

Region where a node is located.

failure-domain.beta.kubernetes.io/zone

AZ where a node is located. Ensure that applications towhich dynamic storage is mounted can be scheduled onlyto the nodes in the AZ to which the storage belongs.

os.architecture Node processor architecture.For example, amd64 indicates a 64-bit AMD processor.



Key Value

os.name Operating system name of a node.For example, EulerOS_2.0_SP2 indicates that the EulerOS2.2 is used.

os.version Kernel version of a node.For example, 3.10.0-327.62.59.83.h112.x86_64

supportContainer Whether a node can run containerized applications.For example, true indicates that the node can runcontainerized applications.

Creating a Node Label

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > Nodes.The node list is displayed. Click Manage Labels in the Operation column.

Step 2 Click Add Label, specify the key and value of the label that you want to create, and clickOK.

For example, to indicate that the node is used to deploy a QA (test) environment, you cancreate a node label in which Key is set to deploy_qa and Value is set to true.

Step 3 After "Label updated successfully." is displayed, click Manage Labels. The label that youhave added is displayed.

----End

Deleting a Node LabelOnly the labels you created can be deleted. Fixed labels cannot be deleted.

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > Nodes.The node list is displayed. Click Manage Labels in the Operation column.

Step 2 Click Delete and click OK to delete the label.

----End

3.10 Deleting a ClusterExercise caution when deleting a cluster because this operation will delete the nodes in thecluster and running applications and services.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > VMClusters.

Step 2 Choose More > Delete. Follow the prompts to delete the cluster.

----End



3.11 Cluster Statuses

Table 3-12 Cluster statuses

Status Description

Creating The cluster is being created and is requesting cloud resources.

Available The cluster is running properly.

Scaling out A node is being added to the cluster.

Scaling in A node is being deleted from the cluster.

Changingspecifications

The maximum number of nodes that can be managed by the clusteris being changed.

Upgrading The cluster is being upgraded.

Unavailable The cluster is not available for use.After a cluster is created, a security group is automatically created.Do not modify the security group. Otherwise, the cluster willbecome unavailable.

Deleting The cluster is being deleted.



Figure 3-3 Cluster statuses

3.12 Monitoring a NodeCCE allows you to monitor the resource usage of a cluster and nodes in the cluster.

Procedure


Step 2 Monitor the cluster resource usage.

1. In the navigation pane, choose Resource Management > VM Clusters. Click the nameof the cluster to be monitored. The cluster details page is displayed.

2. Click the Monitoring tab to view the CPU and memory information.

Step 3 Monitor the resource usage of a node in the cluster.

1. In the navigation pane, choose Resource Management > Nodes. Click the name of thenode to be monitored. The node details page is displayed.

2. Click the Monitoring tab to view the CPU and memory information.

----End



3.13 Managing NamespacesNamespaces enable division of cluster resources and objects among multiple users. Typically,namespaces are best suited for scenarios where a large number of users work across multipleprojects. Multiple namespaces can be created in a single cluster with the data isolated fromeach other. This enables namespaces to share the services of the same cluster withoutaffecting each other.

For example, you can deploy applications in a development environment in one namespace,and deploy applications in a test environment in another namespace.

Prerequisites

You have created at least one cluster. For details, see 3.2 Creating a VM Cluster.

Namespace Types

Namespaces can be created automatically or manually.

l Created automatically by a cluster: When the cluster is started, the default, kube-public,and kube-system namespaces are created by default.– default: Used by default if no namespace is specified.– kube-public: Used for deploying public plug-ins and container templates.– kube-system: Used for deploying the Kubernetes system components.

l Created manually: You can create namespaces as required. For example, you can createdifferent namespaces for a development environment, joint debugging environment, andtest environment. You can also create namespaces for different applications. Forexample, you can create one namespace for login services and one for game services.

Creating a Namespace

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management >Namespaces, and click Create Namespace.

Step 2 Set the parameters for creating a namespace listed in Table 3-13. The parameters marked withan asterisk (*) are mandatory.

Table 3-13 Parameters for creating a namespace


* Namespace Name of the namespace, which must be unique in a cluster.

* Cluster Cluster to which the namespace belongs.

Description Description of the namespace.

Step 3 Click OK.

----End



Using Namespacesl When you create an application, you can select a namespace for it.l When you query applications, select a namespace to view all applications in the

namespace.

Namespace Application Scenariosl Dividing applications into namespaces by environment type

Before being released, an application generally goes through the phases of development,joint debugging, testing, and production. You can create different clusters or differentnamespaces in the same cluster.– Creating clusters for different environments:

Resources cannot be shared among different clusters. A load balancer is required inorder to enable mutual access between services in different environments.

– Creating namespaces in the same cluster for different environments:Applications in the same namespace access each other using service names, whileapplications in different namespaces access each other using service names andnamespace names.Figure 3-4 shows namespaces respectively created for the development, jointdebugging, and testing environments.

Figure 3-4 Dividing applications into namespaces by environment type

l Dividing applications into namespaces by application typeYou are advised to use this method if a large number of applications are deployed in thesame environment. As shown in the following figure, different namespaces are createdfor App 1 and App 2. Applications in a namespace are managed as an application group.Applications in the same namespace access each other using service names, whileapplications in different namespaces access each other using service names andnamespace names.



Figure 3-5 Dividing applications into namespaces by application type

Deleting a NamespaceIf a namespace is deleted, all resources (such as applications, one-off jobs, and ConfigMaps)in this namespace will be also deleted. Exercise caution when deleting a namespace.

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management >Namespaces. The Namespaces page is displayed.

Step 2 In the Clusters drop-down list, select the cluster where the namespace to be deleted islocated.

Step 3 Select the namespace to be deleted and click Delete.

Follow the prompts to delete the namespace. The built-in namespaces of the system cannot bedeleted.

----End



4 Application Management

4.1 Application OverviewYou can create two types of applications on CCE: stateful applications and statelessapplications.

Basic Conceptsl Stateless application instances are independent from each other and provide the same

functions. They support auto scaling and rolling upgrades. Examples of statelessapplications include Nginx and WordPress. For more information on how to create astateless application, see 4.2 Creating a Stateless Application.

l Stateful application instances are dependent on each other and have stable persistentstorage and network identifiers. They support ordered deployment, scaling in, anddeletion. Examples of stateful applications include MySQL HA and etcd applications.For more information on how to create a stateful application, see 4.3 Creating a StatefulApplication.

Applications and ContainersAs shown in Figure 4-1, an application consists of one or more instances. An instanceconsists of one or more containers. Each container corresponds to a container image. Allinstances of a stateless application are identical.

Cloud Container EngineUser Guide 4 Application Management


Figure 4-1 Applications and containers

Application Statuses

Table 4-1 Application statuses

Status Description

Running All instances are in the running state.

Not ready Stateless applications: All containers are in the pending state.Stateful applications: Some containers are in the pending state.

Upgrading The application is being upgraded.

Stopped The application is stopped and the number of instances is now 0.

Deleting The application is being deleted.

Available (Applies to stateless applications) At least one instance is available.Some application instances may be abnormal.

4.2 Creating a Stateless ApplicationStateless application instances are independent from each other and provide the samefunctions. They support auto scaling and rolling upgrades. Examples of stateless applicationsinclude Nginx and WordPress.

Prerequisitesl A cluster is available. For details on how to create a cluster, see 3.2 Creating a VM

Cluster.

NOTE

When creating multiple containerized applications, ensure that each application has a unique port.Otherwise, application deployment will fail.

l To enable access to an application from Internet, ensure that an elastic IP address hasbeen bound to or ELB instances have been configured for at least one node in the cluster.



Creating a Stateless Application on the CCE Console

Step 1 (Optional) If you are creating an application based on your own image, upload the image tothe image management service. For details about how to upload an image, see 11 ImageRepository.

Step 2 In the navigation pane, choose Application Management. Click Create Application, and setSelect Application Type to Stateless Applications.

Step 3 Set basic application parameters as described in Table 4-2. The parameters marked with anasterisk (*) are mandatory.

Table 4-2 Basic application parameters


* ApplicationName

Name of the containerized application to be created. The name mustbe unique.

* Cluster Name Cluster in which the application resides.

* Namespace Namespace in which the application resides. By default, thisparameter is set to default.

Application Group You can manage (start, stop, and delete) applications in batches byadding these applications to the same application group.When you enter an application group name, a user group isautomatically created.

* Instances Number of instances in the application. Each application has at leastone instance. You can specify the number of instances as required.Each application instance consists of the same containers.Configuring multiple instances for an application ensures that theapplication can still run properly even if an instance is faulty.

Description Description of the application.

Step 4 Click Next to add a container.

1. Click Add Container and select the image to be deployed. Click OK.– The My Images tab page displays all images you created.– Third-party Images: CCE allows you to create an application using an image

pulled from a third-party image repository, rather than a public cloud imagerepository or a Docker Hub image repository. When you create an application usinga third-party image, ensure that the node where the application is running canaccess public networks. For details about how to create an application using a third-party image, see 4.10 Using a Third-party Image to Create an Application.n If your image repository does not require authentication, set Authenticate

Secret to No, specify Image Address, for example, nginx:latest, and clickOK.

n If your image repository is accessible only after being authenticated byaccount and password, set Authenticate Secret to Yes. You need to create asecret first and then user a third-party image to create an application.



2. Set image parameters.

Table 4-3 Image parameters


Image Name Imported image. You can click Change Image to update it.

* Image Version Version of the image to be deployed.

* ContainerName

Name of the container. You can modify it.

ContainerResources

For more information about Request and Limit, see 4.5 SettingContainer Specifications.– Request: the amount of resources that CCE will guarantee to a

container.– Limit: the maximum amount of resources that CCE will allow

a container to use. You can set Limit to prevent system faultscaused by container overload.

3. Set environment variables.

Environment variables are set in the container running environment and can be modifiedafter application deployment to ensure the flexibility of applications.

a. Click Add.b. Set Type to Added manually.c. Set Variable Name and Variable/Variable Reference.

4. Set data storage.You can mount a host directory, EVS disk, SFS, and configuration items and secrets tothe corresponding directories of a container instance. For details, see 8 StorageManagement.

NOTE

If you set Allocation Mode to Automatic when adding data storage, the created storage will notbe tagged.

5. Set log policy.Set a policy and log directory for collecting application logs and preventing logs fromexceeding size limits. For details, see 9 Log Management.

6. (Optional) One application instance contains one or more related containers. If yourapplication contains multiple containers, click Add Container and then add containers.

Step 5 Click Next. Then, click Add Access Mode, and set the application access mode.

To enable access to the application from other applications or the Internet, set the applicationaccess mode.

The application access mode determines the network attributes of an application. Applicationswith different access modes can provide different network capabilities.

At present, the following access modes are provided:

l Intra-Cluster Access: An application is accessible to other applications in the samecluster by using an internal domain name.



l Intra-VPC Access: An application is accessible to other applications in the same VPCby using the IP address of the cluster node or the ELB service address of the privatenetwork.

l External Access - Elastic IP Address: An application is accessible to public networksby using an EIP. This access mode is applicable to services that need to be exposed topublic networks. To enable access to an application from the Internet, an elastic IPaddress must be bound to a node in the cluster, and a mapping port number must be set.The port number ranges from 30000 to 32767, for example, the access address can be10.117.117.117:30000.

l External Access - Elastic Load Balancer: An application is accessible to publicnetworks by using an ELB address. This access mode provides higher reliability thanEIP-based access and is applicable to services that need to be exposed to publicnetworks. The access address consists of the ELB service address of the public networkand the configured access port, for example, 10.117.117.117:80.

For details about the access modes, see 5 Application Access Settings.

Step 6 Click OK, and then click Next.

Step 7 (Optional) Configure advanced settings for the application.

1. Configure the upgrade policy, as listed in Table 4-4.

Table 4-4 Upgrade policy

UpgradeMode

Description

In-placeupgrade

In this upgrade mode, the old instance needs to be deleted before anew instance is created. Services are interrupted during the upgrade.

Rollingupgrade

The instance of the old version is gradually replaced with the instanceof the new version. During the upgrade, service traffic is evenlydistributed between new and old instances, so services are notinterrupted.

2. Migration Policy: Provides a time window for the application instances to be

rescheduled to other available nodes when the node where the application instance islocated is unavailable. The default value is 300s.

3. Scheduling policy: You can combine static global scheduling policies or dynamicruntime scheduling policies as required. For details, see 4.6 Affinity and Anti-AffinityScheduling.

4. User-Defined Monitoring: metrics collection mechanism provided by the monitoringsystem. This mechanism allows you to define the names of the metrics to be collectedand the path and port for reporting the metric data when deploying an application. Whenan application is running, the monitoring system regularly collects metric data using thespecified path and port. For details, see 4.8 Interconnection with Prometheus(Monitoring).

5. APM Settings: APM helps you quickly locate application problems and identifyperformance bottlenecks to improve user experience. For details, see 4.9 APM Settings(Performance Bottleneck Analysis).

Step 8 Click Create Now. Click Back to Application List.



In the application list, if the application status is Running, the application has been createdsuccessfully. Application status is not updated in real time. To view the application status,press F5.

Step 9 To access the application in a browser, go to the application list on the ApplicationManagement page. Copy the corresponding External Access Address and paste it into theaddress box in the browser.

NOTE

External access addresses can be obtained only when the application access mode is set to Elastic IPAddress or Elastic Load Balancer.

----End

Creating a Stateless Application Using kubectl

The following procedure uses an Nginx application as an example to describe how to createan application using kubectl.

Prerequisites

You have configured the kubectl commands and connected an ECS to the cluster. For details,see 3.4 Connecting to a Kubernetes Cluster Using kubectl.

Procedure

Step 1 Log in to the ECS on which the kubectl commands have been configured. For details, seeLogin Using an SSH Key.

Step 2 Create and edit the nginx-deployment.yaml file. nginx-deployment.yaml is an example filename, and you can change it as required.

vi nginx-deployment.yaml

The following provides an example of the description file contents. For more information ondeployment, see the Kubernetes documentation.

apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret

Step 3 Create a containerized application.

kubectl create -f nginx-deployment.yaml




https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

If the following information is displayed, the application is being created.

deployment "nginx" created

kubectl get po

If the following information is displayed, the application is running.

NAME READY STATUS RESTARTS AGEicagent-m9dkt 0/0 Running 0 3dnginx-1212400781-qv313 1/1 Running 0 3d

Step 4 If the application needs to be accessed by other nodes in the same cluster, in the same VPC, orin a public network, set the application access mode. For details, see 5 Application AccessSettings.

----End

4.3 Creating a Stateful ApplicationStateful application instances are dependent on each other and have stable persistent storageand network identifiers. They support ordered deployment, scaling in, and deletion. Examplesof stateful applications include MySQL HA and etcd applications.

Prerequisites

A cluster is available. For details on how to create a cluster, see 3.2 Creating a VM Cluster.

NOTE

When creating multiple containerized applications, ensure that each application has a unique port.Otherwise, application deployment will fail.

Procedure

Step 1 (Optional) If you are creating an application based on your own image, upload the image tothe image management service. For details about how to upload an image, see 11 ImageRepository.

Step 2 In the navigation pane, choose Application Management. Click Create Application, and setSelect Application Type to Stateful Applications.

Step 3 Set basic application parameters as listed in Table 4-5. The parameters marked with anasterisk (*) are mandatory.

Table 4-5 Basic application parameters


* ApplicationName


* Cluster Name Cluster in which the application resides.

* Namespace Namespace in which the application resides. By default, thisparameter is set to default.




Application Group You can manage (start, stop, and delete) applications in batches byadding these applications to the same application group.When you enter an application group name, a user group isautomatically created.

* Instances Number of instances in the application. Each application has at leastone instance. You can specify the number of instances as required.Each application instance consists of the same containers.Configuring multiple instances for an application ensures that theapplication can still run properly even if an instance is faulty.

Description Description of the application.


1. Click Add Container and select the image to be deployed. Click OK.– The My Images tab page displays all images you created.– Third-party Images: CCE allows you to create an application using an image







Image Name Imported image. You can click Change Image to update it.

* Image Version Version of the image to be deployed.

* ContainerName

Name of the container. You can modify it.

ContainerResources

For more information about Request and Limit, see 4.5 SettingContainer Specifications.– Request: the amount of resources that CCE will guarantee to a

container.– Limit: the maximum amount of resources that CCE will allow

a container to use. You can set Limit to prevent system faultscaused by container overload.



3. Set environment variables.

Environment variables are set in the container running environment and can be modifiedafter application deployment to ensure the flexibility of applications.

a. Click Add.b. Set Type to Added manually.c. Set Variable Name and Variable/Variable Reference.

4. Set data storage.You can mount a host directory, EVS disk, SFS, and configuration items and secrets tothe corresponding directories of a container instance. For details, see 8 StorageManagement.

NOTE

If you set Allocation Mode to Automatic when adding data storage, the created storage will notbe tagged.

5. Set log policy.Set a policy and log directory for collecting application logs and preventing logs fromexceeding size limits. For details, see 9 Log Management.

6. (Optional) One application instance contains one or more related containers. If yourapplication contains multiple containers, click Add Container and then add containers.

Step 5 Click Next. Set the headless service parameters listed in Table 4-7.

Table 4-7 Headless service parameters


* ServiceName

Name of the service corresponding to the application for mutual accessbetween instances. This service is used for internal discovery of instances,and does not require an independent IP address or load balancing.

* Port Name Name of the container port. You are advised to enter a name that indicatesthe function of the port.

* ContainerPort

Listening port of the container.

Step 6 (Optional) Click Add Access Mode, and set the application access mode.

The application access mode determines the network attributes of an application. Applicationswith different access modes can provide different network capabilities. For details about theaccess modes, see 5 Application Access Settings.

Step 7 Click Next.

Step 8 (Optional) Configure advanced settings for the application.l Upgrade Mode: Rolling Upgrade.l Scheduling policy: You can combine static global scheduling policies or dynamic

runtime scheduling policies as required. For details, see 4.6 Affinity and Anti-AffinityScheduling.



l User-Defined Monitoring: metrics collection mechanism provided by the monitoringsystem. This mechanism allows you to define the names of the metrics to be collectedand the path and port for reporting the metric data when deploying an application. Whenan application is running, the monitoring system regularly collects metric data using thespecified path and port. For details, see 4.8 Interconnection with Prometheus(Monitoring).

l APM Settings: APM helps you quickly locate application problems and identifyperformance bottlenecks to improve user experience. For details, see 4.9 APM Settings(Performance Bottleneck Analysis).

Step 9 Click Create Now. Click Back to Application List.

In the application list, if the application status is Running, the application has been createdsuccessfully. Application status is not updated automatically in real time. To update theapplication status, press F5.

----End

Creating a Stateful Application Using kubectlThe following procedure uses an Nginx application as an example to describe how to createan application using kubectl.

Prerequisites

You have configured the kubectl commands and connected an ECS to the cluster. For details,see 3.4 Connecting to a Kubernetes Cluster Using kubectl.

Procedure

Step 1 Log in to the ECS on which the kubectl commands have been configured. For details, seeLogin Using an SSH Key.

Step 2 Create and edit the etcd-statefulset.yaml file. etcd-statefulset.yaml is an example file name,and you can change it as required.

vi etcd-statefulset.yaml

The following provides an example of the file contents. For more information on StatefulSet,see the Kubernetes document.

apiVersion: apps/v1beta1kind: StatefulSetmetadata: name: etcdspec: replicas: 2 selector: matchLabels: app: etcd serviceName: etcd-svc template: metadata: labels: app: etcd spec: containers: - env: - name: PAAS_APP_NAME value: tesyhhj - name: PAAS_NAMESPACE value: default




https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/

- name: PAAS_PROJECT_ID value: 9632fae707ce4416a0ab1e3e121fe555 image: etcd imagePullPolicy: IfNotPresent name: container-0 updateStrategy: type: RollingUpdate

vi etcd-headless.yaml

apiVersion: v1kind: Servicemetadata: labels: app: etcd name: etcd-svcspec: clusterIP: None ports: - name: etcd-svc port: 3120 protocol: TCP targetPort: 3120 selector: app: etcd sessionAffinity: None type: ClusterIP

Step 3 Create an application and the corresponding headless service.

kubectl create -f etcd-statefulset.yaml


statefulset "etcd" created

kubectl create -f etcd-headless.yaml

If the following information is displayed, the headless service has been created.

service "etcd-svc" created

Step 4 If the application needs to be accessed by other nodes in the same cluster, in the same VPC, orin a public network, set the application access mode. For details, see 5 Application AccessSettings.

----End

4.4 Basic Operations on Applications

Deleting an Application

Delete an application that you do not need to use any longer. Applications cannot be restoredafter being deletion. Exercise caution when you perform this operation.


Step 2 Click More > Delete for the application to be deleted, and follow the prompts to delete thecluster.

Step 3 Click OK.

----End



Upgrading an Application

CCE enables you to quickly upgrade applications by replacing images or image versionswithout interrupting services.

To replace an image or image version, you need to upload the image to the image repositoryin advance.

Step 1 Log in to the CCE console. In the navigation pane, choose Application Management. On thepage that is displayed, click the application to be upgraded. On the Application Details page,click the Upgrading tab.

Step 2 Upgrade the application based on service requirements.

l To replace the image, click Change Image and select a new image.

l To replace the image version, select a version from the Image Version drop-down list.

l To change the container name, click next to Container Name and enter a new name.

l For details on how to configure advanced settings, see Table1 Advanced settings.

Table 4-8 Advanced settings


EnvironmentVariables

Environment variables are set in the container runningenvironment and can be modified after application deployment toensure the flexibility of applications. You can set environmentvariables as follows:1. On the Environment Variables tab page, click Add.2. Set Variable Name and Variable/Variable Reference.

Data Storage This parameter cannot be updated.

Log Policies This parameter cannot be updated.

Step 3 After the upgrade is completed, click Submit.

----End

Monitoring an Application

After an application is created, you can go to the Monitoring page to monitor the CPU usageand memory usage of the container in which the application resides.


Step 2 Click the name of the application to be monitored. The Application Details page is displayed.

Step 3 Click the Instances tab. Click next to an instance to be monitored and click Monitoring.

Step 4 Check the CPU usage and memory usage of the instance.



NOTE

CCE needs time to compute CPU usage. Therefore, when CPU and memory usage are displayed for thefirst time, CPU usage is displayed about one minute later than memory usage

CPU and memory usage are displayed only for instances in the running state.

----End

Adding Labels to Applications

Labels are attached to applications using key-value pairs. Applications with labels attachedcan be easily selected for setting affinity and anti-affinity scheduling rules. You can add labelsto multiple applications or a specified application.

In the following figure, three labels release, env, and role are defined for the applicationsAPP1, APP2, and APP3. The values of these labels vary with applications.

l Label of APP 1: [release:alpha;env:development;role:frontend]l Label of APP 2: [release:beta;env:testing;role:frontend]l Label of APP 3: [release:alpha;env:production;role:backend]

If you set key to role and value to frontend when using application scheduling or anotherfunction, the function will apply to APP1 and APP2.

Figure 4-2 Label example

Step 1 In the navigation pane, choose Application Management.

Step 2 Click the application for which a label is to be added. The Application Details page isdisplayed.

Step 3 Click Manage Labels and Add Label, specify the key and value of the label that you want tocreate, and click OK.

NOTE

A key-value pair must start and end with a letter or digit and consist of a maximum of 63 characters,including letters, digits, hyphens (-), underscores (_), and periods (.).

----End

4.5 Setting Container SpecificationsCCE allows you to set specifications for added containers during application creation. Youcan set Request and Limit for CPU and memory resources used by each instance in anapplication.



NOTE

If you select Request next to CPU and Memory and specify a value for Request, CCE schedules theapplication instance to a node that has the resources specified. If you deselect Request, CCE schedulesan application instance to a random node. If you select Limit and specify a value for Limit, CCE limitsthe resources that can be used by the application instance based on the value specified. If you deselectLimit, CCE does not limit the resources that can be used by the application instance. The application orthe node may be unavailable when the memory resources used by the instance exceed the memory thatthe node can allocate.

l CPU quotas

Table 4-9 CPU quotas


Request Minimum number of CPU cores required by a container. A container isscheduled to a node on which the total number of available CPU coresis greater than or equal to the value specified by Request. Thisparameter does not limit the maximum number of CPU cores availablefor a container.

Limit Maximum number of CPU cores available for a container.

You are advised to configure the CPU quotas as follows: Actual number of CPU coresavailable for a node ≥ Sum of CPU Limits for all containers of the current instance ≥Sum of CPU Limits for all containers of the current instance. For details about the actualnumber of CPU cores available for a node, go to Resource Management > Nodes andobtain the value from the Available CPUs (Cores) column of the corresponding node.

l Memory quotas

Table 4-10 Memory quotas


Request Minimum amount of memory required by a container. A container isscheduled to a node on which the total amount of available memory isgreater than or equal to the value specified by Request.

Limit Maximum amount of memory available for a container. When thememory usage exceeds the configured limit, the instance may berestarted, which affects running of applications.

You are advised to configure the memory quotas as follows: Actual amount of memoryavailable for a node ≥ Sum of memory Limits for all containers of the current instance ≥Sum of memory Limits for all containers of the current instance. For details about theactual amount of memory available for a node, go to Resource Management > Nodesand obtain the value from the Available Memory (GB) column of the correspondingnode.



Configuration Example

In this example, a cluster contains a node with 4 CPU cores and 8-GB memory, and anapplication containing instance 1 and instance 2 has been deployed in the cluster, and theresource quotas are set for instance 1 and instance 2 as follows: {CPU Request, CPU Limit,Memory Request, Memory Limit} = {1 core, 2 cores, 2 GB, 2 GB}

The CPU usage and memory usage of the node are as follows:

l Number of CPU cores available on the node = 4 core – (1 core requested by instance 1+ 1 core requested by instance 1) = 2 cores

l Amount of memory available on the node = 8 GB – (2 GB requested by instance 1 + 2GB core requested by instance 2) = 4 GB

Therefore, the node has 2 CPU cores and 4 GB memory available.

4.6 Affinity and Anti-Affinity Scheduling

Overview

CCE provides a variety of scheduling policies, including static global scheduling policies anddynamic runtime scheduling policies. You can select or combine these strategies as required.CCE provides the following affinity scheduling modes:

When setting Application-Node Affinity and Anti-Affinity and Application-ApplicationAffinity and Anti-Affinity, ensure that the affinity relationships are not mutually exclusive;otherwise, application deployment will fail. For example, application deployment will fail inthe following cases:

l Anti-affinity is configured for two applications. That is, one application is deployed on onenode and a second application is deployed on another node.

l When a third application is deployed on a third node and goes online, affinity isconfigured between this application and the second application.

l Application-AZ Affinity and Anti-Affinity– Affinity with AZs: Applications can be deployed in specific AZs.– Anti-Affinity with AZs: Applications cannot be deployed in specific AZs.

l Application-Node Affinity and Anti-Affinity– Affinity with Nodes: Applications can be deployed on specific nodes.– Anti-Affinity with Nodes: Applications cannot be deployed on specific nodes.

l Application-Application Affinity and Anti-Affinity: Determines whether applicationsare deployed on the same or different nodes.– Affinity with Applications: Applications are deployed on the same node. You can

deploy applications based on service requirements. The nearest route betweencontainers is used to reduce network consumption. For example, Figure 4-3 showsaffinity deployment, in which all apps are deployed on the same node.



Figure 4-3 Application affinity

– Anti-Affinity with Applications: Different applications or multiple instances of thesame application are deployed on different nodes. Anti-affinity deployment formultiple instances of the same application reduces the impact of systembreakdowns. Anti-affinity deployment for applications can prevent interferencebetween the applications.In Figure 4-4, four apps are deployed on four different nodes. The four applicationsare deployed in anti-affinity mode.

Figure 4-4 Application anti-affinity

Deploying an Application on a Specified Node

Affinity settings are configured during application creation. For details on the applicationcreation procedure, see 4.2 Creating a Stateless Application or 4.3 Creating a StatefulApplication.

Step 1 During the application creation process, in the Scheduling Policy area on the ConfigureAdvanced Settings page, choose Application-Node Affinity and Anti-Affinity > Affinitywith Nodes. Click Add.

Step 2 Select the node on which you want to deploy the application, and click OK.

If multiple nodes are selected, the system automatically chooses one of them duringapplication deployment.

----End

Example YAML for Deploying an Application on a Specified Node

This section uses an Nginx application as an example to describe how to deploy anapplication on a specified node using kubectl.



Prerequisites

You have configured the kubectl commands to connect an ECS server to your cluster. Fordetails, see 3.4 Connecting to a Kubernetes Cluster Using kubectl.

Procedure

Create an application and set the affinity attributes for the application as follows. For moreinformation about how to create an application, see Creating a Stateless Application Usingkubectl or Creating a Stateful Application Using kubectl.

apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: nodeName #Label key of a node operator: In values: - test-node-1 #Key value of a node

Deploying an Application with Node Anti-Affinity


Step 1 During the application creation process, in the Scheduling Policy area on the ConfigureAdvanced Settings page, choose Application-Node Affinity and Anti-Affinity > Anti-Affinity with Nodes. Click Add.

Step 2 Select the node on which you do not want to deploy the application, and click OK.

If multiple nodes are selected, the application will not be deployed on any of these nodes.

----End

Example YAML for Deploying an Application with Node Anti-Affinity

This section uses an Nginx application as an example to describe how to deploy anapplication with node anti-affinity using kubectl.



Procedure


Procedure


apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: nodeName # Label key of a node. operator: NotIn # Indicates that the application will not be deployed on the node. values: - test-node-1 # Key value of a node.

Deploying Applications on the Same NodeAffinity settings are configured during application creation. For details on the applicationcreation procedure, see 4.2 Creating a Stateless Application or 4.3 Creating a StatefulApplication.

Step 1 During the application creation process, in the Scheduling Policy area on the ConfigureAdvanced Settings page, choose Application-Application Affinity and Anti-Affinity >Affinity with Applications. Click Add.

Step 2 Select the applications that you want to deploy on the same node as the created application,and click OK.

The created application will be deployed on the same node as the selected applications.

----End



Example YAML for Deploying Applications on the Same NodeThis section uses an Nginx application as an example to describe how to deploy anapplication using kubectl.

Procedure


Procedure


apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret affinity: podAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: app # Label key of an application. operator: In values: - test # Label value of an application.

Deploying Applications on Different NodesAffinity settings are configured during application creation. For details on the applicationcreation procedure, see 4.2 Creating a Stateless Application or 4.3 Creating a StatefulApplication.

Step 1 During the application creation process, in the Scheduling Policy area on the ConfigureAdvanced Settings page, choose Application-Application Affinity and Anti-Affinity >Anti-Affinity with Applications. Click Add.

Step 2 Select the applications that you do not want to deploy on the same node as the createdapplication, and click OK.

The created application and the selected applications will be deployed on different nodes.

----End



Example YAML for Deploying Applications on Different Nodes

This section uses an Nginx application as an example to describe how to deploy anapplication using kubectl.

Procedure


Procedure


apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret affinity: podAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: app # Label key of an application. operator: NotIn values: - test # Label value of an application.

Deploying an Application in a Specified AZ


Step 1 During the application creation process, in the Scheduling Policy area on the Configure

Advanced Settings page, choose Application-AZ Affinity and Anti-Affinity. Click next to Affinity with AZs.

Step 2 Click the AZ in which you want to deploy the application.

The created application will be deployed in the selected AZ.

----End



Example YAML for Deploying an Application in a Specified AZ

This section uses an Nginx application as an example to describe how to deploy anapplication using kubectl.

Procedure


Procedure


apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/availablezone # Label key of a node. operator: NotIn values: - # Key value of a node.

Deploying an Application with AZ Anti-Affinity


Step 1 During the application creation process, in the Scheduling Policy area on the Configure

Advanced Settings page, choose Application-AZ Affinity and Anti-Affinity. Click next to Anti-Affinity with AZs.

Step 2 Click the AZ in which you do not want to deploy the application.

The created application will not be deployed in the selected AZ.

----End



Example YAML for Deploying an Application with AZ Anti-AffinityThis section uses an Nginx application as an example to describe how to deploy anapplication with AZ anti-affinity using kubectl.

Procedure


Procedure

Create an application and set the affinity attributes for the application as follows. For moreinformation about how to create an application, see Creating a Stateless Application Usingkubectl or Creating a Stateful Application Using kubectl.apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/availablezone # Label key of a node. operator: NotIn values: - eu-west-0a # Key value of a node.

4.7 Application ScalingYou can choose either of the following scaling modes based on your service requirements:

l Auto scaling: includes alarm, scheduled, and periodic policies. This mode automaticallyscales in or out instances on an application based on resource usage, scheduled time, orspecified periods.

l Manual scaling: Manually scale in or out instances on an application immediately afterthe application is created.

Auto ScalingYou can define auto scaling policies as required, eliminating the need to repeatedly adjustresources in response to changes in service load and reducing resource and labor costs.



Currently, CCE supports the following types of automatic application scaling policies:

Metric-based Policy: scaling based on the CPU or memory settings. After an application iscreated, instances in this application can be automatically scaled in or out when the number ofCPU cores or memory amount exceeds or is less than a specified value.

Scheduled Policy: Instances in an application can be automatically scaled in or out at aspecified time. This policy is applicable to high traffic scenarios, such as flash sales andpremier shopping events, where a large number of application instances need to be added.

Periodic Policy: Instances in an application can be automatically scaled in or out daily,weekly, or monthly. This policy is applicable to scenarios where traffic changes periodically.

l Metric-based policy: scaling based on the CPU or memory settings.

a. Log in to the CCE console. In the navigation pane, choose ApplicationManagement. Click the application for which the scaling policy is to be set. On theApplication Details page, click the Scaling tab.

b. In the Auto Scaling area, click Add Scaling Policy.

Table 4-11 Parameters for adding a metric-based policy


Policy Name Name of the scaling policy.

Policy Type Type of the policy. Set it to Metric-based policy.

Metric Set it to CPU usage or Physical memory usage.If you set this parameter to Physical memory usage andset the average value to be greater than 70%, the scalingpolicy is triggered when memory usage exceeds 70%.

Trigger Condition

Duration Metric statistics period. Select a value from the drop-downlist box.If the parameter is set to 60s, metric statistics is collectedevery 60 seconds.

Consecutive Times If the parameter is set to 3, the action is triggered ifthreshold is reached for three consecutive measurementperiods.

Action Whether a scale-in or scale-out is triggered.

c. Click OK.d. In the Auto Scaling area, check that the policy has been started.

Figure 4-5 Policy started

When the trigger condition is met, the auto scaling policy starts automatically.l Scheduled policy: scaling at a specified time.



a. Log in to the CCE console. In the navigation pane, choose ApplicationManagement. On the page that is displayed, click the application for which thescaling policy is to be set. On the Application Details page, click the Scaling tab.


Table 4-12 Parameters for adding a scheduled policy



Policy Type Type of the policy. Set this parameter to Scheduled policy.

Trigger Time Time at which the policy is enforced.



When the trigger time is reached, you can see on the Instances tab page that theauto scaling policy has taken effect.

l Periodic policy: scaling at a specified time on a daily, weekly, or monthly basis.

a. Log in to the CCE console. In the navigation pane, choose ApplicationManagement. On the page that is displayed, click the application for which thescaling policy is to be set. On the Application Details page, click the Scaling tab.


Table 4-13 Parameters for adding a periodic policy



Policy Type Type of the policy. Set this parameter to Periodic policy.

Select Time Time at which the policy is enforced.



When the trigger condition is met, the auto scaling policy starts automatically.

Manual Scaling

Step 1 Log in to the CCE console. In the navigation pane, choose Application Management. On thepage that is displayed, click the application to be scaled. On the Application Details page,click the Scaling tab.

Step 2 In the Manual Scaling area, click to modify the number of instances, and click Save. Theinstance scaling takes effect immediately.



Step 3 On the Instances tab page, check that a new instance is being created. When the instancestatus becomes Running, instance scaling is complete.

----End

4.8 Interconnection with Prometheus (Monitoring)CCE allows you to obtain user-defined metrics and display them in Settings > Metrics >User-defined Metrics in the Application Operation Management (AOM) service.

Before customizing monitoring, you must know about the Prometheus and provide GET APIrequests for obtaining user-defined metrics in your application. Currently, only Gaugemetrics of prometheus can be obtained.

User-Defined Metric Monitoring

Step 1 When you create an application, set User-Defined Monitoring on the Configure AdvancedSettings page.

Step 2 Configure the values by referring to Figure 4-6. Reported port and report path of the user-defined metrics must be specified in your exporter. After the configuration, CCE will obtainthe user-defined metric data in response to the GET request "http://PodIP:reported port/reportpath", for example, http://192.168.1.19:8080/metrics.

Figure 4-6 Setting user-defined metric monitoring

Table 4-14 Parameter description

Parameter Description Mandatory(Yes/No)

ReportPath

URL provided by the exporter for CCE to obtain user-definedmetric data.The path consists of letters, digits, backslashes (/), andunderscores (_), and must start with a backslash (/). Forexample, /metrics.

Yes

Report Port Port provided by the exporter for CCE to obtain user-definedmetric data.The port number is an integer from 1 to 65535. For example,8080.

Yes



https://prometheus.io/

https://prometheus.io/docs/concepts/metric_types/

https://prometheus.io/docs/concepts/metric_types/

Parameter Description Mandatory(Yes/No)

MonitoringMetrics

Name of the user-defined metric provided by the exporter.The name of a user-defined metric is a string of 5 to 100characters. Only letters, digits, and underscores (_) areallowed. The format is as follows: ["User-defined metricname 1","User-defined metric name 2"]. Use commas (,) toseparate multiple user-defined metric names. For example,["cpu_usage","mem_usage"].l If this parameter is not configured, CCE obtains all user-

defined metric data.l If this parameter is configured, for example,

["cpu_usage","mem_usage"], CCE filters user-definedmetrics and obtains only the data of cpu_usage andmem_usage.

No

----End

4.9 APM Settings (Performance Bottleneck Analysis)Currently, monitoring capabilities such as call chain and topology for Java applications aresupported. If you want to monitor the status of a Java application, select JAVA probe andenter a monitoring group name.

NOTE

If you have not enabled the Application Performance Management (APM) service, click click here tosubscribe, select the number of application instances as prompted, and click Buy Now. Afterconfirming the order, click Submit as prompted.

Setting Java Application Monitoring

Step 1 When you create an application, set APM Settings on the Configure Advanced Settingspage and select JAVA probe. The APM service is enabled and the probe is installed on thenode, which consumes a small number of resources. The probe provides monitoringcapabilities, such as call chain, topology, SQL analysis, and stack tracing for Javaapplications.

Step 2 Enter a monitoring group name, for example, testapp. If one or more monitoring groups exist,you can select one from the drop-down list box.

----End

4.10 Using a Third-party Image to Create an ApplicationCCE allows you to pull images from a third-party image repository to create applications.

Generally, you are required to pass authentication using your account and password beforeaccessing a third-party image repository. The secret authentication mode is used for pulling



images from a CCE container. Therefore, you must create a secret for accessing the imagerepository before pulling images.

PrerequisitesWhen you create an application using a third-party image, ensure that the node where theapplication is running can access public networks.

Creating an Application on the GUI

Step 1 Create a secret for accessing a third-party image repository.

In the navigation pane, choose Configuration Center > Secrets. Click Create Secret, and setType to kubernetes.io/dockerconfigjson. For more information, see 7.3 Creating a Secret.In Secret Data, set Username and Password to those used to log in to the third-party imagerepository.

Step 2 Create an application. For more information, see 4.2 Creating a Stateless Application or 4.3Creating a Stateful Application. If you use a third-party image to create an application, setthe parameters as follows:l Set Authenticate Secret to Yes.l Select the secret created in Step 1.l Enter the image address.

Step 3 Click OK.

----End

Creating an Application using kubectl

Step 1 Configure kubectl. For details, see 3.4 Connecting to a Kubernetes Cluster Using kubectl.

Step 2 Log in to the ECS server where kubectl is configured.

Step 3 Create a secret of the dockerconfigjson type using kubectl.kubectl create secret docker-registry myregistrykey --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL

In the preceding commands, myregistrykey indicates the secret name, and other parametersare described as follows:l DOCKER_REGISTRY_SERVER: address of a third-party image repository, for

example, www.3rdregistry.com or 10.10.10.10:443l DOCKER_USER: account used for logging in to a third-party image repositoryl DOCKER_PASSWORD: password used for logging in to a third-party image

repositoryl DOCKER_EMAIL: email of a third-party image repository

Step 4 Use a third-party image to create an application.

The secret of the dockerconfigjson type is used for authentication when you obtain a privateimage. The following is an example of using the myregistrykey for authentication.apiVersion: v1kind: Podmetadata:



name: foo namespace: defaultspec: containers: - name: foo image: www.3rdregistry.com/janedoe/awesomeapp:v1 imagePullSecrets: - name: myregistrykey #Use the secret created in step 3.

----End



5 Application Access Settings

5.1 OverviewCCE provides the following access modes that allow access between applications in differentscenarios:

l Intra-Cluster AccessAn application can be accessed by other applications in the same cluster using an internaldomain name. The internal domain name is in the format of <User-defined accessmode>.<Namespace of the application>.svc.cluster.local, for example,nginx.default.svc.cluster.local.

l Intra-VPC AccessAn application can be accessed by other applications in the same VPC. The applicationcan be accessed through the IP address of the cluster node or the service address of theELB of the private network. The main scenarios are as follows: Other applications in thesame VPC in the cloud need to access the application in the kubernetes cluster.

l External Access - Elastic IP AddressAn EIP is used to access applications from a public network. This access mode isapplicable to services that need to be exposed to a public network in the system. In thisaccess mode, an EIP must be bound to a node in the cluster, and a port must be mappedto the node. The port range is 30000–32767. For example, the access address could be10.0.0.0:30000.

l External Access - Elastic Load BalancerThis access mode is applicable to services that need to be exposed to public networks.Compared with EIP-based access, ELB allows access to applications from a publicnetwork with higher reliability. The access address consists of the IP address of an ELBservice in the public network, followed by the configured access port number, forexample, 10.117.117.117:80.

5.2 Intra-Cluster AccessAn application can be accessed by other applications in the same cluster using an internaldomain name. The internal domain name is in the format of <User-defined access

Cloud Container EngineUser Guide 5 Application Access Settings


mode>.<Namespace of the application>.svc.cluster.local, for example,nginx.default.svc.cluster.local.

Figure 5-1 shows the mapping relationships between access channels, container ports, and anaccess port.

Figure 5-1 Intra-cluster access

Methods for Setting the Access ModeYou can set the access mode using either of the following two methods:

l Set the access mode when creating an application. For details, see Creating anApplication on the CCE Console and Implementing Intra-Cluster Access Usingkubectl.

l Set the access mode after creating an application. This has no impact on the applicationstatus and takes effect immediately. To set the access mode, perform the following steps:

a. In the navigation pane of the CCE console, choose Application Management.Click the application name. On the application details page that is displayed, clickthe Access Mode tab, and then click Add Access Mode.

b. Set the access mode. For details, see Creating an Application on the CCEConsole.

Creating an Application on the CCE Console

Step 1 Create an application. For details, see 4.2 Creating a Stateless Application or 4.3 Creating aStateful Application. In the Set Application Access step, click Add Access Mode, and setthe parameters as follows:l Service Name: Specify a service name. You can use the application name as the service

name.l Access Mode: Select Intra-cluster access.l Protocol: Select a protocol used by the service.l Container Port: Specify a port on which the application listens. The Nginx application

listens on port 80.



l Access Port: Specify a port to map a container port to the cluster's virtual IP address.The port range is 1–65535. The port will be used when the application is accessed usingthe cluster's virtual IP address.

Step 2 Click OK, and then click Next. On the Configure Advanced Settings page that is displayed,click Create Now.

Step 3 Click View Application Details. On the Access Mode tab page, obtain the access address, forexample, 10.247.74.100:2.

Step 4 Log in to any node in the cluster where the application is located. For details, see LoginUsing an SSH Key.

Step 5 Run the curl command to check whether the application can be accessed normally. You canperform the verification by using the IP address or domain name.l IP address

curl 10.247.74.100:210.247.74.100:2 is the access address obtained in Step 3.If the following information is displayed, the application is accessible.<html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>

<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p></body></html>

l Domain namecurl nginx.default.svc.cluster.local:2nginx.default.svc.cluster.local is the domain name access address obtained in Step 3.If the following information is displayed, the application is accessible.<html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed and





working. Further configuration is required.</p>



----End

Implementing Intra-Cluster Access Using kubectl

This section uses an Nginx application as an example to describe how to implement intra-cluster access using kubectl.

Prerequisites

You have configured the kubectl command and connected an ECS server to the cluster. Fordetails, see 3.4 Connecting to a Kubernetes Cluster Using kubectl.

Procedure

Step 1 Log in to the ECS server on which the kubectl commands have been configured. For details,see Login Using an SSH Key.

Step 2 Create and edit the nginx-deployment.yaml and nginx-clusterip-svc.yaml files.

You can change the file names as required.

vi nginx-deployment.yamlapiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret

vi nginx-ClusterIp-svc.yamlapiVersion: v1kind: Servicemetadata: labels: app: nginx name: nginx-clusteripspec: ports: - name: service0




port: 80 # Access port set on the CCE console. protocol: TCP targetPort: 80 # Container port set on the CCE console. selector: app: nginx type: ClusterIP # Access type set on the CCE console. ClusterIP refers to the cluster virtual IP address.

Step 3 Create an application.




kubectl get po


NAME READY STATUS RESTARTS AGEetcd-0 0/1 ImagePullBackOff 0 27micagent-m9dkt 0/0 Running 0 3dnginx-2601814895-znhbr 1/1 Running 0 15s

Step 4 Create a service.

kubectl create -f nginx-ClusterIp-svc.yaml

If the following information is displayed, the service is being created.

service "nginx-clusterip" created

kubectl get svc

If the following information is displayed, the service has been created, and a cluster IPaddress has been generated.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEetcd-svc ClusterIP None <none> 3120/TCP 30mkubernetes ClusterIP 10.247.0.1 <none> 443/TCP 3dnginx-clusterip ClusterIP 10.247.200.134 <none> 80/TCP 20s

Step 5 Log in to any node in the cluster where the application is located. For details, see LoginUsing an SSH Key.

Step 6 Run the curl command to check whether the application can be accessed normally. You canperform the verification by using the IP address or domain name.l IP address

curl 10.247.200.134:80If the following information is displayed, the application is accessible.<html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed and





working. Further configuration is required.</p>



l Domain namecurl nginx-clusterip.default.svc.cluster.local:80If the following information is displayed, the application is accessible.<html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>



----End

5.3 Intra-VPC AccessAn application is accessible to other applications in the same VPC by using the IP address ofa cluster node or the ELB service IP address of a private network.

Typical scenario: Applications in a kubernetes cluster are accessed by other applications in thesame VPC.

The following two intra-VPC access modes are available:

l Using the IP address of a cluster node, as shown in Figure 5-2.l Using the ELB service IP address of a private network, as shown in Figure 5-3. This

mode provides higher reliabilities than the preceding access mode.



Figure 5-2 Intra-VPC access (by using the IP address of a cluster node)

Figure 5-3 Intra-VPC access (by using the ELB service IP address of a private network)




l Set the access mode when creating an application. For details, see Creating anApplication on the CCE Console and Implementing Intra-VPC Access Usingkubectl.



b. Set the access mode by following the procedure in Creating an Application on theCCE Console.

Creating an Application on the CCE ConsoleThe following procedure uses an Nginx application as an example.


name.l Access Mode: Select Intra-VPC access.

– If Intra-VPC load balancing is disabled, nodes in the cluster are accessible usingthe node IP address.

– If Intra-VPC load balancing is enabled, nodes in the cluster are accessible usingELB instances. If no ELB instances are available, click Create a classic ELBinstance or Create an enhanced ELB instance and create one.

Set Type to Private network when creating a classic load balancer.

l Protocol: Select a protocol used by the service.l Container Port: Specify a port on which the application listens. The Nginx application

listens on port 80.l Access Port:

– Access a node in a cluster using the IP address of the node: Specify a port to map acontainer port to the node's private IP address. The port range is 30000–32767.The port will be used when the application is accessed using the node's private IPaddress. You are advised to select Automatically generated.n Automatically generated: The system automatically assigns a port number.n Specified port: Specify a fixed node port. The port range is 30000–32767.

Ensure that the port is unique in the same cluster.– Access a node in a cluster using the private IP address of the elastic load balancer:

Specify a port to map a container port to the load balancer's port. The port range is1–65535. When the private network load balancing IP address is used to access the



application. The port will be used when the application is accessed using the privateIP address of the elastic load balancer.

Step 2 Click OK, and then click Next. On the Configure Advanced Settings page that is displayed,click Create Now.

Step 3 Click View Application Details. On the Access Mode tab page, obtain the access address, forexample: 192.168.0.160:30358.

Step 4 On the homepage of the management console, choose Computing > Elastic Cloud Server.

Step 5 Find any ECS server in the same VPC, and confirm that the security group is open to the IPaddress and port to be connected.

Figure 5-4 Confirming that the security group is open

Step 6 Click Remote Login. On the login page that is displayed, enter the username and password.

Step 7 Run the curl command to check whether the application can be accessed normally.

NOTE

If a node is accessed by using a private IP address, a cluster virtual IP address is also allocated.Therefore, you can verify whether the application is accessible using the cluster virtual IP address. Bydefault, the cluster virtual IP address access port is the same as the container port. In this example, theaccess port is port 80.

curl 192.168.0.160:30358

192.168.0.160:30358 is the access address obtained in Step 3.

If the following information is displayed, the application is accessible.

<html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>


<p><em>Thank you for using nginx.</em></p>



</body></html>

----End

Implementing Intra-VPC Access Using kubectlThis section uses an Nginx application as an example to describe how to implement intra-VPC access using kubectl.

Prerequisites


Procedure


Step 2 Create and edit the nginx-deployment.yaml and nginx-nodeport-svc.yaml files.

You can change the file names as required.


vi nginx-nodeport-svc.yamlapiVersion: v1kind: Servicemetadata: labels: app: nginx name: nginx-nodeportspec: ports: - name: service nodePort: 30000 # Access port set on the CCE console. If this parameter is not specified, the system automatically allocates an access port. port: 80 # Cluster virtual IP address access port. protocol: TCP targetPort: 80 # Container port set on the CCE console. selector: app: nginx




type: NodePort # Access type set on the CCE console. NodePort refers to the node's private IP address.





kubectl get po


NAME READY STATUS RESTARTS AGEetcd-0 0/1 ImagePullBackOff 0 48micagent-m9dkt 0/0 Running 0 3dnginx-2601814895-qhxqv 1/1 Running 0 9s


kubectl create -f nginx-nodeport-svc.yaml

If the following information is displayed, the service is being created.

service "nginx-nodeport" created

kubectl get svc

If the following information is displayed, the service has been created.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEetcd-svc ClusterIP None <none> 3120/TCP 49mkubernetes ClusterIP 10.247.0.1 <none> 443/TCP 3dnginx-nodeport NodePort 10.247.4.225 <none> 80:30000/TCP 7s

Step 5 Run the curl command to check whether the application can be accessed normally.

curl 192.168.2.240:30000

192.168.2.240 is the IP address of any node in the cluster, and 30000 is the port number of thenode.

If the following information is displayed, the application is accessible.

<html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>


<p><em>Thank you for using nginx.</em></p>



</body></html>

----End

5.4 External Access - Elastic IP AddressAn application is accessible to public networks using an EIP. This access mode is applicableto services that need to be exposed to public networks. To enable access to an applicationfrom the Internet, an EIP must be bound to a node in the cluster, and a mapping port numbermust be set. The port number must be in the 30000–32767 range. For example, the accessaddress could be 10.117.117.117:30000.

Figure 5-5 Elastic IP Address


l Set the access mode when creating an application. For details, see Creating anApplication on the CCE Console and Implementing Public Network Access (EIP)Using kubectl.



b. Set the access mode by following the procedure in Creating an Application on theCCE Console.

Creating an Application on the CCE ConsoleThe following procedure uses an Nginx application as an example.




name.l Access Mode: Select External access.l Access Type: Select EIP. Ensure that at least one node in the cluster has been bound to

an EIP.l Protocol: Select a protocol used by the service.l Container Port: Specify a port on which the application listens. The Nginx application

listens on port 80.l Access Port: Specify a port to map a container port to an EIP. The port range is 30000–

32767. The port will be used when the application is accessed using the EIP. You areadvised to select Automatically generated.– Automatically generated: The system automatically assigns a port number.– Specified port: Specify a fixed node port. The port range is 30000–32767. Ensure

that the port is unique in the same cluster.

Step 2 Click OK. Click Next. On the Configure Advanced Settings page that is displayed, clickCreate Now.


Step 4 Click the access address to go to the login page.


----End

Implementing Public Network Access (EIP) Using kubectl

This section uses an Nginx application as an example to describe how to implement publicnetwork access using kubectl.

Prerequisites


Procedure




Step 2 Create and edit the nginx-deployment.yaml file and the nginx-eip-svc.yaml file. The filenames are used as examples, and you can change them as required.


vi nginx-eip-svc.yamlapiVersion: v1kind: Servicemetadata: annotations: service.protal.kubernetes.io/access-ip: 10.78.44.60 # EIP. At least one node in the cluster has been bound to this EIP. service.protal.kubernetes.io/type: EIP # Set the external access type to Elastic IP Address. labels: app: nginx name: nginx-eipspec: ports: - name: service0 nodePort: 30000 # Access port set on the CCE console. If this parameter is not specified, the system automatically allocates an access port. port: 80 # Cluster virtual IP address access port. protocol: TCP targetPort: 80 # Container port set on the CCE console. selector: app: nginx type: NodePort # The EIP must be based on an NodePort service.



If the following information is displayed, the application is being created.deployment "nginx" created

kubectl get po

If the following information is displayed, the application is running.NAME READY STATUS RESTARTS AGEetcd-0 0/1 ImagePullBackOff 0 59m




icagent-m9dkt 0/0 Running 0 3dnginx-2601814895-sf71t 1/1 Running 0 8s


kubectl create -f nginx-eip-svc.yaml


service "nginx-eip" created

kubectl get svc

If the following information is displayed, the service access mode has been set successfully.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEetcd-svc ClusterIP None <none> 3120/TCP 59mkubernetes ClusterIP 10.247.0.1 <none> 443/TCP 3dnginx-eip NodePort 10.247.120.135 <none> 80:30000/TCP 7s

Step 5 In the address bar of your browser, enter 10.78.44.60:30000 and press Enter.

10.78.44.60 is the EIP, and 30000 is the node port number obtained in the previous step.


----End

5.5 External Access - Elastic Load BalancerThis access mode is accessible to public networks using an ELB address. This access modeprovides higher reliability than EIP-based access and is applicable to services that need to beexposed to public networks. The access address consists of the ELB service address of thepublic network, followed by the access port number, for example, 10.117.117.117:80.



Figure 5-8 Elastic Load Balancer

Methods for Setting the Access Mode

You can set the access mode using either of the following two methods:

l Set the access mode when creating an application. For details, see Creating anApplication on the CCE Console and Implementing Public Network Access (ELB)Using kubectl.



b. Set the access mode. For details, see Creating an Application on the CCEConsole.

Creating an Application on the CCE Console

The following procedure uses an Nginx application as an example.


name.l Access Mode: Select External access.



l Access Type: Select ELB. You must create an ELB instance first. Currently, classic andenhanced ELB instances are supported.

l Health Check: When Classic load balancer is selected, you can manually configure thehealth check port.– By default, this function is disabled and the service port is used for health check.– If the health check port is different from the service port, enable this function and

manually configure the health check port.n Health Check Agreement: Select a value based on the health check protocol. If

the protocol type is UDP, ensure that the security group rule is correctlyconfigured. For details, see 14.2 How Do I Enable ICMP Security GroupRules?

n Health Check Container port: Listening port of the actual container healthcheck.

n Health Check Access port: The value of the input port must be different fromthat of the port mapping.

l Protocol: Select a protocol used by the service.l Container Port: Specify a port on which the application listens. The Nginx application

listens on port 80.l Access Port: Specify a port to map a container port to the IP address of an ELB service.

The port range is 1–65535. The port will be used when the application is accessed usingthe IP address of an ELB service.

Step 2 Click OK. Click Next. On the Configure Advanced Settings page that is displayed, clickCreate Now.


Step 4 Click the access address to go to the login page.

----End

Implementing Public Network Access (ELB) Using kubectlThis section uses an Nginx application as an example to describe how to implement publicnetwork access using kubectl.

Prerequisites


Procedure


Step 2 Create and edit the nginx-deployment.yaml file and the nginx-elb-svc.yaml file. The filenames are used as examples, and you can change them as required.

vi nginx-deployment.yaml

apiVersion: extensions/v1beta1kind: Deploymentmetadata:




name: nginxspec: replicas: 1 selector: matchLabels: app: nginx strategy: type: RollingUpdate template: metadata: labels: app: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx imagePullSecrets: - name: default-secret

vi nginx-elb-svc.yaml

l Enhanced Load BalancerapiVersion: v1kind: Servicemetadata: annotations: kubernetes.io/elb.class: union kubernetes.io/elb.id: a172d66c-e42f-4276-aa23-9258113478f6 labels: app: nginx name: nginxspec: loadBalancerIP: 10.78.42.242 # IP address of the an ELB service in a public network. ports: - name: service0 nodePort: 31540 # Access port set on the CCE console. If this parameter is not specified, the system automatically allocates an access port. port: 80 # Cluster virtual IP address access port, which has been registered with an ELB service. protocol: TCP targetPort: 80 # Container port set on the CCE console. selector: app: nginx type: LoadBalancer # The EIP must be based on an NodePort service.

l Classic Load BalancerapiVersion: v1kind: Servicemetadata: labels: app: nginx name: nginxspec: loadBalancerIP: 10.78.42.242 # IP address of the an ELB service in a public network. ports: - name: service0 nodePort: 31540 # Access port set on the CCE console. If this parameter is not specified, the system automatically allocates an access port. port: 80 # Cluster virtual IP address access port, which has been registered with an ELB service. protocol: TCP targetPort: 80 # Container port set on the CCE console. selector: app: nginx type: LoadBalancer # The EIP must be based on an NodePort service.







kubectl get po


NAME READY STATUS RESTARTS AGEetcd-0 0/1 ImagePullBackOff 0 1hicagent-m9dkt 0/0 Running 0 3dnginx-2601814895-c1xhw 1/1 Running 0 6s


kubectl create -f nginx-elb-svc.yaml


service "nginx" created

kubectl get svc

If the following information is displayed, the service access mode has been set successfully,and the application is accessible.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEetcd-svc ClusterIP None <none> 3120/TCP 1hkubernetes ClusterIP 10.247.0.1 <none> 443/TCP 3dnginx LoadBalancer 10.247.130.196 10.4.10.230 80:31540/TCP 51s

Step 5 In the address bar of your browser, enter 10.4.10.230 and press Enter. In this example,10.4.10.230 is the IP address of the ELB instance.

The Nginx application is accessible.


----End



6 Job Management

6.1 Creating a One-off JobsA one-off job is executed only once immediately after being deployed. Before creating anapplication, you can execute a one-off job to upload an image or software package to thesoftware repository.

PrerequisitesNodes have been added. For more information, see 3.6 Creating a Node in a VM Cluster.

Procedure

Step 1 (Optional) If you use a private container image to create your one-off job, upload thecontainer image to the image repository.

For details about how to use the image repository, see 11 Image Repository.

Step 2 Log in to the CCE console, choose Job Management > One-off Jobs, and click Create Job.

Step 3 Configure the basic job information listed in Table 6-1. The parameters marked with anasterisk (*) are mandatory.

Table 6-1 Basic job information


* Job Name Name of a new job. The name must be unique.

* Cluster Cluster to which a new job belongs.

* Namespace Namespace to which a job belongs.

Description Description of a job.


1. Click Select Container Image and select the image to be deployed.

Cloud Container EngineUser Guide 6 Job Management


– The My Images tab page displays all images you created.

– Third-party Images: CCE allows you to create an application using an imagepulled from a third-party image repository, rather than a public cloud imagerepository or a Docker Hub image repository. When you create an application usinga third-party image, ensure that the node where the application is running canaccess public networks. For details about how to create an application using a third-party image, see 4.10 Using a Third-party Image to Create an Application.

n If your image repository does not require authentication, set AuthenticateSecret to No, specify Image Address, for example, nginx:latest, and clickOK.





Image Image to be imported. You can click Change Image to changeyour image.

* Image Version Image version to be deployed.

* ContainerName

Name of the container. You can modify this parameter.

ContainerResources

– Request: the amount of resources that CCE will guarantee to acontainer.

– Limit: the maximum amount of resources that CCE will allowa container to use. You can set Limit to prevent system faultscaused by container overload.

For more information on Request and Limit, see section 4.5Setting Container Specifications.

3. (Optional) Configure advanced settings.




Environment variables are set in the container runningenvironment and can be modified after job deployment to ensurethe flexibility of jobs.1. On the Environment Variables tab page, click Add.2. Set Variable Name and Variable/Variable Reference.

Data Storage You can mount a host directory, EVS disk, SFS, andconfiguration items and secrets to the corresponding directories ofa container instance. For details, see 8 Storage Management.




Log Policies Set a log policy and log path for collecting application logs andpreventing logs from being over-sized. For details, see 9 LogManagement.

4. (Optional) One job instance contains one or more related containers. If your job containsmultiple containers, click Add Container and then add containers.

Step 5 Click Create Now.

If the status is Execution completed, the job has been created successfully.

----End

Creating a Job Using kubectl

A job has the following configuration parameters:

l spec.template: has exactly the same schema as a pod.

l RestartPolicy: can only be set to Never or OnFailure.

l For a single-pod job, the job ends after the pod runs successfully by default.

l .spec.completions: indicates the number of pods that need to run successfully to end ajob. The default value is 1.

l .spec.parallelism: indicates the number of pods that run concurrently. The default valueis 1.

l spec.backoffLimit: indicates the maximum number of retries performed if a pod fails.When the limit is reached, the pod will not try again.

l .spec.activeDeadlineSeconds: indicates the running time of pods. Once the time isreached, all pods of the job are terminated. The priority of .spec.activeDeadlineSecondsis higher than that of .spec.backoffLimit. That is, if a job reachesthe .spec.activeDeadlineSeconds, the spec.backoffLimit is ignored.

Based on the .spec.completions and .spec.Parallelism settings, jobs are classified into thefollowing types.

Table 6-4 Job types

Job Type Description Example

One-shot jobs A single pod runs once untilsuccessful termination.

Database migration

Jobs with a fixedcompletioncount

One pod runs until reaching thespecified completions count.

Work queue processing pod

Parallel jobswith a fixedcompletioncount

Multiple pods run until reachingthe specified completions count.

Multiple pods processing from acentralized work queue



Job Type Description Example

Parallel jobs One or more pods run untilsuccessful termination.

Multiple pods processing from acentralized work queue

The following is an example job, which calculates π till the 2000th digit and prints the output.

apiVersion: batch/v1kind: Jobmetadata: name: pi-with-timeoutspec: completions: 50 # Indicates that 50 pods need to run to end the job. In this example, the value of π is printed 50 times. parallelism: 5 # Indicates 5 parallel pods. backoffLimit: 5 # Indicates that a pod that fails retries a maximum of 5 times. template: spec: containers: - name: pi image: perl command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"] restartPolicy: Never

Related Operations

After a one-off job is created, you can perform operations listed in Table 6-5.

Table 6-5 Other operations

Operation Description

Deleting a one-off job 1. Select the job to be deleted and click Delete in theOperation column.

2. Click OK.Deleted jobs cannot be restored. Therefore, exercise cautionwhen deleting a job.

6.2 Creating a Cron JobA cron job is a short-lived job that runs at a specified time. You can perform timesynchronization for all active nodes at a fixed time point.

Prerequisites

Nodes have been added. For more information, see 3.6 Creating a Node in a VM Cluster.

Procedure

Step 1 (Optional) If you use a private container image to create your containerized cron job, uploadthe container image to the image repository.



For details about how to use the image repository, see 11 Image Repository.

Step 2 Log in to the CCE console, choose Job Management > Cron Jobs, and click Create Job.

Step 3 Configure the basic job information listed in Table 6-6. The parameters marked with anasterisk (*) are mandatory.

Table 6-6 Basic job information


* Job Name Name of a new job. The name must be unique.

* Cluster Cluster to which a new job belongs.

* Namespace Namespace to which a cron job belongs.

Description Description of a job.

Step 4 Click Next.

Step 5 Set the scheduling rule.

Table 6-7 Scheduling rule parameters


* ConcurrencyPolicy

The following three modes are supported:l Allow: New cron jobs can be created continuously.l Forbid: A new job cannot be created before the previous job is

complete.l Replace: A new job replaces the previous job when it is time to

create the new job but the previous job is not complete.

* Timing Rule Specifies the time when a new cron job is executed.

Job Records You can set the number of job execution records (successful or failed)that can be retained.


1. Click Select Container Image and select the image to be deployed.– The My Images tab page displays all images you created.– Third-party Images: CCE allows you to create an application using an image









Image Image to be imported. You can click Change Image to changeyour image.

* Image Version Image version to be deployed.

* ContainerName

Name of the container. You can modify this parameter.

ContainerResources

– Request: the amount of resources that CCE will guarantee to acontainer.

– Limit: the maximum amount of resources that CCE will allowa container to use. You can set Limit to prevent system faultscaused by container overload.

For more information on Request and Limit, see section 4.5Setting Container Specifications.

3. (Optional) Configure advanced settings.




Environment variables are set in the container runningenvironment and can be modified after job deployment to ensurethe flexibility of jobs.1. On the Advanced settings tab page, click Add.2. Set Variable Name and Variable/Variable Reference.

4. (Optional) One job instance contains one or more related containers. If your job contains

multiple containers, click Add Container and then add containers.

Step 7 Click Create Now.

If the status is Started, the cron job has been created successfully.

----End

Creating a Cron Job Using kubectl

A Cron job has the following configuration parameters:

l .spec.schedule: takes a Cron format string, for example, 0 * * * * or @hourly, as thetime scheduled for creating or executing jobs.



https://en.wikipedia.org/wiki/Cron

l .spec.jobTemplate: specifies jobs to be run, and has exactly the same schema as a job.For details, see Creating a Job Using kubectl.

l .spec.startingDeadlineSeconds: specifies the deadline for starting a job.

l .spec.concurrencyPolicy: specifies how to treat concurrent executions of a job createdby the Cron job. The following options are supported:

– Allow (default value): allows concurrently running jobs.

– Forbid: forbids concurrent runs, skipping next run if previous has not finished yet.

– Replace: cancels the currently running job and replaces it with a new one.

The following is an example Cron job, which is saved in the cronjob.yaml file.

apiVersion: batch/v2alpha1kind: CronJobmetadata: name: hellospec: schedule: "*/1 * * * *" jobTemplate: spec: template: spec: containers: - name: hello image: busybox args: - /bin/sh - -c - date; echo Hello from the Kubernetes cluster restartPolicy: OnFailure

1. Run the following command to create a Cron job.$ kubectl create -f cronjob.yamlcronjob.batch/hello created

2. After the creation, run the following commands to view the running status of the job.$ kubectl get cronjobNAME SCHEDULE SUSPEND ACTIVE LAST-SCHEDULEhello */1 * * * * False 0 <none>$ kubectl get jobsNAME DESIRED SUCCESSFUL AGEhello-1202039034 1 1 49s$ pods=$(kubectl get pods --selector=job-name=hello-1202039034 --output=jsonpath={.items..metadata.name} -a)$ kubectl logs $podsMon Aug 29 21:34:09 UTC 2016Hello from the Kubernetes cluster$ kubectl delete cronjob hellocronjob "hello" deleted

Deleting a Cron job will not automatically delete its jobs. You can delete the jobs by runningthe kubectl delete job command.

Related Operations

After a cron job is created, you can perform operations listed in Table 6-10.





Stopping a cron job 1. Select the job to be stopped and click Stop in the Operationcolumn.

2. Click OK.

Deleting a cron job 1. Select the job to be deleted and click Delete in theOperation column.

2. Click OK.Deleted jobs cannot be restored. Therefore, exercise cautionwhen deleting a job.



7 Configuration Center

7.1 Creating a Configuration ItemA configuration item is a type of resource that stores configuration information required by anapplication. Its content is user-defined. After creating configuration items, you can use themas files or environment variables in a containerized application.

Configuration items allow you to decouple configuration files from container images toenhance the portability of containerized applications.

Benefits of configuration items:

l Manage configurations of different environments and services.l Deploy applications in different environments. Multiple versions are supported for

configuration files so that you can update and roll back applications easily.l Quickly import your configurations files to containers.

PrerequisitesCluster and node resources have been created. For more information, see 3.2 Creating a VMCluster.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Configuration Center >ConfigMaps, and click Create ConfigMap.

Step 2 Configuration items can be created manually or by uploading a configuration file.l To create a configuration item manually, set the parameters for creating a configuration

item listed in Table 7-1. The parameters marked with an asterisk (*) are mandatory.

Cloud Container EngineUser Guide 7 Configuration Center


Table 7-1 Parameters for creating a configuration item


* Name Name of a configuration item, which must be unique in anamespace.

* Cluster Cluster that will use the configuration item you create.

* Namespace Namespace to which the configuration item belongs. If youdo not specify this parameter, the value default is used bydefault.

Description Description of the configuration item.

Configuration Data The application configuration data can be used in a containeror used to store the configuration data. Key indicates a filename. Value indicates the content in the file.1. Click Add Data.2. Set Key and Value.

Configuration Labels Labels are attached to objects such as applications, nodes,and services in key-value pairs.Labels define the identifiable attributes of these objects andare used to manage and select the objects.1. Click Add Label.2. Set Key and Value.

l To create configuration items by uploading a file, perform the following steps:

NOTE

When creating configuration items by uploading a file, ensure that the resource description file hasbeen created. CCE supports files in JSON or yaml format. For more information, seeConfiguration Item Requirements for ConfigMap.

a. Choose a cluster from the Cluster drop-down list.b. Choose the namespace to which the configuration item belongs from the

Namespace drop-down list. If you do not specify this parameter, the value defaultis used by default.

c. Click Upload, select the created ConfigMap resource file, and click Open.


The new configuration item is displayed in the configuration item list.

----End

Configuration Item Requirements for ConfigMapA ConfigMap resource file can be in JSON or YAML format, and the file size cannot exceed2 MB.

l JSON formatThe file name is configmap.json and the following shows a configuration example.



{ "kind": "ConfigMap", "apiVersion": "v1", "metadata": { "name": "paas-broker-app-017", "namespace": "test", "enable": true }, "data": { "context": "{\"applicationComponent\":{\"properties\":{\"custom_spec\":{}},\"node_name\":\"paas-broker-app\",\"stack_id\":\"0177eae1-89d3-cb8a-1f94-c0feb7e91d7b\"},\"softwareComponents\":[{\"properties\":{\"custom_spec\":{}},\"node_name\":\"paas-broker\",\"stack_id\":\"0177eae1-89d3-cb8a-1f94-c0feb7e91d7b\"}]}" }}

l YAML formatThe file name is configmap.yaml and the following shows a configuration example.apiVersion: v1kind: ConfigMapmetadata: name: test-configmapdata: data-1: value-1 data-2: value-2

Creating a Configuration Item Using kubectl

Step 1 Follow the instructions in 3.4 Connecting to a Kubernetes Cluster Using kubectl toconfigure the kubectl command to connect an ECS server to the cluster.

Step 2 Create and edit the cce-configmap.yaml file.

vi cce-configmap.yaml

apiVersion: v1kind: ConfigMapmetadata: name: cce-configmapdata: SPECIAL_LEVEL: Hello SPECIAL_TYPE: CCE

Step 3 Run the following commands to create a configuration item.

kubectl create -f cce-configmap.yaml

kubectl get cm

NAME DATA AGE cce-configmap 3 3hcce-configmap1 3 7m

----End

Related OperationsAfter creating a configuration item, you can update or delete it as described in Table 7-2.



Table 7-2 Related operations


Updating a configurationitem

1. Select a desired configuration item and click Modify.2. Modify the configuration parameters. For more

information about the parameters, see Table 7-1.3. Click Modify.

Deleting a configurationitem

Select the configuration you want to delete and click Delete.Follow the prompts to delete the configuration item.

7.2 Using a Configuration ItemThe following example shows how to use ConfigMap.

apiVersion: v1kind: ConfigMapmetadata: name: cce-configmapdata: SPECIAL_LEVEL: Hello SPECIAL_TYPE: CCE

When ConfigMap is used in a pod, the pod and ConfigMap must be in the same cluster andnamespace.

Mounting a Configuration Item to an Application Data Volume

The configuration item can also be used in a data volume. You only need to mount theconfiguration item to an application when creating the application. After the mounting iscomplete, a configuration file with key as the file name and value as the file content isgenerated.

apiVersion: v1 kind: Pod metadata: name: configmap-pod-4 spec: containers: - name: test-container image: busybox command: [ "/bin/sh", "-c", "ls /etc/config/" ] ## List names of files in this directory. volumeMounts: - name: config-volume mountPath: /etc/config ## Mount the configuration item to the /etc/config directory. volumes: - name: config-volume configMap: name: cce-configmap restartPolicy: Never



After the pod is run, the SPECIAL_LEVEL and SPECIAL_TYPE files are generated inthe /etc/config directory. The contents of the files are Hello and CCE, respectively. Also, thefollowing file names will be displayed.

SPECIAL_TYPE SPECIAL_LEVEL

To mount ConfigMap to a data volume, you can also perform operations on the CCE console.When creating an application, add a container image. Then, select Data Storage > LocalDisks, click Add Local Disk, and select ConfigMap. For details, see ConfigMap.

7.3 Creating a SecretA secret is a type of resource that holds sensitive data, such as authentication and keyinformation. All content is user-defined. After creating secrets, you can use them as files orenvironment variables in a containerized application.

PrerequisitesCluster and node resources have been created. For more information, see 3.2 Creating a VMCluster.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Configuration Center > Secrets,and click Create Secret.

Step 2 Create a secret manually or by uploading a file.l To create a secret manually, set the parameters for creating a secret listed in Table 7-3.

The parameters marked with an asterisk (*) are mandatory.

Table 7-3 Parameters for creating a Key


* Name Name of the secret you create, which must be unique.

* Cluster Cluster that will use the secret you create.

* Namespace Namespace to which the configuration item belongs. If youdo not specify this parameter, the value default is used bydefault.




* Type Type of the secret you create.– Opaque: common secret– kubernetes.io/dockerconfigjson: a secret that stores the

authentication information required for pulling imagesfrom a private repository.

– IngressTLS: a secret that stores the certificate required bythe Layer-7 load balancing service.

– Other: Another type of secret, which is specifiedmanually.

Description Description of a secret.

Secret Data Application secret data can be used in containers.– If the secret is of the Opaque type:

1. Click Add Data.2. Enter the key and value. The value must be encoded to

Base64. For details about the encoding method, seeBase64 Encoding.

– If the secret is of the kubernetes.io/dockerconfigjson type,enter the account name and password of the private imagerepository.

– If the secret is of the IngressTLS type, upload certificatefiles and private key files.

Secret Labels Labels are attached to objects such as applications, nodes,and services in key-value pairs.Labels define the identifiable attributes of these objects andare used to manage and select the objects.1. Click Add Label.2. Set Key and Value.

l To create a Secret resource by uploading a file, perform the following steps:

NOTE

When creating a resource by uploading a file, ensure that the resource description file has beencreated. CCE supports files in JSON or yaml format. For more information, see Secret ResourceFile Configuration.

a. Choose a cluster from the Cluster drop-down list.

b. Select the corresponding cluster namespace.

c. Click Upload, select the created secret resource file, and click Open.


The new secret is displayed in the key list.

----End



Secret Resource File ConfigurationThis section describes configuration examples of secret resource description files.

For example, you can retrieve the username and password for an application through a secret.

l YAML formatThe secret.yaml file is defined as shown below. The value must be encoded to Base64.For details about the encoding method, see Base64 Encoding.apiVersion: v1 kind: Secret metadata: name: mysecret # secret name namespace: default # By default, the namespace is default. data: username: my_username password: ****** # The Base64 coding scheme is required. The method is as follows: echo -n "Content to be encoded" | base64. type: Opaque # You are advised not to change the value of type.

l JSON formatThe secret.json file is defined as shown below.{ "apiVersion": "v1", "kind": "Secret", "metadata": { "name": "mysecret", "namespace": "default" }, "data": { "password": "******", # The Base64 coding scheme is required. The method is as follows: echo -n "Content to be encoded" | base64 "username": "my_username" }, "type": "Opaque"}

NOTE

Set the username and password to the actual user name and password.

Creating a Secret Using kubectl

Step 1 According to 3.4 Connecting to a Kubernetes Cluster Using kubectl, configure the kubectlcommand to connect an ECS server to the cluster.

Step 2 Create and edit the cce-secrets.yaml file based on the Base64 encoding method.

vi cce-secret.yamlapiVersion: v1kind: Secretmetadata: name: mysecrettype: Opaquedata: username: mysecret password: ****** # The Base64 coding scheme is required. The method is as follows: echo -n "Content to be encoded" | base64

Step 3 Create a secret.

kubectl create -f cce-secret.yaml

You can query the secret after creation.



kubectl get secret

----End

Base64 EncodingTo encrypt a character string to Base64, you can run the echo -n encoding content | base64command. The following is an example.

root@ubuntu:~# echo -n "Content to be encoded" | base64******

Related OperationsAfter creating a configuration item, you can update or delete it as described in Table 7-4.

NOTE

The secret list contains system secret resources that can be queried only. The system secret resourcescannot be updated or deleted.

Table 7-4 Related Operations


Updating a secret 1. Select a key you want to modify and click Modify.2. Modify the secret data. For more information, see Table

7-3.3. Click Modify Secret.

Deleting a secret Select the secret you want to delete and click Delete.Follow the prompts to delete the secret.

Deleting secrets in batches 1. Select the secrets to be deleted.2. Click Delete above the secret list.3. Follow the prompts to delete the secrets.

7.4 Using a SecretThe following is an example showing how to use a secret.apiVersion: v1kind: Secretmetadata: name: mysecrettype: Opaquedata: username: mysecret password: ****** # The Base64 coding scheme is required. The method is as follows: echo -n "Content to be encoded" | base64

When a secret is used in a pod, the pod and secret must be in the same cluster and namespace.



Configuring the Data Volume of a PodA secret can be used as a file in a pod. As shown in the following example, the username andpassword of the mysecret secret are saved in the /etc/foo directory as files.

apiVersion: v1kind: Podmetadata: name: mypodspec: containers: - name: mypod image: redis volumeMounts: - name: foo mountPath: "/etc/foo" readOnly: true volumes: - name: foo secret: secretName: mysecret

In addition, you can specify the directory and permission to access a secret. The username isstored in the /etc/foo/my-group/my-username directory of the container.

apiVersion: v1kind: Podmetadata: name: mypodspec: containers: - name: mypod image: redis volumeMounts: - name: foo mountPath: "/etc/foo" volumes: - name: foo secret: secretName: mysecret items: - key: username path: my-group/my-username mode: 511

To mount a secret to a data volume, you can also perform operations on the CCE console.When creating an application, set advanced settings for the container, choose Data Storage >Local Disks, click Add Local Disk, and select Secret. For details, see Secret.



8 Storage Management

8.1 OverviewStorage is a component that provides storage for containerized applications. Multiple types ofstorage are supported.

Selecting a Storage TypeYou can use the following types of storage when creating an application:

l Local disksThe following types of local disk volumes are available: HostPath, EmptyDir,ConfigMap, and Secret. A HostPath volume mounts a specified host path to a path of thecontainer for persistent data storage. An EmptyDir volume mounts a default temporarypath to a path of the container for temporary data storage. For details, see section 8.2Using Local Hard Disks. You can also mount the ConfigMap and secret to thecontainer. For details, see 7 Configuration Center.

l EVS disksCCE supports creation of EVS disks and mounting of the disks to a path of a container.In addition, the EVS disks in a container are migrated during container migration. TheEVS disks are used to store data persistently. For details, see section 8.3 Using EVSDisks.

l SFS file systemsCCE supports creating SFS file systems and mounting them to a path of a container. Thefile storage volumes created by the underlying SFS service can also be used. The SFSdata volumes are applicable to persistent storage for frequent read/write in multiplescenarios, including media processing, content management, big data analysis, andapplication analysis. For details, see section 8.4 Using SFS File Systems.

8.2 Using Local Hard Disks

Application ScenariosLocal hard disks are applicable to the following scenarios:

Cloud Container EngineUser Guide 8 Storage Management


l HostPath: Mount the file directory of the host where a container is located to thespecified mounting point of the container. If the container needs to access /etc/hosts, useHostPath to map /etc/hosts.

l EmptyDir: Used for temporary storage. The lifecycle is the same as that of a containerinstance. When a container instance disappears, EmptyDir will be deleted and the datawill be permanently lost.

l ConfigMap: Keys in the configuration items of ConfigMap are mapped to a container sothat configuration files can be mounted to the specified container directory. For detailson how to create ConfigMap, see section 7.1 Creating a Configuration Item. Fordetails about ConfigMap usage, see section 7.2 Using a Configuration Item.

l Secret: Secret data is mounted to a path of the container. A secret is a type of resourcethat holds sensitive data, such as authentication and key information. All content is user-defined. For details about how to create a secret, see section 7.3 Creating a Secret. Fordetails about secret usage, see section 7.4 Using a Secret.

HostPathThe file or directory of the host is mounted to the container. HostPath is used to storecontainerized application logs that need to be stored permanently, or to store containerizedapplications that need to access internal data structure of the Docker engine on the host.

Step 1 Create an application by following the procedure in section 4.2 Creating a StatelessApplication or section 4.3 Creating a Stateful Application. Choose Data Storage > LocalDisks. On the page that is displayed, click Add Local Disk.

Step 2 Set parameters for adding a local disk, as listed in Table 8-1.

Table 8-1 Volume type set to HostPath


Volume Type Type of the local disk to be mounted. It is set to HostPath here.

* Host Path Path of the host to which the local volume is to be mounted, forexample, /etc/hosts.




Add ContainerPath

1. Click Add Container Path.2. Enter the container path to which the data volume is mounted.

NOTICE– Do not mount a data volume to a system directory such as / or /var/

run; otherwise, the container becomes abnormal. You are advised tomount the data volume to an empty directory. If the directory is notempty, ensure that the directory does not contain any files that affectcontainer startup; otherwise, the files will be replaced. As a result, thecontainer cannot be properly started and the application creation willfail.

– When the data volume is mounted to a high-risk directory, you areadvised to use a low-permission account to start the container;otherwise, high-risk files on the host machine may be damaged.

3. Set permissions.– Read-only: only allows you to read data volumes in the

container path.– Read/Write: allows you to modify the data volumes in the

container path. Newly written data will not be migrated duringcontainer migration; otherwise, data loss occurs.

4. Click OK.

----End

EmptyDir

In this scenario, a volume is automatically created when a container is allocated to a node, andthe volume is empty by default. In a pod, all containers can read and write the same files inEmptyDir. When the pod is deleted from a node, data in EmptyDir will be deletedpermanently. In general, EmptyDir is used for high-speed storage of temporary data.


Step 2 Set parameters for adding a local disk, as shown in Table 8-2.

Table 8-2 Volume type set to EmptyDir


Volume Type Type of the local disk to be mounted. It is set to EmptyDir here.

Storage MediaType

l Deselect In-memory storage: Data is stored in disks, which isapplicable to a large amount of data with low requirements onreading and writing efficiency.

l Select In-memory storage: Data is stored in memory, which isapplicable to a small amount of data with high requirements onreading and writing efficiency.




Add Container Path 1. Click Add Container Path.2. Enter the container path to which the data volume is mounted.


run. This action may cause a container error to occur. You are advisedto mount the data volume to an empty directory. If the directory is notempty, ensure that the directory does not contain any files that affectcontainer startup; otherwise, the files will be replaced, making itimpossible for the container to be properly started. The applicationcreation will fail.


3. Set permissions.– Read-only: only allows you to read data volumes in the

container path.– Read/Write: allows you to modify the data volumes in the

container path. To prevent data loss, newly written data willnot be migrated during container migration.

4. Click OK.

----End

ConfigMapCCE separates the application codes from configuration files. The configMap volume is usedto process application configuration parameters. You need to create application configurationsin advance. For details, see 7.1 Creating a Configuration Item.



Table 8-3 Volume type set to ConfigMap


Volume Type Type of the local disk to be mounted. It is set to ConfigMap here.

* ConfigMap Select the desired configMap name.NOTE

ConfigMap must be created in advance. For details, see 7.1 Creating aConfiguration Item to create a configMap.








3. Set the permission to Read-only, which indicates that datavolumes in the path are read-only.

4. Click OK.

----End

SecretMount the data in the key to the specified container. The content of the key is user-defined.You need to create application configurations in advance. For more information, see 7.3Creating a Secret.



Table 8-4 Volume type set to Secret


Volume Type Type of the local disk to be mounted. It is set to Secret here.

* Secret Select the desired secret name.NOTE

Secret must be created in advance. For details, see 7.3 Creating a Secret tocreate a Secret.








3. Set the permission to Read-only, which indicates that datavolumes in the path are read-only.

4. Click OK.

----End

8.3 Using EVS DisksTo meet data persistency requirements, CCE allows EVS disks to be mounted to containers.By using EVS disks, you can mount a remote file directory of the storage system to thecontainer so that data in the data volume is permanently stored. Even if the container isdeleted, only the mount relationship is deleted. The data in the data volume is still stored inthe storage system.

Application Scenarios

Currently, EVS disks of two specifications are supported: common I/O, and ultra-high I/O.

l Common I/O: The backend storage is provided by the SATA storage media. CommonI/O is applicable to scenarios where large capacity and low read/write rate are required,and the volume of transactions is low. Examples include development testing andenterprise office applications.

l Ultra-high I/O: The backend storage is provided by the SSD storage media. Super-highI/O is applicable to scenarios where high performance, high read/write rate, and data-intensive applications are required. Examples include NoSQL, relational database, anddata warehouse (such as SAP HANA).

Creating Storage Volumes of EVS Disks

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management > Storageand then click Create in the navigation pane.

Step 2 Configure basic disk information, as shown in Table 8-5.



Table 8-5 Basic disk information


PVC Name Name of the PersistentVolumeClaim (PVC). A storage volume isautomatically created when a PVC is created. One PVCcorresponds to one storage volume. The name of a storage volumeis automatically generated when a PVC is created.

Cluster Name Cluster where the new EVS disk is deployed.

Namespace Namespace where the EVS disk is deployed.

Type Type of the new EVS disk.

Disk Capacity (GB) Capacity of the new EVS disk.

Access Mode ReadWriteMany: The volume can be mounted as read-write bymany nodes.

AZ Physical location where resources use independent power suppliesand networks. AZs are physically isolated but interconnectedthrough an internal network.

Step 3 Click Create Now. Confirm order details, click Submit, click Back to Storage List and waituntil the EVS disk has been created successfully.

After the EVS disk has been created successfully, the created storage is displayed in thestorage list. If the storage status is Bound, the storage has been created successfully.

Step 4 Click the storage name. On the storage details page that is displayed, view the storageinformation such as the PVC name and creation time.

----End

Using Storage Volumes of EVS Disks

Step 1 Create an application by following the procedure in section 4.2 Creating a StatelessApplication or section 4.3 Creating a Stateful Application. Choose Data Storage > CloudStorage. On the page that is displayed, click Add Cloud Storage.

Step 2 Set the storage type to EVS.

Table 8-6 Parameters required for mounting EVS disks


Storage Type EVS: The usage of an EVS disk is the same as that of a traditional disk.EVS disks have higher data reliability and I/O throughput and are moreuser-friendly than traditional disks. EVS disks are available for filesystems, databases, and system software or applications that requireblock storage devices.

Allocation Mode




Manual Select an existing storage volume. If no storage volume is available,follow the prompts to create one.

Automatic A storage volume is created automatically. You need to enter thestorage capacity.NOTE

A storage volume that is created automatically will not be tagged.

1. If you have selected EVS as the storage type, first select an AZ.This must be the same as the CCE node where the container willrun.

2. Select a storage sub-type.– Common I/O: EVS disks that have common I/O and use SATA.– Ultra-high I/O: EVS disks that have super-high I/O and use SSD.

3. Enter the storage capacity in GB. Ensure that the storage capacityquota is not exceeded; otherwise, creation will fail.

Add ContainerPath

1. Click Add Container Path.2. Container Path: Enter the container path to which the data volume is

mounted.NOTICE– Do not mount a data volume to a system directory such as / or /var/run.

This action may cause a container error to occur. You are advised tomount the data volume to an empty directory. If the directory is notempty, ensure that the directory does not contain any files that affectcontainer startup; otherwise, the files will be replaced, making itimpossible for the container to be properly started. The applicationcreation will fail.

– When the data volume is mounted to a high-risk directory, you areadvised to use a low-permission account to start the container; otherwise,high-risk files on the host machine may be damaged.

3. Set permissions.– Read-only: only allows you to read data volumes in the container

path.– Read/Write: allows you to modify the data volumes in the

container path. However, newly written data will not be migratedduring container migration.

Step 3 Click OK.

----End

Attaching and Mounting EVS Disks

CCE allows you to import existing EVS disks.

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management >Storage. On the EVS tab page, click Import.

Step 2 Select one or more EVS disks that you want to attach and mount.



Step 3 Click OK.

----End

Unbinding an EVS Disk

After an EVS disk is successfully created or attached, the EVS disk is automatically bound tothe current cluster and cannot be used by other clusters. After the EVS disk is unbound fromthe cluster, it can be attached and used by other clusters.

If the EVS disk has been mounted to an application, the EVS disk cannot be unbound fromthe cluster.

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management >Storage. In the EVS disk list, click Unbind next to the target EVS disk.

Step 2 In the dialog box that is displayed, click OK.

----End

Creating an EVS Disk Using kubectl

CCE supports the creation of EVS disks in the form of PersistentVolumeClaim (PVC).

Prerequisites


Procedure


Step 2 Run the following commands to configure the pvc-evs-auto-example.yaml file, which isused to create a PVC.

touch pvc-evs-auto-example.yaml

vi pvc-evs-auto-example.yaml

The following shows an example of creating an EVS disk.

apiVersion: v1kind: PersistentVolumeClaimmetadata: annotations: volume.beta.kubernetes.io/storage-class: sas # Storage type; Currently, EVS supports ssd, and sata labels: failure-domain.beta.kubernetes.io/region: failure-domain.beta.kubernetes.io/zone: name: pvc-evs-auto-example # PVC name namespace: defaultspec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi # Storage capacity in Gi.

In the preceding example:




l volume.beta.kubernetes.io/storage-class is the EVS disk type. Currently, ultra-high I/O(SSD), and common I/O (SATA) are supported.

l failure-domain.beta.kubernetes.io/region indicates the region where the cluster islocated.

l failure-domain.beta.kubernetes.io/zone indicates the AZ where the EVS disk iscreated. It must be the same as the AZ planned for the application.

l name indicates the name of the PVC to be created.l storage indicates the storage capacity in Gi.

Step 3 Run the following command to create a PVC.

kubectl create -f pvc-evs-auto-example.yaml

After the command is executed, you can go to Resource Management > Storage > EVS toview the EVS disk. Alternatively, you can view the EVS disk by volume name on the EVSconsole.

----End

Creating a PV/PVC for an Existing EVS Disk Using kubectlCCE allows you to use an existing EVS disk to create a PersistentVolume (PV) and bind thePV to the PVC.

Step 1 Log in to the EVS console, and obtain the ID of the existing EVS disk.

Step 2 Log in to the ECS server on which kubectl commands have been configured. For details, seeLogin Using an SSH Key.

Step 3 Run the following command to configure the pv-evs-example.yaml file, which is used tocreate a PV.

vi pv-evs-example.yaml

The following shows an example of creating a PV using an EVS disk.

apiVersion: v1kind: PersistentVolumemetadata: labels: failure-domain.beta.kubernetes.io/region: failure-domain.beta.kubernetes.io/zone: name: pv-evs-example namespace: defaultspec: accessModes: - ReadWriteMany capacity: storage: 10Gi flexVolume: driver: huawei.com/fuxivol fsType: ext4 options: fsType: ext4 volumeID: 0992dbda-6340-470e-a74e-4f0db288ed82 persistentVolumeReclaimPolicy: Delete storageClassName: sata

In the preceding example:l failure-domain.beta.kubernetes.io/region indicates the region where the cluster is

located.




l failure-domain.beta.kubernetes.io/zone indicates the AZ where the EVS disk iscreated. It must be the same as the AZ planned for the application.

l driver indicates the driver for the storage to be mounted. Set this parameter tohuawei.com/fuxivol for EVS disks.

l volumeID: indicates the ID of the EVS disk.l storage indicates the storage capacity in Gi.l storageClassName is the EVS disk type. Currently, ultra-high I/O (SSD), and common

I/O (SATA) are supported.

Step 4 Run the following command to create a PV.

kubectl create -f pv-sfs-example.yaml

Step 5 Run the following commands to configure the pvc-evs-example.yaml file, which is used tocreate a PVC.

touch pvc-evs-example.yaml

vi pvc-evs-example.yaml

apiVersion: v1kind: PersistentVolumeClaimmetadata: annotations: volume.beta.kubernetes.io/storage-class: sata # Storage type; Currently, EVS supports sas, ssd, and sata labels: failure-domain.beta.kubernetes.io/region: failure-domain.beta.kubernetes.io/zone: name: pvc-evs-auto-example # PersistentVolumeClaim name namespace: defaultspec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi # Storage capacity in Gi. volumeName: pv-evs-example


kubectl create -f pvc-evs-example.yaml

After the command is executed, go to Resource Management > Storage > EVS to view theEVS disk. Alternatively, you can view the EVS disk by volume name on the EVS console.

----End

Mounting an EVS Disk Using kubectlAfter an EVS disk is created or imported to the CCE console, you can mount it in anapplication.

EVS disks cannot be mounted across AZs. Before mounting, you can run the kubectl get pvccommand to query the available PVCs in the partition where the current cluster is located.



Step 1 Run the following commands to configure the evs-pod-example.yaml file, which is used tocreate a pod.

touch evs-pod-example.yaml

vi evs-pod-example.yaml

The following shows an example of mounting an EVS disk.

apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: evs-pod-example namespace: defaultspec: replicas: 1 selector: matchLabels: app: evs-pod-example template: metadata: labels: app: evs-pod-example spec: containers: - image: nginx:1.1 name: container-0 volumeMounts: - mountPath: /tmp name: pvc-evs-example restartPolicy: Always volumes: - name: pvc-evs-example persistentVolumeClaim: claimName: pvc-evs-auto-example


l name is the name of the pod to be created.

l app is the name of a pod application.

l mountPath is the mount path in a container. In the example, the EVS disk is mounted tothe /tmp directory.

l spec.template.spec.containers.volumeMounts.name andspec.template.spec.volumes.name must be consistent because they have a mappingrelationship.

Step 2 Run the following command to create a pod.

kubectl create -f evs-pod-example.yaml

After the pod is created, choose Resource Management > Storage > EVS on the CCEconsole to view the binding relationship between the application and PVC.

----End

Related Operations

After the EVS disk is created, you can perform operations described in Table 8-7.





Deleting an EVS disk 1. Select the EVS disk to be deleted and click Delete in theOperation column.

2. Follow the prompts to delete the EVS disk.

8.4 Using SFS File SystemsSFS file systems apply to a wide range of scenarios, including media processing, contentmanagement, big data, and analytic applications.

Creating a File System

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management >Storage.

Step 2 Click the SFS tab and then click Create.

Step 3 Configure basic information in the page, the meanings of the corresponding parameters aslisted in Table 8-8.

Table 8-8 Basic file storage information


PVC Name Name of the PVC. A storage volume is automatically created whena PVC is created. One PVC corresponds to one storage volume.The name of a storage volume is automatically generated when aPVC is created.

Cluster Name Cluster where the file storage is deployed.

Namespace Namespace where the SFS is located.

Total Capacity (GB) Volume of the file system to be created.

Access Mode ReadWriteMany

Step 4 Click Create Now. Confirm order details, click Submit, click Back to Storage List, and waituntil the SFS has been created successfully.

Step 5 After the SFS has been created successfully, the created storage is displayed in the storage list.If the storage status is Bound, the storage has been created successfully.

Step 6 Click the storage name. On the storage details page that is displayed, view the storageinformation such as the mounting details and creation time.

----End



Using File Systems

Step 1 Create an application by following the procedure in section 4.2 Creating a StatelessApplication or section 4.3 Creating a Stateful Application. On the Data Storage > CloudStorage tab page, click Add Cloud Storage.

Step 2 Select SFS as the storage type.

Table 8-9 Parameters for mounting file storage


Storage Type SFS. This storage type applies to a wide range of scenarios, includingmedia processing, content management, big data, and applicationanalysis.

Allocation Mode

Manual Select an existing storage volume. If no storage volume is available,follow the prompts to create one.

Automatic A storage volume is created automatically. You need to enter thestorage capacity.NOTE

A storage volume that is created automatically will not be tagged.

1. Select the storage subtype.The file storage subtype is NFS.

2. Enter the storage capacity, measured in GB. Ensure that the storagecapacity quota is not exceeded; otherwise, creation will fail.

Add ContainerPath

1. Click Add Container Path.2. Container Path: Enter the container path to which the data volume is

mounted.NOTICE– Do not mount a data volume to a system directory such as / or /var/run.


– When the data volume is mounted to a high-risk directory, you areadvised to use a low-permission account to start the container; otherwise,high-risk files on the host machine may be damaged.

3. Set permissions.– Read-only: only allows you to read data volumes in the container

path.– Read/Write: allows you to modify the data volumes in the

container path. However, newly written data will not be migratedduring container migration.



Step 3 Click OK.

----End

Attaching and Mounting File Storage VolumesCCE allows you to import existing file storage volumes.

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management >Storage. On the SFS tab page, click Import.

Step 2 Select one or more file storage volumes that you want to attach and mount.

Step 3 Click OK.

----End

Unbinding an SFS File SystemAfter an SFS file system is successfully created or imported, the SFS file system isautomatically bound to the current cluster and cannot be used by other clusters. After the SFSfile system is unbound from the cluster, other clusters can import and use the file system.

If the SFS file system has been mounted to an application, the SFS file system cannot beunbound from the cluster.

Step 1 Log in to the CCE console. In the navigation pane, choose Resource Management >Storage. In the file storage volume list, click Unbind next to the target file storage volume.

Step 2 In the dialog box that is displayed, click OK.

----End

Creating a File System Using kubectlCCE supports the creation of file storage in the form of PersistentVolumeClaim (PVC).

Prerequisites


Procedure


Step 2 Run the following commands to configure the pvc-sfs-auto-example.yaml file, which is usedto create a PVC.

touch pvc-sfs-auto-example.yaml

vi pvc-sfs-auto-example.yaml

The following shows an example of creating a file storage.

apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations:




volume.beta.kubernetes.io/storage-class: nfs-rw name: pvc-sfs-auto-example namespace: default spec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi


l volume.beta.kubernetes.io/storage-class indicates the file storage type. Currently, thestandard file protocol type (nfs-rw) is supported.

l name indicates the name of the PVC to be created.l storage indicates the storage capacity in Gi.


kubectl create -f pvc-sfs-auto-example.yaml

After the command is executed, a file storage is created in the VPC to which the clusterbelongs. Choose Resource Management > Storage > SFS or log in to the SFS console toview the file system.

----End

Creating a PV/PVC for an Existing File System Using kubectl

CCE allows you to use an existing file system to create a PV and bind the PV to the PVC.

Prerequisites

You have configured the kubectl command and connected an ECS server to the cluster. Fordetails, see 3.4 Connecting to a Kubernetes Cluster Using kubectl

Procedure

Step 1 Log in to the SFS console, and obtain the ID of the file system.


Step 3 Run the following command to configure the pv-sfs-example.yaml file, which is used tocreate a PV.

vi pv-sfs-example.yaml

Copy the following yaml commands to the pv-sfs-example.yaml file.

apiVersion: v1kind: PersistentVolumemetadata: name: pv-sfs-example namespace: defaultspec: accessModes: - ReadWriteMany capacity: storage: 10Gi flexVolume: driver: huawei.com/fuxinfs fsType: nfs




options: deviceMountPath: fsType: nfs volumeID: f6976f9e-2493-419b-97ca-d7816008d91c persistentVolumeReclaimPolicy: Delete storageClassName: nfs-ro

Parameter description:l driver indicates the driver for the storage to be mounted. Set this parameter to

huawei.com/fuxinfs for file systems.l volumeID indicates the ID of the file system.l storage indicates the capacity of the file system.l storageClassName indicates the read/write mode (nfs-rw or nfs-ro) supported by the

file system.

Step 4 Run the following command to create a PV.

kubectl create -f pv-sfs-example.yaml

Step 5 Run the following commands to configure the pvc-sfs-example.yaml file, which is used tocreate a PVC.

touch pvc-sfs-example.yaml

vi pvc-sfs-example.yaml

Copy the following yaml commands to the pvc-sfs-example.yaml file.

apiVersion: v1kind: PersistentVolumeClaimmetadata: annotations: volume.beta.kubernetes.io/storage-class: nfs-rw volume.beta.kubernetes.io/storage-provisioner: flexvolume-huawei.com/fuxinfs name: pvc-sfs-example namespace: defaultspec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi

Parameter description:l volume.beta.kubernetes.io/storage-class indicates the file system type. Currently, the

file systems (nfs-rw and nfs-ro) compatible with the standard file protocol are supported.l volume.beta.kubernetes.io/storage-provisioner indicates the plug-in for creating the

file system. This parameter is fixed to flexvolume-huawei.com/fuxinfs.l volumeName indicates the PV name.


kubectl create -f pvc-sfs-example.yaml

----End

Mounting a File System Using kubectl

Step 1 Run the following commands to configure the sfs-pod-example.yaml file, which is used tocreate a pod.



touch sfs-pod-example.yaml

vi sfs-pod-example.yaml

The following shows an example of mounting file storage.

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: sfs-pod-example # Application name namespace: default spec: replicas: 1 selector: matchLabels: app: sfs-pod-example template: metadata: labels: app: sfs-pod-example spec: containers: - image: nginx:1.1 name: container-0 volumeMounts: - mountPath: /tmp # Mounting path name: pvc-sfs-example restartPolicy: Always volumes: - name: pvc-sfs-example persistentVolumeClaim: claimName: pvc-sfs-auto-example # Mount the PVC.


l name is the name of the pod to be created.l app is the name of a pod application.l mountPath is the mount path in a container. In the example, the mount path is /tmp.l spec.template.spec.containers.volumeMounts.name and

spec.template.spec.volumes.name must be consistent because they have a mappingrelationship.

Step 2 Run the following command to create a pod.

kubectl create -f sfs-pod-example.yaml

After the pod is created, choose Resource Management > Storage > SFS on the CCEconsole to view the binding relationship between the application and PVC.

----End

Related OperationsAfter the file storage is created, you can perform the operation described in Table 8-10.





Deleting a file storage 1. Select the name of the storage to be deleted and clickDelete in the Operation column.

2. Follow the prompts to delete the file storage.



9 Log Management

CCE allows you to configure policies for collecting and analyzing application logsperiodically to prevent logs from being over-sized.

9.1 Collecting Standard Output Logs of Containers

Procedure

Step 1 When creating a containerized application, add a container and expand Log Policies.

Step 2 By default, no configuration is performed. In this case, the standard output logs of thecontainer are collected by default. The following uses Nginx as an example to describe how tocreate an application.

Step 3 View logs.

After the application is created, access the nginx application. On the Application O&M tab,select All instances.

----End

9.2 Collecting Logs in a Specified Path of a Container

Procedure

Step 1 When creating a containerized application, upload an image in Set Containers and select LogPolicies.

Step 2 Click Add Log Policy. Set the parameters for configuring a log policy based on applicationrequirements. The following uses an nginx application as an example.

Cloud Container EngineUser Guide 9 Log Management


Figure 9-1 Adding a log policy

Table 9-1 Parameters for adding a log policy


Storage Type Currently, only HostPath is supported.

* Host Path Enter the log storage path on the host.

Add Container Path

Container Path 1. Click Add Container Path.2. Enter the container path to which the data volume is mounted.

NOTICE– Do not mount a data volume to a system directory such as / or /var/run.



Extended HostPath

None: No extended path is configured.




Aging Period l Hourly: Log files are scanned every hour. If a log file exceeds 20MB, it will be dumped to a historical file in the directory where thelog file is saved and then will be cleared.

l Daily: Log files are scanned every day. If a log file exceeds 20MB, it will be dumped to a historical file in the directory where thelog file is saved and then will be cleared.

l Weekly: Log files are scanned every week. If a log file exceeds 20MB, it will be dumped to a historical file in the directory where thelog file is saved and then will be cleared.

Step 3 Click OK. An application is created.

Step 4 View logs.

After the application is created, access the nginx application. On the Application O&M tab,select All instances.

----End



10 Container Orchestration

10.1 Basic Concepts

Helm

Helm is a tool similar to apt/yum/homebrew for Kubernetes. It is used for managingKubernetes charts and installing Kubernetes applications. If you deploy applications toKubernetes, Helm makes it easy to version those deployments, package it, make a release ofit, and deploy, delete, upgrade and even rollback those deployments as charts.

Chart

A chart is a collection of files that describe a set of Kubernetes resources required for runningKubernetes applications, tools, or services. The relationship between Chart and Helm issimilar to that between rpm and yum.

Release

A release is a chart instance running on Kubernetes. A single chart can be installed manytimes into the same cluster, and create many different releases. Each release can contain oneor more Kubernetes resource objects.

10.2 Preparing a Template PackageTwo methods are available to prepare a template package:

l Customizing a Template Packagel Using a Kubernetes Official Template Package

Template Package Specifications

The following uses the Redis application as an example. Prepare the Redis applicationtemplate package according to the template package specifications.

l Naming Requirement

Cloud Container EngineUser Guide 10 Container Orchestration


A template package is named in the format of application name-major versionnumber.minor version number.revision number.tgz, for example, redis-0.4.2.tgz.

NOTE

l The main version number, minor version number, and revision number must be integers. Theymust be ≥ 0 and ≤ 99.

l The main version number and minor version number are mandatory, and the revision numberis optional.

l Directory StructureThe directory structure of a template package is as follows:redis/ templates/ values.yaml README.md Chart.yaml .helmignoreTable 10-1 lists the parameters of the directory structure of a template package. Theparameters marked with an asterisk (*) are mandatory.

Table 10-1 Parameters of the directory structure of a template package


*templates All templates

*values.yaml Configuration parameters essential to describe the template

README.md Markdown file that contains the following:l Applications or services provided by Chart.l Prerequisites for running Chart.l Configurations in the values.yaml file.l Information about Chart installation and configuration.

*Chart.yaml Basic information about the template.

.helmignore Files or data that does not need to read templates duringapplication installation.

Customizing a Template Package

Step 1 Customize the content of a template package as required.

For details about how to create a template package, see https://github.com/kubernetes/helm/blob/master/docs/charts.md.

Step 2 Set the template package directory structure and name the template package based on therequirements defined in Template Package Specifications.

----End

Using a Kubernetes Official Template Package

Step 1 Access https://github.com/kubernetes/charts to obtain the required community templatepackage.



https://github.com/kubernetes/helm/blob/master/docs/charts.md

https://github.com/kubernetes/helm/blob/master/docs/charts.md

https://github.com/kubernetes/charts

Step 2 Check whether the package meets Template Package Specifications.

Step 3 Log in to a Linux machine.

Step 4 Upload the template package obtained in Step 1.

Step 5 Run the following command to compress the template package.l If the Helm client is not installed on the Linux machine, run the following command:

tar pzcf {name}-{version}.tgz {name}/In the preceding command,{name} indicates the actual name of the template package.{version} indicates the actual version of the template package.

l If the Helm client is installed on the Linux machine, run the following command:helm package {name}/In the preceding command, {name} indicates the actual name of the template package.After package, run the following command to check whether the compressed package iscorrect.helm lint {name}-{version}.tgzThe compressed package is correct if the following information is displayed:==> Linting {name}-{version}.tgzLint OK

----End

10.3 Uploading a Template PackageTo prepare for later application creation, you must upload the template to Chart Marketplace> Charts.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Chart Marketplace > Charts, andclick Upload Chart.

Step 2 In the Chart Package area, click , select the application package to be uploaded, and clickUpload.

----End

Follow-up Procedure

After a template is created, you can perform operations listed in Table 10-2 on the Chartspage.



Installing aTemplate

Click Install to install the template for creating applications. For details,see 10.4 Creating an Application Using Template.




Updating aTemplate

Click Update to update the template version. After the updating process,only the content of the template is updated, and the version of thetemplate is not updated. The procedure is similar to that of uploading atemplate.

Downloading aTemplate

Click More > Download to download the template to the local host.

Deleting aTemplate

On the template details page, click Delete in the upper right corner todelete the template.CAUTION

Once a template is deleted, it cannot be restored.

10.4 Creating an Application Using Template

PrerequisitesAt least one cluster has been created. For more information, see 3.2 Creating a VM Cluster.

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Chart Marketplace > Charts.

Step 2 Select the template uploaded in 10.3 Uploading a Template Package and click Install tocreate an application based on the template.

Step 3 Set the installation parameters listed in Table 10-3. The parameters marked with an asterisk(*) are mandatory.

Table 10-3 Parameters for creating an application


* ApplicationName


Chart Name Default name of the template.

* Chart Version Default version of the template.

* Cluster Cluster to which the application is deployed.

* Namespace Namespace to which the application is deployed.

* Description Enter the application description.




Advanced Settings You can import and replace the values.yaml file or directly edit thetemplate parameters online.NOTE

An imported values.yaml file must comply with YAML specifications, that is,KEY:VALUE format. The fields in the file are not restricted.

1. Click Import Configuration File.2. Select the corresponding values.yaml file and click Open.

Step 4 After the configuration is complete, click Install Now.

Step 5 Confirm the order and click Submit.

Step 6 Click Go to Application List to view the running status of the template application, or clickGo to Application Details to view the details of the template application.

After the application is created, click View Access Mode to view the application accessmode.

----End

10.5 Using an EVS DiskThe CCE uses plug-ins to connect to EVS disks to support persistent storage.

The following example shows how to define an EVS disk in a chart. When creating the chartapplication, the container dynamically creates a 10 Gi EVS disk and attaches it to thecontainer.

Currently, the CCE supports only creating EVS disks in a dynamic way.

apiVersion: apps/v1beta1kind: StatefulSetmetadata: name: {{ .Release.Name }}-slavespec: updateStrategy: type: "RollingUpdate" serviceName: {{ .Release.Name }}-slave-headless replicas: 1 template: metadata: labels: app: {{ .Release.Name }}-slave type: slave release: "{{ .Release.Name }}" spec: containers: - name: {{ .Release.Name }}-slave image: {{ .Values.chartimage.app_image }} volumeMounts: - mountPath: /redis-data name: {{ .Release.Name }}-slave



- mountPath: /opt/rancher/ name: utility - mountPath: /etc/redis/ name: redis-conf ports: - containerPort: 6379 volumeClaimTemplates: - metadata: labels: app: {{ .Release.Name }}-slave type: slave release: "{{ .Release.Name }}" name: {{ .Release.Name }}-slave annotations: "volume.beta.kubernetes.io/storage-class": sas "volume.beta.kubernetes.io/storage-provisioner": flexvolume-huawei.com/fuxivol spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 10Gi

Table 10-4 Key parameters


*annotations Used for console display. volume.beta.kubernetes.io/storage-classindicates the EVS disk type (SAS, SATA, or SSD). For details, see thedefinition of the EVS service. The value ofvolume.beta.kubernetes.io/storage-provisioner is fixed atflexvolume-huawei.com/fuxivol.

*accessModes EVS access mode. Three options are available:l ReadWriteOncel ReadOnlyManyl ReadWriteMany

*resource.request.storage

Size of the EVS disk, in Gi. The minimum value is 10.

10.6 Using Load BalancersThe service type of a load balancer can be used in a chart. Its definition method is the same asthat of the community.

To display the type of the load balancer on the CCE console, add the following annotation tothe corresponding resource type chart:

apiVersion: apps/v1beta1kind: StatefulSetmetadata: name: {{ .Release.Name }}-master annotations: "service.protal.kubernetes.io/access-ip": "10.4.4.14:8888" "service.protal.kubernetes.io/type": LoadBalancerspec: ......



Table 10-5 Key parameters


*annotations Used for console display. service.protal.kubernetes.io/access-ipindicates the IP address and exposed port number of the load balancer.The value of service.protal.kubernetes.io/type is fixed atLoadBalancer.

10.7 Upgrading a Template-based Application

Procedure

Step 1 Log in to the CCE console. In the navigation pane, choose Chart Marketplace > InstalledApplications.

Step 2 Click Upgrade for the application to be upgraded.

Step 3 Select the corresponding template version and follow the prompts to modify the templateparameters.

Step 4 Click Upgrade.

If the application status on the template application list page is Upgrade succeeded, theapplication has been successfully upgraded

----End

10.8 Rolling Back a Template-based Application

Procedure


Step 2 Click More > Roll Back next to the template application to be rolled back, select the templateapplication version to be rolled back, and click OK.

If the application status is Rollback succeeded in the template application list page, theapplication is rolled back successfully.

----End

10.9 Uninstalling Template-based Applications

Procedure


Step 2 Click More > Uninstall for the application you want to uninstall.



To uninstall multiple applications at a time, click Uninstall Application.

Step 3 Click OK.

Uninstalled applications cannot be restored. Therefore, exercise caution when uninstalling anapplication.

----End



11 Image Repository

Image Repository is a service provided by Software Repository for Container (SWR) servicefor storing and managing Docker container images. Image Repository allows you to easilystore, manage, and deploy Docker container images.

Uploading an ImageUpload images on SoftWare Repository. For details about how to upload an image, see ImageManagement.

Using an ImageAfter the image is uploaded successfully, you can choose an image from My Images to createan application on CCE. The following uses a game application as an example.

Step 1 Log in to the CCE console. In the navigation pane, choose Application Management. ClickCreate Application, and set Select Application Type to Stateless Applications.

Step 2 Set the following parameters, and retain the default settings for other parameters:l Application Name: gamel Cluster Name: Cluster in which the application residesl Instances: 1


Click Add Container and select the image to be deployed. Click OK.

Step 4 Create an application. For details, see 4.2 Creating a Stateless Application or 4.3 Creating aStateful Application.

----End

Cloud Container EngineUser Guide 11 Image Repository


https://support.huaweicloud.com/en-us/usermanual-swr/swr_01_0011.html

https://support.huaweicloud.com/en-us/usermanual-swr/swr_01_0011.html

12 Application O&M

After creating applications on CCE, you can operate and maintain the applications on AOM.This following introduces several AOM O&M scenarios by using the nginx as an example.

AOM is the one-stop platform for O&M personnel to monitor application and resourcerunning statuses in real time. By analyzing dozens of metrics, alarms, and logs, you canquickly locate root causes to ensure smooth running of services.

l Create threshold rules for metrics of these resources to monitor changes of certainresources. For details, see Creating Threshold Rules.

l Use the dashboard to learn comprehensive information in real time during routine O&M.You can create and add concerned contents to the dashboard. For details, see Creating aDashboard.

l Perform routine preventive maintenance inspection (PMI). For details, see MonitoringApplications.

Creating Threshold Rules

Threshold rules define upper and lower thresholds for metrics. When these rules are met,AOM reports threshold alarms. It can also send resource change information to you by shortmessage service (SMS) or email, so you are able to rapidly detect and handle abnormalities toensure resource running.

Step 1 In the navigation pane, choose Alarm Center > Threshold Rules and click Add Single-resource Threshold.

Step 2 Select a resource to be monitored and its metrics in the metric tree, configure the parameters,and click Next.

Step 3 Configure Threshold Name, Threshold Condition, Consecutive Period(s), AlarmSeverity, and other basic parameters.

NOTE

l Threshold Condition: Trigger condition of a threshold-crossing alarm. A threshold conditionconsists of two parts: determination condition (≥, ≤, >, and <) and threshold value. For example, ifThreshold Condition is set to ≥ 80, a threshold-crossing alarm will be generated when the actualmetric value exceeds 80.

l Consecutive Period(s): When the metric value meets the threshold condition for a specified numberof consecutive periods, a threshold-crossing alarm will be generated.

Cloud Container EngineUser Guide 12 Application O&M


Step 4 Configure a notification policy. If you do not need to receive SMS or email notifications, skipthis step.

1. Before configuring a notification policy, you need to create a topic, configure a topicpolicy, and add related subscribers to the topic.AOM has interconnected with Simple Message Notification (SMN). Click create a topicand access the SMN console. On the SMN console, create a topic, and then configure thetopic policy by referring to Figure 12-1. Otherwise, notifications will fail to be sent.Then, add related subscribers, that, is, receivers of notifications (via SMS or email). Inthis way, when an exception occurs in a resource, AOM can broadcast the status changeinformation of the threshold rule to the subscribers in real time, so that they can obtaininformation such as the resource running status in time and take measures to avoidservice loss caused by resource problems.After the configuration is complete, select the created topic from the Topic drop-downlist box.

2. Configure Trigger Condition, that is, the trigger condition for sending a notification.You can select multiple trigger conditions. For example, if you want to receivenotifications about threshold status changing from normal to other statuses, select bothThreshold crossing and Insufficient data. If you want to receive notifications upon anythreshold status change, select all trigger conditions.

Figure 12-1 Configuring a topic policy

Step 5 Click Submit to complete the configuration.

----End



Creating a Dashboard

During routine O&M, you can create a dashboard and add clusters, application metrics, andstatus graphs to the dashboard to learn comprehensive information. You can also add metricsfor routine O&M to the customized dashboard so that you can perform routine check withoutre-selecting metrics.

The dashboard can display metric data and status data. For different metric data, differenticons can be added based on demands. To monitor change trends or compare metrics, you cancreate line graphs. To learn the latest values, you can create digit graphs. The following showshow to create a dashboard:

Step 1 In the navigation pane, choose View Management > Dashboard. Click Create Dashboard.On the Create Dashboard page that is displayed, enter a dashboard name and click OK.

Step 2 Add the line graph, digit graph, threshold-crossing status, host status, and service status to thedashboard based on demands. The following shows how to add a line graph:

1. Select a graph adding mode: On the Select Which to Add page that is displayed, clickCreate below Metric Data.

2. Select the type of the metric graph: On the Add Metric Graph page that is displayed,select Line graph and then click Next.

3. Select metrics and set metric statistical methods, and click OK.

NOTE

To create multiple graphs of the same type, for example, to create multiple line graphs of differentmetrics, click Action and then select Copy in the upper right corner of the created graph. Then, clickAction and select Edit to modify metrics. In this way, you can create multiple graphs rapidly.

Step 3 After adding the graph, click Save on the right of the page.

----End

Monitoring Applications

Application monitoring adopts the hierarchical drill-down design. The hierarchy is as follows:application list > application details > instance details > container/process details. That is,applications, instances, containers, and processes are interconnected. Their hierarchicalrelationships and health status are directly displayed on the GUI.

On the details page of each layer, resource alarms, logs, and host information are associated todisplay alarm statistics, host statuses, and the next-level resource list for further analysis.

Step 1 In the navigation pane, choose Metrics > Application.

Step 2 Click the to-be-queried application in the application list or configure filter criteria to find theto-be-queried application. Click the application name. The Application Overview page isdisplayed.

In the upper right corner of the page, select a statistical period from the drop-down list. Youcan view details about application monitoring during the selected period.

Click Add Metric Monitoring Graph on the right of Metric Monitoring Graphs tocustomize the display of metric graphs. This helps you monitor concerned metrics and viewmetric trends in real time.



In the Instances list, view information about all instances of the application. Click an instancename. On the Instances Overview page that is displayed, monitor the application instance.Click an IP address. On the Process Overview page that is displayed, monitor the process.

Step 3 Click an instance name. On the Instances Overview page that is displayed, monitor theapplication instance.

Step 4 Click a container name. On the Container Overview page that is displayed, monitor thecontainer.

When you select Add Threshold from the More drop-down list in a metric graph, you can seta threshold rule for the metric.

When you select Details from the More drop-down list in a metric graph, the Metrics page isdisplayed. You can adjust the statistical cycle and time range to view the metric graph indifferent dimensions.

In the Threshold-Crossing Alarms area, you can view application metric statistics ofthreshold-crossing alarms of different alarm severities.

Step 5 In the navigation pane, choose Threshold-Crossing Alarms to view or set threshold ruleinformation.

Step 6 In the navigation pane, choose Running Logs to view run logs of the application.

----End



13 Kubectl Usage Guide

Table 13-1 kubectl usage guide

Category kubectl Usage

Connecting toa cluster

Connecting to the Kubernetes Cluster Using kubectl

Creating anapplication

Creating a Stateless Application Using kubectl

Creating a Stateful Application Using kubectl

Applicationaffinity andanti-affinityscheduling

Example YAML for Deploying an Application on a Specified Node

Example YAML for Deploying an Application with Node Anti-Affinity

Example YAML for Deploying Applications on the Same Node

Example YAML for Deploying Applications on Different Nodes

Example YAML for Deploying an Application in a Specified AZ

Example YAML for Deploying an Application with AZ Anti-Affinity

Applicationaccess modesettings

Implementing Intra-Cluster Access Using kubectl

Implementing Intra-VPC Access Using kubectl

Implementing Public Network Access (EIP) Using kubectl

Implementing Public Network Access (ELB) Using kubectl

Taskmanagement

Creating a Job Using kubectl

Creating a Cron Job Using kubectl

Configurationcenter

Creating a Configuration Item Using kubectl

Creating a Secret Using kubectl

Storagemanagement

Creating an EVS Disk Using kubectl

Mounting an EVS Disk Using kubectl

Cloud Container EngineUser Guide 13 Kubectl Usage Guide


Category kubectl Usage

Creating a File System Using kubectl

Mounting a File System Using kubectl

Cloud Container EngineUser Guide 13 Kubectl Usage Guide


14 Reference

14.1 Formula for Calculating the Reserved Resources of aNode

Some of the resources on the node need to run some necessary Kubernetes systemcomponents and resources to make the node as part of your cluster. Therefore, the totalnumber of node resources and the number of assignable node resources in Kubernetes aredifferent. The larger the node specifications, the more the containers deployed on the node.Therefore, Kubernetes needs to reserve more resources.

To ensure node stability, CCE cluster nodes reserve some resources for Kubernetescomponents (such as kubelet, kube-proxy, and docker) based on node specifications.

CCE calculates the resources that can be allocated to user nodes as follows:

Allocatable = Capacity - Reserved - Eviction Threshold

That is, Configurable amount on the node = Total amount – Reserved amount –Eviction threshold.

l The rules for reserving the node memory are as follows:

a. total_mem ≤ 4 GB, reserved_value = total_mem x 25%b. 4 GB < total_mem ≤ 8 GB, reserved_value = 4 GB x 25% + (total_mem – 4 GB) x

20%c. 8 GB < total_mem ≤ 16 GB, reserved_value = 4 GB x 25% + 4 GB x 20% +

(total_mem – 8 GB) x 10%d. 16 GB < total_mem ≤ 128 GB, reserved_value = 4 GB x 25% + 4 GB x 20% + 8

GB x 10% + (total_mem – 16 GB) x 6%e. total_mem > 128 GB, reserved_value = 4 GB x 25% + 4 GB x 20% + 8 GB x 10%

+ 112 GB x 6% + (total_mem – 128 GB) x 2%

In the preceding information, total_mem indicates the total memory andreserved_value indicates the reserved memory.

l The rules for reserving the node CPU are as follows:

a. total_cpu ≤ 1 core, reserved_value = total_cpu x 6%

Cloud Container EngineUser Guide 14 Reference


b. 1 core < total_cpu ≤ 2 core, reserved_value = 1 core x 6% + (total_cpu – 1 core) x1%

c. 2 core < total_cpu ≤ 4 core, reserved_value = 1 core x 6% + 1 core x 1% +(total_cpu – 2 core) x 0.5%

d. total_cpu > 4 core, reserved_value = 1 core x 6% + 1 core x 1% + 2 core x 0.5% +(total_cpu – 4 core) x 0.25%

In the preceding information, total_cpu indicates the total CPU, and reserved_valueindicates the reserved CPU.

l CCE reserves an extra 100Mi for kubelet eviction.

14.2 How Do I Enable ICMP Security Group Rules?If the health check protocol of a load balancer is UDP, you must enable the ICMP securitygroup rules of the backend server.

Procedure

Step 1 Log in to the ECS console, find the ECS corresponding to the node in the CCE cluster, andclick the ECS name. The ECS details page is displayed.

NOTE

You only need to change the security group rules for any node in the cluster where the application islocated. Add rules and do not modify the original security group rules.

Step 2 On the Security Groups tab page, click Modify Security Group Rule.

Step 3 Click Add Rule to add an inbound rule for the ECS. For details, see . Click OK.

----End

14.3 Connecting to a Kubernetes Cluster Using Helm

Prerequisitesl A VM that has a Helm client and a kubectl client installed is available.l An EIP has been bound to the target cluster. You can perform the following operations to

bind an EIP to the cluster:

a. Log in to the CCE console. In the navigation pane, choose Resource Management> VM Clusters. Click Kubectl for the target cluster.

b. Bind an EIP to the cluster by referring to 3.4 Connecting to a Kubernetes ClusterUsing kubectl.

Procedure

Step 1 Log in to the VM and run the following command to create a file namedtiller_service_account.yaml:

vi tiller_service_account.yaml

Copy the following content to the file. Parameter values do not need to be modified.



apiVersion: v1kind: ServiceAccountmetadata: name: tiller namespace: kube-system---apiVersion: rbac.authorization.k8s.io/v1beta1kind: ClusterRoleBindingmetadata: name: tillerroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-adminsubjects: - kind: ServiceAccount name: tiller namespace: kube-system

Step 2 Run the following command to create a service account for Tiller:

kubectl create -f tiller_service_account.yaml

Step 3 Install Helm and Tiller.

helm init --service-account tiller

Step 4 After you have connected to the target Kubernetes cluster using Helm, you can use Helm todeploy charts. In the following steps, fc-tank-0.1.0.tgz is deployed as an example.

1. Log in to the VM where the Helm client is installed. Run the following commands tocheck the version of the installed Helm.helm versionInformation similar to the following is displayed:

2. Deploy fc-tank-0.1.0.tgz.helm install fc-tank-0.1.0.tgz

Run the kubectl get pods command to check the pod status. If the status is Running, thedeployment is successful.

3. View all releases of the chart.helm list



If you want to delete or uninstall the solid-grizzly release, run the following command:helm delete solid-grizzly

NOTE

For more information about how to use Helm, visit the official Helm website.

----End



https://medium.com/@gajus/the-missing-ci-cd-kubernetes-component-helm-package-manager-1fe002aac680

user guide - huawei cloud · user guide issue 01 date 2019-07-04 huawei technologies co., ltd. ......

Documents