OpsAny Community Edition — Kubernetes Cluster Deployment

OpsAny Enterprise Subscription and Enterprise Solution editions include implementation of Kubernetes cluster deployments. For the Community Edition you can use the documentation and scripts we provide to build the cluster yourself.

We provide Helm-based deployment of OpsAny into a Kubernetes cluster. You may want to learn the relevant tooling first; for Helm, see the official documentation at https://helm.sh/docs/.

The main flow for deploying OpsAny on a Kubernetes cluster is:

• Understand Kubernetes (if you do not, do not deploy).
• Understand Helm.
• Prepare a Kubernetes cluster (version >= 1.20), install Helm, and verify it.
• Prepare an Nginx Ingress Controller and an NFS-based Storage Class.
• Deploy the OpsAny Community Edition components one by one and verify each.
• Sign in to OpsAny Community Edition and use it.

Tip

Before starting, we need to agree on the following. If these conditions are not met, do not start — time is precious:

• Domain-name access only, domain-name access only, domain-name access only (do not deploy on Kubernetes without a domain name).
• If any step fails, fix it before continuing; otherwise you will only waste time.
• If you are not familiar with Kubernetes, give up and choose the simpler Docker container deployment. (Do you know what DaemonSet + NodeSelector produces? If not, we consider you unfamiliar.)
• OpsAny Community Edition on Kubernetes supports domain-name access through Ingress only. Prepare a domain name in advance and point it at the host running the Ingress Controller. (Confused? Do not deploy!)

Preparing the Kubernetes Cluster and Deployment Host

1. You need to prepare a Kubernetes cluster yourself, version >= 1.20. Buying a managed container service in the cloud is recommended. If you use your own hosts, you can deploy it yourself; see:
   • Deploying a Kubernetes cluster with kubeadm
   • Automated Kubernetes cluster deployment with SaltStack
2. You need a deployment host to run the deployment and installation scripts. This document simply reuses the Kubernetes Master node.
3. Please confirm again that you understand Kubernetes fundamentals and that the Kubernetes version is >= 1.20. Earlier versions are incompatible and the deployment cannot complete.
4. Prepare an access domain name for the Ingress configuration. Do not start without one.

Installing and configuring kubectl

Deploying on a Kubernetes Master node is recommended — it saves you the kubectl installation and configuration steps.

Deploy on the Kubernetes Master node

No need to install kubectl; you only need to deploy Helm.

1. Deploy the Helm client

   $ cd /usr/local/src
   $ wget https://get.helm.sh/helm-v3.11.1-linux-amd64.tar.gz
   $ tar zxf helm-v3.11.1-linux-amd64.tar.gz
   $ mv linux-amd64/helm /usr/local/bin/
   

2. Verify the installation

   $ helm version
   version.BuildInfo{Version:"v3.2.4", GitCommit:"b29d20baf09943e134c2fa5e1e1cab3bf93315fa", GitTreeState:"clean", GoVersion:"go1.13.7"}
   

Deploy on a dedicated host

1. Install the kubectl client

# Set the client version to match the server version
$ K8S_VERSION=1.20.11
$ cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.tencent.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
EOF
$ yum install -y kubectl-${K8S_VERSION}

1. Configure kubeconfig

For a managed public-cloud Kubernetes cluster, enable public access to the API Server, download the kubeconfig file, and replace the address in the file with the public IP.

$ mkdir -p ~/.kube/
$ vim ~/.kube/config

1. Deploy the Helm client

$ cd /usr/local/src
$ wget https://get.helm.sh/helm-v3.11.1-linux-amd64.tar.gz
$ tar zxf helm-v3.11.1-linux-amd64.tar.gz
$ mv linux-amd64/helm /usr/local/bin/

1. Verify the Helm installation

$ helm version
version.BuildInfo{Version:"v3.2.4", GitCommit:"b29d20baf09943e134c2fa5e1e1cab3bf93315fa", GitTreeState:"clean", GoVersion:"go1.13.7"}

Deploying NFS file sharing

NFS stores OpsAny's shared data — uploaded scripts and files, customized images, and so on. Daily backups must cover this data. If your cluster already has a usable Storage Class, you can skip this step, but you must then change the Storage Class name in every subsequent configuration. To keep things simple, do not change the NFS shared path; the following operations assume it.

1. Install and configure NFS

# For CentOS
$ yum install -y nfs-utils rpcbind

# For Ubuntu
$ apt install -y nfs-server rpcbind

1. Configure NFS

$ mkdir -p /data/k8s-nfs
$ vim /etc/exports
/data/k8s-nfs *(rw,sync,no_root_squash)

1. Start NFS and verify

# For CentOS
$ systemctl enable rpcbind nfs
$ systemctl start rpcbind nfs

# For Ubuntu
$ systemctl enable rpcbind nfs-server
$ systemctl start rpcbind nfs-server

# Change 192.168.56.11 to your NFS host's IP address
$ NFS_SERVER=192.168.56.11
$ showmount -e ${NFS_SERVER}
Export list for 192.168.56.11:
/data/k8s-nfs *
Output like the above means NFS is configured.

Initializing the Deployment Environment

Initialization has two steps: - Edit the configuration file and write the correct settings into it. - Run the script, which uses those settings to update every configuration file required for deployment in bulk.

Configuring the installation environment

1. Install the packages used by initialization

CentOS 7.x

# Note: if you paste and run the whole block, read the output carefully — network issues sometimes cause a command to fail without you noticing.
curl -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
yum install -y git uuid wget mariadb jq python3 python3-pip python3-devel ntpdate
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
ntpdate time1.aliyun.com
pip3 install -U pip setuptools -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip3 install requests==2.25.1 grafana-api==1.0.3 mysql-connector==2.2.9 SQLAlchemy==1.4.22 bcrypt==4.0.1 \
         -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com

CentOS 8.x

# Note: if you paste and run the whole block, read the output carefully — network issues sometimes cause a command to fail without you noticing.
dnf -y install mariadb jq git uuid
systemctl enable --now docker
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
pip3 install -U pip setuptools -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip3 install requests==2.25.1 grafana-api==1.0.3 mysql-connector==2.2.9 SQLAlchemy==1.4.22 bcrypt==4.0.1 \
         -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com

Ubuntu

# Note: if you paste and run the whole block, read the output carefully — network issues sometimes cause a command to fail without you noticing.
# step 1: install the required system tools
sudo apt-get -y update
sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common
sudo apt-get -y install jq uuid wget mysql-client git ntpdate python3 python3-pip
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
ntpdate time1.aliyun.com
pip3 install -U pip setuptools -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip3 install requests==2.25.1 grafana-api==1.0.3 mysql-connector==2.2.9 SQLAlchemy==1.4.22 bcrypt==4.0.1 \
         -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com

1. Clone the code

Gitee

$ cd /opt && git clone --depth=1 https://gitee.com/unixhot/opsany-paas.git

Github

$ cd /opt && git clone --depth=1 https://github.com/unixhot/opsany-paas.git

1. Edit the configuration file

2. Step1: generate the configuration file from the template

# Copy the configuration template to create the configuration file
$ cd /opt/opsany-paas/install && cp install-k8s.config.example install-k8s.config

• Step2: adjust the configuration file to your needs

If you use external MySQL, Redis, or MongoDB, update the database addresses, usernames, and passwords. If everything runs inside the current Kubernetes cluster, you only need to change the setting below.

$ vim install-k8s.config
# Set this to the domain name used to access OpsAny. Kubernetes deployment uses Ingress, so a domain name is mandatory.
DOMAIN_NAME=demo.opsany.com
# To keep things simple, do not change INSTALL_PATH; everything below assumes the default /data/opsany.

1. Run the initialization script

The initialization script reads the configuration file, updates the Helm values.yaml and the service configuration files, and creates the corresponding NFS mount paths.

# This script reads install-k8s.config and generates all PaaS deployment configuration files into the Kubernetes directory under the installation path.
$ cd /opt/opsany-paas/install/
$ ./paas-k8s-install.sh install

Creating the storage and ingress components

1. Create the Storage Class

OpsAny uses NFS as its default shared storage for uploaded files (the uploads directory), log files (the logs directory), and so on.

• Step1: create a dedicated namespace for OpsAny

$ kubectl create ns opsany

• Step2: install nfs-subdir-external-provisioner with Helm. nfs.server is the NFS IP address; nfs.path is the shared path.

# Change 192.168.56.11 to your NFS host's IP address
$ NFS_SERVER=192.168.56.11
$ source install-k8s.config

# Install the NFS provisioner, which lets the StorageClass create PVs automatically.
helm install nfs-provisioner ${INSTALL_PATH}/kubernetes/helm/opsany-base/nfs-subdir-external-provisioner \
    --set storageClass.name=nfs-sc-opsany \
    --set nfs.server=${NFS_SERVER} \
    --set nfs.path=/data/k8s-nfs \
    --set storageClass.defaultClass=true \
    --set image.repository=registry.cn-beijing.aliyuncs.com/opsany/nfs-subdir-external-provisioner \
    --set image.tag=v4.0.2 \
    -n opsany

• Step3: check the created Pod and Storage Class. All later storage uses the StorageClass named nfs-sc-opsany.

# Wait until the Pod is running before continuing
$ kubectl get pod -n opsany
NAME                                               READY   STATUS    RESTARTS   AGE
nfs-subdir-external-provisioner-5679c5d797-prn7p   1/1     Running   0          67s

1. Create the shared PVs and PVCs for uploads, logs, and proxy

$ sed -i "s/LOCAL_IP/${NFS_SERVER}/g" ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-nfs.yaml
$ kubectl apply -f ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-nfs.yaml -n opsany

1. Deploy the Nginx Ingress Controller

If you are using a managed public-cloud product, you do not need Ingress — you can change the service type of opsany-paas-openresty to LoadBalancer later. Otherwise deploy the Nginx Ingress Controller as the access entry point.

# Label the host where the Nginx Ingress Controller should run.
$ kubectl label node linux-node2.example.com edgenode=true

# Create the Ingress Controller; it runs as DaemonSet + NodeSelector by default.
$ kubectl apply -f  ${INSTALL_PATH}/kubernetes/helm/opsany-paas/nginx-ingress.yaml

# Continue once the Ingress Controller is running.
$ kubectl get pod -n ingress-nginx
NAME                                   READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-wpq2d   0/1     Completed   0          10d
ingress-nginx-admission-patch-kdgmr    0/1     Completed   0          10d
ingress-nginx-controller-f68tj         1/1     Running     0          10d

Deploying OpsAny Community Edition with Helm

Deploying the OpsAny base services

Deploying by hand makes the whole process and the services involved clear, and — most importantly — lets you spot and fix problems in time. The document looks long only because much of it is checking results. You must wait until each created Pod is Ready before continuing.

1. Deploy the MySQL database

2. We recommend using a cloud RDS product or a MySQL instance managed by your production DBA, which lets you skip the first two steps.

# Step1: deploy MySQL
$ helm install opsany-base-mysql ${INSTALL_PATH}/kubernetes/helm/opsany-base/mysql -n opsany
$ kubectl get pod -n opsany | grep "opsany-base-mysql-"

# Step2: once the Pod is Ready, enter it and test the MySQL login with the password from k8s-install.config.
$ kubectl exec -it opsany-base-mysql-0 -n opsany -- /bin/bash
$ mysql -u root -p
> show databases;
> exit;
> exit

# Step3: initialize the schema. This creates the MySQL databases for PaaS and SaaS and takes about 2 minutes.
cd /opt/opsany-paas/install/
./paas-k8s-install.sh mysql

1. Deploy the MongoDB database

2. We recommend using a cloud RDS product or a MongoDB instance managed by your production DBA, which lets you skip the first two steps.

# Step1: deploy MongoDB
$ helm install opsany-base-mongodb ${INSTALL_PATH}/kubernetes/helm/opsany-base/mongodb -n opsany
$ kubectl get pod -n opsany | grep "opsany-base-mongodb-"

# Step2: once the Pod is Ready, enter it and test the MongoDB login with the password from k8s-install.config.
$ kubectl exec -it opsany-base-mongodb-0 -n opsany -- /bin/bash
$ mongosh -u root -p
> show databases;
> exit;
$ exit

# Step3: initialize the schema. This creates the MongoDB databases used by the SaaS platform and takes about 1 minute.
$ cd /opt/opsany-paas/install/
$ ./paas-k8s-install.sh mongodb

1. Deploy the Redis cache

2. We recommend using a cloud RDS product or a Redis instance managed by your production DBA.

# Step1: deploy Redis
$ helm install opsany-base-redis ${INSTALL_PATH}/kubernetes/helm/opsany-base/redis -n opsany
$ kubectl get pod -n opsany | grep "opsany-base-redis-"

# Step2: once the Pod is Ready, enter it and test the Redis login. If you changed the password in the configuration file, use it here.
$ kubectl exec -it opsany-base-redis-master-0 -n opsany -- /bin/bash
$ redis-cli
127.0.0.1:6379> auth OpsAny2020
127.0.0.1:6379> info
127.0.0.1:6379> exit
exit

The steps above deploy the base services OpsAny needs: MySQL, MongoDB, and Redis. Make sure they all run correctly before continuing. If anything is wrong, stop — there is no point continuing.

$ kubectl get pod -n opsany
NAME                                                              READY   STATUS    RESTARTS   AGE
nfs-provisioner-nfs-subdir-external-provisioner-5d44f75dd4z98kf   1/1     Running   0          12m
opsany-base-mongodb-0                                             1/1     Running   0          4m12s
opsany-base-mysql-0                                               1/1     Running   0          6m52s
opsany-base-redis-master-0                                        1/1     Running   0          63s

---

Deploying the OpsAny PaaS platform

1. Deploy the Login service

Login handles unified authentication. Requests that go directly to a PaaS service port are redirected to Login for authentication.

# Step1: check whether the Login configuration file was generated correctly. If not, all later configuration is probably wrong too — the initialization failed, so go back to the previous section.
$ cat ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-login/settings_production.py.login
Check that BK_COOKIE_DOMAIN and the PASSWORD entry under DATABASES are correct.

# Step2: deploy the Login service with Helm.
$ helm install opsany-paas-login ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-login/ -n opsany

# Step3: after 30-60 seconds, check that the Login Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-login-"

# Step4: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
LOGIN_SVC=$(kubectl get svc -n opsany | grep 'opsany-paas-login' | awk -F ' ' '{print $3}')
curl --head http://${LOGIN_SVC}:8003/healthz/

1. Deploy the ESB service

ESB is the enterprise service bus. It depends on MySQL, Redis, the PaaS service, and the Login service.

# Step1: deploy the ESB service with Helm.
$ helm install opsany-paas-esb ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-esb/ -n opsany

# Step2: after 30-60 seconds, check that the ESB Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-esb-"

# Step3: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
ESB_SVC=$(kubectl get svc -n opsany | grep 'opsany-paas-esb' | awk -F ' ' '{print $3}')
curl --head http://${ESB_SVC}:8002/healthz/

1. Deploy the Appengine service

Appengine is the backend module of the PaaS platform that hosts SaaS applications. It depends on MySQL and the PaaS service.

# Step1: deploy the AppEngine service with Helm.
$ helm install opsany-paas-appengine ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-appengine/ -n opsany

# Step2: after 30-60 seconds, check that the AppEngine Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-appengine-"

# Step3: test access. 【HTTP/1.1 200 OK】 means normal; a 5XX status code is an error.
AppEngine_SVC=$(kubectl get svc -n opsany | grep 'opsany-paas-appengine' | awk -F ' ' '{print $3}')
curl --head http://${AppEngine_SVC}:8000/v1/healthz/

1. Deploy the PaaS service

The PaaS service provides the developer center and the web workbench of the PaaS platform. It uses MySQL and depends on the Login and AppEngine services, listening on port 8001.

# Step1: deploy the PaaS service with Helm.
$ helm install opsany-paas-paas ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-paas -n opsany

# Step2: after 30-60 seconds, check that the PaaS Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-paas-"

# Step3: test access. 【HTTP/1.1 200 OK】 means normal; 4XX and 5XX status codes are errors.
PAAS_SVC=$(kubectl get svc -n opsany | grep 'opsany-paas-paas' | awk -F ' ' '{print $3}')
curl --head http://${PAAS_SVC}:8001/healthz/

1. Deploy the Guacd RDP proxy

Guacd lets the Bastion Host connect to Windows remote desktops over RDP and to network devices over Telnet.

# Step1: deploy the Guacd RDP service with Helm.
$ helm install opsany-paas-guacd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-guacd -n opsany

# Step2: after 30-60 seconds, check that the Guacd Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-guacd-"

1. Deploy the Bastion Host WebSocket service

The WebSocket service carries the Bastion Host's underlying communication; if it fails, the Bastion Host becomes unusable.

# Step1: deploy the WebSocket service with Helm.
$ helm install opsany-paas-websocket ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-websocket/ -n opsany

# Step2: after 30-60 seconds, check that the WebSocket Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-websocket-"

1. Deploy the OpsAny Grafana service

OpsAny integrates Grafana for data visualization.

# Step1: deploy the Grafana service with Helm.
$ helm install opsany-paas-grafana ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-grafana/ -n opsany

# Step2: after 30-60 seconds, check that the Grafana Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-grafana-"

# Step3: test access. 【HTTP/1.1 200 OK】 means normal; a 5XX status code is an error.
GRAFANA_SVC=$(kubectl get svc -n opsany | grep 'opsany-paas-grafana' | awk -F ' ' '{print $3}')
curl --head -k https://${GRAFANA_SVC}:8007/grafana/login

1. Deploy the OpenResty service

OpenResty is the entry point for OpsAny; all requests are forwarded through it.

# Step1: deploy the OpenResty service with Helm.
$ helm install opsany-paas-openresty ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/ -n opsany

# Step2: after 30-60 seconds, check that the OpenResty Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-paas-openresty-"

# Step3: test access. 【HTTP/1.1 200 OK】 means normal; a 5XX status code is an error.
OpenResty_SVC=$(kubectl get svc -n opsany | grep 'opsany-paas-openresty' | awk -F ' ' '{print $3}')
curl --head http://{$OpenResty_SVC}

1. Access and test the OpsAny PaaS platform

By now you have deployed every service the OpsAny PaaS needs. Open a browser and test access through the domain name — use the DOMAIN_NAME set in install-k8s.config. - Access address: https://$DOMAIN_NAME/ - Default username: admin - Default password: admin

Note: do not change the password yet. It is changed automatically after the SaaS platform is installed; watch the messages printed when installation finishes.

---

Deploying the OpsAny SaaS platform

1. Run the script to generate the configuration files. If it fails, do not proceed.

# Generate configuration files for each SaaS
cd /opt/opsany-paas/install/
./saas-ce-k8s-install.sh install
source /opt/opsany-paas/install/install-k8s.config

1. Deploy unified permissions — rbac

# Step1: deploy the rbac service with Helm.
$ helm install opsany-saas-rbac ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-rbac/ -n opsany

# Step2: after 30-60 seconds, check that the rbac Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-rbac-"

# Step3: update the Openresty configuration to enable rbac access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7001/opsany-saas-rbac:7001/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
RBAC_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-rbac' | awk -F ' ' '{print $3}')
curl --head http://${RBAC_SVC}:7001/o/rbac/healthz/

1. Deploy the Workbench — workbench

# Step1: deploy the workbench service with Helm.
$ helm install opsany-saas-workbench ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-workbench/ -n opsany

# Step2: after 30-60 seconds, check that the workbench Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-workbench-"

# Step3: update the Openresty configuration to enable workbench access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7002/opsany-saas-workbench:7002/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code workbench

> Note: if the step above fails, do not continue — it means workbench cannot reach the rbac service through ESB, which is usually a network problem.

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
WORKBENCH_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-workbench' | awk -F ' ' '{print $3}')
curl --head http://${WORKBENCH_SVC}:7002/o/workbench/healthz/

1. Deploy the resource platform — cmdb

# Step1: deploy the cmdb service with Helm.
$ helm install opsany-saas-cmdb ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-cmdb/ -n opsany

# Step2: after 30-60 seconds, check that the cmdb Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-cmdb-"

# Step3: update the Openresty configuration to enable cmdb access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7003/opsany-saas-cmdb:7003/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code cmdb

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
CMDB_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-cmdb' | awk -F ' ' '{print $3}')
curl --head http://${CMDB_SVC}:7003/o/cmdb/healthz/

1. Deploy the Control Platform — control

# Step1: deploy the control service with Helm.
$ helm install opsany-saas-control ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-control/ -n opsany

# Step2: after 30-60 seconds, check that the control Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-control-"

# Step3: update the Openresty configuration to enable control access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7004/opsany-saas-control:7004/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code control

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
CONTROL_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-control' | awk -F ' ' '{print $3}')
curl --head http://${CONTROL_SVC}:7004/o/control/healthz/

1. Deploy the Job Platform — job

# Step1: deploy the job service with Helm.
$ helm install opsany-saas-job ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-job/ -n opsany

# Step2: after 30-60 seconds, check that the job Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-job-"

# Step3: update the Openresty configuration to enable job access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7005/opsany-saas-job:7005/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code job

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
JOB_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-job' | awk -F ' ' '{print $3}')
curl --head http://${JOB_SVC}:7005/o/job/healthz/

1. Deploy the Cloud Management Platform — cmp

# Step1: deploy the cmp service with Helm.
$ helm install opsany-saas-cmp ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-cmp/ -n opsany

# Step2: after 30-60 seconds, check that the cmp Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-cmp-"

# Step3: update the Openresty configuration to enable cmp access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7007/opsany-saas-cmp:7007/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code cmp

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
CMP_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-cmp' | awk -F ' ' '{print $3}')
curl --head http://${CMP_SVC}:7007/o/cmp/healthz/

1. Deploy the Bastion Host — bastion

# Step1: deploy the bastion service with Helm.
$ helm install opsany-saas-bastion ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-bastion/ -n opsany

# Step2: after 30-60 seconds, check that the bastion Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-bastion-"

# Step3: update the Openresty configuration to enable bastion access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7008/opsany-saas-bastion:7008/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code bastion

# Step5: test Bastion Host access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
BASTION_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-bastion' | awk -F ' ' '{print $3}')
curl --head http://${BASTION_SVC}:7008/o/bastion/healthz/

# Step6: test WebSocket access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
WEBSOCKET_SVC=$(kubectl get svc -n opsany | grep 'opsany-paas-websocket' | awk -F ' ' '{print $3}')
curl --head http://${WEBSOCKET_SVC}:8004/healthz/

1. Deploy basic monitoring — monitor

# Step1: deploy the monitor service with Helm.
$ helm install opsany-saas-monitor ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-monitor/ -n opsany

# Step2: after 30-60 seconds, check that the monitor Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-monitor-"

# Step3: update the Openresty configuration to enable monitor access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7006/opsany-saas-monitor:7006/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code monitor

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
MONITOR_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-monitor' | awk -F ' ' '{print $3}')
curl --head http://${MONITOR_SVC}:7006/o/monitor/healthz/

1. Deploy the Application Platform — devops

# Step1: deploy the devops service with Helm.
$ helm install opsany-saas-devops ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-devops/ -n opsany

# Step2: after 30-60 seconds, check that the devops Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-devops-"

# Step3: update the Openresty configuration to enable devops access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7009/opsany-saas-devops:7009/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code devops

# Step5: test Application Platform access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
DEVOPS_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-devops' | awk -F ' ' '{print $3}')
curl --head http://${DEVOPS_SVC}:7009/o/devops/healthz/

1. Deploy the code repository — code

# Step1: deploy the code service with Helm.
$ helm install opsany-saas-code ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-code/ -n opsany

# Step2: after 30-60 seconds, check that the code Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-code-"

# Step3: update the Openresty configuration to enable code access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7010/opsany-saas-code:7010/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code code

# Step5: test code repository access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
CODE_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-code' | awk -F ' ' '{print $3}')
curl --head http://${CODE_SVC}:7010/o/code/healthz/

1. Deploy the Application Platform — pipeline

# Step1: deploy the pipeline service with Helm.
$ helm install opsany-saas-pipeline ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-pipeline/ -n opsany

# Step2: after 30-60 seconds, check that the pipeline Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-pipeline-"

# Step3: update the Openresty configuration to enable pipeline access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7017/opsany-saas-pipeline:7017/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code pipeline

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
PIPELINE_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-pipeline' | awk -F ' ' '{print $3}')
curl --head http://${PIPELINE_SVC}:7017/o/pipeline/healthz/

1. Deploy the Application Platform — repo

# Step1: deploy the repo service with Helm.
$ helm install opsany-saas-repo ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-repo/ -n opsany

# Step2: after 30-60 seconds, check that the repo Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-repo-"

# Step3: update the Openresty configuration to enable repo access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7020/opsany-saas-repo:7020/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code repo

# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
REPO_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-repo' | awk -F ' ' '{print $3}')
curl --head http://${REPO_SVC}:7020/o/repo/healthz/

1. Deploy the Application Platform — deploy

# Step1: deploy the deploy service with Helm.
$ helm install opsany-saas-deploy ${INSTALL_PATH}/kubernetes/helm/opsany-saas/opsany-saas-deploy/ -n opsany

# Step2: after 30-60 seconds, check that the deploy Pod is Running.
$ kubectl get pod -n opsany | grep "opsany-saas-deploy-"

# Step3: update the Openresty configuration to enable deploy access
cd ${INSTALL_PATH}/kubernetes/helm/opsany-paas/opsany-paas-openresty/
sed -i 's/127.0.0.1:7018/opsany-saas-deploy:7018/g' opsany_paas.conf
helm upgrade opsany-paas-openresty . -n opsany
OPENRESTY_POD=$(kubectl get pod -n opsany | grep opsany-paas-openresty | awk '{print $1}')
kubectl delete pod ${OPENRESTY_POD} -n opsany
kubectl get pod -n opsany | grep opsany-paas-openresty

# Step4: sync users from RBAC
cd /opt/opsany-paas/saas/
python3 sync-user-script.py --domain https://${DOMAIN_NAME} --paas_username admin --paas_password admin --app_code deploy


# Step5: test access. 【HTTP/1.1 200 OK】 means normal; any other status code is an error.
DEPLOY_SVC=$(kubectl get svc -n opsany | grep 'opsany-saas-deploy' | awk -F ' ' '{print $3}')
curl --head http://${DEPLOY_SVC}:7018/o/deploy/healthz/

1. Deploy the OpsAny Proxy service

The Proxy service manages operations targets — hosts, switches, routers, databases, and more. Managed hosts must be able to reach ports 8011, 8012, 4505, and 4506 on the Proxy, so it needs special handling. There are two options:

• Option 1: deploy it into the Kubernetes cluster and expose the ports outside the cluster (not yet tested).

• Option 2: deploy it separately on a host inside or outside the Kubernetes cluster (recommended). See: Standalone Proxy deployment

• Deploy the monitoring service

OpsAny supports data collection with Zabbix and Prometheus. You can deploy them into Kubernetes yourself, but we recommend using our scripts to deploy them directly on a host.

#
cd /opt/opsany-paas/install/
./zabbix-install.sh 7.0

Initializing the OpsAny platform

1. Run the initialization script

OpsAny initialization performs many setup tasks for you: - Workbench — imports the ticket workflow sample and ticket samples. - Control Platform — Collection Management — creates the built-in controller (Proxy). - Initializes user information for every SaaS platform. - Imports the built-in jobs of the Job Platform. - Imports the built-in scripts of the Job Platform.

# The admin password is changed when this finishes.
cd /opt/opsany-paas/install/
./saas-ce-k8s-install.sh init

1. Access the platform and complete the basic setup.

When installation finishes, the console prints the access address and password. Open it in Chrome.

• Access address: https://$DOMAIN_NAME
• Username: admin

• Password: for security, it is changed automatically after installation and printed; it is also stored in $INSTALL_PATH/conf/.passwd_env.

• Place the Agent and the built-in documentation

# Download the Agent package into the NFS volume
cd /data/k8s-nfs/opsany-uploads/
wget https://opsany.oss-cn-beijing.aliyuncs.com/opsany-agent-2.2.0.tar.gz
tar zxf opsany-agent-2.2.0.tar.gz

# Download the built-in documentation into the NFS volume
cd /data/k8s-nfs/opsany-uploads/
wget https://opsany.oss-cn-beijing.aliyuncs.com/opsany-docs-2.2.2.tar.gz
tar zxf opsany-docs-2.2.2.tar.gz

1. OpsAny is now ready to use. We strongly recommend starting with the Quick Start, onboarding resources first, then exploring other features.

After signing in, configure the "controller (proxy)" and the "Grafana integration" under 【Control Platform】 → 【Collection Management】 before the platform works properly.

• Quick Start

If you run into problems, check the FAQ first.

• FAQ

If you need basic monitoring, you also need to integrate Zabbix.

• Basic monitoring deployment

If you want DevOps features, continue with this deployment.

• Application Platform deployment

To replace the HTTPS certificate after deployment, follow this.

• Replacing the HTTPS certificate

Offline deployment is possible after the online deployment is complete.

• Offline deployment guide