## Getting Started

This document introduces how to create a kubernetes cluster locally using kind and how to deploy the basic skywalking components to the cluster.

### Prerequisites
- [docker](https://docs.docker.com/get-docker/) >= v20.10.6
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) >= v1.21.0
- [kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) >= v0.20.0
- [swctl](https://github.com/apache/skywalking-cli?tab=readme-ov-file#install) >= v0.10.0

### Step1: Create a kubernetes cluster locally using kind

> Note: If you have a kubernetes cluster (> v1.21.10) already, you can skip this step.

Here we create a kubernetes cluster with 1 control-plane node and 1 worker nodes.

```shell
$ cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  image: kindest/node:v1.21.10
- role: worker
  image: kindest/node:v1.21.10
EOF
```

<details>
  <summary>Expected output</summary>

```shell
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.21.10) 🖼
 ✓ Preparing nodes 📦 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
 ✓ Joining worker nodes 🚜
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
```
</details>

Check all pods in the cluster.

```shell
$ kubectl get pods -A
```

<details>
  <summary>Expected output</summary>

```shell
NAMESPACE            NAME                                         READY   STATUS    RESTARTS   AGE
kube-system          coredns-558bd4d5db-h5gxt                     1/1     Running   0          106s
kube-system          coredns-558bd4d5db-lhnvz                     1/1     Running   0          106s
kube-system          etcd-kind-control-plane                      1/1     Running   0          116s
kube-system          kindnet-fxlkm                                1/1     Running   0          106s
kube-system          kindnet-vmcvl                                1/1     Running   0          91s
kube-system          kube-apiserver-kind-control-plane            1/1     Running   0          116s
kube-system          kube-controller-manager-kind-control-plane   1/1     Running   0          116s
kube-system          kube-proxy-nr4f4                             1/1     Running   0          91s
kube-system          kube-proxy-zl4h2                             1/1     Running   0          106s
kube-system          kube-scheduler-kind-control-plane            1/1     Running   0          116s
local-path-storage   local-path-provisioner-74567d47b4-kmtjh      1/1     Running   0          106s
```
</details>

### Step2: Build the operator image

Check into the root directory of SWCK and build the operator image as follows.

```shell
$ cd operator
# Build the operator image
$ make docker-build
```

You will get the operator image `controller:latest` as follows.

```shell
$ docker images         
REPOSITORY     TAG        IMAGE ID       CREATED          SIZE
controller     latest     84da7509092a   22 seconds ago   53.6MB
```

Load the operator image into the kind cluster or push the image to a registry that
your kubernetes cluster can access.

```shell
$ kind load docker-image controller
```
or
```shell
$ docker push $(YOUR_REGISTRY)/controller
```

### Step3: Deploy operator on the kubernetes cluster

Install the CRDs as follows.

```shell
$ make install
```

Check the CRDs are installed successfully.

<details>
  <summary>Expected output</summary>

```shell
kubectl get crd | grep skywalking
banyandbs.operator.skywalking.apache.org                 2023-11-05T03:30:43Z
fetchers.operator.skywalking.apache.org                  2023-11-05T03:30:43Z
javaagents.operator.skywalking.apache.org                2023-11-05T03:30:43Z
oapserverconfigs.operator.skywalking.apache.org          2023-11-05T03:30:43Z
oapserverdynamicconfigs.operator.skywalking.apache.org   2023-11-05T03:30:43Z
oapservers.operator.skywalking.apache.org                2023-11-05T03:30:43Z
satellites.operator.skywalking.apache.org                2023-11-05T03:30:43Z
storages.operator.skywalking.apache.org                  2023-11-05T03:30:43Z
swagents.operator.skywalking.apache.org                  2023-11-05T03:30:43Z
uis.operator.skywalking.apache.org                       2023-11-05T03:30:43Z
```
</details>

Deploy the SWCK operator to the cluster.

```shell
$ make deploy
```

Or deploy the SWCK operator to the cluster with your own image.

```shell
$ make deploy OPERATOR_IMG=$(YOUR_REGISTRY)/controller
```

Get the status of the SWCK operator pod.

```shell
$ kubectl get pod -n skywalking-swck-system
NAME                                                 READY   STATUS    RESTARTS   AGE
skywalking-swck-controller-manager-5f5bbd4fd-9wdw6   2/2     Running   0          34s
```

### Step4: Deploy skywalking componentes on the kubernetes cluster

Create the `skywalking-system` namespace.

```shell
$ kubectl create namespace skywalking-system
```

Deploy the skywalking components to the cluster.

```shell
$ cat <<EOF | kubectl apply -f -
apiVersion: operator.skywalking.apache.org/v1alpha1
kind: OAPServer
metadata:
  name: skywalking-system
  namespace: skywalking-system
spec:
  version: 9.5.0
  instances: 1
  image: apache/skywalking-oap-server:9.5.0
  service:
    template:
      type: ClusterIP
---
apiVersion: operator.skywalking.apache.org/v1alpha1
kind: UI
metadata:
  name: skywalking-system
  namespace: skywalking-system
spec:
  # spec.kind selects the UI flavor. Default is "horizon" (next-gen UI).
  # Use "booster" to deploy the legacy apache/skywalking-ui image.
  kind: booster
  version: 9.5.0
  instances: 1
  image: apache/skywalking-ui:9.5.0
  OAPServerAddress: http://skywalking-system-oap.skywalking-system:12800
  service:
    template:
      type: ClusterIP
    ingress:
      host: demo.ui.skywalking
EOF
```

#### UI flavors (`spec.kind`)

The `UI` resource supports two flavors via `spec.kind`:

| `spec.kind`           | Default image                         | Listens on | OAP wire-up                                    |
|-----------------------|---------------------------------------|------------|------------------------------------------------|
| `horizon` *(default)* | `apache/skywalking-horizon-ui:<ver>`  | `8081`     | YAML config mounted at `/app/horizon.yaml`     |
| `booster`             | `apache/skywalking-ui:<ver>`          | `8080`     | `SW_OAP_ADDRESS` env var                       |

When `spec.kind: horizon`, the operator additionally:

- Creates `ConfigMap/<name>-ui-horizon` with key `horizon.yaml`, populated from `spec.OAPServerAddress` (→ `oap.queryUrl`), `spec.OAPServerAdminAddress` (default `http://<name>-oap.<ns>:17128` → `oap.adminUrl`), and `spec.OAPServerZipkinAddress` (default `<OAPServerAddress>/zipkin` → `oap.zipkinUrl`).
- Mounts that ConfigMap as `/app/horizon.yaml` (read-only) on the Horizon UI container.
- Mounts an `emptyDir` at `/data` for Horizon's writable state (audit log, setup state, alarm state, wire-debug log). State is lost on pod restart.

Set `spec.config` to a raw `horizon.yaml` string to fully override the operator-generated config (e.g. to configure local users, LDAP, RBAC, or session settings). When set, the operator-generated fields above are ignored.

Note: the `OAPServer` Service exposes port `17128` (admin) in addition to `12800`/`11800`/`1234` so Horizon UI can reach runtime-rule, DSL/MQE debug, and inspect endpoints. The admin port must be enabled in your OAP configuration for Horizon's admin features to work.

Minimal horizon example (defaults pick image, admin URL, and Zipkin URL):

```yaml
apiVersion: operator.skywalking.apache.org/v1alpha1
kind: UI
metadata:
  name: skywalking-system
  namespace: skywalking-system
spec:
  kind: horizon              # default — can be omitted
  version: 9.5.0
  instances: 1
  OAPServerAddress: http://skywalking-system-oap.skywalking-system:12800
  service:
    template:
      type: ClusterIP
    ingress:
      host: demo.ui.skywalking
```

Check the status of the skywalking components.

```shell
$ kubectl get pod -n skywalking-system      
NAME                                     READY   STATUS    RESTARTS   AGE
skywalking-system-oap-68bd877f57-fhzdz   1/1     Running   0          6m23s
skywalking-system-ui-6db8579b47-rphtl    1/1     Running   0          6m23s
```

### Step5: Use the java agent injector to inject the java agent into the application pod

Label the namespace where the application pod is located with `swck-injection=enabled`.

```shell
$ kubectl label namespace skywalking-system swck-injection=enabled
```

Create the application pod.

> Note: The application pod must be labeled with `swck-java-agent-injected=true` and the `agent.skywalking.apache.org/collector.backend_service` annotation must be set to the address of the OAP server. For more configurations, please refer to the [guide](./java-agent-injector.md#use-annotations-to-overlay-default-agent-configuration). 

```shell
$ cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  namespace: skywalking-system
spec:
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      labels:
        # enable the java agent injector
        swck-java-agent-injected: "true"
        app: demo
      annotations:
        agent.skywalking.apache.org/collector.backend_service: "skywalking-system-oap.skywalking-system:11800"
    spec:
      containers:
      - name: demo1
        imagePullPolicy: IfNotPresent
        image: ghcr.io/apache/skywalking-swck-spring-demo:v0.0.1
        command: ["java"]
        args: ["-jar","/app.jar"]
        ports:
          - containerPort: 8085
        readinessProbe:
          httpGet:
            path: /hello
            port: 8085
          initialDelaySeconds: 3
          periodSeconds: 3
          failureThreshold: 10
---
apiVersion: v1
kind: Service
metadata:
  name: demo
  namespace: skywalking-system
spec:
  type: ClusterIP
  ports:
  - name: 8085-tcp
    port: 8085
    protocol: TCP
    targetPort: 8085
  selector:
    app: demo
EOF
```

Check the status of the application pod and make
sure the java agent is injected into the application pod.


```shell
$ kubectl get pod -n skywalking-system -l app=demo -ojsonpath='{.items[0].spec.initContainers[0]}'
```

<details>
  <summary>Expected output</summary>

```shell
{"args":["-c","mkdir -p /sky/agent \u0026\u0026 cp -r /skywalking/agent/* /sky/agent"],"command":["sh"],"image":"apache/skywalking-java-agent:8.16.0-java8","imagePullPolicy":"IfNotPresent","name":"inject-skywalking-agent","resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/sky/agent","name":"sky-agent"},{"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount","name":"kube-api-access-4qk26","readOnly":true}]}
```
</details>

Also, you could check the final java agent configurations with the following command.

```shell
$ kubectl get javaagent -n skywalking-system -l app=demo -oyaml
```

<details>
  <summary>Expected output</summary>

```shell
apiVersion: v1
items:
- apiVersion: operator.skywalking.apache.org/v1alpha1
  kind: JavaAgent
  metadata:
    creationTimestamp: "2023-11-19T05:34:03Z"
    generation: 1
    labels:
      app: demo
    name: app-demo-javaagent
    namespace: skywalking-system
    ownerReferences:
    - apiVersion: apps/v1
      blockOwnerDeletion: true
      controller: true
      kind: ReplicaSet
      name: demo-75d8d995cc
      uid: 8cb64abc-9b50-4f67-9304-2e09de476168
    resourceVersion: "21515"
    uid: 6cbafb3d-9f43-4448-95e8-bda1f7c72bc3
  spec:
    agentConfiguration:
      collector.backend_service: skywalking-system-oap.skywalking-system:11800
      optional-plugin: webflux|cloud-gateway-2.1.x
    backendService: skywalking-system-oap.skywalking-system:11800
    podSelector: app=demo
    serviceName: Your_ApplicationName
  status:
    creationTime: "2023-11-19T05:34:03Z"
    expectedInjectiedNum: 1
    lastUpdateTime: "2023-11-19T05:34:46Z"
    realInjectedNum: 1
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""
```
</details>

If you want to check the logs of the java agent, you can run the following command.

```shell
$ kubectl logs -f -n skywalking-system -l app=demo -c inject-skywalking-agent
```


### Step6: Check the application metrics in the skywalking UI

First, port-forward the demo service to your local machine.

```shell
$ kubectl port-forward svc/demo 8085:8085 -n skywalking-system
```

Then, trigger the application to generate some metrics.

```shell
$ for i in {1..10}; do curl http://127.0.0.1:8085/hello && echo ""; done
```

After that, you can port-forward the skywalking UI to your local machine.

```shell
$ kubectl port-forward svc/skywalking-system-ui 8080:80 -n skywalking-system
```

Open the skywalking UI in your browser and navigate to `http://127.0.0.1:8080` to check the application metrics.

<details>
  <summary>Expected output</summary>

![ui](https://skywalking.apache.org/doc-graph/swck/demo-ui.png)
</details>


Also, if you want to expose the external metrics to the kubernetes HPA, you can follow the [guide](./custom-metrics-adapter.md) to deploy the custom metrics adapter and you may get some inspiration from the 
[e2e test](../test/e2e/oap-agent-adapter-hpa/e2e.yaml).