我的书签
添加书签
移除书签Create a crun demo for KubeEdge
1. Setup Cloud Side (KubeEdge Master Node)
Install Go
1. $ wget https://golang.org/dl/go1.17.3.linux-amd64.tar.gz
2. $ tar xzvf go1.17.3.linux-amd64.tar.gz
4. $ export PATH=/home/${user}/go/bin:$PATH
5. $ go version
6. go version go1.17.3 linux/amd64
Install CRI-O
Please see CRI-O Installation Instructions.
1. # Create the .conf file to load the modules at bootup
2. cat <<EOF | sudo tee /etc/modules-load.d/crio.conf
3. overlay
4. br_netfilter
5. EOF
7. sudo modprobe overlay
8. sudo modprobe br_netfilter
10. # Set up required sysctl params, these persist across reboots.
11. cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
12. net.bridge.bridge-nf-call-iptables = 1
13. net.ipv4.ip_forward = 1
14. net.bridge.bridge-nf-call-ip6tables = 1
15. EOF
17. sudo sysctl --system
18. export OS="xUbuntu_20.04"
19. export VERSION="1.21"
20. cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
21. deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /
22. EOF
23. cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
24. deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /
25. EOF
27. curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/libcontainers.gpg add -
28. curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/libcontainers-cri-o.gpg add -
30. sudo apt-get update
31. sudo apt-get install cri-o cri-o-runc
33. sudo systemctl daemon-reload
34. sudo systemctl enable crio --now
35. sudo systemctl status cri-o
output:
1. $ sudo systemctl status cri-o
2. ● crio.service - Container Runtime Interface for OCI (CRI-O)
3. Loaded: loaded (/lib/systemd/system/crio.service; enabled; vendor preset: enabled)
4. Active: active (running) since Mon 2021-12-06 13:46:29 UTC; 16h ago
5. Docs: https://github.com/cri-o/cri-o
6. Main PID: 6868 (crio)
7. Tasks: 14
8. Memory: 133.2M
9. CGroup: /system.slice/crio.service
10. └─6868 /usr/bin/crio
12. Dec 07 06:04:13 master crio[6868]: time="2021-12-07 06:04:13.694226800Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.4.1" id=1dbb722e-f031-410c-9f45-5d4b5760163e name=/runtime.v1alpha2.ImageServic>
13. Dec 07 06:04:13 master crio[6868]: time="2021-12-07 06:04:13.695739507Z" level=info msg="Image status: &{0xc00047fdc0 map[]}" id=1dbb722e-f031-410c-9f45-5d4b5760163e name=/runtime.v1alpha2.ImageService/ImageSta>
14. Dec 07 06:09:13 master crio[6868]: time="2021-12-07 06:09:13.698823984Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.4.1" id=661b754b-48a4-401b-a03f-7f7a553c7eb6 name=/runtime.v1alpha2.ImageServic>
15. Dec 07 06:09:13 master crio[6868]: time="2021-12-07 06:09:13.703259157Z" level=info msg="Image status: &{0xc0004d98f0 map[]}" id=661b754b-48a4-401b-a03f-7f7a553c7eb6 name=/runtime.v1alpha2.ImageService/ImageSta>
16. Dec 07 06:14:13 master crio[6868]: time="2021-12-07 06:14:13.707778419Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.4.1" id=8c7e4d36-871a-452e-ab55-707053604077 name=/runtime.v1alpha2.ImageServic>
17. Dec 07 06:14:13 master crio[6868]: time="2021-12-07 06:14:13.709379469Z" level=info msg="Image status: &{0xc000035030 map[]}" id=8c7e4d36-871a-452e-ab55-707053604077 name=/runtime.v1alpha2.ImageService/ImageSta>
18. Dec 07 06:19:13 master crio[6868]: time="2021-12-07 06:19:13.713158978Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.4.1" id=827b6315-f145-4f76-b8da-31653d5892a2 name=/runtime.v1alpha2.ImageServic>
19. Dec 07 06:19:13 master crio[6868]: time="2021-12-07 06:19:13.714030148Z" level=info msg="Image status: &{0xc000162bd0 map[]}" id=827b6315-f145-4f76-b8da-31653d5892a2 name=/runtime.v1alpha2.ImageService/ImageSta>
20. Dec 07 06:24:13 master crio[6868]: time="2021-12-07 06:24:13.716746612Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.4.1" id=1d53a917-4d98-4723-9ea8-a2951a472cff name=/runtime.v1alpha2.ImageServic>
21. Dec 07 06:24:13 master crio[6868]: time="2021-12-07 06:24:13.717381882Z" level=info msg="Image status: &{0xc00042ce00 map[]}" id=1d53a917-4d98-4723-9ea8-a2951a472cff name=/runtime.v1alpha2.ImageService/ImageSta>
Install and Creating a cluster with kubeadm for K8s
Please see Creating a cluster with kubeadm.
Install K8s
1. sudo apt-get update
2. sudo apt-get install -y apt-transport-https curl
3. echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
5. sudo apt update
6. K_VER="1.21.0-00"
7. sudo apt install -y kubelet=${K_VER} kubectl=${K_VER} kubeadm=${K_VER}
8. sudo apt-mark hold kubelet kubeadm kubectl
Create a cluster with kubeadm
1. #kubernetes scheduler requires this setting to be done.
2. $ sudo swapoff -a
3. $ sudo vim /etc/fstab
4. mark contain swapfile of row
6. $ cat /etc/cni/net.d/100-crio-bridge.conf
7. {
8. "cniVersion": "0.3.1",
9. "name": "crio",
10. "type": "bridge",
11. "bridge": "cni0",
12. "isGateway": true,
13. "ipMasq": true,
14. "hairpinMode": true,
15. "ipam": {
16. "type": "host-local",
17. "routes": [
18. { "dst": "0.0.0.0/0" },
19. { "dst": "1100:200::1/24" }
20. ],
21. "ranges": [
22. [{ "subnet": "10.85.0.0/16" }],
23. [{ "subnet": "1100:200::/24" }]
24. ]
25. }
26. }
27. $ export CIDR=10.85.0.0/16
28. $ sudo kubeadm init --apiserver-advertise-address=192.168.122.160 --pod-network-cidr=$CIDR --cri-socket=/var/run/crio/crio.sock
30. $ mkdir -p $HOME/.kube
31. $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
32. $ sudo chown $(id -u):$(id -g) $HOME/.kube/config
output:
1. Your Kubernetes control-plane has initialized successfully!
3. To start using your cluster, you need to run the following as a regular user:
5. mkdir -p $HOME/.kube
6. sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
7. sudo chown $(id -u):$(id -g) $HOME/.kube/config
9. You should now deploy a Pod network to the cluster.
10. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
11. /docs/concepts/cluster-administration/addons/
13. You can now join any number of machines by running the following on each node
14. as root:
16. kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>
To make kubectl work for your non-root user, run these commands, which are also part of the kubeadm init output:
1. mkdir -p $HOME/.kube
2. sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
3. sudo chown $(id -u):$(id -g) $HOME/.kube/config
Setup KubeEdge Master Node
Please see Deploying using Keadm.
IMPORTANT NOTE:
- At least one of kubeconfig or master must be configured correctly, so that it can be used to verify the version and other info of the k8s cluster.
- Please make sure edge node can connect cloud node using local IP of cloud node, or you need to specify public IP of cloud node with —advertise-address flag.
- --advertise-address(only work since 1.3 release) is the address exposed by the cloud side (will be added to the SANs of the CloudCore certificate), the default value is the local IP.
1. wget https://github.com/kubeedge/kubeedge/releases/download/v1.8.0/keadm-v1.8.0-linux-amd64.tar.gz
2. tar xzvf keadm-v1.8.0-linux-amd64.tar.gz
3. cd keadm-v1.8.0-linux-amd64/keadm/
4. sudo ./keadm init --advertise-address=192.168.122.160 --kube-config=/home/${user}/.kube/config
output:
1. Kubernetes version verification passed, KubeEdge installation will start...
2. ...
3. KubeEdge cloudcore is running, For logs visit: /var/log/kubeedge/cloudcore.log
2. Setup Edge Side (KubeEdge Worker Node)
You can use the CRI-O install.sh script to install CRI-O and crun on Ubuntu 20.04.
1. wget -qO- https://raw.githubusercontent.com/second-state/wasmedge-containers-examples/main/crio/install.sh | bash
Install Go on Edge Side
1. $ wget https://golang.org/dl/go1.17.3.linux-amd64.tar.gz
2. $ tar xzvf go1.17.3.linux-amd64.tar.gz
4. $ export PATH=/home/${user}/go/bin:$PATH
5. $ go version
6. go version go1.17.3 linux/amd64
Get Token From Cloud Side
Run keadm gettoken in cloud side will return the token, which will be used when joining edge nodes.
1. $ sudo ./keadm gettoken --kube-config=/home/${user}/.kube/config
2. 27a37ef16159f7d3be8fae95d588b79b3adaaf92727b72659eb89758c66ffda2.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1OTAyMTYwNzd9.JBj8LLYWXwbbvHKffJBpPd5CyxqapRQYDIXtFZErgYE
Download Kubeedge and join edge nodes
Please see Setting different container runtime with CRI and Deploying using Keadm.
1. $ wget https://github.com/kubeedge/kubeedge/releases/download/v1.8.0/keadm-v1.8.0-linux-amd64.tar.gz
2. $ tar xzvf keadm-v1.8.0-linux-amd64.tar.gz
3. $ cd keadm-v1.8.0-linux-amd64/keadm/
5. $ sudo ./keadm join \
6. --cloudcore-ipport=192.168.122.160:10000 \
7. --edgenode-name=edge \
8. --token=b4550d45b773c0480446277eed1358dcd8a02a0c214646a8082d775f9c447d81.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Mzg4ODUzNzd9.A9WOYJFrgL2swVGnydpb4gMojyvyoNPCXaA4rXGowqU \
9. --remote-runtime-endpoint=unix:///var/run/crio/crio.sock \
10. --runtimetype=remote \
11. --cgroupdriver=systemd
Output:
1. Host has mosquit+ already installed and running. Hence skipping the installation steps !!!
2. ...
3. KubeEdge edgecore is running, For logs visit: /var/log/kubeedge/edgecore.log
Get Edge Node Status From Cloud Side
Output:
1. kubectl get node
2. NAME STATUS ROLES AGE VERSION
3. edge Ready agent,edge 10s v1.19.3-kubeedge-v1.8.2
4. master Ready control-plane,master 68m v1.21.0
3. Enable kubectl logs Feature
Before metrics-server deployed, kubectl logs feature must be activated, please see here.
4. Run a simple WebAssembly app
We can run the WebAssembly-based image from Docker Hub in the Kubernetes cluster.
Cloud Side
1. $ kubectl run -it --restart=Never wasi-demo --image=hydai/wasm-wasi-example:with-wasm-annotation --annotations="module.wasm.image/variant=compat" /wasi_example_main.wasm 50000000
3. Random number: -1694733782
4. Random bytes: [6, 226, 176, 126, 136, 114, 90, 2, 216, 17, 241, 217, 143, 189, 123, 197, 17, 60, 49, 37, 71, 69, 67, 108, 66, 39, 105, 9, 6, 72, 232, 238, 102, 5, 148, 243, 249, 183, 52, 228, 54, 176, 63, 249, 216, 217, 46, 74, 88, 204, 130, 191, 182, 19, 118, 193, 77, 35, 189, 6, 139, 68, 163, 214, 231, 100, 138, 246, 185, 47, 37, 49, 3, 7, 176, 97, 68, 124, 20, 235, 145, 166, 142, 159, 114, 163, 186, 46, 161, 144, 191, 211, 69, 19, 179, 241, 8, 207, 8, 112, 80, 170, 33, 51, 251, 33, 105, 0, 178, 175, 129, 225, 112, 126, 102, 219, 106, 77, 242, 104, 198, 238, 193, 247, 23, 47, 22, 29]
5. Printed from wasi: This is from a main function
6. This is from a main function
7. The env vars are as follows.
8. The args are as follows.
9. /wasi_example_main.wasm
10. 50000000
11. File content is This is in a file
The WebAssembly app of pod successfully deploy to edge node.
1. $ kubectl describe pod wasi-demo
3. Name: wasi-demo
4. Namespace: default
5. Priority: 0
6. Node: edge/192.168.122.229
7. Start Time: Mon, 06 Dec 2021 15:45:34 +0000
8. Labels: run=wasi-demo
9. Annotations: module.wasm.image/variant: compat
10. Status: Succeeded
11. IP:
12. IPs: <none>
13. Containers:
14. wasi-demo:
15. Container ID: cri-o://1ae4d0d7f671050331a17e9b61b5436bf97ad35ad0358bef043ab820aed81069
16. Image: hydai/wasm-wasi-example:with-wasm-annotation
17. Image ID: docker.io/hydai/wasm-wasi-example@sha256:525aab8d6ae8a317fd3e83cdac14b7883b92321c7bec72a545edf276bb2100d6
18. Port: <none>
19. Host Port: <none>
20. Args:
21. /wasi_example_main.wasm
22. 50000000
23. State: Terminated
24. Reason: Completed
25. Exit Code: 0
26. Started: Mon, 06 Dec 2021 15:45:33 +0000
27. Finished: Mon, 06 Dec 2021 15:45:33 +0000
28. Ready: False
29. Restart Count: 0
30. Environment: <none>
31. Mounts:
32. /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bhszr (ro)
33. Conditions:
34. Type Status
35. Initialized True
36. Ready False
37. PodScheduled True
38. Volumes:
39. kube-api-access-bhszr:
40. Type: Projected (a volume that contains injected data from multiple sources)
41. TokenExpirationSeconds: 3607
42. ConfigMapName: kube-root-ca.crt
43. ConfigMapOptional: <nil>
44. DownwardAPI: true
45. QoS Class: BestEffort
46. Node-Selectors: <none>
47. Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
48. node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
49. Events:
50. Type Reason Age From Message
51. ---- ------ ---- ---- -------
Edge Side
1. $ sudo crictl ps -a
2. CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
3. 1ae4d0d7f6710 0423b8eb71e312b8aaa09a0f0b6976381ff567d5b1e5729bf9b9aa87bff1c9f3 16 minutes ago Exited wasi-demo 0 2bc2ac0c32eda
4. 1e6c7cb6bc731 k8s.gcr.io/kube-proxy@sha256:2a25285ff19f9b4025c8e54dac42bb3cd9aceadc361f2570489b8d723cb77135 18 minutes ago Running kube-proxy 0 8b7e7388ad866
That’s it.
5. Demo Run Screen Recording
