评论

This is a demo project required by SRE role.
The candidate should be able to complete the project independently in two days and well document the procedure in a practical and well understanding way.
It is not guaranteed that all tasks can be achieved as expected, in which circumstance, the candidate should trouble shoot the issue, conclude based on findings and document which/why/how.

Task 0: Install a ubuntu 18.04 server 64-bit
either in a physical machine or a virtual machine (choose any hypervisor like virtualbox, vmware workstation/fusion, kvm, hyper-v, …. )
Use virtualbox as a quick start: https://www.virtualbox.org/
http://releases.ubuntu.com/18.04/
http://releases.ubuntu.com/18.04/ubuntu-18.04.5-live-server-amd64.iso

https://segmentfault.com/a/1190000022468063

https://nsrc.org/workshops/2014/btnog/raw-attachment/wiki/Track2Agenda/ex-virtualbox-portforward-ssh.htm

For virtualbox VM, use network NAT and forward required ports to host machine, like:
22->22222 for ssh
80->28080 for gitlab
8081/8082->28081/28082 for go app
31080/31081->31080/31081 for go app in k8s

Task 1: Update system
ssh to guest machine from host machine ($ ssh user@localhost -p 22222) and update the system to the latest
https://help.ubuntu.com/18.04/serverguide/apt.html
upgrade the kernel to the 18.04 latest
Note: You may need to change the default apt repo to local to avoid network issue, like:
# cat /etc/apt/sources.list
deb http://mirrors.163.com/ubuntu bionic main restricted
deb http://mirrors.163.com/ubuntu bionic-updates main restricted
deb http://mirrors.163.com/ubuntu bionic universe
deb http://mirrors.163.com/ubuntu bionic-updates universe
deb http://mirrors.163.com/ubuntu bionic multiverse
deb http://mirrors.163.com/ubuntu bionic-updates multiverse
deb http://mirrors.163.com/ubuntu bionic-backports main restricted universe multiverse
deb http://mirrors.163.com/ubuntu bionic-security main restricted
deb http://mirrors.163.com/ubuntu bionic-security universe
deb http://mirrors.163.com/ubuntu bionic-security multiverse

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
mv /etc/apt/sources.list /etc/apt/sources.list.bak
echo 'deb http://mirrors.163.com/ubuntu bionic main restricted
deb http://mirrors.163.com/ubuntu bionic-updates main restricted
deb http://mirrors.163.com/ubuntu bionic universe
deb http://mirrors.163.com/ubuntu bionic-updates universe
deb http://mirrors.163.com/ubuntu bionic multiverse
deb http://mirrors.163.com/ubuntu bionic-updates multiverse
deb http://mirrors.163.com/ubuntu bionic-backports main restricted universe multiverse
deb http://mirrors.163.com/ubuntu bionic-security main restricted
deb http://mirrors.163.com/ubuntu bionic-security universe
deb http://mirrors.163.com/ubuntu bionic-security multiverse' > /cat/etc/apt/sources.list

sudo apt-get update

sudo apt-get upgrade

Task 2: install gitlab-ce version in the host
https://about.gitlab.com/install/#ubuntu?version=ce
Expect output: Gitlab is up and running at http://127.0.0.1 (no tls or FQDN required)
Access it from host machine http://127.0.0.1:28080

参考：https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-gitlab-on-ubuntu-18-04

1
2
3
4
5
6
7
sudo apt update
sudo apt install ca-certificates curl openssh-server postfix
curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh

bash /tmp/script.deb.sh
apt install gitlab-ce

Task 3: create a demo group/project in gitlab
named demo/go-web-hello-world (demo is group name, go-web-hello-world is project name).
Use golang to build a hello world web app (listen to 8081 port)
Check-in the code to mainline (need to use git).
https://golang.org/
https://gowebexamples.com/hello-world/
Expect source code at http://127.0.0.1:28080/demo/go-web-hello-world

Task 4: build the app and expose ($ go run) the service to 28081 port
Expect output from host machine:
curl http://127.0.0.1:28081
Go Web Hello World!

Task 5: install docker
https://docs.docker.com/install/linux/docker-ce/ubuntu/
Note: To avoid default docker repo access issue (could not pull images), you can add mirror registry like this:
# cat /etc/docker/daemon.json
{
“registry-mirrors”: [“https://hub-mirror.c.163.com"]
}
# systemctl daemon-reload
# systemctl restart docker

1
2
3
4
5
6
7
apt install  ca-certificates  curl  gnupg  lsb-release
mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

Task 6: run the app in container
build a docker image ($ docker build) for the web app and run that in a container ($ docker run), expose the service to 28082 (-p)
https://docs.docker.com/engine/reference/commandline/build/
Check in the Dockerfile into gitlab
Expect output from host machine:
curl http://127.0.0.1:28082
Go Web Hello World!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "\n Go Web Hello World!\n")
    })

    http.ListenAndServe(":8081", nil)
}

# Create build stage based on buster image
FROM golang:alpine
# Create working directory under /app
WORKDIR "/app"
# Install any required modules
#RUN go mod download
# Copy over Go source code
COPY hello.go /app/hello.go
RUN go build hello.go
# Run the Go build and output binary under hello_go_http
# Make sure to expose the port the HTTP server is using
EXPOSE 8082
# Run the app binary when we run the container
ENTRYPOINT ["./hello"]


docker build -t goweb .
docker run -d -p 8083:8082  --name goweb goweb
docker tag goweb ganshizhong/go-web-hello-world:v0.1
docker push ganshizhong/go-web-hello-world:v0.1
docker pull ganshizhong/go-web-hello-world:v0.1

Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io: Temporary failure in name resolution
Q：删除所有none镜像
docker rmi docker images | grep "<none>" | awk '{print $3}'
参考： https://blog.csdn.net/weixin_42581414/article/details/105954828

Task 7: push image to dockerhub
tag the docker image using your_dockerhub_id/go-web-hello-world:v0.1 and push it to docker hub (https://hub.docker.com/)
Expect output: https://hub.docker.com/repository/docker/your_dockerhub_id/go-web-hello-world

Task 8: document the procedure in a MarkDown file
create a README.md file in the gitlab repo and add the technical procedure above (0-7) in this file

Task 9: install a single node Kubernetes cluster using kubeadm
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
Check in the admin.conf file into the gitlab repo

1
2
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

https://blog.csdn.net/qq_44895681/article/details/107413950

https://blog.csdn.net/H12590400327/article/details/103740602

https://www.jianshu.com/p/8e78e0abddf9

https://blog.csdn.net/weixin_43168190/article/details/107227626

https://www.cnblogs.com/dream397/p/13814166.html

https://www.quwenqing.com/archives/1899.html

1
2
kubeadm init --image-repository=registry.aliyuncs.com/google_containers

Task 10: deploy the hello world container
in the kubernetes above and expose the service to nodePort 31080
Expect output:
curl http://127.0.0.1:31080
Go Web Hello World!
Check in the deployment yaml file or the command line into the gitlab repo

https://galaxyyao.github.io/2019/06/24/%E5%AE%B9%E5%99%A8-8-Kubernetes%E5%AE%9E%E6%88%98-k8s%E6%A0%B8%E5%BF%83%E6%A6%82%E5%BF%B5%E4%B9%8BNode-Pod%E4%B8%8EDeployment/

https://kubernetes.io/zh-cn/docs/reference/kubectl/cheatsheet/

https://kubernetes.io/zh-cn/docs/tasks/debug/debug-application/_print/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
kubectl get nodes # 节点，相当于一台容器
kubectl describe node 'node-name'
kubectl 

kubectl run myweb --image=ganshizhong/go-web-hello-world:v0.1 --port=8082  -o yaml > goweb.ymal


kubectl expose deployment goweb  --port=31080 --target-port=8082


kubectl rollout restart deployment goweb

kubectl edit svc -n kkubernetes-dashboard

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2022-09-16T02:38:43Z"
  labels:
    app: goweb
  name: goweb
  namespace: default
  resourceVersion: "30824"
  uid: a9c5fab5-3c4a-49b7-bb80-cdbb58e72e3d
spec:
  clusterIP: 10.111.191.86
  clusterIPs:
  - 10.111.191.86
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - nodePort: 51828
    port: 31080
    protocol: TCP
    targetPort: 8082
  selector:
    app: goweb
  sessionAffinity: None
  type: NodePort # 将`ClusterIP`替换为 `NodePort` 会自动产生一个nodePort端口，修改即可
status:
  loadBalancer: {}

Task 11: install kubernetes dashboard
and expose the service to nodeport 31081
https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
Expect output: https://127.0.0.1:31081 (asking for token)

Task 12: generate token for dashboard login in task 11
figure out how to generate token to login to the dashboard and publish the procedure to the gitlab.

1
2
3
https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md

kubectl -n kubernetes-dashboard create token admin-user

Task 13: publish your work
push all files/procedures in your local gitlab repo to remote github repo (e.g. https://github.com/your_github_id/go-web-hello-world)
if this is for an interview session, please send it to bo.cui@ericsson.com, no later than two calendar days after the interview.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-go-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-go-app
  template:
    metadata:
      labels:
        app: my-go-app
    spec:
      containers:
        - name: go-app-container
          image: registry.cn-shanghai.aliyuncs.com/your_hubname_xxxx/servertest:1.0.0
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              memory: "512Mi"
              cpu: "500m"
            requests:
              memory: "64Mi"
              cpu: "100m"
          ports:
          - containerPort: 8000
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2022-09-16T02:06:39Z"
  labels:
    run: deplyment
  name: deplyment
  namespace: default
  resourceVersion: "13869"
  uid: b6f661e5-a718-4774-bfcb-5921e5058d04
spec:
  containers:
  - args:
    - myweb
    image: ganshizhong/go-web-hello-world:v0.1
    imagePullPolicy: IfNotPresent
    name: deplyment
    ports:
    - containerPort: 8082
      protocol: TCP
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-dglgf
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: kube-api-access-dglgf
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          expirationSeconds: 3607
          path: token
      - configMap:
          items:
          - key: ca.crt
            path: ca.crt
          name: kube-root-ca.crt
      - downwardAPI:
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
            path: namespace
status:
  phase: Pending
  qosClass: BestEffort

报错

1
2
3
4
5
6
7
8
9
10
11
单节点，preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling

https://www.cnblogs.com/fat-girl-spring/p/14538991.html

root@demo:/home/gsz# kubectl describe node demo |grep Taint
Taints:             node-role.kubernetes.io/control-plane:NoSchedule
root@demo:/home/gsz# kubectl taint node demo node-role.kubernetes.io/control-plane:NoSchedule-
node/demo untainted
root@demo:/home/gsz# kubectl describe node demo |grep Taint
Taints:             <none>

甘士忠Sharing is meaningful.

Long life and long code.