Skip to main content

Deployment readiness (environment)

1. Overview

When the project is developed, we need to deploy, and we need to describe three deployment modes, server deployments, docker deployments, k8s deployment

Before deploying, we have installed relevant intermediaries. Our cicd is based primarily on gitlab and jenkins (three ways will be used), mirror storage using harbor(docker, k8s deployment needs to be used), a k8s cluster environment (k8s deployment).

Middleware for service use (mysql, redis, es etc.) will be deployed in srv-data.com, if you are online using cloud service and if you build it better than k8s.

So we need to configure the following：

Server NameRole
deploy-server.comDeployment of gitlab, jenkins, harbor (prepackaged docker, docker-compose)
srv-data.comDeployment of mysql, redis, es et al., simulate an independent environment, k8s internal connection to this server
nginx-gateway.comGateway, independent from k8s cluster
k8s clusterK8s Cluster

2. Build gitlab

2.1. Build gitlab

Creating folders

$ mkdir gitlab && cd gitlab
$ vim docker-compose.yml

docker-compose.yml

version: "3"

services:
  gitlab:
    image: "twang2218/gitlab-ce-zh"
    container_name: "gitlab"
    restart: always
    hostname: "192.168.1.180" #部署机器的ip,非容器ip(因为是本地不是线上所以用ip，线上的话可以用域名)
    environment:
      TZ: "Asia/Shanghai"
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://192.168.1.180'  #使用这个地址访问gitlab web ui(因为是本地不是线上所以用ip，线上的话可以用域名)
        gitlab_rails['gitlab_shell_ssh_port'] = 2222 #ssh clone代码地址
        unicorn['port'] = 8888 #gitlab一个内部端口
    ports:
      - "80:80" #web 80 端口
      #- '443:443'      #web 443 端口,本次未使用就不开放了
      - "2222:22" #ssh 检出代码 端口
    volumes:
      - ./etc:/etc/gitlab #Gitlab配置文件目录
      - ./data:/var/opt/gitlab #Gitlab数据目录
      - ./logs:/var/log/gitlab #Gitlab日志目录

Run

$  docker-compose up -d

This execution may take a little long! Let's go to the cup to rest again!

2.2 Visit gitlab

Visit http://192.168.1.103 (i.e. http://"docker-compose")

gitlab

Account default is root

2.3 Create project k8scode

gitlab

2.4 Configure ssh public key

Click on the arrow under the avatar position, "Settings"

gitlab
gitlab

Configure your own public key, click "Add key" (your own search will not be generated by the public key, is not detailed here)

2.5 Upload Project

Click on the project, go back to the project just created and upload the k8scode project to this repository ssh:/git@192.168.180:2222/root/k8scode.git is enough to finish our gitlab build.

3、harbor

3.1 Deployment of harbor

Download harbo harbo https://github.com/goharbor/harbor/releases/download/v2.2.0/harbor-offline-installer-v2.2.0.tgz

Enter the harbor folder after downloading

$ cd harbor && cp harbor.yml.tmpl harbor.yml

We open harbor.yml, modify the following

hostname: 192.168.1.180 #修改为本机ip，不能使用localhost、127.0.0.1

http:
  port: 8077 #改一下http端口8077

#https: #暂时将https注释掉，我们先不通过https只铜鼓http
#  port: 443
#  certificate: /your/certificate/path
#  private_key: /your/private/key/path

data_volume: /root/harbor/data #修改一下数据目录位置

log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: /root/harbor/log #修改一下日志目录位置

Run "sudo ./install.sh" directly to wait a little longer.

3.2 Visit harbor

Browser input http://192.168.1.1180:8077

Account: admin

Password: Harbor12345 (recorded in harbor.yml, default is Harbor12345)

gitlab

Login successful

gitlab

This is our halbor build.

4、jenkins

4.1 Deployment of jenkins

Creating folders

$ mkdir jenkins && cd jenkins
$ vim docker-compose.yml

docker-compose.yml

version: "3"
services:
  jenkins:
    image: "jenkins/jenkins:lts"
    container_name: jenkins
    restart: always
    environment:
      - TZ=Asia/Shanghai
    user: root
    ports:
      - "8989:8080"
      - "50000:50000"
    volumes:
      - "./jenkins_home:/var/jenkins_home"
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/usr/bin/docker:/usr/bin/docker"
      - "/root/port.sh:/root/port.sh"

[Note]: /root/port.sh is for subsequent k8s deployment

#!/bin/sh

case $1 in
"identity-api") echo 1001
;;
"identity-rpc") echo 1101
;;
"usercenter-api") echo 1002
;;
"usercenter-rpc") echo 1102
;;
"message-mq") echo 1207
;;
"mqueue-rpc") echo 1106
;;
"order-api") echo 1004
;;
"order-mq") echo 1204
;;
"order-rpc") echo 1104
;;
"payment-api") echo 1005
;;
"payment-rpc") echo 1105
;;
"travel-api") echo 1003
;;
"travel-rpc") echo 1103
esac

Run

$ docker-compose up -d

This time is not slow. Can drink a cup of coffee

4.2 Mount Tools

1）将 goctl 复制到 jenkins 容器中

$ docker cp $GOPATH/bin/goctl jenkins:/usr/local/bin
$ docker exec -it jenkins /bin/sh #进入jenkins 容器
$ goctl -v   #验证成功
goctl version 1.3.0-20220201 linux/amd64

2）将 kubectl 文件复制到 jenkins 容器中

$ curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
$ sudo chmod a+x kubectl
$ docker cp kubectl jenkins:/usr/local/bin
$ docker exec -it jenkins /bin/sh #进入jenkins 容器
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3" .....

3）将 k8s 的配置.kube/config 复制到 jenkins 容器

$ docker cp ~/.kube jenkins:/root/ #前提是家目录下的.kube文件夹中存在k8s的config配置
$ docker exec -it jenkins /bin/sh #进入jenkins 容器
$ kubectl ge ns
default              Active   43m
kube-node-lease      Active   43m
kube-public          Active   43m
kube-system          Active   43m
local-path-storage   Active   43m

These 4 parts above can also be hit directly in the mirror, and I leave it to you yourself.

4.3 Visit to jenkins

http://192.168.1.180:8989

gitlab

The first visit does not panic. Let you wait a little while it is ready to jump to the landing page when it is ready.

The following interface is ready because our directory is mounted. We see your native jenkins_home/secrets/initialAdminPassword password, enter the next step

gitlab

Select "Install Recommended Plugins"

gitlab

Then wait until the plugin is installed

gitlab

4.4 Create User

root

root

gitlab

4.5 Deployment Complete

gitlab

Deployed to this jenkins

4.6 Add credentials

Click on the left menu "Manage Jenkins"

gitlab

Click "Manage Credentials"

gitlab

Click on the triangle after "Global" and then click "Add Credits"

gitlab

Enter the "Add Credits" page, type we choose "SSH Username with private key",Usernameis a gitlab identifier, after which you add pipeline you know this logo is self-defined on behalf of gitlab credentials, rivate Key`, a private key configured in gitlab (previously we have a public key corresponding to gitlab, here is our own private key), our voucher is for jenkins to go to gitlab free of cryptography.

gitlab

Suffice it.

4.7 Add harbor repository configuration

Go to homepage, click on the menu on the left Manage Jenkins -> Configure System

gitlab

Swipe down toGlobal Propertiesentry, add docker private repository information such as graphdockerUsername,dockeruser password,dockerprivate repository address

gitlab

Click to save

4.8 Configure git

EnterManage Jenkins->Global Tool Configuration, find Git entry, fill jenkins in the machine git executable path; if not, download Git plugin in jenkins plugin management and don't need to be taken into custody (graph below)

gitlab

Git Parameter plugin to configure pipline

Tap "System Configuration" -> "Plugin Management"

gitlab

Then click "Optional Plugins", type "Git Parameter" in the search, like the one

gitlab

Finished installing and restarting to complete this jenkin.

5、k8s

The deployment of k8s is not described. Use kubeadm, rancher, kind to install it, or buy cloud container services, all with a k8s cluster.

References

Contents