Forem: majun

Play with CVM: Gitlab installation and construction

majun — Tue, 05 Jul 2022 03:30:12 +0000

This tutorial will explain how to build Gitlab service in Docker mode relying on Tencent Cloud Host (CVM).

0. Introduction to Gitlab

Gitlab (https://gitlab.com) is a complete set of DevOps tools, through which developers can easily manage projects, source code management, continuous integration/deployment, development document maintenance and even security monitoring, and these important tasks Covers the entire development cycle of the software.

Gitlab's code submission mode is fully compatible with Git, and the cooperation mode is similar to Github. Merge Request in Gitlab can be roughly analogous to Github's Pull Request. This makes it easy for developers to get started with almost no concept learning cost.

When do we need Gitlab (or a similar tool)? The answer is that the earlier the project, the better. The core functions of Gitlab are source code submission records and continuous integration, which are like "ledgers" and "rulers" for software projects, and their importance is self-evident. For developers, it is convenient to carry out distributed collaboration, such as branch feature development, etc., thereby improving efficiency, confidence, and sense of achievement; for project managers, it is easy to analyze and track problems, measure progress, and plan target features.

Git provides a complete set of software, which is essentially a Ruby on Rails WebApp (some background components are now implemented in Go). In the early days, its installation was relatively cumbersome, because it had many components, such as servers, databases, message queues, log management, etc. Although it was easy to use, it had a slight threshold. With the popularization of Docker technology, now we can easily install, configure and deploy through its official image. Next, we will build Gitlab service from scratch on Tencent Cloud Server CVM , let's get started~

1. Preparations

Before starting to install Gitlab, we need to prepare some initial environments, mainly including: cloud server + domain name + certificate + Nginx server installation.

Cloud server: Gitlab's official recommendation is not less than 4GB of memory, here we choose S4.MEDIUM4 of Tencent cloud server (standard S4, 2 cores 4GB).

Operating system: CentOS-7.5 is selected here, other systems such as Ubuntu are also no problem at all, and the operation is similar.

Domain name: It can be purchased from any domain name provider (such as Tencent Cloud Domain Name ). The following is an example of master-cvm.yangyang.cloud.

Then we have the web server and SSL certificate installed, which can be easily verified through the Nginx test page similar to the following.

For SSL certificate application and advanced Nginx installation, please refer to the previous article: Play with CVM: Web Service Construction .

2. Docker installation

As a platform for container management, Docker has been widely used in service deployment and other fields. Container is a lightweight virtualization solution, relying on OS-level virtualization technologies such as overlayfs, namespace under Linux, and cgroups, and its performance is more prominent than VM-based virtualization. But more importantly, it is more convenient to install and configure software through Docker, so we need to simply install Docker first.

Here we take the example of installing the latest docker-ce version. If the ee version is installed, it is similar. Refer to the official documentation here: https://docs.docker.com/install/linux/docker-ce/centos/

Install dependencies and set up repository sources

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo

Install docker-ce

In our environment, it takes about 1 minute, and Tencent Cloud's network generally feels pretty good.

# The latest stable version
yum install docker-ce docker-ce-cli containerd.io
# Or the specified version, such as:
# yum install docker-ce-18.09.6 docker-ce-cli-18.09.6 containerd.io

The version installed in this article is: docker-ce-18.09.6 (the latest).

start the docker service

systemctl start docker

Docker service verification

Start the hello-world container for simple verification.

docker run hello-world docker run hello-world docker run hello-world docker run hello-world

Indeed, the installation of Docker is becoming more and more convenient.

3. Install Gitlab

Official documentation: https://docs.gitlab.com/omnibus/docker/

Download the docker image of gitlab to the local

It takes about 2.5 minutes on Tencent Cloud.

# The latest stable version
docker pull gitlab/gitlab-ce
# or the specified version, such as:
docker pull gitlab/gitlab-ce:11.11.2-ce.0

The version installed in this article is: gitlab-ce:11.11.2-ce.0 (the latest).

Start the gitlab container

docker run --detach \
  --hostname master-cvm.yangyang.cloud \
  --env GITLAB_OMNIBUS_CONFIG="external_url 'https://master-cvm.yangyang.cloud/gitlab'; gitlab_rails['gitlab_shell_ssh_port'] = 55522;" \
  --publish 127.0.0.1:55523:443 --publish 55522:22 \
  --name gitlab    \
  --restart always \
  --volume /data/gitlab/config:/etc/gitlab   \
  --volume /data/gitlab/logs:/var/log/gitlab \
  --volume /data/gitlab/data:/var/opt/gitlab \
  gitlab/gitlab-ce:latest

The meaning of each parameter:

detach: let the container run in the background;
always: always restart after failure;
env: pass environment variables, pay special attention here, the external_url value passed should be consistent with the domain name and routing under Nginx;
publish: port forwarding –pubish 55522:22 will forward port 22 of the container to port 55522 of the cloud server;
volume: The volume mount directory of the custom container, Gitlab needs three: configuration config, log logs and data data;

After the container is started, we can docker ps view the container status through the command, as shown in the figure:

It can be seen that our gitlab container is running normally.

In addition, you can also use docker logs to view the startup log of the container to troubleshoot potential errors.

docker logs -f --tail 50 gitlab

Nginx settings

Note that the configuration of the path (location) and port number (proxy_pass) should match the docker run command above.

server {
    listen 80;
    server_name master-cvm.yangyang.cloud;

    return 301 https://$host$request_uri;
}

server {
    listen 443 http2 ssl;
    listen [::]:443 http2 ssl;

    server_name master-cvm.yangyang.cloud;

    ssl on;
    ssl_certificate /etc/letsencrypt/live/master-cvm.yangyang.cloud/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/master-cvm.yangyang.cloud/privkey.pem;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
    ssl_prefer_server_ciphers on;

    location /gitlab/ {
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_pass https://127.0.0.1:55523;
    }

    location / {
        root    /usr/share/nginx/html;
    }
}

reboot

systemctl restart nginx.service

Try accessing it from a browser~ For example: https://master-cvm.yangyang.cloud/gitlab/

At this time, the interface to reset the root password will appear, and the installation is successful.

4. Initialization and installation verification

After Gitlab is installed, we can use it on the web side. The first is to set the root password. Note that only the root user can access the administrator panel and view many global configurations such as users and projects.

create user

After setting the root password, you will be redirected to the login page. Although you can use root + the password you just set to log in directly, it is more recommended to register the first user and use this user for daily operations.

Upload the key of the user's local development machine

set sshkey

Consistent with Github, users need to upload the user's ssh key to push code through the ssh protocol. Execute the following command on development and copy it:

ssh-keygen
cat ~/.ssh/id_rsa.pub

User avatar in the upper right corner -> "Settings" -> "SSH Keys" in the left column to manage and add keys.

5. Common operations

Create projects, create users and group management

Create a project group (optional)

In Gitlab, the namespace that manages each project (Project) is a group (Group) and its nested subgroup (Subgroup), which can be simply understood as different "directories" where the project is located.

Gitlab creates a default project group with the same name for each user, so projects can be created directly.

Create project

Here is the familiar Github-like interface:

Because the key has been uploaded before, we can clone the project locally through SSH

git clone ssh://git@master-cvm.yangyang.cloud:55522/example-group/my-webapp.git

Note that the url here will contain the ssh port set by my door

It is also supported to select the https protocol when cloning the code, but it is not as convenient to use ssh to enter the user name and password.

6. CI and CI-Runner installation

Gitlab's support for CI (Continuous Integration/Deployment) is very mature. Personally think it is easier to understand and use than jenkins. The so-called continuous integration is to do pre-set tasks (such as code compilation and packaging, automated testing, release, etc.) on a specific machine at a specific point in time (such as code Merge). The program that executes the task is called Gitlab Runner, which executes the component of the task, and it does not necessarily run on the host where gitlab is located, as long as it can communicate with the network.

For the installation process, refer to the official documentation here: https://docs.gitlab.com/runner/install/linux-repository.html

The installation of Gitlab's CI-Runner is very simple and straightforward, and subsequent use (such as configuration tasks) can be started in just three steps.

Update repository source

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh | bash

Install

yum install gitlab-runner

register

Why register? In fact, it is to let gitlab know which machines are running runner. The url and token of gitlab are required.

You can find it in the project's "Settings" -> "CI/CD" -> "Runners" and copy it, as shown in the figure

We can register directly with the gitlab-runner register command.

The type of executor is selected according to the needs, and the shell is more commonly used.

Later, you can set specific CI tasks in .gitlab-ci.yml under the project, and view the execution results of the tasks on the Pipeline page of Gitlab-CI.

7. bonus: git/gitlab based development process

The pictures on Gitlab's official website are quoted here to illustrate the development workflow.

It can be seen that Gitlab classifies a series of work on the branch code before merging into the release trunk as a CI process, including automatic build testing, etc.; and the automated work after merging into the trunk is called a CD process, including deployment to the production environment, etc. .

The problems to be dealt with in actual automated deployment are usually much more complicated than this picture, and we can further experience this evolving process in the process of project development.

At this point, this tutorial has come to an end. I believe that following here, you can already build your own Gitlab service on CVM by yourself, and you are well prepared for the next journey.

Let's enjoy the fun of playing CVM together~
Enjoy Yourself~

This article is from Tencent Cloud Computing Community, please indicate the source for reprinting: https://computeinit.com/archives/2415

How to Build Web Services Relying on Tencent Cloud CVM

majun — Tue, 05 Jul 2022 03:16:46 +0000

This tutorial will explain how to build cutting-edge secure and high-performance web services relying on Tencent Cloud Hosting.

0. Environment preparation

Before we start, we have prepared the following two types of resources.

One Tencent Cloud instance:

Created on demand in the Tencent Cloud CVM product homepage . This article uses a cloud server with the model S4.SMALL2. Note that you need to check the "free allocation of public network IP" when purchasing. The operating system we choose is CentOS 7. Of course, for other systems such as Fedora/Ubuntu, most of the steps in this article are common. All commands below are executed inside this instance.

One Tencent Cloud domain name:

On the Tencent Cloud domain name registration page , select Register. Pick a domain name you like ~ http://my-awesome-domain.com is used below .

1. Set domain name resolution

Adding a domain name resolution record means associating a domain name record with a cloud server on a public cloud.

Tencent Cloud's cloud resolution products can easily manage our domain name resolution work.
The addition of all parsing records can be set in the cloud parsing console .

Record type: select "A", this type of resolution record can be associated with IP and domain name;
Record value: the public network IP of the CVM cloud host;
Host record: the domain name above the third level we need, such as filling in web, that is, the domain name "web" .my-awesome-domain.com" to the specified IP.

Tencent Cloud's resolution takes effect very quickly, so we can log in to CVM through domain name records, such as:

ssh root@web.my-awesome-domain.com

2 Apply for SSL certificate

Let's apply for a Let's Encrypt certificate. It can be easily done through the official Certbot tool.Certbot is essentially a client of the ACME protocol , which is specially used for developers to automatically manage the certificate application process.

Install Certbot

yum install certbot

At the same time, relevant dependent libraries, such as openssl, will be installed. Just use apt install under Debian/Ubuntu.

Certificate Application

certbot certonly --standalone -n -m my-email-address@example.com --agree-tos -d web.my-awesome-domain.com

The application execution process takes about ten seconds, as shown below:

Certificate Application Result

After success, certificate-related files will be generated in the /etc/letsencrypt/live/ http://web.my-awesome-domain.com/ directory: certificate file fullchain.pem and certificate private key file privkey.pem , later in Nginx They will be used in the configuration.

The certificate that is set to automatically renew

The certificate application will expire after 90 days, but Certbot comes with a tool for re-applying (renew) certificate regularly: certbot-renew. We don't have to worry about certificate expiration by starting this scheduled task through the systemctl command.

systemctl start certbot-renew.timer

3 Install Nginx

There are two common ways to install Nginx: through distribution package management tools, or through source code compilation and installation. If the former is used, only:

yum install nginx # Debian/Ubuntu下：apt install nginx

Then skip this section and start the configuration process in the next section.

However, in the current mainstream distributions (such as Centos7/Ubuntu18, etc.), due to the relatively low version of nginx/openssl and other software packages, they will not be able to support features such as TLSv1.3, so please choose between features according to your needs.

Then, let's explain in detail how to install the latest version of Nginx through source code . The latest stable version is 1.16.0. Be careful to install the latest stable version as much as possible, too old versions do not support many features, such as HTTP/2 (supported after 1.10) and TLSv1.3 (supported after 1.15).

The latest version of the software is usually not in the software repository of the distribution's package management tools (such as Yum, APT), but requires us to compile and install the source code. But for us CVM players, this is not a problem at all. Let's experience greater freedom and flexibility with me.

We choose to complete the Nginx installation in the /opt directory, which is usually a suitable choice. Of course, whatever you are used to working with directory is fine.

cd /opt

Installation related dependencies

Here are mainly compilers, PCRE packages and zlib packages

yum install gcc pcre-devel zlib-devel

(It needs to be done with apt install under Debian/Ubuntu system, and the corresponding package names are libpcre3-dev and zlib1g-dev)

Download the openssl source code

To download the latest version of the openssl library, version 1.1.1b. This is because the TLS protocol and encryption and decryption in Nginx are done by external libraries (such as libssl/libcrypto, etc.), and they are all implemented in the openssl project. The default openssl of the system is relatively old and cannot support the latest HTTP/2 and TLS features.

It only takes two steps: download and unzip. No need to compile and install.

wget https://www.openssl.org/source/openssl-1.1.1b.tar.gz
tar -zxvf openssl-1.1.1b.tar.gz

Source code to compile Nginx

Download, compile and install Nginx, version 1.16.0.

wget http://nginx.org/download/nginx-1.16.0.tar.gz
tar -zxvf nginx-1.16.0.tar.gz
cd nginx-1.16.0

Configure the compilation options, note that here we need to specify the code directory of openssl, and Nginx will compile the parts required for compiling openssl by the way. Its options here focus on enabling the http/2 and ssl modules. For other options, if you want to make changes later, you only need to reconfigure and compile. The source installation is so convenient and willful.

./configure \
  --pid-path=/run/nginx.pid   \
  --with-http_v2_module      \
  --with-http_ssl_module     \
  --with-openssl=/opt/openssl-1.1.1b

Compile and install

make &amp;&amp; make install

Nginx will be installed by default in the /usr/local/nginx directory (also specified by the prefix compile option).

At this point, we have completed the installation of Nginx. actually implement

/usr/local/nginx/sbin/nginx

The Nginx service can be started. But wait, let's get the job done a little more gracefully.

Configure the Nginx service as a systemd system service

Edit the file: /lib/systemd/system/nginx.service , add the following content

[Unit]
Description=The nginx HTTP and reverse proxy server
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/bin/rm -f /run/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
KillSignal=SIGQUIT
TimeoutStopSec=5
KillMode=process
PrivateTmp=true

[Install]
WantedBy=multi-user.target

then execute

systemctl daemon-reload
systemctl enable nginx.service

We can then manage Nginx services through systemctl commands, such as restart, reload, etc.

systemctl restart nginx.service

4. Configure Nginx

Edit the server section in nginx.conf (or similar configuration file), set ssl-related parameters such as certificate/key, and redirect the HTTP service of port 80 to port 443 of HTTPS. details as follows:

server {
    listen       443 ssl http2;
    server_name  web.my-awesome-domain.com;

    ssl_certificate "/etc/letsencrypt/live/web.my-awesome-domain.com/fullchain.pem";
    ssl_certificate_key "/etc/letsencrypt/live/web.my-awesome-domain.com/privkey.pem";
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout  10m;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:!aNULL:!MD5:!RC4:!DHE;
    ssl_prefer_server_ciphers on;

    add_header Strict-Transport-Security "max-age=31536000";

    location / {
        root   html;
        index  index.html;
    }
}

server {
    listen       80;
    server_name  web.my-awesome-domain.com;
    if ($host = web.my-awesome-domain.com) {
        return 301 https://$host$request_uri;
    }
}

Note: We support http2, and for the SSL protocol, we support both the current stable TLSv1.2 and the latest TLSv1.3.

systemctl restart nginx.service

Then restart the service and you're done!

This article is from Tencent Cloud Computing Community, please indicate the source for reprinting: https://computeinit.com/archives/2405