今日天气
  1. 博客/

Gitlab pre-receive WebHook 的添加与使用

·2219 字·5 分钟· ·
devops gitlab webhook shell
Johny
作者
Johny
熟练的 云原生搬砖师
Table of Contents

环境说明
#

版本使用说明
#

  • Gitlab version: 13.10.2

    gitlab 服务器安装教程请参考 文档,本文档将示例在 gitlab 中如何添加和配置 pre-receive 钩子,并演示针对 git push 操作做相关的限制操作。

文档参考
#

Pre-receive 钩子概述
#

当我们需要对 git 提交的相关内容做校验时,可以使用服务端的 pre-receive 钩子。相对于客户端的 pre-commit 钩子来说,它便于统一管理维护。pre-commit 需要将处理脚本添加到客户端代码仓库的 .git/hooks 目录下,当 clone 后 需要重新 进行添加,所以通常采用服务端的 pre-receive 来替代。

pre-receive 可以对 push 中的内容做校验,如果校验脚本正常退出(exit 0)即允许push,反之会拒绝客户端进行push操作。

在 gitlab 服务端中除了 pre-receive 钩子外还可以添加其他两种钩子即 pre-receiveupdate 钩子,详细文档说明请 参考

Pre-receive 钩子添加
#

pre-receive 脚本的使用可以是 shell、python、ruby 等任何可执行程序,需要注意它的权限。pre-receive 钩子会从标准输入获取三个参数:旧版本号新版本号分支。同时还提供了内置的许多环境变量可以使用,详情可以参考官方文档

Gitlab 服务端开启 自定义全局钩子选项
#

下面示例中的 gitlab 使用 docker 进行部署,数据通过映射持久卷的形式进行保存。

gitlab 中有 全局 的 webhook 和 针对 单个项目生效的 webhook,针对单个项目生效的 hook 我后面将会有说明。

$ pwd
/application/gitlab

$ ll
total 4
drwxrwxr-x  3 root root  239 Apr  9 16:38 config
drwxr-xr-x 20 root root 4096 Apr 13 10:14 data
drwxr-xr-x 20 root root  326 Apr  9 16:41 logs

vi config/gitlab.rb 
gitlab_shell['custom_hooks_dir'] = "/etc/gitlab/hooks"  # 设置全局钩子文件夹,默认这行在配置文件中是注释的

mkdir -p config/hooks/pre-receive.d/  # 创建文件夹

docker exec -it gitlab-ce bash -c 'ls /etc/gitlab/hooks'  # 查看容器中此文件夹是否存在,并验证映射关系是否正确
pre-receive.d 

docker exec -it gitlab-ce bash -c 'gitlab-ctl reconfigure' # 配置重载生效,不执行此项配置将不会生效,重启容器也没有用。

docker restart gitlab-ce   # 重启一下相关容器

image-20210413110811283

添加全局 commit 字节长度验证
#

此脚本为 shell 脚本,脚本示例如下所示

cat config/hooks/pre-receive.d/commit-length 

#!/bin/bash
#pre-receive script
#set -x #for debugging
LG=18  # 字节长度

validate_ref()
{
    # --- Arguments
    oldrev=$(git rev-parse $1)
    newrev=$(git rev-parse $2)
    refname="$3"

    commitList=`git rev-list $oldrev..$newrev`
        #echo $commitList
    split=($commitList)
    for s in ${split[@]}
    do
        echo "@@@@@@@"
        echo "$s"
        msg=`git cat-file commit $s | sed '1,/^$/d'`
        echo $msg
        if [ ${#msg} -lt "$LG" ];then
            echo "!!! Commit message length less than 18"
            exit 1
        else
            echo "bigger than $LG by commit-length check"
        fi
    done

}

fail=""

# Allow dual mode: run from the command line just like the update hook, or
# if no arguments are given then run as a hook script
if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
    # Output to the terminal in command line mode - if someone wanted to
    # resend an email; they could redirect the output to sendmail
    # themselves
    PAGER= validate_ref $2 $3 $1
else
    while read oldrev newrev refname
    do
        validate_ref $oldrev $newrev $refname
    done
fi

if [ -n "$fail" ]; then
    exit $fail
fi

docker exec -it gitlab-ce bash -c 'chmod 777 -R /etc/gitlab/hooks/'

这里需要注意一下 1 个 中文字符 等于 3个 字节,也就是说最少需要 8 中文字符,当然 空格 也会被计入,如想排除空格,可修改一下脚本对应逻辑即可,这里不多赘述实现方法,熟悉 shell 脚本的人应该不难。

这里的 权限 一定要,进行更改一下,否则将导致 hook 脚本不生效。

验证脚本是否可行

gitlab dashboard 验证

image-20210413112643927

可以看到,已经提交不上去了,只是 dashboard 中无 message 错误输出。

命令行验证

cd /tmp
git clone http://gitlab.treesir.pub/gitlab-instance-7c228ebb/Monitoring.git
touch test.md
git add ./*
git commit -m "aaa"

git push origin master  # 可以看到 命令行是有正常输出的。
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 266 bytes | 266.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: @@@@@@@
remote: d1d9d8d5cad9ece2e82e4a9252efacb271c4fd0f
remote: aaa
remote: !!! Commit message length less than 18
To http://gitlab.treesir.pub/gitlab-instance-7c228ebb/Monitoring.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'http://gitlab.treesir.pub/gitlab-instance-7c228ebb/Monitoring.git'

基于针对单个项目的 hook
#

有些时候我们的限制不想针对于全局,只想针对于某个项目组,或者组中的某些项目。下面示例,创建一个新的仓库,并对这个仓库添加独立的 hook 限制。全局钩子局部钩子 的脚本方式是一样的,只是放置的路径不同,全局即我们上面配置的容器中 /etc/gitlab/hooks 而 局部的 是在 项目绝对路径/custom_hooks/ 下,可能局部的钩子,这样说有点抽象,下面我演示一下就清楚了。

添加局部 hook,强制 git commit 输入 jira issue id
#

此 hook 常见与 gitlab 与 jira 进行集成后,并且 jira issue id 是有规律的,我们可以通过正则进行过滤匹配即可。

shell 脚本如下所示
#

cat custom_hooks/pre-receive.d/require-jira-issue 
#!/bin/bash
#
# Reject pushes that contain commits with messages that do not adhere
# to the defined regex.

# This can be a useful pre-receive hook [1] if you want to ensure every
# commit is associated with a ticket ID.
#
# As an example this hook ensures that the commit message contains a
# JIRA issue formatted as [JIRA-<issue number>].
#
# [1] https://help.github.com/en/enterprise/user/articles/working-with-pre-receive-hooks
#

set -e

zero_commit='0000000000000000000000000000000000000000'
msg0_regex='^DT-[0-9]+ '
msg1_regex='^Merge'

while read -r oldrev newrev refname; do

        # Branch or tag got deleted, ignore the push
    [ "$newrev" = "$zero_commit" ] && continue

    # Calculate range for new branch/updated branch
    [ "$oldrev" = "$zero_commit" ] && range="$newrev" || range="$oldrev..$newrev"

        for commit in $(git rev-list "$range" --not --all); do
                if ! git log --max-count=1 --format=%B $commit | egrep -iq "$msg0_regex|$msg1_regex"; then
                        echo "ERROR:"
                        echo "ERROR: Your push was rejected because the commit"
                        echo "ERROR: $commit in ${refname#refs/heads/}"
                        echo "ERROR: is missing the JIRA Issue. Example: 'DT-123 This is a commit example.'"
                        echo "ERROR:"
                        echo "ERROR: Please fix the commit message and push again."
                        echo "ERROR: https://help.github.com/en/articles/changing-a-commit-message"
                        echo "ERROR"
                        exit 1
                fi
        done

done

找到对应项目在磁盘中的路径
#

从 gitlab 多少版本开始,项目的路径就是已 hash 形式存在磁盘中,目的好像是为了提高磁盘io性能,我看官方文档好像是这样说的。

创建一个新的项目

image-20210413114851496

找到刚才这个项目的 对应 hash 路径

image-20210413114933852

image-20210413115002149
image-20210413115025924

获得 hash 路径即 @hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git

添加钩子

 cd /application/gitlab/data/git-data/repositories/ \
 && cd @hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git  # 刚才找到的路径

image-20210413115213400

创建 hook 文件夹

mkdir -p custom_hooks/pre-receive.d/ 

vim custom_hooks/pre-receive.d/require-jira-issue # 将上面的脚本内容 添加至文件夹中

tree custom_hooks
custom_hooks/
└── pre-receive.d
    └── require-jira-issue

1 directory, 1 file


chmod 777 -R ./custom_hooks  # 授予权限,同理权限未设置将导致,hook无效。

测试效果
#

git clone http://gitlab.treesir.pub/root/test-commit.git

cd test-commit


touch test.md

git add ./*

git commit -m "test"

git push # 可以看到下面输出 脚本已生效。
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 205 bytes | 205.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: ERROR:
remote: ERROR: Your push was rejected because the commit
remote: ERROR: 38bcec39ffd4132c495a22af0ed1485fbe8852f4 in master
remote: ERROR: is missing the JIRA Issue. Example: 'DT-123 This is a commit example.'
remote: ERROR:
remote: ERROR: Please fix the commit message and push again.
remote: ERROR: https://help.github.com/en/articles/changing-a-commit-message
remote: ERROR
To http://gitlab.treesir.pub/root/test-commit.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'http://gitlab.treesir.pub/root/test-commit.git

如果项目一多了话,一个一个的添加还是有些麻烦,可以使用 gitlab 提供的 api 功能 进行一键查询对应的 hash path,并添加对应的脚本,不过需要我们有一定的 脚本编写能力。

其他钩子
#

限制上传文件的类型
#

cat custom_hooks/pre-receive.d/block-files 

#!/usr/bin/env bash

#
# Pre-receive hook that will block any new commits that contain files ending
# with .gz, .zip or .tgz
#
# More details on pre-receive hooks and how to apply them can be found on
# https://help.github.com/enterprise/admin/guides/developer-workflow/managing-pre-receive-hooks-on-the-github-enterprise-appliance/
#

zero_commit="0000000000000000000000000000000000000000"

# Do not traverse over commits that are already in the repository
# (e.g. in a different branch)
# This prevents funny errors if pre-receive hooks got enabled after some
# commits got already in and then somebody tries to create a new branch
# If this is unwanted behavior, just set the variable to empty
excludeExisting="--not --all"

while read oldrev newrev refname; do
  # echo "payload"
  echo $refname $oldrev $newrev

  # branch or tag get deleted
  if [ "$newrev" = "$zero_commit" ]; then
    continue
  fi

  # Check for new branch or tag
  if [ "$oldrev" = "$zero_commit" ]; then
    span=`git rev-list $newrev $excludeExisting`
  else
    span=`git rev-list $oldrev..$newrev $excludeExisting`
  fi

  for COMMIT in $span;
  do
    for FILE  in `git log -1 --name-only --pretty=format:'' $COMMIT`;
    do
      case $FILE in
      *.zip|*.gz|*.tgz|*.exe )
        echo "Hello there! We have restricted committing that filetype(*.zip|*.gz|*.tgz|*.exe). Please see Dave in IT to discuss alternatives."
        exit 1
        ;;
      esac
    done
  done
done
exit 0

此脚本中对 .zip、.gz、.tgz、.exe 类型的文件做了 block 处理。

相关文章

Gitlab 和 jira 之间的集成
·257 字·1 分钟·
devops jira gitlab
使用 Docker 部署 Gitlab,及常用优化项的说明
·1417 字·3 分钟·
docker docker-compose devops gitlab centos7
Centos 7 Yum 安装 Jenkins 及常用配置的说明
·1419 字·3 分钟·
devops jenkins ci-cd install
Jira & confluence 安装 并 配置 破解激活
·1146 字·3 分钟·
devops jira install confluence
在 Kubernetes 中部署 nfs storageClass
·1026 字·3 分钟·
devops k8s storage-class nfs
初探 Traefik ingress gateway
·1594 字·4 分钟·
k8s devops traefik