众所周知的原因,在国内访问Jenkins的插件站点不是很稳定,经常访问很慢或者超时。

如果你网上搜索Jenkins更换国内源等关键字,大多数文章都会告诉你做以下三个步骤:

  • 修改"Manage Jenkins"--->"Manage Plugins"--->"Advanced" --->"Update Site" URL为国内源,如https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json
  • /var/jenkins_home/updates/default.json文件内容中https://updates.jenkins.io/download替换为国内源,如https://mirrors.tuna.tsinghua.edu.cn/jenkins
  • 重启Jenkins

第一步有一个坑:https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json下载的是始终是最新的插件,和旧版本的Jenkins是不兼容的,要替换为https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/dynamic-stable-VERSION/update-center.json才能装到匹配的插件版本,其中VERSION是当前Jenkins的版本。如果不是LTS版本,则dynamic-stable-VERSION要换成dynamic-VERSION

第二步修改了之后,过一段时间你想再装插件你会发现/var/jenkins_home/updates/default.json这个文件又被恢复成官方的了。此举并不是一劳永逸的,因为Jenkins会定时更新这个文件。

那么有没有更好的办法呢?答案是有的,下面就教大家如何一劳永逸地将Jenkins插件站点换为你想要的任何镜像站点。

方法示意图:

Jenkins.png

  • 修改hosts,使updates.jenkins.io指向nginx的IP
  • 配置nginx反向代理,指向你所希望访问的Jenkins插件站点

需要解决一个问题,Jenkins源站点是https协议的,Jenkins会校验SSL证书的有效性,因此我这里将使用nginx的sub_filter模块将update-center.json返回的内容修改为http协议的。

- 阅读剩余部分 -

前言

禅道作为常用的项目管理工具,在很多公司都有使用。其官方文档中介绍的部署方式主要是安装LAMP集成环境的方式来部署的,虽然其官方也提供了Docker部署,但是他提供的Docker镜像(easysoft/zentao)是集成了httpd+php+mysql在里面的,不满足容器的最佳实践原则。

在容器时代,禅道所推崇的部署方式显然显得比较的落后了,其官方更没有关于部署到Kubernetes的相关文档。社区论坛是有一些提问的,但是提供的方案大多都是使用 easysoft/zentao 这个仓的镜像。

本来是打算自己制作镜像的,在翻阅资料的时候发现 easysoft/quickon-zentao 这个仓,看了一下是满足我的需求的。

部署架构

zetao.drawio.png

- 阅读剩余部分 -

Rancher创建的Kubernetes集群(RKE),kubernetes版本为1.24.x。在Rancher上安装了Monitoring应用,无法查看Pod监控指标数据。Grafana显示为no data

grafana-pod-nodata.png

在Prometheus界面上通过PromQL查询,发现指标数据缺失container、image、name、namespace、pod等标签,如下:

prom-query.png

- 阅读剩余部分 -

在linux中,解析json一般都会使用jq,jq固然好用,但是要额外安装就不太友好了。如果你在网上搜索通过shell来解析json的方法,你会发现都是些啥玩意儿。既然如此,那就动手撸一个吧。

有以下特性:

  • 她能兼容jq的路径表达式

    • .name
    • .[0].name
    • .[0][0].name
    • ...
  • 纯shell,其实用的是awk。纯bash不好搞,就曲线救一下国呗

代码如下,拿去不谢。转载请标出处。转载请标出处。转载请标出处。

- 阅读剩余部分 -

正在运行的gitlab集群突然抽风,具体故障现象如下。

环境信息:gitlab部署在k8s集群内,采用官方helm包部署,gitlab版本为14.10.x。

故障现象:项目相关的操作有几率报错

  • 新建:创建项目有几率失败,并报错
  • 查看项目:进入项目内,有几率刷不出代码树,并报错,报错信息 An error occurred while fetching folder content.
  • 删除项目:项目删不掉

查看Gitaly服务的日志,发现是Praefect服务调用Gitaly的健康检查接口报错,错误关键信息为 PermissionDenied。

{"correlation_id":"01G7BX3DR59H35EYHPMKHNVBC0","error":"rpc error: code = PermissionDenied desc = permission denied","grpc.code":"PermissionDenied","grpc.meta.auth_version":"v2","grpc.meta.deadline_type":"unknown","grpc.meta.method_type"
:"unary","grpc.method":"Check","grpc.request.deadline":"2022-07-07T08:39:54.207","grpc.request.fullMethod":"/grpc.health.v1.Health/Check","grpc.request.payload_bytes":0,"grpc.response.payload_bytes":0,"grpc.service":"grpc.health.v1.Healt
h","grpc.start_time":"2022-07-07T08:39:53.208","grpc.time_ms":0.285,"level":"warning","msg":"finished unary call with code PermissionDenied","peer.address":"10.42.1.134:51266","pid":12,"span.kind":"server","system":"grpc","time":"2022-07-07T08:39:53.208Z"}
{"correlation_id":"01G7BX3EQJZJFT996MD2KF2HXW","error":"rpc error: code = PermissionDenied desc = permission denied","grpc.code":"PermissionDenied","grpc.meta.auth_version":"v2","grpc.meta.deadline_type":"unknown","grpc.meta.method_type":"unary","grpc.method":"Check","grpc.request.deadline":"2022-07-07T08:39:55.212","grpc.request.fullMethod":"/grpc.health.v1.Health/Check","grpc.request.payload_bytes":0,"grpc.response.payload_bytes":0,"grpc.service":"grpc.health.v1.Health","grpc.start_time":"2022-07-07T08:39:54.213","grpc.time_ms":0.201,"level":"warning","msg":"finished unary call with code PermissionDenied","peer.address":"10.42.1.134:51266","pid":12,"span.kind":"server","system":"grpc","time":"2022-07-07T08:39:54.213Z"}

*** /var/log/gitaly/gitaly_ruby_json.log ***
{"type":"gitaly-ruby","grpc.start_time":"2022-07-07T08:39:53Z","grpc.time_ms":0.286,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":35,"correlation_id":"c486ca8b2bbc3eb6736d533c38cf6017","time":"2022-07-07T08:39:53.642Z"}
{"type":"gitaly-ruby","grpc.start_time":"2022-07-07T08:39:53Z","grpc.time_ms":17.012,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":36,"correlation_id":"71bff8c80424d633989f5daeb29111c3","time":"2022-07-07T08:39:53.658Z"}

*** /var/log/gitaly/gitaly.log ***
{"correlation_id":"01G7BX3FPZ8BTMCA0RY5754JGJ","error":"rpc error: code = PermissionDenied desc = permission denied","grpc.code":"PermissionDenied","grpc.meta.auth_version":"v2","grpc.meta.deadline_type":"unknown","grpc.meta.method_type":"unary","grpc.method":"Check","grpc.request.deadline":"2022-07-07T08:39:56.217","grpc.request.fullMethod":"/grpc.health.v1.Health/Check","grpc.request.payload_bytes":0,"grpc.response.payload_bytes":0,"grpc.service":"grpc.health.v1.Health","grpc.start_time":"2022-07-07T08:39:55.218","grpc.time_ms":0.135,"level":"warning","msg":"finished unary call with code PermissionDenied","peer.address":"10.42.1.134:51266","pid":12,"span.kind":"server","system":"grpc","time":"2022-07-07T08:39:55.218Z"}
{"correlation_id":"01G7BX3GPCRD1RPMN7F3WN6X14","error":"rpc error: code = PermissionDenied desc = permission denied","grpc.code":"PermissionDenied","grpc.meta.auth_version":"v2","grpc.meta.deadline_type":"unknown","grpc.meta.method_type":"unary","grpc.method":"Check","grpc.request.deadline":"2022-07-07T08:39:57.222","grpc.request.fullMethod":"/grpc.health.v1.Health/Check","grpc.request.payload_bytes":0,"grpc.response.payload_bytes":0,"grpc.service":"grpc.health.v1.Health","grpc.start_time":"2022-07-07T08:39:56.223","grpc.time_ms":0.189,"level":"warning","msg":"finished unary call with code PermissionDenied","peer.address":"10.42.1.134:51266","pid":12,"span.kind":"server","system":"grpc","time":"2022-07-07T08:39:56.223Z"}

解决:服务器之间做好时间同步就好了

官方issue: Permission denied between Gitlab and Praefect