mirrors/diun
1
0
Fork 0
mirror of https://github.com/crazy-max/diun.git synced 2025-12-21 13:23:09 +01:00
Code Issues Packages Projects Releases Wiki Activity

Allow overriding os and architecture when watching

Browse Source 
Move insecure_tls and timeout options to registry option
Rename Bolt bucket
Change default schedule
Review registry client
This commit is contained in:
CrazyMax
2019-06-08 02:50:46 +02:00
parent c88e352dd1
commit 5942e39b83
25 changed files with 335 additions and 299 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.res/compose/docker-compose.yml
View File

@@ -11,5 +11,5 @@ services:
- "TZ=Europe/Paris" - "TZ=Europe/Paris"
- "LOG_LEVEL=info" - "LOG_LEVEL=info"
- "LOG_JSON=false" - "LOG_JSON=false"
- "RUN_ONCE=false" - "RUN_STARTUP=false"
restart: always restart: always

8
CHANGELOG.md
View File

@@ -1,5 +1,13 @@
# Changelog # Changelog
## 0.3.0 (2019/06/08)
* Allow overriding os and architecture when watching
* Move `insecure_tls` and `timeout` options to registry option
* Rename Bolt bucket
* Change default schedule
* Review registry client
## 0.2.0 (2019/06/05) ## 0.2.0 (2019/06/05)
* Don't skip repo analysis if default tag not found * Don't skip repo analysis if default tag not found

64
README.md
View File

@@ -22,6 +22,7 @@
* Allow to watch a full Docker repository and report new tags * Allow to watch a full Docker repository and report new tags
* Include and exclude filters with regular expression for tags * Include and exclude filters with regular expression for tags
* Internal cron implementation through go routines * Internal cron implementation through go routines
* Allow overriding os and architecture when watching
* Beautiful email report * Beautiful email report
* Webhook notification * Webhook notification
* Enhanced logging * Enhanced logging
@@ -54,7 +55,7 @@ Flags:
--timezone="UTC" Timezone assigned to Diun. --timezone="UTC" Timezone assigned to Diun.
--log-level="info" Set log level. --log-level="info" Set log level.
--log-json Enable JSON logging output. --log-json Enable JSON logging output.
--run-once Run once on startup. --run-startup Run on startup.
--docker Enable Docker mode. --docker Enable Docker mode.
--version Show application version. --version Show application version.
``` ```
@@ -69,7 +70,7 @@ Flags:
* `--timezone <timezone>` : Timezone assigned to Diun. _Optional_. (default: `UTC`). * `--timezone <timezone>` : Timezone assigned to Diun. _Optional_. (default: `UTC`).
* `--log-level <level>` : Log level output. _Optional_. (default: `info`). * `--log-level <level>` : Log level output. _Optional_. (default: `info`).
* `--log-json` : Enable JSON logging output. _Optional_. (default: `false`). * `--log-json` : Enable JSON logging output. _Optional_. (default: `false`).
* `--run-once` : Run once on startup. _Optional_. (default: `false`). * `--run-startup` : Run on startup. _Optional_. (default: `false`).
## Configuration ## Configuration
@@ -80,7 +81,9 @@ db:
path: diun.db path: diun.db
watch: watch:
schedule: 0 */30 * * * * schedule: 0 0 * * * *
os: linux
arch: amd64
notif: notif:
mail: mail:
@@ -102,21 +105,23 @@ notif:
Authorization: Token123456 Authorization: Token123456
timeout: 10 timeout: 10
reg_creds: registries:
aregistrycred: someregistryoptions:
username: foo username: foo
password: bar password: bar
another: timeout: 20
onemore:
username: foo2 username: foo2
password: bar2 password: bar2
insecure_tls: true
items: items:
- -
image: docker.io/crazymax/nextcloud:latest image: docker.io/crazymax/nextcloud:latest
reg_cred_id: aregistrycred registry_id: someregistryoptions
- -
image: jfrog-docker-reg2.bintray.io/jfrog/artifactory-oss:4.0.0 image: jfrog-docker-reg2.bintray.io/jfrog/artifactory-oss:4.0.0
reg_cred_id: another registry_id: onemore
- -
image: quay.io/coreos/hyperkube image: quay.io/coreos/hyperkube
- -
@@ -129,7 +134,9 @@ items:
* `db` * `db`
* `path`: Path to Bolt database file where images analysis are stored. Flag `--docker` force this path to `/data/diun.db` (default: `diun.db`). * `path`: Path to Bolt database file where images analysis are stored. Flag `--docker` force this path to `/data/diun.db` (default: `diun.db`).
* `watch` * `watch`
* `schedule`: [CRON expression](https://godoc.org/github.com/crazy-max/cron#hdr-CRON_Expression_Format) to schedule Diun watcher. _Optional_. (default: `0 */30 * * * *`). * `schedule`: [CRON expression](https://godoc.org/github.com/crazy-max/cron#hdr-CRON_Expression_Format) to schedule Diun watcher. _Optional_. (default: `0 0 * * * *`).
* `os`: OS to use for choosing images. _Optional_. (default: `linux`).
* `arch`: Architecture to use for choosing images. _Optional_. (default: `amd64`).
* `notif` * `notif`
* `mail` * `mail`
* `enable`: Enable email reports (default: `false`). * `enable`: Enable email reports (default: `false`).
@@ -147,17 +154,17 @@ items:
* `method`: HTTP method (default: `GET`). **required** * `method`: HTTP method (default: `GET`). **required**
* `headers`: Map of additional headers to be sent. * `headers`: Map of additional headers to be sent.
* `timeout`: Timeout specifies a time limit for the request to be made. (default: `10`). * `timeout`: Timeout specifies a time limit for the request to be made. (default: `10`).
* `reg_creds`: Map of registry credentials to use with items. Key is the ID and value is a struct with the following fields: * `registries`: Map of registry options to use with items. Key is the ID and value is a struct with the following fields:
* `username`: Registry username. * `username`: Registry username.
* `password`: Registry password. * `password`: Registry password.
* `timeout`: Timeout is the maximum amount of time for the TCP connection to establish. 0 means no timeout (default: `10`).
* `insecure_tls`: Allow contacting docker registry over HTTP, or HTTPS with failed TLS verification (default: `false`).
* `items`: Slice of items to watch with the following fields: * `items`: Slice of items to watch with the following fields:
* `image`: Docker image to watch using `registry/path:tag` format. If registry is omitted, `docker.io` will be used. If tag is omitted, `latest` will be used. **required** * `image`: Docker image to watch using `registry/path:tag` format. If registry is omitted, `docker.io` will be used. If tag is omitted, `latest` will be used. **required**
* `reg_cred_id`: Registry credential ID from `reg_creds` to use. * `registry_id`: Registry ID from `registries` to use.
* `insecure_tls`: Allow contacting docker registries over HTTP, or HTTPS with failed TLS verification (default: `false`).
* `watch_repo`: Watch all tags of this `image` repository (default: `false`). * `watch_repo`: Watch all tags of this `image` repository (default: `false`).
* `include_tags`: List of regular expressions to include tags. Can be useful if you use `watch_repo`. * `include_tags`: List of regular expressions to include tags. Can be useful if you enable `watch_repo`.
* `exclude_tags`: List of regular expressions to exclude tags. Can be useful if you use `watch_repo`. * `exclude_tags`: List of regular expressions to exclude tags. Can be useful if you enable `watch_repo`.
* `timeout`: Timeout is the maximum amount of time for the TCP connection to establish. 0 means no timeout (default: `10`).
## Docker ## Docker
@@ -168,7 +175,7 @@ Environment variables can be used within your container :
* `TZ` : Timezone assigned * `TZ` : Timezone assigned
* `LOG_LEVEL` : Log level output (default `info`) * `LOG_LEVEL` : Log level output (default `info`)
* `LOG_JSON`: Enable JSON logging output (default `false`) * `LOG_JSON`: Enable JSON logging output (default `false`)
* `RUN_ONCE`: Run once on startup (default `false`) * `RUN_STARTUP`: Run on startup (default `false`)
Docker compose is the recommended way to run this image. Copy the content of folder [.res/compose](.res/compose) in `/opt/diun/` on your host for example. Edit the compose and config file with your preferences and run the following commands : Docker compose is the recommended way to run this image. Copy the content of folder [.res/compose](.res/compose) in `/opt/diun/` on your host for example. Edit the compose and config file with your preferences and run the following commands :
@@ -184,20 +191,39 @@ $ docker run -d --name diun \
-e "TZ=Europe/Paris" \ -e "TZ=Europe/Paris" \
-e "LOG_LEVEL=info" \ -e "LOG_LEVEL=info" \
-e "LOG_JSON=false" \ -e "LOG_JSON=false" \
-e "RUN_ONCE=false" \ -e "RUN_STARTUP=false" \
-v "$(pwd)/data:/data" \ -v "$(pwd)/data:/data" \
-v "$(pwd)/diun.yml:/diun.yml:ro" \ -v "$(pwd)/diun.yml:/diun.yml:ro" \
crazymax/diun:latest crazymax/diun:latest
``` ```
## Mail notification sample ## Notifications
If you choose `webhook` notification, a HTTP request is sent with a JSON format response that looks like:
```json
{
"diun_version": "0.3.0",
"status": "new",
"image": "docker.io/crazymax/swarm-cronjob:0.2.1",
"mime_type": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:5913d4b5e8dc15430c2f47f40e43ab2ca7f2b8df5eee5db4d5c42311e08dfb79",
"created": "2019-01-24T10:26:49.152006005Z",
"architecture": "amd64",
"os": "linux"
}
```
And here is an email sample if you add `mail` notification:
![](.res/notif-mail.png) ![](.res/notif-mail.png)
## TODO ## TODO
* [ ] Scan Dockerfile * [ ] Watch images inside Dockerfile and Compose files
* [ ] Watch images from Docker daemon * [ ] Watch images from Docker daemon
* [ ] Watch starred repo on DockerHub and Quay
* [ ] Fetch image size
## How can I help ? ## How can I help ?

6
cmd/main.go
View File

@@ -31,7 +31,7 @@ func main() {
kingpin.Flag("timezone", "Timezone assigned to Diun.").Envar("TZ").Default("UTC").StringVar(&flags.Timezone) kingpin.Flag("timezone", "Timezone assigned to Diun.").Envar("TZ").Default("UTC").StringVar(&flags.Timezone)
kingpin.Flag("log-level", "Set log level.").Envar("LOG_LEVEL").Default("info").StringVar(&flags.LogLevel) kingpin.Flag("log-level", "Set log level.").Envar("LOG_LEVEL").Default("info").StringVar(&flags.LogLevel)
kingpin.Flag("log-json", "Enable JSON logging output.").Envar("LOG_JSON").Default("false").BoolVar(&flags.LogJson) kingpin.Flag("log-json", "Enable JSON logging output.").Envar("LOG_JSON").Default("false").BoolVar(&flags.LogJson)
kingpin.Flag("run-once", "Run once on startup.").Envar("RUN_ONCE").Default("false").BoolVar(&flags.RunOnce) kingpin.Flag("run-startup", "Run on startup.").Envar("RUN_STARTUP").Default("false").BoolVar(&flags.RunStartup)
kingpin.Flag("docker", "Enable Docker mode.").Envar("DOCKER").Default("false").BoolVar(&flags.Docker) kingpin.Flag("docker", "Enable Docker mode.").Envar("DOCKER").Default("false").BoolVar(&flags.Docker)
kingpin.UsageTemplate(kingpin.CompactUsageTemplate).Version(version).Author("CrazyMax") kingpin.UsageTemplate(kingpin.CompactUsageTemplate).Version(version).Author("CrazyMax")
kingpin.CommandLine.Name = "diun" kingpin.CommandLine.Name = "diun"
@@ -77,8 +77,8 @@ func main() {
log.Fatal().Err(err).Msg("Cannot initialize Diun") log.Fatal().Err(err).Msg("Cannot initialize Diun")
} }
// Run once // Run on startup
if flags.RunOnce { if flags.RunStartup {
diun.Run() diun.Run()
} }

6
go.mod
View File

@@ -5,7 +5,7 @@ require (
github.com/Microsoft/go-winio v0.4.12 // indirect github.com/Microsoft/go-winio v0.4.12 // indirect
github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc // indirect github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc // indirect
github.com/containers/image v1.5.1 github.com/containers/image v1.5.1
github.com/containers/storage v1.12.8 // indirect github.com/containers/storage v1.12.9 // indirect
github.com/crazy-max/cron v1.2.2 github.com/crazy-max/cron v1.2.2
github.com/docker/distribution v2.7.1+incompatible // indirect github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v1.13.1 // indirect github.com/docker/docker v1.13.1 // indirect
@@ -24,12 +24,10 @@ require (
github.com/opencontainers/go-digest v1.0.0-rc1 github.com/opencontainers/go-digest v1.0.0-rc1
github.com/opencontainers/image-spec v1.0.1 // indirect github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/opencontainers/runc v0.1.1 // indirect github.com/opencontainers/runc v0.1.1 // indirect
github.com/prometheus/client_golang v0.9.3 // indirect github.com/prometheus/client_golang v0.9.4 // indirect
github.com/rs/zerolog v1.14.3 github.com/rs/zerolog v1.14.3
github.com/sirupsen/logrus v1.4.2 // indirect github.com/sirupsen/logrus v1.4.2 // indirect
github.com/stretchr/testify v1.3.0 // indirect
go.etcd.io/bbolt v1.3.2 go.etcd.io/bbolt v1.3.2
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3 // indirect
gopkg.in/alecthomas/kingpin.v2 v2.2.6 gopkg.in/alecthomas/kingpin.v2 v2.2.6
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect
gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df // indirect gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df // indirect

31
go.sum
View File

@@ -6,7 +6,6 @@ github.com/Masterminds/sprig v2.16.0+incompatible h1:QZbMUPxRQ50EKAq3LFMnxddMu88
github.com/Masterminds/sprig v2.16.0+incompatible/go.mod h1:y6hNFY5UBTIWBxnzTeuNhlNS5hqE0NB0E6fgfo2Br3o= github.com/Masterminds/sprig v2.16.0+incompatible/go.mod h1:y6hNFY5UBTIWBxnzTeuNhlNS5hqE0NB0E6fgfo2Br3o=
github.com/Microsoft/go-winio v0.4.12 h1:xAfWHN1IrQ0NJ9TBC0KBZoqLjzDTr1ML+4MywiUOryc= github.com/Microsoft/go-winio v0.4.12 h1:xAfWHN1IrQ0NJ9TBC0KBZoqLjzDTr1ML+4MywiUOryc=
github.com/Microsoft/go-winio v0.4.12/go.mod h1:VhR8bwka0BXejwEJY73c50VrPtXAaKcyvVC4A4RozmA= github.com/Microsoft/go-winio v0.4.12/go.mod h1:VhR8bwka0BXejwEJY73c50VrPtXAaKcyvVC4A4RozmA=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc h1:cAKDfWh5VpdgMhJosfJnn5/FoN2SRZ4p7fJNX58YPaU= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc h1:cAKDfWh5VpdgMhJosfJnn5/FoN2SRZ4p7fJNX58YPaU=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf h1:qet1QNfXsQxTZqLG4oE62mJzwPIB8+Tee4RNCL9ulrY= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf h1:qet1QNfXsQxTZqLG4oE62mJzwPIB8+Tee4RNCL9ulrY=
@@ -16,20 +15,18 @@ github.com/aokoli/goutils v1.0.1/go.mod h1:SijmP0QR8LtwsmDs8Yii5Z/S4trXFGFC2oO5g
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0 h1:HWo1m869IqiPhD389kmkxeTalrjNbbJTC8LXupb+sl0= github.com/beorn7/perks v1.0.0 h1:HWo1m869IqiPhD389kmkxeTalrjNbbJTC8LXupb+sl0=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc h1:TP+534wVlf61smEIq1nwLLAjQVEK2EADoW3CX9AuT+8= github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc h1:TP+534wVlf61smEIq1nwLLAjQVEK2EADoW3CX9AuT+8=
github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y= github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
github.com/containers/image v1.5.1 h1:ssEuj1c24uJvdMkUa2IrawuEFZBP12p6WzrjNBTQxE0= github.com/containers/image v1.5.1 h1:ssEuj1c24uJvdMkUa2IrawuEFZBP12p6WzrjNBTQxE0=
github.com/containers/image v1.5.1/go.mod h1:8Vtij257IWSanUQKe1tAeNOm2sRVkSqQTVQ1IlwI3+M= github.com/containers/image v1.5.1/go.mod h1:8Vtij257IWSanUQKe1tAeNOm2sRVkSqQTVQ1IlwI3+M=
github.com/containers/storage v1.12.8 h1:5js4CV+oEW0E9pOA/cJcMsY2V7xLlMhIqFZmQlVpTuo= github.com/containers/storage v1.12.9 h1:kYUE0EBpYv9zwW+MEgXBDsoY5FzmHE5PNKXhWbAkLKg=
github.com/containers/storage v1.12.8/go.mod h1:+RirK6VQAqskQlaTBrOG6ulDvn4si2QjFE1NZCn06MM= github.com/containers/storage v1.12.9/go.mod h1:+RirK6VQAqskQlaTBrOG6ulDvn4si2QjFE1NZCn06MM=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/crazy-max/cron v1.2.2 h1:DQB06Nbb9Lah7UrFaRthTzJABto+qRThKcIZLlTOczA= github.com/crazy-max/cron v1.2.2 h1:DQB06Nbb9Lah7UrFaRthTzJABto+qRThKcIZLlTOczA=
github.com/crazy-max/cron v1.2.2/go.mod h1:1VehsRAaLIq0DQZP1LSRFzKx2+ar2fCpysK7qoqQt1M= github.com/crazy-max/cron v1.2.2/go.mod h1:1VehsRAaLIq0DQZP1LSRFzKx2+ar2fCpysK7qoqQt1M=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug= github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker-credential-helpers v0.6.2 h1:CrW9H1VMf3a4GrtyAi7IUJjkJVpwBBpX0+mvkvYJaus= github.com/docker/docker-credential-helpers v0.6.2 h1:CrW9H1VMf3a4GrtyAi7IUJjkJVpwBBpX0+mvkvYJaus=
@@ -50,7 +47,6 @@ github.com/go-gomail/gomail v0.0.0-20160411212932-81ebce5c23df h1:Bao6dhmbTA1KFV
github.com/go-gomail/gomail v0.0.0-20160411212932-81ebce5c23df/go.mod h1:GJr+FCSXshIwgHBtLglIg9M2l2kQSi6QjVAngtzI08Y= github.com/go-gomail/gomail v0.0.0-20160411212932-81ebce5c23df/go.mod h1:GJr+FCSXshIwgHBtLglIg9M2l2kQSi6QjVAngtzI08Y=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -71,6 +67,7 @@ github.com/imdario/mergo v0.3.7 h1:Y+UAYTZ7gDEuOfhxKWy+dvb5dRQ6rJjFSdX2HZY1/gI=
github.com/imdario/mergo v0.3.7/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/imdario/mergo v0.3.7/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/jaytaylor/html2text v0.0.0-20180606194806-57d518f124b0 h1:xqgexXAGQgY3HAjNPSaCqn5Aahbo5TKsmhp8VRfr1iQ= github.com/jaytaylor/html2text v0.0.0-20180606194806-57d518f124b0 h1:xqgexXAGQgY3HAjNPSaCqn5Aahbo5TKsmhp8VRfr1iQ=
github.com/jaytaylor/html2text v0.0.0-20180606194806-57d518f124b0/go.mod h1:CVKlgaMiht+LXvHG173ujK6JUhZXKb2u/BQtjPDIvyk= github.com/jaytaylor/html2text v0.0.0-20180606194806-57d518f124b0/go.mod h1:CVKlgaMiht+LXvHG173ujK6JUhZXKb2u/BQtjPDIvyk=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/konsorten/go-windows-terminal-sequences v1.0.1 h1:mweAR1A6xJ3oS2pRaGiHgQ4OO8tzTaLawm8vnODuwDk= github.com/konsorten/go-windows-terminal-sequences v1.0.1 h1:mweAR1A6xJ3oS2pRaGiHgQ4OO8tzTaLawm8vnODuwDk=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
@@ -86,8 +83,9 @@ github.com/mattn/go-runewidth v0.0.3 h1:a+kO+98RDGEfo6asOGMmpodZq4FNtnGP54yps8Bz
github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/olekukonko/tablewriter v0.0.1 h1:b3iUnf1v+ppJiOfNX4yxxqfWKMQPZR5yoh8urCTFX88= github.com/olekukonko/tablewriter v0.0.1 h1:b3iUnf1v+ppJiOfNX4yxxqfWKMQPZR5yoh8urCTFX88=
github.com/olekukonko/tablewriter v0.0.1/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo= github.com/olekukonko/tablewriter v0.0.1/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2iki3E3Ii+WN7gQ= github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2iki3E3Ii+WN7gQ=
@@ -102,18 +100,16 @@ github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3 h1:9iH4JKXLzFbOAdtqv/a+j8aewx2Y8lAjAydhbaScPF8= github.com/prometheus/client_golang v0.9.4 h1:Y8E/JaaPbmFSW2V81Ab/d8yZFYQQGbni1b1jPcG9Y6A=
github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= github.com/prometheus/client_golang v0.9.4/go.mod h1:oCXIBxdI62A4cR6aTRJCgetEjecSIYzOEaeAn4iYEpM=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90 h1:S/YWwWx/RA8rT8tKFRuGUZhuA90OyIBpPCXkcbwU8DE= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90 h1:S/YWwWx/RA8rT8tKFRuGUZhuA90OyIBpPCXkcbwU8DE=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= github.com/prometheus/common v0.4.1 h1:K0MGApIoQvMw27RTdJkPbr3JZ7DNbtxQNyi5STVM6Kw=
github.com/prometheus/common v0.4.0 h1:7etb9YClo3a6HjLzfl6rIQaU+FDfi0VSX39io3aQ+DM= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084 h1:sofwID9zm4tzrgykg80hfFph1mryUeLRsUfoocVVmRY= github.com/prometheus/procfs v0.0.2 h1:6LJUbpNm42llc4HRCuvApCSWB/WfhuNo9K98Q9sNGfs=
github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ= github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ=
github.com/rs/zerolog v1.14.3 h1:4EGfSkR2hJDB0s3oFfrlPqjU1e4WLncergLil3nEKW0= github.com/rs/zerolog v1.14.3 h1:4EGfSkR2hJDB0s3oFfrlPqjU1e4WLncergLil3nEKW0=
github.com/rs/zerolog v1.14.3/go.mod h1:3WXPzbXEEliJ+a6UFE4vhIxV8qR1EML6ngzP9ug4eYg= github.com/rs/zerolog v1.14.3/go.mod h1:3WXPzbXEEliJ+a6UFE4vhIxV8qR1EML6ngzP9ug4eYg=
@@ -124,7 +120,6 @@ github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeV
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4= github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf h1:pvbZ0lM0XWPBqUKqFU8cmavspvIl9nulOYwdy6IFRRo= github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf h1:pvbZ0lM0XWPBqUKqFU8cmavspvIl9nulOYwdy6IFRRo=
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf/go.mod h1:RJID2RhlZKId02nZ62WenDCkgHFerpIOmW0iT7GKmXM= github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf/go.mod h1:RJID2RhlZKId02nZ62WenDCkgHFerpIOmW0iT7GKmXM=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -141,14 +136,12 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2 h1:VklqNMn3ovrHsnt90Pveol
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/net v0.0.0-20181029044818-c44066c5c816/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181029044818-c44066c5c816/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a h1:oWX7TPOiFAMXLq8o0ikBYfCJVlRHBcsciT5bXOrH628=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3 h1:0GoQqolDA55aaLxZyTzK/Y2ePZzZTUrRacwib7cNsYQ=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894 h1:Cz4ceDQGXuKRnVBDTS23GTn/pU5OE2C0WrNTOYK1Uuc= golang.org/x/sys v0.0.0-20190422165155-953cdadca894 h1:Cz4ceDQGXuKRnVBDTS23GTn/pU5OE2C0WrNTOYK1Uuc=

127
internal/app/diun.go
View File

@@ -1,6 +1,7 @@
package app package app
import ( import (
"encoding/json"
"fmt" "fmt"
"sync/atomic" "sync/atomic"
"time" "time"
@@ -10,7 +11,8 @@ import (
"github.com/crazy-max/diun/internal/model" "github.com/crazy-max/diun/internal/model"
"github.com/crazy-max/diun/internal/notif" "github.com/crazy-max/diun/internal/notif"
"github.com/crazy-max/diun/internal/utl" "github.com/crazy-max/diun/internal/utl"
"github.com/crazy-max/diun/pkg/registry" "github.com/crazy-max/diun/pkg/docker"
"github.com/crazy-max/diun/pkg/docker/registry"
"github.com/hako/durafmt" "github.com/hako/durafmt"
"github.com/rs/zerolog/log" "github.com/rs/zerolog/log"
) )
@@ -18,7 +20,6 @@ import (
// Diun represents an active diun object // Diun represents an active diun object
type Diun struct { type Diun struct {
cfg *config.Config cfg *config.Config
reg *registry.Client
db *db.Client db *db.Client
notif *notif.Client notif *notif.Client
locker uint32 locker uint32
@@ -26,12 +27,6 @@ type Diun struct {
// New creates new diun instance // New creates new diun instance
func New(cfg *config.Config) (*Diun, error) { func New(cfg *config.Config) (*Diun, error) {
// Registry client
regcli, err := registry.New()
if err != nil {
return nil, err
}
// DB client // DB client
dbcli, err := db.New(cfg.Db) dbcli, err := db.New(cfg.Db)
if err != nil { if err != nil {
@@ -46,7 +41,6 @@ func New(cfg *config.Config) (*Diun, error) {
return &Diun{ return &Diun{
cfg: cfg, cfg: cfg,
reg: regcli,
db: dbcli, db: dbcli,
notif: notifcli, notif: notifcli,
}, nil }, nil
@@ -63,104 +57,95 @@ func (di *Diun) Run() {
// Iterate items // Iterate items
for _, item := range di.cfg.Items { for _, item := range di.cfg.Items {
image, err := registry.ParseImage(item.Image) reg, err := docker.NewRegistryClient(docker.RegistryOptions{
Os: di.cfg.Watch.Os,
Arch: di.cfg.Watch.Arch,
Username: item.Registry.Username,
Password: item.Registry.Password,
Timeout: time.Duration(item.Registry.Timeout) * time.Second,
InsecureTLS: item.Registry.InsecureTLS,
})
if err != nil { if err != nil {
log.Error().Err(err).Str("image", item.Image).Msg("Cannot parse image") log.Error().Err(err).Str("image", item.Image).Msg("Cannot create registry client")
continue continue
} }
opts := &registry.Options{ image, err := di.analyzeImage(item.Image, item, reg)
Image: image, if err != nil {
Username: item.RegCred.Username, log.Error().Err(err).Str("image", item.Image).Msg("Cannot analyze image")
Password: item.RegCred.Password,
InsecureTLS: item.InsecureTLS,
} }
if err := di.analyzeImage(item, opts); err != nil { if image.Domain != "" && item.WatchRepo {
log.Error().Err(err).Str("image", opts.Image.String()).Msg("Cannot analyze image") di.analyzeRepo(image, item, reg)
}
if item.WatchRepo {
di.analyzeRepo(item, opts)
} }
} }
} }
func (di *Diun) analyzeImage(item model.Item, opts *registry.Options) error { func (di *Diun) analyzeImage(imageStr string, item model.Item, reg *docker.RegistryClient) (registry.Image, error) {
if !di.isIncluded(opts.Image.Tag, item.IncludeTags) { image, err := registry.ParseImage(imageStr)
log.Warn().Str("image", opts.Image.String()).Msgf("Tag %s not included", opts.Image.Tag) if err != nil {
return nil return registry.Image{}, fmt.Errorf("cannot parse image name %s: %v", item.Image, err)
} else if di.isExcluded(opts.Image.Tag, item.ExcludeTags) {
log.Warn().Str("image", opts.Image.String()).Msgf("Tag %s excluded", opts.Image.Tag)
return nil
} }
log.Debug().Str("image", opts.Image.String()).Msgf("Analyzing") if !di.isIncluded(image.Tag, item.IncludeTags) {
liveAna, err := di.reg.Inspect(opts) log.Warn().Str("image", image.String()).Msgf("Tag %s not included", image.Tag)
if err != nil { return image, nil
return err } else if di.isExcluded(image.Tag, item.ExcludeTags) {
log.Warn().Str("image", image.String()).Msgf("Tag %s excluded", image.Tag)
return image, nil
} }
dbAna, err := di.db.GetAnalysis(opts.Image) log.Debug().Str("image", image.String()).Msgf("Fetching manifest")
liveManifest, err := reg.Manifest(image)
if err != nil { if err != nil {
return err return image, err
}
b, _ := json.MarshalIndent(liveManifest, "", " ")
log.Debug().Msg(string(b))
dbManifest, err := di.db.GetManifest(image)
if err != nil {
return image, err
} }
status := model.ImageStatusUnchange status := model.ImageStatusUnchange
if dbAna.Name == "" { if dbManifest.Name == "" {
status = model.ImageStatusNew status = model.ImageStatusNew
log.Info().Str("image", opts.Image.String()).Msgf("New image found") log.Info().Str("image", image.String()).Msgf("New image found")
} else if !liveAna.Created.Equal(*dbAna.Created) { } else if !liveManifest.Created.Equal(*dbManifest.Created) {
status = model.ImageStatusUpdate status = model.ImageStatusUpdate
log.Info().Str("image", opts.Image.String()).Msgf("Image update found") log.Info().Str("image", image.String()).Msgf("Image update found")
} else { } else {
log.Debug().Str("image", opts.Image.String()).Msgf("No changes") log.Debug().Str("image", image.String()).Msgf("No changes")
return nil return image, nil
} }
if err := di.db.PutAnalysis(opts.Image, liveAna); err != nil { if err := di.db.PutManifest(image, liveManifest); err != nil {
return err return image, err
} }
log.Debug().Str("image", opts.Image.String()).Msg("Analysis saved to database") log.Debug().Str("image", image.String()).Msg("Manifest saved to database")
di.notif.Send(model.NotifEntry{ di.notif.Send(model.NotifEntry{
Status: status, Status: status,
Image: opts.Image, Image: image,
Analysis: liveAna, Manifest: liveManifest,
}) })
return nil return image, nil
} }
func (di *Diun) analyzeRepo(item model.Item, opts *registry.Options) { func (di *Diun) analyzeRepo(image registry.Image, item model.Item, reg *docker.RegistryClient) {
tags, err := di.reg.Tags(opts) tags, err := reg.Tags(image)
if err != nil { if err != nil {
log.Error().Err(err).Str("image", opts.Image.String()).Msg("Cannot retrieve tags") log.Error().Err(err).Str("image", image.String()).Msg("Cannot retrieve tags")
return return
} }
log.Debug().Str("image", opts.Image.String()).Msgf("%d tag(s) found", len(tags)) log.Debug().Str("image", image.String()).Msgf("%d tag(s) found in repository", len(tags))
for _, tag := range tags { for _, tag := range tags {
if tag == opts.Image.Tag { imageStr := fmt.Sprintf("%s/%s:%s", image.Domain, image.Path, tag)
continue if _, err := di.analyzeImage(imageStr, item, reg); err != nil {
} log.Error().Err(err).Str("image", imageStr).Msg("Cannot analyze image")
simage := fmt.Sprintf("%s/%s:%s", opts.Image.Domain, opts.Image.Path, tag)
image, err := registry.ParseImage(simage)
if err != nil {
log.Error().Err(err).Str("image", simage).Msg("Cannot parse image")
continue
}
opts := &registry.Options{
Image: image,
Username: opts.Username,
Password: opts.Password,
InsecureTLS: opts.InsecureTLS,
}
if err := di.analyzeImage(item, opts); err != nil {
log.Error().Err(err).Str("image", image.String()).Msg("Cannot analyze image")
continue continue
} }
} }

40
internal/config/config.go
View File

@@ -18,13 +18,13 @@ import (
// Config holds configuration details // Config holds configuration details
type Config struct { type Config struct {
Flags model.Flags Flags model.Flags
App model.App App model.App
Db model.Db `yaml:"db,omitempty"` Db model.Db `yaml:"db,omitempty"`
Watch model.Watch `yaml:"watch,omitempty"` Watch model.Watch `yaml:"watch,omitempty"`
Notif model.Notif `yaml:"notif,omitempty"` Notif model.Notif `yaml:"notif,omitempty"`
RegCreds map[string]model.RegCred `yaml:"reg_creds,omitempty"` Registries map[string]model.Registry `yaml:"registries,omitempty"`
Items []model.Item `yaml:"items,omitempty"` Items []model.Item `yaml:"items,omitempty"`
} }
// Load returns Configuration struct // Load returns Configuration struct
@@ -44,7 +44,9 @@ func Load(fl model.Flags, version string) (*Config, error) {
Path: "diun.db", Path: "diun.db",
}, },
Watch: model.Watch{ Watch: model.Watch{
Schedule: "0 */30 * * * *", Schedule: "0 0 * * * *",
Os: "linux",
Arch: "amd64",
}, },
Notif: model.Notif{ Notif: model.Notif{
Mail: model.Mail{ Mail: model.Mail{
@@ -89,19 +91,23 @@ func (cfg *Config) Check() error {
} }
cfg.Db.Path = path.Clean(cfg.Db.Path) cfg.Db.Path = path.Clean(cfg.Db.Path)
for id, regCred := range cfg.RegCreds { for id, reg := range cfg.Registries {
if regCred.Username == "" || regCred.Password == "" { if err := mergo.Merge(&reg, model.Registry{
return fmt.Errorf("username and password required for registry credentials '%s'", id) InsecureTLS: false,
Timeout: 10,
}); err != nil {
return fmt.Errorf("cannot set default registry values for %s: %v", id, err)
} }
cfg.Registries[id] = reg
} }
for key, item := range cfg.Items { for key, item := range cfg.Items {
if item.RegCredID != "" { if item.RegistryID != "" {
regCred, found := cfg.RegCreds[item.RegCredID] reg, found := cfg.Registries[item.RegistryID]
if !found { if !found {
return fmt.Errorf("registry credentials '%s' not found", item.RegCredID) return fmt.Errorf("registry ID '%s' not found", item.RegistryID)
} }
cfg.Items[key].RegCred = regCred cfg.Items[key].Registry = reg
} }
for _, includeTag := range item.IncludeTags { for _, includeTag := range item.IncludeTags {
@@ -116,9 +122,7 @@ func (cfg *Config) Check() error {
} }
} }
if err := mergo.Merge(&cfg.Items[key], model.Item{ if err := mergo.Merge(&cfg.Items[key], item); err != nil {
Timeout: 10,
}); err != nil {
return err return err
} }
} }

21
internal/db/client.go
View File

@@ -6,7 +6,8 @@ import (
"time" "time"
"github.com/crazy-max/diun/internal/model" "github.com/crazy-max/diun/internal/model"
"github.com/crazy-max/diun/pkg/registry" "github.com/crazy-max/diun/pkg/docker"
"github.com/crazy-max/diun/pkg/docker/registry"
"github.com/rs/zerolog/log" "github.com/rs/zerolog/log"
bolt "go.etcd.io/bbolt" bolt "go.etcd.io/bbolt"
) )
@@ -17,7 +18,7 @@ type Client struct {
cfg model.Db cfg model.Db
} }
const bucket = "analysis" const bucket = "manifest"
// New creates new db instance // New creates new db instance
func New(cfg model.Db) (*Client, error) { func New(cfg model.Db) (*Client, error) {
@@ -52,24 +53,24 @@ func (c *Client) Close() error {
return c.DB.Close() return c.DB.Close()
} }
// GetAnalysis returns Docker image analysis // GetManifest returns Docker image manifest
func (c *Client) GetAnalysis(image registry.Image) (registry.Inspect, error) { func (c *Client) GetManifest(image registry.Image) (docker.Manifest, error) {
var ana registry.Inspect var manifest docker.Manifest
err := c.View(func(tx *bolt.Tx) error { err := c.View(func(tx *bolt.Tx) error {
b := tx.Bucket([]byte(bucket)) b := tx.Bucket([]byte(bucket))
if entryBytes := b.Get([]byte(image.String())); entryBytes != nil { if entryBytes := b.Get([]byte(image.String())); entryBytes != nil {
return json.Unmarshal(entryBytes, &ana) return json.Unmarshal(entryBytes, &manifest)
} }
return nil return nil
}) })
return ana, err return manifest, err
} }
// PutAnalysis add Docker image analysis in db // PutManifest add Docker image manifest in db
func (c *Client) PutAnalysis(image registry.Image, analysis registry.Inspect) error { func (c *Client) PutManifest(image registry.Image, manifest docker.Manifest) error {
entryBytes, _ := json.Marshal(analysis) entryBytes, _ := json.Marshal(manifest)
err := c.Update(func(tx *bolt.Tx) error { err := c.Update(func(tx *bolt.Tx) error {
b := tx.Bucket([]byte(bucket)) b := tx.Bucket([]byte(bucket))

2
internal/logging/logger.go
View File

@@ -28,7 +28,7 @@ func Configure(fl *model.Flags, location *time.Location) {
w = os.Stdout w = os.Stdout
} }
log.Logger = zerolog.New(w).With().Timestamp().Logger() log.Logger = zerolog.New(w).With().Caller().Timestamp().Logger()
logLevel, err := zerolog.ParseLevel(fl.LogLevel) logLevel, err := zerolog.ParseLevel(fl.LogLevel)
if err != nil { if err != nil {

14
internal/model/flags.go
View File

@@ -2,11 +2,11 @@ package model
// Flags holds flags from command line // Flags holds flags from command line
type Flags struct { type Flags struct {
Cfgfile string Cfgfile string
Populate bool Populate bool
Timezone string Timezone string
LogLevel string LogLevel string
LogJson bool LogJson bool
RunOnce bool RunStartup bool
Docker bool Docker bool
} }

14
internal/model/item.go
View File

@@ -2,12 +2,10 @@ package model
// Item holds item configuration for a Docker image // Item holds item configuration for a Docker image
type Item struct { type Item struct {
Image string `yaml:"image,omitempty"` Image string `yaml:"image,omitempty" json:",omitempty"`
RegCredID string `yaml:"reg_cred_id,omitempty"` RegistryID string `yaml:"registry_id,omitempty" json:",omitempty"`
InsecureTLS bool `yaml:"insecure_tls,omitempty"` WatchRepo bool `yaml:"watch_repo,omitempty" json:",omitempty"`
WatchRepo bool `yaml:"watch_repo,omitempty"` IncludeTags []string `yaml:"include_tags,omitempty" json:",omitempty"`
IncludeTags []string `yaml:"include_tags,omitempty"` ExcludeTags []string `yaml:"exclude_tags,omitempty" json:",omitempty"`
ExcludeTags []string `yaml:"exclude_tags,omitempty"` Registry Registry `yaml:"-" json:"-"`
Timeout int `yaml:"timeout,omitempty"`
RegCred RegCred `json:"-"`
} }

11
internal/model/notif.go
View File

@@ -1,6 +1,9 @@
package model package model
import "github.com/crazy-max/diun/pkg/registry" import (
"github.com/crazy-max/diun/pkg/docker"
"github.com/crazy-max/diun/pkg/docker/registry"
)
// Notif holds data necessary for notification configuration // Notif holds data necessary for notification configuration
type Notif struct { type Notif struct {
@@ -10,7 +13,7 @@ type Notif struct {
// NotifEntry represents a notification entry // NotifEntry represents a notification entry
type NotifEntry struct { type NotifEntry struct {
Status ImageStatus `json:"status,omitempty"` Status ImageStatus `json:"status,omitempty"`
Image registry.Image `json:"image,omitempty"` Image registry.Image `json:"image,omitempty"`
Analysis registry.Inspect `json:"analysis,omitempty"` Manifest docker.Manifest `json:"manifest,omitempty"`
} }

7
internal/model/regcred.go
View File

@@ -1,7 +0,0 @@
package model
// RegCred holds registry credential
type RegCred struct {
Username string `yaml:"username,omitempty"`
Password string `yaml:"password,omitempty"`
}

9
internal/model/registry.go Normal file
View File

@@ -0,0 +1,9 @@
package model
// Registry holds registry configuration
type Registry struct {
Username string `yaml:"username,omitempty" json:",omitempty"`
Password string `yaml:"password,omitempty" json:",omitempty"`
InsecureTLS bool `yaml:"insecure_tls,omitempty" json:",omitempty"`
Timeout int `yaml:"timeout,omitempty" json:",omitempty"`
}

2
internal/model/watch.go
View File

@@ -3,4 +3,6 @@ package model
// Watch holds data necessary for watch configuration // Watch holds data necessary for watch configuration
type Watch struct { type Watch struct {
Schedule string `yaml:"schedule,omitempty"` Schedule string `yaml:"schedule,omitempty"`
Os string `yaml:"os,omitempty"`
Arch string `yaml:"arch,omitempty"`
} }

26
internal/notif/webhook/client.go
View File

@@ -41,23 +41,23 @@ func (c *Client) Send(entry model.NotifEntry) error {
} }
body, err := json.Marshal(struct { body, err := json.Marshal(struct {
Version string `json:"diun_version,omitempty"` Version string `json:"diun_version"`
Status string `json:"status,omitempty"` Status string `json:"status"`
Image string `json:"image,omitempty"` Image string `json:"image"`
MIMEType string `json:"mime_type,omitempty"` MIMEType string `json:"mime_type"`
Digest digest.Digest `json:"digest,omitempty"` Digest digest.Digest `json:"digest"`
Date *time.Time `json:"date,omitempty"` Created *time.Time `json:"created"`
Architecture string `json:"architecture,omitempty"` Architecture string `json:"architecture"`
Os string `json:"os,omitempty"` Os string `json:"os"`
}{ }{
Version: c.app.Version, Version: c.app.Version,
Status: string(entry.Status), Status: string(entry.Status),
Image: entry.Image.String(), Image: entry.Image.String(),
MIMEType: entry.Analysis.MIMEType, MIMEType: entry.Manifest.MIMEType,
Digest: entry.Analysis.Digest, Digest: entry.Manifest.Digest,
Date: entry.Analysis.Created, Created: entry.Manifest.Created,
Architecture: entry.Analysis.Architecture, Architecture: entry.Manifest.Architecture,
Os: entry.Analysis.Os, Os: entry.Manifest.Os,
}) })
if err != nil { if err != nil {
return err return err

59
pkg/docker/client.go Normal file
View File

@@ -0,0 +1,59 @@
package docker
import (
"context"
"time"
"github.com/containers/image/types"
)
// RegistryClient represents an active docker registry object
type RegistryClient struct {
ctx context.Context
cancel context.CancelFunc
sysCtx *types.SystemContext
}
// RegistryOptions holds docker registry object options
type RegistryOptions struct {
Os string
Arch string
Username string
Password string
InsecureTLS bool
Timeout time.Duration
}
// NewRegistryClient creates new docker registry client instance
func NewRegistryClient(opts RegistryOptions) (*RegistryClient, error) {
// Context
ctx := context.Background()
var cancel context.CancelFunc = func() {}
if opts.Timeout > 0 {
ctx, cancel = context.WithTimeout(ctx, opts.Timeout)
}
// Auth
auth := &types.DockerAuthConfig{}
if opts.Username != "" {
auth = &types.DockerAuthConfig{
Username: opts.Username,
Password: opts.Password,
}
}
// Sys context
sysCtx := &types.SystemContext{
OSChoice: opts.Os,
ArchitectureChoice: opts.Arch,
DockerAuthConfig: auth,
DockerDaemonInsecureSkipTLSVerify: opts.InsecureTLS,
DockerInsecureSkipTLSVerify: types.NewOptionalBool(opts.InsecureTLS),
}
return &RegistryClient{
ctx: ctx,
cancel: cancel,
sysCtx: sysCtx,
}, nil
}

27
pkg/docker/image.go Normal file
View File

@@ -0,0 +1,27 @@
package docker
import (
"fmt"
"strings"
"github.com/containers/image/docker"
"github.com/containers/image/types"
)
func (c *RegistryClient) newImage(imageStr string) (types.ImageCloser, error) {
if !strings.HasPrefix(imageStr, "//") {
imageStr = fmt.Sprintf("//%s", imageStr)
}
ref, err := docker.ParseReference(imageStr)
if err != nil {
return nil, fmt.Errorf("invalid image name %s: %v", imageStr, err)
}
img, err := ref.NewImage(c.ctx, c.sysCtx)
if err != nil {
return nil, err
}
return img, nil
}

34
pkg/registry/inspect.go → pkg/docker/manifest.go
View File

@@ -1,13 +1,14 @@
package registry package docker
import ( import (
"time" "time"
"github.com/containers/image/manifest" "github.com/containers/image/manifest"
"github.com/crazy-max/diun/pkg/docker/registry"
"github.com/opencontainers/go-digest" "github.com/opencontainers/go-digest"
) )
type Inspect struct { type Manifest struct {
Name string Name string
Tag string Tag string
MIMEType string MIMEType string
@@ -20,39 +21,38 @@ type Inspect struct {
Layers []string Layers []string
} }
// Inspect inspects a Docker image // Manifest returns the manifest for a specific image
func (c *Client) Inspect(opts *Options) (Inspect, error) { func (c *RegistryClient) Manifest(image registry.Image) (Manifest, error) {
ctx, cancel := c.timeoutContext(opts.Timeout) defer c.cancel()
defer cancel()
img, _, err := c.newImage(ctx, opts) imgCls, err := c.newImage(image.String())
if err != nil { if err != nil {
return Inspect{}, err return Manifest{}, err
} }
defer img.Close() defer imgCls.Close()
rawManifest, _, err := img.Manifest(ctx) rawManifest, _, err := imgCls.Manifest(c.ctx)
if err != nil { if err != nil {
return Inspect{}, err return Manifest{}, err
} }
imgInspect, err := img.Inspect(ctx) imgInspect, err := imgCls.Inspect(c.ctx)
if err != nil { if err != nil {
return Inspect{}, err return Manifest{}, err
} }
imgDigest, err := manifest.Digest(rawManifest) imgDigest, err := manifest.Digest(rawManifest)
if err != nil { if err != nil {
return Inspect{}, err return Manifest{}, err
} }
imgTag := imgInspect.Tag imgTag := imgInspect.Tag
if imgTag == "" { if imgTag == "" {
imgTag = opts.Image.Tag imgTag = image.Tag
} }
return Inspect{ return Manifest{
Name: img.Reference().DockerReference().Name(), Name: imgCls.Reference().DockerReference().Name(),
Tag: imgTag, Tag: imgTag,
MIMEType: manifest.GuessMIMEType(rawManifest), MIMEType: manifest.GuessMIMEType(rawManifest),
Digest: imgDigest, Digest: imgDigest,

1
pkg/docker/registry/README.md Normal file
View File

@@ -0,0 +1 @@
This is a copy of github.com/genuinetools/reg/registry as of commit 2bd71359e708deb5e9e1c4e5d882dde99ce23b85

2
pkg/registry/image.go → pkg/docker/registry/image.go
View File

@@ -1,5 +1,3 @@
// Source: https://github.com/genuinetools/reg/blob/master/registry/image.go
package registry package registry
import ( import (

26
pkg/docker/tags.go Normal file
View File

@@ -0,0 +1,26 @@
package docker
import (
"github.com/containers/image/docker"
"github.com/crazy-max/diun/pkg/docker/registry"
)
type Tags []string
// Tags returns tags of a Docker repository
func (c *RegistryClient) Tags(image registry.Image) (Tags, error) {
defer c.cancel()
imgCls, err := c.newImage(image.String())
if err != nil {
return nil, err
}
defer imgCls.Close()
tags, err := docker.GetRepositoryTags(c.ctx, c.sysCtx, imgCls.Reference())
if err != nil {
return nil, err
}
return Tags(tags), err
}

69
pkg/registry/client.go
View File

@@ -1,69 +0,0 @@
package registry
import (
"context"
"fmt"
"strings"
"time"
"github.com/containers/image/docker"
"github.com/containers/image/types"
)
// Client represents an active registry object
type Client struct{}
type Options struct {
Image Image
Username string
Password string
InsecureTLS bool
Timeout time.Duration
}
// New creates new registry instance
func New() (*Client, error) {
return &Client{}, nil
}
func (c *Client) timeoutContext(timeout time.Duration) (context.Context, context.CancelFunc) {
ctx := context.Background()
var cancel context.CancelFunc = func() {}
if timeout > 0 {
ctx, cancel = context.WithTimeout(ctx, timeout)
}
return ctx, cancel
}
func (c *Client) newImage(ctx context.Context, opts *Options) (types.ImageCloser, *types.SystemContext, error) {
image := opts.Image.String()
if !strings.HasPrefix(opts.Image.String(), "//") {
image = fmt.Sprintf("//%s", opts.Image.String())
}
ref, err := docker.ParseReference(image)
if err != nil {
return nil, nil, fmt.Errorf("invalid image name %s: %v", image, err)
}
auth := &types.DockerAuthConfig{}
if opts.Username != "" {
auth = &types.DockerAuthConfig{
Username: opts.Username,
Password: opts.Password,
}
}
sys := &types.SystemContext{
DockerAuthConfig: auth,
DockerDaemonInsecureSkipTLSVerify: opts.InsecureTLS,
DockerInsecureSkipTLSVerify: types.NewOptionalBool(opts.InsecureTLS),
}
img, err := ref.NewImage(ctx, sys)
if err != nil {
return nil, nil, err
}
return img, sys, nil
}

26
pkg/registry/tags.go
View File

@@ -1,26 +0,0 @@
package registry
import (
"github.com/containers/image/docker"
)
type Tags []string
// Tags returns tags of a Docker repository
func (c *Client) Tags(opts *Options) (Tags, error) {
ctx, cancel := c.timeoutContext(opts.Timeout)
defer cancel()
img, sys, err := c.newImage(ctx, opts)
if err != nil {
return nil, err
}
defer img.Close()
tags, err := docker.GetRepositoryTags(ctx, sys, img.Reference())
if err != nil {
return nil, err
}
return Tags(tags), err
}