mirrors/diun
1
0
Fork 0
mirror of https://github.com/crazy-max/diun.git synced 2025-12-21 21:33:22 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #749 from crazy-max/entry-metadata

Browse Source 
Entry metadata field
This commit is contained in:
CrazyMax
2022-12-29 11:45:26 +01:00
committed by GitHub
parent 0cf39615b2 073e08ee11
commit 858a9589a7
22 changed files with 297 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/config/watch.md
View File

@@ -31,7 +31,7 @@ Maximum number of workers that will execute tasks concurrently. (default `10`)
### `schedule` ### `schedule`
[CRON expression](https://godoc.org/github.com/robfig/cron#hdr-CRON_Expression_Format) to schedule Diun. [CRON expression](https://pkg.go.dev/github.com/crazy-max/cron/v3#hdr-CRON_Expression_Format) to schedule Diun.
!!! warning !!! warning
Remove this setting if you want to run Diun directly. Remove this setting if you want to run Diun directly.

1
docs/faq.md
View File

@@ -82,6 +82,7 @@ Templating is supported with the following fields:
| `.Entry.Manifest.Labels` | Image labels | | `.Entry.Manifest.Labels` | Image labels |
| `.Entry.Manifest.Layers` | Image layers | | `.Entry.Manifest.Layers` | Image layers |
| `.Entry.Manifest.Platform` | Platform that the image is runs on. e.g. `linux/amd64` | | `.Entry.Manifest.Platform` | Platform that the image is runs on. e.g. `linux/amd64` |
| `.Entry.Metadata` | Key-value pair of image metadata specific to each provider |
## Authentication against the registry ## Authentication against the registry

9
docs/notif/script.md
View File

@@ -3,7 +3,7 @@
You can call a script when a notification occured. Following environment variables will be passed: You can call a script when a notification occured. Following environment variables will be passed:
``` ```
DIUN_VERSION=3.0.0 DIUN_VERSION=4.24.0
DIUN_ENTRY_STATUS=new DIUN_ENTRY_STATUS=new
DIUN_HOSTNAME=myserver DIUN_HOSTNAME=myserver
DIUN_ENTRY_PROVIDER=file DIUN_ENTRY_PROVIDER=file
@@ -13,6 +13,13 @@ DIUN_ENTRY_MIMETYPE=application/vnd.docker.distribution.manifest.list.v2+json
DIUN_ENTRY_DIGEST=sha256:216e3ae7de4ca8b553eb11ef7abda00651e79e537e85c46108284e5e91673e01 DIUN_ENTRY_DIGEST=sha256:216e3ae7de4ca8b553eb11ef7abda00651e79e537e85c46108284e5e91673e01
DIUN_ENTRY_CREATED=2020-03-26 12:23:56 +0000 UTC DIUN_ENTRY_CREATED=2020-03-26 12:23:56 +0000 UTC
DIUN_ENTRY_PLATFORM=linux/amd64 DIUN_ENTRY_PLATFORM=linux/amd64
DIUN_ENTRY_METADATA_CTN_COMMAND=diun serve
DIUN_ENTRY_METADATA_CTN_CREATEDAT=2022-12-29 10:46:20 +0100 CET
DIUN_ENTRY_METADATA_CTN_ID=7c71187fad11aa06f951dee0ebd6382ee0030a8228929fc7ea2fccc18f940788
DIUN_ENTRY_METADATA_CTN_NAMES=diun
DIUN_ENTRY_METADATA_CTN_SIZE=0B
DIUN_ENTRY_METADATA_CTN_STATE=running
DIUN_ENTRY_METADATA_CTN_STATUS=Up Less than a second (health: starting)
``` ```
## Configuration ## Configuration

13
docs/notif/webhook.md
View File

@@ -35,7 +35,7 @@ The JSON response will look like this:
```json ```json
{ {
"diun_version": "4.0.0", "diun_version": "4.24.0",
"hostname": "myserver", "hostname": "myserver",
"status": "new", "status": "new",
"provider": "file", "provider": "file",
@@ -44,7 +44,16 @@ The JSON response will look like this:
"mime_type": "application/vnd.docker.distribution.manifest.list.v2+json", "mime_type": "application/vnd.docker.distribution.manifest.list.v2+json",
"digest": "sha256:216e3ae7de4ca8b553eb11ef7abda00651e79e537e85c46108284e5e91673e01", "digest": "sha256:216e3ae7de4ca8b553eb11ef7abda00651e79e537e85c46108284e5e91673e01",
"created": "2020-03-26T12:23:56Z", "created": "2020-03-26T12:23:56Z",
"platform": "linux/amd64" "platform": "linux/amd64",
"metadata": {
"ctn_command": "diun serve",
"ctn_createdat": "2022-12-29 10:22:15 +0100 CET",
"ctn_id": "0dbd10e15b31add2c48856fd34451adabf50d276efa466fe19a8ef5fbd87ad7c",
"ctn_names": "diun",
"ctn_size": "0B",
"ctn_state": "running",
"ctn_status": "Up Less than a second (health: starting)"
}
} }
``` ```

37
docs/providers/docker.md
View File

@@ -173,15 +173,28 @@ Include created and exited containers too (default `false`).
You can configure more finely the way to analyze the image of your container through Docker labels: You can configure more finely the way to analyze the image of your container through Docker labels:
| Name | Default | Description | | Name | Default | Description |
|---------------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------| |---------------------|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
| `diun.enable` | | Set to true to enable image analysis of this container | | `diun.enable` | | Set to true to enable image analysis of this container |
| `diun.regopt` | | [Registry options](../config/regopts.md) name to use | | `diun.regopt` | | [Registry options](../config/regopts.md) name to use |
| `diun.watch_repo` | `false` | Watch all tags of this container image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `diun.watch_repo` | `false` | Watch all tags of this container image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) |
| `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update` | | `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update` |
| `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` | | `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` |
| `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them | | `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them |
| `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` | | `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` |
| `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` |
| `diun.hub_link` | _automatic_ | Set registry hub link for this image | | `diun.hub_link` | _automatic_ | Set registry hub link for this image |
| `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) | | `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) |
| `diun.metadata.*` | See [below](#default-metadata) | Additional metadata that can be used in [notification template](../faq.md#notification-template) (e.g. `diun.metadata.foo=bar`) |
## Default metadata
| Key | Description |
|-------------------------------|-------------------------|
| `diun.metadata.ctn_id` | Container ID |
| `diun.metadata.ctn_names` | Container names |
| `diun.metadata.ctn_command` | Container command |
| `diun.metadata.ctn_createdat` | Container creation date |
| `diun.metadata.ctn_state` | Container state |
| `diun.metadata.ctn_status` | Container status |
| `diun.metadata.ctn_size` | Container size |

2
docs/providers/dockerfile.md
View File

@@ -37,6 +37,7 @@ providers:
# syntax=docker/dockerfile:1.2 # syntax=docker/dockerfile:1.2
# diun.platform=linux/amd64 # diun.platform=linux/amd64
# diun.metadata.foo=bar
FROM alpine:latest FROM alpine:latest
# diun.watch_repo=true # diun.watch_repo=true
@@ -117,3 +118,4 @@ The following annotations can be added as comments before the target instruction
| `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` |
| `diun.hub_link` | _automatic_ | Set registry hub link for this image | | `diun.hub_link` | _automatic_ | Set registry hub link for this image |
| `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) | | `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) |
| `diun.metadata.*` | | Additional metadata that can be used in [notification template](../faq.md#notification-template) (e.g. `metadata.foo=bar`) |

4
docs/providers/file.md
View File

@@ -48,7 +48,10 @@ providers:
regopt: myregistry regopt: myregistry
# Watch coreos/hyperkube image on quay.io (Quay) and assume latest tag. # Watch coreos/hyperkube image on quay.io (Quay) and assume latest tag.
# Add foo=bar metadata to be used in notification template.
- name: quay.io/coreos/hyperkube - name: quay.io/coreos/hyperkube
metadata:
foo: bar
# Watch crazymax/swarm-cronjob image and assume docker.io registry and latest tag # Watch crazymax/swarm-cronjob image and assume docker.io registry and latest tag
# with registry options named 'docker.io/crazymax' (image selector). # with registry options named 'docker.io/crazymax' (image selector).
@@ -186,3 +189,4 @@ The configuration file(s) defines a slice of images to analyze with the followin
| `platform.os` | _automatic_ | Operating system to use as custom platform | | `platform.os` | _automatic_ | Operating system to use as custom platform |
| `platform.arch` | _automatic_ | CPU architecture to use as custom platform | | `platform.arch` | _automatic_ | CPU architecture to use as custom platform |
| `platform.variant` | _automatic_ | Variant of the CPU to use as custom platform | | `platform.variant` | _automatic_ | Variant of the CPU to use as custom platform |
| `metadata.*` | | Additional metadata that can be used in [notification template](../faq.md#notification-template) (e.g. `metadata.foo=bar`) |

36
docs/providers/kubernetes.md
View File

@@ -282,15 +282,27 @@ Enable watch by default. If false, pods that don't have `diun.enable: "true"` an
You can configure more finely the way to analyze the image of your pods through Kubernetes annotations: You can configure more finely the way to analyze the image of your pods through Kubernetes annotations:
| Name | Default | Description | | Name | Default | Description |
|---------------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------| |---------------------|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
| `diun.enable` | | Set to true to enable image analysis of this pod | | `diun.enable` | | Set to true to enable image analysis of this pod |
| `diun.regopt` | | [Registry options](../config/regopts.md) name to use | | `diun.regopt` | | [Registry options](../config/regopts.md) name to use |
| `diun.watch_repo` | `false` | Watch all tags of this pod image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `diun.watch_repo` | `false` | Watch all tags of this pod image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) |
| `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update`. | | `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update`. |
| `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` | | `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` |
| `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them | | `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them |
| `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` | | `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` |
| `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` |
| `diun.hub_link` | _automatic_ | Set registry hub link for this image | | `diun.hub_link` | _automatic_ | Set registry hub link for this image |
| `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) | | `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) |
| `diun.metadata.*` | See [below](#default-metadata) | Additional metadata that can be used in [notification template](../faq.md#notification-template) (e.g. `diun.metadata.foo=bar`) |
## Default metadata
| Key | Description |
|-------------------------------|-------------------|
| `diun.metadata.pod_name` | Pod name |
| `diun.metadata.pod_status` | Pod status |
| `diun.metadata.pod_namespace` | Pod namespace |
| `diun.metadata.pod_createdat` | Pod creation date |
| `diun.metadata.ctn_name` | Container name |
| `diun.metadata.ctn_command` | Container command |

38
docs/providers/nomad.md
View File

@@ -197,15 +197,29 @@ Enable watch by default. If false, tasks that don't have `diun.enable = true` in
You can configure more finely the way to analyze the image of your tasks through Nomad meta attributes or service tags: You can configure more finely the way to analyze the image of your tasks through Nomad meta attributes or service tags:
| Name | Default | Description | | Name | Default | Description |
|---------------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------| |---------------------|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
| `diun.enable` | | Set to true to enable image analysis of this task | | `diun.enable` | | Set to true to enable image analysis of this task |
| `diun.regopt` | | [Registry options](../config/regopts.md) name to use | | `diun.regopt` | | [Registry options](../config/regopts.md) name to use |
| `diun.watch_repo` | `false` | Watch all tags of this task image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `diun.watch_repo` | `false` | Watch all tags of this task image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) |
| `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update`. | | `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update`. |
| `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` | | `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` |
| `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them | | `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them |
| `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` | | `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` |
| `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` |
| `diun.hub_link` | _automatic_ | Set registry hub link for this image | | `diun.hub_link` | _automatic_ | Set registry hub link for this image |
| `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) | | `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) |
| `diun.metadata.*` | See [below](#default-metadata) | Additional metadata that can be used in [notification template](../faq.md#notification-template) (e.g. `diun.metadata.foo=bar`) |
## Default metadata
| Key | Description |
|--------------------------------|-----------------|
| `diun.metadata.job_id` | Job ID |
| `diun.metadata.job_name` | Job name |
| `diun.metadata.job_status` | Job status |
| `diun.metadata.job_namespace` | Job namespace |
| `diun.metadata.taskgroup_name` | Task group name |
| `diun.metadata.task_name` | Task name |
| `diun.metadata.task_driver` | Task driver |
| `diun.metadata.task_user` | Task user |

34
docs/providers/swarm.md
View File

@@ -175,15 +175,25 @@ Enable watch by default. If false, services that don't have `diun.enable=true` l
You can configure more finely the way to analyze the image of your service through Docker labels: You can configure more finely the way to analyze the image of your service through Docker labels:
| Name | Default | Description | | Name | Default | Description |
|---------------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------| |---------------------|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
| `diun.enable` | | Set to true to enable image analysis of this service | | `diun.enable` | | Set to true to enable image analysis of this service |
| `diun.regopt` | | [Registry options](../config/regopts.md) name to use | | `diun.regopt` | | [Registry options](../config/regopts.md) name to use |
| `diun.watch_repo` | `false` | Watch all tags of this service image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `diun.watch_repo` | `false` | Watch all tags of this service image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) |
| `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update`. | | `diun.notify_on` | `new;update` | Semicolon separated list of status to be notified: `new`, `update`. |
| `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` | | `diun.sort_tags` | `reverse` | [Sort tags method](../faq.md#tags-sorting-when-using-watch_repo) if `diun.watch_repo` enabled. One of `default`, `reverse`, `semver`, `lexicographical` |
| `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them | | `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them |
| `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` | | `diun.include_tags` | | Semicolon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` |
| `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semicolon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` |
| `diun.hub_link` | _automatic_ | Set registry hub link for this image | | `diun.hub_link` | _automatic_ | Set registry hub link for this image |
| `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) | | `diun.platform` | _automatic_ | Platform to use (e.g. `linux/amd64`) |
| `diun.metadata.*` | See [below](#default-metadata) | Additional metadata that can be used in [notification template](../faq.md#notification-template) (e.g. `diun.metadata.foo=bar`) |
## Default metadata
| Key | Description |
|-------------------------------|-----------------------|
| `diun.metadata.svc_id` | Service ID |
| `diun.metadata.svc_createdat` | Service creation date |
| `diun.metadata.svc_updatedat` | Service update date |
| `diun.metadata.ctn_name` | Container name |

53
internal/app/diun.go
View File

@@ -151,37 +151,11 @@ func (di *Diun) Run() {
}, ants.WithLogger(new(logging.AntsLogger))) }, ants.WithLogger(new(logging.AntsLogger)))
defer di.pool.Release() defer di.pool.Release()
// Docker provider for _, job := range di.providersJobs() {
for _, job := range dockerPrd.New(di.cfg.Providers.Docker).ListJob() {
di.createJob(job) di.createJob(job)
} }
// Swarm provider
for _, job := range swarmPrd.New(di.cfg.Providers.Swarm).ListJob() {
di.createJob(job)
}
// Kubernetes provider
for _, job := range kubernetesPrd.New(di.cfg.Providers.Kubernetes).ListJob() {
di.createJob(job)
}
// File provider
for _, job := range filePrd.New(di.cfg.Providers.File).ListJob() {
di.createJob(job)
}
// Dockerfile provider
for _, job := range dockerfilePrd.New(di.cfg.Providers.Dockerfile).ListJob() {
di.createJob(job)
}
// Nomad provider
for _, job := range nomadPrd.New(di.cfg.Providers.Nomad).ListJob() {
di.createJob(job)
}
di.wg.Wait() di.wg.Wait()
log.Info(). log.Info().
Int("added", entries.CountNew). Int("added", entries.CountNew).
Int("updated", entries.CountUpdate). Int("updated", entries.CountUpdate).
@@ -202,3 +176,26 @@ func (di *Diun) Close() {
log.Warn().Err(err).Msg("Cannot close database") log.Warn().Err(err).Msg("Cannot close database")
} }
} }
func (di *Diun) providersJobs() []model.Job {
var jobs []model.Job
for _, job := range dockerPrd.New(di.cfg.Providers.Docker).ListJob() {
jobs = append(jobs, job)
}
for _, job := range swarmPrd.New(di.cfg.Providers.Swarm).ListJob() {
jobs = append(jobs, job)
}
for _, job := range kubernetesPrd.New(di.cfg.Providers.Kubernetes).ListJob() {
jobs = append(jobs, job)
}
for _, job := range filePrd.New(di.cfg.Providers.File).ListJob() {
jobs = append(jobs, job)
}
for _, job := range dockerfilePrd.New(di.cfg.Providers.Dockerfile).ListJob() {
jobs = append(jobs, job)
}
for _, job := range nomadPrd.New(di.cfg.Providers.Nomad).ListJob() {
jobs = append(jobs, job)
}
return jobs
}

1
internal/app/job.go
View File

@@ -169,6 +169,7 @@ func (di *Diun) runJob(job model.Job) (entry model.NotifEntry) {
Status: model.ImageStatusError, Status: model.ImageStatusError,
Provider: job.Provider, Provider: job.Provider,
Image: job.RegImage, Image: job.RegImage,
Metadata: job.Image.Metadata,
} }
sublog := log.With(). sublog := log.With().

23
internal/model/image.go
View File

@@ -4,17 +4,18 @@ import "github.com/crazy-max/diun/v4/pkg/registry"
// Image holds image configuration // Image holds image configuration
type Image struct { type Image struct {
Name string `yaml:"name,omitempty" json:",omitempty"` Name string `yaml:"name,omitempty" json:",omitempty"`
Platform ImagePlatform `yaml:"platform,omitempty" json:",omitempty"` Platform ImagePlatform `yaml:"platform,omitempty" json:",omitempty"`
RegOpt string `yaml:"regopt,omitempty" json:",omitempty"` RegOpt string `yaml:"regopt,omitempty" json:",omitempty"`
WatchRepo bool `yaml:"watch_repo,omitempty" json:",omitempty"` WatchRepo bool `yaml:"watch_repo,omitempty" json:",omitempty"`
NotifyOn []NotifyOn `yaml:"notify_on,omitempty" json:",omitempty"` NotifyOn []NotifyOn `yaml:"notify_on,omitempty" json:",omitempty"`
MaxTags int `yaml:"max_tags,omitempty" json:",omitempty"` MaxTags int `yaml:"max_tags,omitempty" json:",omitempty"`
SortTags registry.SortTag `yaml:"sort_tags,omitempty" json:",omitempty"` SortTags registry.SortTag `yaml:"sort_tags,omitempty" json:",omitempty"`
IncludeTags []string `yaml:"include_tags,omitempty" json:",omitempty"` IncludeTags []string `yaml:"include_tags,omitempty" json:",omitempty"`
ExcludeTags []string `yaml:"exclude_tags,omitempty" json:",omitempty"` ExcludeTags []string `yaml:"exclude_tags,omitempty" json:",omitempty"`
HubTpl string `yaml:"hub_tpl,omitempty" json:",omitempty"` HubTpl string `yaml:"hub_tpl,omitempty" json:",omitempty"`
HubLink string `yaml:"hub_link,omitempty" json:",omitempty"` HubLink string `yaml:"hub_link,omitempty" json:",omitempty"`
Metadata map[string]string `yaml:"metadata,omitempty" json:",omitempty"`
} }
// ImagePlatform holds image platform configuration // ImagePlatform holds image platform configuration

1
internal/model/notif.go
View File

@@ -27,6 +27,7 @@ type NotifEntry struct {
Provider string `json:"provider,omitempty"` Provider string `json:"provider,omitempty"`
Image registry.Image `json:"image,omitempty"` Image registry.Image `json:"image,omitempty"`
Manifest registry.Manifest `json:"manifest,omitempty"` Manifest registry.Manifest `json:"manifest,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
} }
// Notif holds data necessary for notification configuration // Notif holds data necessary for notification configuration

30
internal/msg/client.go
View File

@@ -91,16 +91,17 @@ func (c *Client) RenderHTML() (title []byte, body []byte, err error) {
// RenderJSON returns a notification message as JSON // RenderJSON returns a notification message as JSON
func (c *Client) RenderJSON() ([]byte, error) { func (c *Client) RenderJSON() ([]byte, error) {
return json.Marshal(struct { return json.Marshal(struct {
Version string `json:"diun_version"` Version string `json:"diun_version"`
Hostname string `json:"hostname"` Hostname string `json:"hostname"`
Status string `json:"status"` Status string `json:"status"`
Provider string `json:"provider"` Provider string `json:"provider"`
Image string `json:"image"` Image string `json:"image"`
HubLink string `json:"hub_link"` HubLink string `json:"hub_link"`
MIMEType string `json:"mime_type"` MIMEType string `json:"mime_type"`
Digest digest.Digest `json:"digest"` Digest digest.Digest `json:"digest"`
Created *time.Time `json:"created"` Created *time.Time `json:"created"`
Platform string `json:"platform"` Platform string `json:"platform"`
Metadata map[string]string `json:"metadata"`
}{ }{
Version: c.opts.Meta.Version, Version: c.opts.Meta.Version,
Hostname: c.opts.Meta.Hostname, Hostname: c.opts.Meta.Hostname,
@@ -112,12 +113,17 @@ func (c *Client) RenderJSON() ([]byte, error) {
Digest: c.opts.Entry.Manifest.Digest, Digest: c.opts.Entry.Manifest.Digest,
Created: c.opts.Entry.Manifest.Created, Created: c.opts.Entry.Manifest.Created,
Platform: c.opts.Entry.Manifest.Platform, Platform: c.opts.Entry.Manifest.Platform,
Metadata: c.opts.Entry.Metadata,
}) })
} }
// RenderEnv returns a notification message as environment variables // RenderEnv returns a notification message as environment variables
func (c *Client) RenderEnv() []string { func (c *Client) RenderEnv() []string {
return []string{ var metadataEnvs []string
for k, v := range c.opts.Entry.Metadata {
metadataEnvs = append(metadataEnvs, fmt.Sprintf("DIUN_ENTRY_METADATA_%s=%s", strings.ToUpper(k), v))
}
return append([]string{
fmt.Sprintf("DIUN_VERSION=%s", c.opts.Meta.Version), fmt.Sprintf("DIUN_VERSION=%s", c.opts.Meta.Version),
fmt.Sprintf("DIUN_HOSTNAME=%s", c.opts.Meta.Hostname), fmt.Sprintf("DIUN_HOSTNAME=%s", c.opts.Meta.Hostname),
fmt.Sprintf("DIUN_ENTRY_STATUS=%s", string(c.opts.Entry.Status)), fmt.Sprintf("DIUN_ENTRY_STATUS=%s", string(c.opts.Entry.Status)),
@@ -128,5 +134,5 @@ func (c *Client) RenderEnv() []string {
fmt.Sprintf("DIUN_ENTRY_DIGEST=%s", c.opts.Entry.Manifest.Digest), fmt.Sprintf("DIUN_ENTRY_DIGEST=%s", c.opts.Entry.Manifest.Digest),
fmt.Sprintf("DIUN_ENTRY_CREATED=%s", c.opts.Entry.Manifest.Created), fmt.Sprintf("DIUN_ENTRY_CREATED=%s", c.opts.Entry.Manifest.Created),
fmt.Sprintf("DIUN_ENTRY_PLATFORM=%s", c.opts.Entry.Manifest.Platform), fmt.Sprintf("DIUN_ENTRY_PLATFORM=%s", c.opts.Entry.Manifest.Platform),
} }, metadataEnvs...)
} }

51
internal/provider/common.go
View File

@@ -2,21 +2,29 @@ package provider
import ( import (
"fmt" "fmt"
"regexp"
"strconv" "strconv"
"strings" "strings"
"github.com/containerd/containerd/platforms" "github.com/containerd/containerd/platforms"
"github.com/crazy-max/diun/v4/internal/model" "github.com/crazy-max/diun/v4/internal/model"
"github.com/crazy-max/diun/v4/pkg/registry" "github.com/crazy-max/diun/v4/pkg/registry"
"github.com/pkg/errors"
)
var (
metadataKeyChars = `a-zA-Z0-9_`
metadataKeyRegexp = regexp.MustCompile(`^[` + metadataKeyChars + `]+$`)
) )
// ValidateImage returns a standard image through Docker labels // ValidateImage returns a standard image through Docker labels
func ValidateImage(image string, labels map[string]string, watchByDef bool) (img model.Image, err error) { func ValidateImage(image string, metadata, labels map[string]string, watchByDef bool) (img model.Image, err error) {
if i := strings.Index(image, "@sha256:"); i > 0 { if i := strings.Index(image, "@sha256:"); i > 0 {
image = image[:i] image = image[:i]
} }
img = model.Image{ img = model.Image{
Name: image, Name: image,
Metadata: metadata,
NotifyOn: model.NotifyOnDefaults, NotifyOn: model.NotifyOnDefaults,
SortTags: registry.SortTagReverse, SortTags: registry.SortTagReverse,
} }
@@ -34,14 +42,14 @@ func ValidateImage(image string, labels map[string]string, watchByDef bool) (img
} }
for key, value := range labels { for key, value := range labels {
switch key { switch {
case "diun.regopt": case key == "diun.regopt":
img.RegOpt = value img.RegOpt = value
case "diun.watch_repo": case key == "diun.watch_repo":
if img.WatchRepo, err = strconv.ParseBool(value); err != nil { if img.WatchRepo, err = strconv.ParseBool(value); err != nil {
return img, fmt.Errorf("cannot parse %s value of label %s", value, key) return img, fmt.Errorf("cannot parse %s value of label %s", value, key)
} }
case "diun.notify_on": case key == "diun.notify_on":
if len(value) == 0 { if len(value) == 0 {
break break
} }
@@ -53,7 +61,7 @@ func ValidateImage(image string, labels map[string]string, watchByDef bool) (img
} }
img.NotifyOn = append(img.NotifyOn, notifyOn) img.NotifyOn = append(img.NotifyOn, notifyOn)
} }
case "diun.sort_tags": case key == "diun.sort_tags":
if value == "" { if value == "" {
break break
} }
@@ -62,19 +70,19 @@ func ValidateImage(image string, labels map[string]string, watchByDef bool) (img
return img, fmt.Errorf("unknown sort tags type %q", value) return img, fmt.Errorf("unknown sort tags type %q", value)
} }
img.SortTags = sortTags img.SortTags = sortTags
case "diun.max_tags": case key == "diun.max_tags":
if img.MaxTags, err = strconv.Atoi(value); err != nil { if img.MaxTags, err = strconv.Atoi(value); err != nil {
return img, fmt.Errorf("cannot parse %s value of label %s", value, key) return img, fmt.Errorf("cannot parse %s value of label %s", value, key)
} }
case "diun.include_tags": case key == "diun.include_tags":
img.IncludeTags = strings.Split(value, ";") img.IncludeTags = strings.Split(value, ";")
case "diun.exclude_tags": case key == "diun.exclude_tags":
img.ExcludeTags = strings.Split(value, ";") img.ExcludeTags = strings.Split(value, ";")
case "diun.hub_tpl": case key == "diun.hub_tpl":
img.HubTpl = value img.HubTpl = value
case "diun.hub_link": case key == "diun.hub_link":
img.HubLink = value img.HubLink = value
case "diun.platform": case key == "diun.platform":
platform, err := platforms.Parse(value) platform, err := platforms.Parse(value)
if err != nil { if err != nil {
return img, fmt.Errorf("cannot parse %s platform of label %s", value, key) return img, fmt.Errorf("cannot parse %s platform of label %s", value, key)
@@ -84,8 +92,27 @@ func ValidateImage(image string, labels map[string]string, watchByDef bool) (img
Arch: platform.Architecture, Arch: platform.Architecture,
Variant: platform.Variant, Variant: platform.Variant,
} }
case strings.HasPrefix(key, "diun.metadata."):
mkey := strings.TrimPrefix(key, "diun.metadata.")
if len(mkey) == 0 || len(value) == 0 {
break
}
if err := validateMetadataKey(mkey); err != nil {
return img, errors.Wrapf(err, "invalid metadata key %q", mkey)
}
if img.Metadata == nil {
img.Metadata = map[string]string{}
}
img.Metadata[mkey] = value
} }
} }
return img, nil return img, nil
} }
func validateMetadataKey(key string) error {
if !metadataKeyRegexp.MatchString(key) {
return errors.Errorf("only %q are allowed", metadataKeyChars)
}
return nil
}

37
internal/provider/docker/container.go
View File

@@ -1,12 +1,17 @@
package docker package docker
import ( import (
"fmt"
"reflect" "reflect"
"strings"
"time"
"github.com/crazy-max/diun/v4/internal/model" "github.com/crazy-max/diun/v4/internal/model"
"github.com/crazy-max/diun/v4/internal/provider" "github.com/crazy-max/diun/v4/internal/provider"
"github.com/crazy-max/diun/v4/pkg/docker" "github.com/crazy-max/diun/v4/pkg/docker"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
"github.com/docker/go-units"
) )
func (c *Client) listContainerImage() []model.Image { func (c *Client) listContainerImage() []model.Image {
@@ -86,7 +91,7 @@ func (c *Client) listContainerImage() []model.Image {
Str("ctn_image", imageName). Str("ctn_image", imageName).
Interface("ctn_labels", ctn.Labels). Interface("ctn_labels", ctn.Labels).
Msg("Validate image") Msg("Validate image")
image, err := provider.ValidateImage(imageName, ctn.Labels, *c.config.WatchByDefault) image, err := provider.ValidateImage(imageName, metadata(ctn), ctn.Labels, *c.config.WatchByDefault)
if err != nil { if err != nil {
c.logger.Error().Err(err). c.logger.Error().Err(err).
@@ -109,3 +114,33 @@ func (c *Client) listContainerImage() []model.Image {
return list return list
} }
func metadata(ctn types.Container) map[string]string {
return map[string]string{
"ctn_id": ctn.ID,
"ctn_names": formatNames(ctn.Names),
"ctn_command": ctn.Command,
"ctn_createdat": time.Unix(ctn.Created, 0).String(),
"ctn_state": ctn.State,
"ctn_status": ctn.Status,
"ctn_size": formatSize(ctn.SizeRw, ctn.SizeRootFs),
}
}
func formatNames(names []string) string {
namesStripPrefix := make([]string, len(names))
for i, s := range names {
namesStripPrefix[i] = s[1:]
}
return strings.Join(namesStripPrefix, ",")
}
func formatSize(sizeRw, sizeRootFs int64) string {
srw := units.HumanSizeWithPrecision(float64(sizeRw), 3)
sv := units.HumanSizeWithPrecision(float64(sizeRootFs), 3)
sf := srw
if sizeRootFs > 0 {
sf = fmt.Sprintf("%s (virtual %s)", srw, sv)
}
return sf
}

2
internal/provider/dockerfile/image.go
View File

@@ -32,7 +32,7 @@ func (c *Client) listExtImage() (list []model.Image) {
Interface("dfile_comments", fromImage.Comments). Interface("dfile_comments", fromImage.Comments).
Int("dfile_line", fromImage.Line). Int("dfile_line", fromImage.Line).
Msg("Validate image") Msg("Validate image")
image, err := provider.ValidateImage(fromImage.Name, c.extractLabels(fromImage.Comments), true) image, err := provider.ValidateImage(fromImage.Name, nil, c.extractLabels(fromImage.Comments), true)
if err != nil { if err != nil {
c.logger.Error().Err(err). c.logger.Error().Err(err).
Str("dfile_image", fromImage.Name). Str("dfile_image", fromImage.Name).

15
internal/provider/kubernetes/pod.go
View File

@@ -2,10 +2,12 @@ package kubernetes
import ( import (
"reflect" "reflect"
"strings"
"github.com/crazy-max/diun/v4/internal/model" "github.com/crazy-max/diun/v4/internal/model"
"github.com/crazy-max/diun/v4/internal/provider" "github.com/crazy-max/diun/v4/internal/provider"
"github.com/crazy-max/diun/v4/pkg/k8s" "github.com/crazy-max/diun/v4/pkg/k8s"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
) )
@@ -39,7 +41,7 @@ func (c *Client) listPodImage() []model.Image {
Str("ctn_image", ctn.Image). Str("ctn_image", ctn.Image).
Msg("Validate image") Msg("Validate image")
image, err := provider.ValidateImage(ctn.Image, pod.Annotations, *c.config.WatchByDefault) image, err := provider.ValidateImage(ctn.Image, metadata(pod, ctn), pod.Annotations, *c.config.WatchByDefault)
if err != nil { if err != nil {
c.logger.Error().Err(err). c.logger.Error().Err(err).
Str("pod_name", pod.Name). Str("pod_name", pod.Name).
@@ -64,3 +66,14 @@ func (c *Client) listPodImage() []model.Image {
return list return list
} }
func metadata(pod v1.Pod, ctn v1.Container) map[string]string {
return map[string]string{
"pod_name": pod.Name,
"pod_status": pod.Status.String(),
"pod_namespace": pod.Namespace,
"pod_createdat": pod.CreationTimestamp.String(),
"ctn_name": ctn.Name,
"ctn_command": strings.Join(ctn.Command, " "),
}
}

15
internal/provider/nomad/task.go
View File

@@ -96,7 +96,7 @@ func (c *Client) listTaskImages() []model.Image {
// Finally, merge task meta values // Finally, merge task meta values
labels = updateMap(labels, task.Meta) labels = updateMap(labels, task.Meta)
image, err := provider.ValidateImage(imageName, labels, *c.config.WatchByDefault) image, err := provider.ValidateImage(imageName, metadata(job, taskGroup, task), labels, *c.config.WatchByDefault)
if err != nil { if err != nil {
c.logger.Error(). c.logger.Error().
Err(err). Err(err).
@@ -124,3 +124,16 @@ func (c *Client) listTaskImages() []model.Image {
return list return list
} }
func metadata(job *nomad.JobListStub, taskGroup *nomad.TaskGroup, task *nomad.Task) map[string]string {
return map[string]string{
"job_id": job.ID,
"job_name": job.Name,
"job_status": job.Status,
"job_namespace": job.Namespace,
"taskgroup_name": *taskGroup.Name,
"task_name": task.Name,
"task_driver": task.Driver,
"task_user": task.User,
}
}

12
internal/provider/swarm/service.go
View File

@@ -7,6 +7,7 @@ import (
"github.com/crazy-max/diun/v4/internal/provider" "github.com/crazy-max/diun/v4/internal/provider"
"github.com/crazy-max/diun/v4/pkg/docker" "github.com/crazy-max/diun/v4/pkg/docker"
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/swarm"
) )
func (c *Client) listServiceImage() []model.Image { func (c *Client) listServiceImage() []model.Image {
@@ -36,7 +37,7 @@ func (c *Client) listServiceImage() []model.Image {
Str("ctn_image", svc.Spec.TaskTemplate.ContainerSpec.Image). Str("ctn_image", svc.Spec.TaskTemplate.ContainerSpec.Image).
Msg("Validate image") Msg("Validate image")
image, err := provider.ValidateImage(svc.Spec.TaskTemplate.ContainerSpec.Image, svc.Spec.Labels, *c.config.WatchByDefault) image, err := provider.ValidateImage(svc.Spec.TaskTemplate.ContainerSpec.Image, metadata(svc), svc.Spec.Labels, *c.config.WatchByDefault)
if err != nil { if err != nil {
c.logger.Error().Err(err). c.logger.Error().Err(err).
Str("svc_name", svc.Spec.Name). Str("svc_name", svc.Spec.Name).
@@ -58,3 +59,12 @@ func (c *Client) listServiceImage() []model.Image {
return list return list
} }
func metadata(svc swarm.Service) map[string]string {
return map[string]string{
"svc_id": svc.ID,
"svc_createdat": svc.CreatedAt.String(),
"svc_updatedat": svc.UpdatedAt.String(),
"ctn_name": svc.Spec.Name,
}
}

1
test/dockerfile1/mount/Dockerfile
View File

@@ -11,6 +11,7 @@ COPY --from=crazymax/yasu / /
# diun.watch_repo=true # diun.watch_repo=true
# diun.include_tags=^\d+\.\d+\.\d+$ # diun.include_tags=^\d+\.\d+\.\d+$
# diun.platform=linux/amd64 # diun.platform=linux/amd64
# diun.metadata.foo=bar
RUN --mount=type=bind,target=.,rw \ RUN --mount=type=bind,target=.,rw \
--mount=type=bind,from=docker/buildx-bin:0.6.0,source=/buildx,target=/usr/bin/buildx \ --mount=type=bind,from=docker/buildx-bin:0.6.0,source=/buildx,target=/usr/bin/buildx \
yasu --version yasu --version