From 75eea45fdb5c0a5d3f0245c51a352101cc67f536 Mon Sep 17 00:00:00 2001 From: CrazyMax <1951866+crazy-max@users.noreply.github.com> Date: Thu, 29 Apr 2021 01:03:46 +0200 Subject: [PATCH] Note about watch_repo setting (#348) Co-authored-by: CrazyMax --- docs/faq.md | 5 +- docs/install/docker.md | 45 +++++- docs/install/podman.md | 186 ---------------------- docs/providers/docker.md | 14 +- docs/providers/file.md | 5 +- docs/providers/kubernetes.md | 30 ++-- docs/providers/swarm.md | 11 +- docs/user-guides/docker-file-providers.md | 4 +- mkdocs.yml | 1 - 9 files changed, 82 insertions(+), 219 deletions(-) delete mode 100644 docs/install/podman.md diff --git a/docs/faq.md b/docs/faq.md index 349e7290..cff80357 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -47,7 +47,7 @@ should be: ## No image found in manifest list for architecture, variant, OS -If you encounter this kind of warning, you are probably using the [file provider](providers/file.md) containing an +If you encounter this kind of warning, you are probably using the [file provider](providers/file.md) for an image with an erroneous or empty platform. If the platform is not filled in, it will be deduced automatically from the information of your operating system on which Diun is running. @@ -122,6 +122,9 @@ regopts: Or you can tweak the [`schedule` setting](config/watch.md#schedule) with something like `0 */6 * * *` (every 6 hours). +!!! warning + Also be careful with the `watch_repo` setting as it will fetch manifest for **ALL** tags available for the image. + ## Profiling Diun provides a simple way to manage runtime/pprof profiling through [`--profiler-path` and `--profiler` flags](usage/cli.md#options): diff --git a/docs/install/docker.md b/docs/install/docker.md index 5c9640ff..e29e00ab 100644 --- a/docs/install/docker.md +++ b/docs/install/docker.md @@ -6,7 +6,7 @@ Diun provides automatically updated Docker :whale: images within several registr | Registry | Image | |--------------------------------------------------------------------------------------------------|---------------------------------| -| [Docker Hub](https://hub.docker.com/r/crazymax/diun/) | `crazymax/diun` | +| [Docker Hub](https://hub.docker.com/r/crazymax/diun/) | `crazymax/diun` | | [GitHub Container Registry](https://github.com/users/crazy-max/packages/container/package/diun) | `ghcr.io/crazy-max/diun` | It is possible to always use the latest stable tag or to use another service that handles updating Docker images. @@ -60,16 +60,12 @@ services: - "DIUN_WATCH_WORKERS=20" - "DIUN_WATCH_SCHEDULE=0 */6 * * *" - "DIUN_PROVIDERS_DOCKER=true" - - "DIUN_PROVIDERS_DOCKER_WATCHSTOPPED=true" labels: - "diun.enable=true" - "diun.watch_repo=true" restart: always ``` -!!! note - If you are running [Podman](https://podman.io/), please check the appropriate [page](./podman.md) - Edit this example with your preferences and run the following commands to bring up Diun: ```shell @@ -87,7 +83,6 @@ docker run -d --name diun \ -e "DIUN_WATCH_WORKERS=20" \ -e "DIUN_WATCH_SCHEDULE=0 */6 * * *" \ -e "DIUN_PROVIDERS_DOCKER=true" \ - -e "DIUN_PROVIDERS_DOCKER_WATCHSTOPPED=true" \ -v "$(pwd)/data:/data" \ -v "/var/run/docker.sock:/var/run/docker.sock" \ -l "diun.enable=true" \ @@ -132,3 +127,41 @@ providers: docker: watchStopped: true ``` + +## Podman + +If you're using [Podman](https://podman.io/) there are some small changes: + +```yaml +version: "3.5" + +services: + diun_app: + image: docker.io/crazymax/diun:latest + container_name: diun + volumes: + - "./data:/data" + # root mode + - "/run/podman/podman.sock:/var/run/docker.sock" + # rootless mode + #- "$XDG_RUNTIME_DIR/podman/podman.sock:/var/run/docker.sock" + #- "/run/user/1000/podman/podman.sock:/var/run/docker.sock" + environment: + - "TZ=Europe/Paris" + - "LOG_LEVEL=info" + - "LOG_JSON=false" + - "DIUN_WATCH_WORKERS=20" + - "DIUN_WATCH_SCHEDULE=0 */6 * * *" + - "DIUN_PROVIDERS_DOCKER=true" + labels: + - "diun.enable=true" + - "diun.watch_repo=true" + restart: always +``` + +Edit this example with your preferences and run the following commands to bring up Diun with [`podman-compose`](https://github.com/containers/podman-compose): + +```shell +podman-compose up -d +podman-compose logs -f +``` diff --git a/docs/install/podman.md b/docs/install/podman.md deleted file mode 100644 index c2e0bc20..00000000 --- a/docs/install/podman.md +++ /dev/null @@ -1,186 +0,0 @@ -# Installation with Podman - -## About - -[Podman](https://podman.io) is now the default container runtime in some, *mainly RPM-based*, Linux distributions like RHEL. - -Podman needs only a config tweak to work with diun as it is backwards-compatible with the Docker API. - -## Prerequisites - -!!! warning - Package `podman-docker` is not available in debian-based environment, only native method is supported - -First of all you need to check if you have the `podman-docker` package installed with the following command: - -```shell -rpm -q podman-docker -``` - -### Native podman method - -!!! warning - Only install via `podman-compose`/`podman run` will work, due to file location change between podman and docker - -!!! note - You need [Podman Compose](https://github.com/containers/podman-compose) to process `docker-compose` files - -Enable and start the socket: - -```shell -# monitoring root instance -sudo systemctl enable podman.socket -sudo systemctl start podman.socket -# monitoring user instance -systemctl enable --user podman.socket -systemctl start --user podman.socket - -Without `podman-docker`, you need to modfiy the socket location - -Replace this line in `docker-compose.yml` or in `podman run` *(keep the `-v`)* : - -```yaml - - "/var/run/docker.sock:/var/run/docker.sock" -``` - -By *(without `-` on `podman run`)* : - -```yaml -# When monitoring root instance : - - "/run/podman/podman.sock:/var/run/docker.sock" -# When monitoring rootless podman : - - "$XDG_RUNTIME_DIR/podman/podman.sock:/var/run/docker.sock" - # You need to modify $XDG_RUNTIME_DIR by the real value of the variable - # for the given user -``` - -`podman-compose` requires you to modify the container name to correctly start the pod - -```diff -- container_name: diun -+ container_name: diun_app #or any name -``` - -Replacing the image name by the qualified name of the image is advised - -```diff -- crazymax/diun:latest -+ docker.io/crazymax/diun:latest -``` - -#### Examples - -##### Rootful (`root` user) - - -??? example "rootful docker-compose.yml" - ```yaml - version: "3.5" - - services: - diun: - image: docker.io/crazymax/diun:latest - container_name: diun_app - volumes: - - "./data:/data" - - "/run/podman/podman.sock:/var/run/docker.sock" - environment: - - "TZ=Europe/Paris" - - "LOG_LEVEL=info" - - "LOG_JSON=false" - - "DIUN_WATCH_WORKERS=20" - - "DIUN_WATCH_SCHEDULE=0 */6 * * *" - - "DIUN_PROVIDERS_DOCKER=true" - - "DIUN_PROVIDERS_DOCKER_WATCHSTOPPED=true" - labels: - - "diun.enable=true" - - "diun.watch_repo=true" - restart: always - ``` - -??? example "rootful podman run" - ```shell - podman run -d --name diun \ - -e "TZ=Europe/Paris" \ - -e "LOG_LEVEL=info" \ - -e "LOG_JSON=false" \ - -e "DIUN_WATCH_WORKERS=20" \ - -e "DIUN_WATCH_SCHEDULE=0 */6 * * *" \ - -e "DIUN_PROVIDERS_DOCKER=true" \ - -e "DIUN_PROVIDERS_DOCKER_WATCHSTOPPED=true" \ - -v "$(pwd)/data:/data" \ - -v "/run/podman/podman.sock:/var/run/docker.sock" \ - -l "diun.enable=true" \ - -l "diun.watch_repo=true" \ - docker.io/crazymax/diun:latest - ``` - -##### Rootless - -Grab the value of `$XDG_RUNTIME_DIR` - -```shell -echo $XDG_RUNTIME_DIR -# Usually /run/user/1000, but can vary -``` - -With `/run/user/1000` as an example - - -??? example "rootless docker-compose.yml" - ```yaml - version: "3.5" - - services: - diun: - image: docker.io/crazymax/diun:latest - container_name: diun_app - volumes: - - "./data:/data" - - "/run/user/1000/podman/podman.sock:/var/run/docker.sock" - environment: - - "TZ=Europe/Paris" - - "LOG_LEVEL=info" - - "LOG_JSON=false" - - "DIUN_WATCH_WORKERS=20" - - "DIUN_WATCH_SCHEDULE=0 */6 * * *" - - "DIUN_PROVIDERS_DOCKER=true" - - "DIUN_PROVIDERS_DOCKER_WATCHSTOPPED=true" - labels: - - "diun.enable=true" - - "diun.watch_repo=true" - restart: always - ``` - -??? example "rootless podman run" - ```shell - podman run -d --name diun \ - -e "TZ=Europe/Paris" \ - -e "LOG_LEVEL=info" \ - -e "LOG_JSON=false" \ - -e "DIUN_WATCH_WORKERS=20" \ - -e "DIUN_WATCH_SCHEDULE=0 */6 * * *" \ - -e "DIUN_PROVIDERS_DOCKER=true" \ - -e "DIUN_PROVIDERS_DOCKER_WATCHSTOPPED=true" \ - -v "$(pwd)/data:/data" \ - -v "/run/user/1000/podman/podman.sock:/var/run/docker.sock" \ - -l "diun.enable=true" \ - -l "diun.watch_repo=true" \ - docker.io/crazymax/diun:latest - ``` - -### Podman-docker method - -!!! warning - If you have this package installed, you will only be able to monitor `root` podman instance. - You will **NOT** be able to monitor any user *rootless* instance. - Therefore this is not the recommanded method. - -Enable and start the socket with : - -```shell -sudo systemctl enable podman.socket -sudo systemctl start podman.socket -``` - -And you're good to go with default configuration and binary install if needed diff --git a/docs/providers/docker.md b/docs/providers/docker.md index acbaa550..87d2e003 100644 --- a/docs/providers/docker.md +++ b/docs/providers/docker.md @@ -2,13 +2,15 @@ ## About -The Docker provider allows you to analyze the containers of your Docker instance to extract images found and check for updates on the registry. +The Docker provider allows you to analyze the containers of your Docker instance to extract images found and check +for updates on the registry. ## Quick start In this section we quickly go over a basic docker-compose file using your local docker provider. -Here we use a single Docker provider with a minimum configuration to analyze labeled containers (watch by default disabled), even stopped ones, of your local Docker instance. +Here we use a single Docker provider with a minimum configuration to analyze labeled containers +(watch by default disabled), even stopped ones, of your local Docker instance. Now let's create a simple docker-compose file with Diun and some simple services: @@ -28,7 +30,6 @@ services: - "DIUN_WATCH_WORKERS=20" - "DIUN_WATCH_SCHEDULE=0 */6 * * *" - "DIUN_PROVIDERS_DOCKER=true" - - "DIUN_PROVIDERS_DOCKER_WATCHSTOPPED=true" restart: always cloudflared: @@ -45,11 +46,12 @@ services: - "TUNNEL_DNS_UPSTREAM=https://1.1.1.1/dns-query,https://1.0.0.1/dns-query" labels: - "diun.enable=true" - - "diun.watch_repo=true" restart: always ``` -As an example we use [crazymax/cloudflared:latest](https://github.com/crazy-max/docker-cloudflared) Docker image. A few [labels](#docker-labels) are added to configure the image analysis of this container for Diun. Now start this composition with `docker-composes up -d` and take a look at the logs: +As an example we use [crazymax/cloudflared:latest](https://github.com/crazy-max/docker-cloudflared) Docker image. +A few [labels](#docker-labels) are added to configure the image analysis of this container for Diun. Now start this +composition with `docker-composes up -d` and take a look at the logs: ``` $ docker-compose logs -f @@ -171,7 +173,7 @@ You can configure more finely the way to analyze the image of your container thr |-------------------------------|---------------|---------------| | `diun.enable` | | Set to true to enable image analysis of this container | | `diun.regopt` | | [Registry options](../config/regopts.md) name to use | -| `diun.watch_repo` | `false` | Watch all tags of this container image | +| `diun.watch_repo` | `false` | Watch all tags of this container image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them | | `diun.include_tags` | | Semi-colon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semi-colon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | diff --git a/docs/providers/file.md b/docs/providers/file.md index 2ca70205..b215c16d 100644 --- a/docs/providers/file.md +++ b/docs/providers/file.md @@ -108,7 +108,8 @@ providers: - name: docker.bintray.io/jfrog/xray-mongo:3.2.6 ``` -Here we want to analyze all tags of `crazymax/cloudflared` and `docker.bintray.io/jfrog/xray-mongo:3.2.6` tag. Now let's start Diun: +Here we want to analyze all tags of `crazymax/cloudflared` and `docker.bintray.io/jfrog/xray-mongo:3.2.6` tag. +Now let's start Diun: ``` $ diun --config diun.yml @@ -173,7 +174,7 @@ The configuration file(s) defines a slice of images to analyze with the followin |-------------------------------|----------------------------------|---------------| | `name` | `latest` | Docker image name to watch using `registry/path:tag` format. If registry omitted, `docker.io` will be used and if tag omitted, `latest` will be used | | `regopt` | | [Registry options](../config/regopts.md) name to use | -| `watch_repo` | `false` | Watch all tags of this image | +| `watch_repo` | `false` | Watch all tags of this image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `max_tags` | `0` | Maximum number of tags to watch if `watch_repo` enabled. `0` means all of them | | `include_tags` | | List of regular expressions to include tags. Can be useful if you enable `watch_repo` | | `exclude_tags` | | List of regular expressions to exclude tags. Can be useful if you enable `watch_repo` | diff --git a/docs/providers/kubernetes.md b/docs/providers/kubernetes.md index 941d5041..7a1298c8 100644 --- a/docs/providers/kubernetes.md +++ b/docs/providers/kubernetes.md @@ -2,13 +2,15 @@ ## About -The Kubernetes provider allows you to analyze the pods of your Kubernetes cluster to extract images found and check for updates on the registry. +The Kubernetes provider allows you to analyze the pods of your Kubernetes cluster to extract images found and check +for updates on the registry. ## Quick start In this section we quickly go over a basic deployment using your local Kubernetes cluster. -Here we use our local Kubernetes provider with a minimum configuration to analyze annotated pods (watch by default disabled). +Here we use our local Kubernetes provider with a minimum configuration to analyze annotated pods (watch by default +disabled). Now let's create a simple pod for Diun: @@ -115,7 +117,6 @@ spec: run: nginx annotations: diun.enable: "true" - diun.watch_repo: "true" spec: containers: - name: nginx @@ -124,7 +125,8 @@ spec: - containerPort: 80 ``` -As an example we use [nginx](https://hub.docker.com/_/nginx/) Docker image. A few [annotations](#kubernetes-annotations) are added to configure the image analysis of this pod for Diun. We can now start these 2 pods: +As an example we use [nginx](https://hub.docker.com/_/nginx/) Docker image. A few [annotations](#kubernetes-annotations) +are added to configure the image analysis of this pod for Diun. We can now start these 2 pods: ``` kubectl apply -f diun.yml @@ -172,13 +174,20 @@ The Kubernetes server endpoint as URL. !!! abstract "Environment variables" * `DIUN_PROVIDERS_KUBERNETES_ENDPOINT` -Kubernetes server endpoint as URL, which is only used when the behavior based on environment variables described below does not apply. +Kubernetes server endpoint as URL, which is only used when the behavior based on environment variables described below +does not apply. -When deployed into Kubernetes, Diun reads the environment variables `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` or `KUBECONFIG` to create the endpoint. +When deployed into Kubernetes, Diun reads the environment variables `KUBERNETES_SERVICE_HOST` and +`KUBERNETES_SERVICE_PORT` or `KUBECONFIG` to create the endpoint. -The access token is looked up in `/var/run/secrets/kubernetes.io/serviceaccount/token` and the SSL CA certificate in `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`. They are both provided automatically as mounts in the pod where Diun is deployed. +The access token is looked up in `/var/run/secrets/kubernetes.io/serviceaccount/token` and the SSL CA certificate +in `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`. They are both provided automatically as mounts in the +pod where Diun is deployed. -When the environment variables are not found, Diun tries to connect to the Kubernetes API server with an external-cluster client. In which case, the endpoint is required. Specifically, it may be set to the URL used by `kubectl proxy` to connect to a Kubernetes cluster using the granted authentication and authorization of the associated kubeconfig. +When the environment variables are not found, Diun tries to connect to the Kubernetes API server with an +external-cluster client. In which case, the endpoint is required. Specifically, it may be set to the URL used +by `kubectl proxy` to connect to a Kubernetes cluster using the granted authentication and authorization of the +associated kubeconfig. ### `token` @@ -254,7 +263,8 @@ Array of namespaces to watch (default all namespaces). ### `watchByDefault` -Enable watch by default. If false, pods that don't have `diun.enable: "true"` annotation will be ignored (default `false`). +Enable watch by default. If false, pods that don't have `diun.enable: "true"` annotation will be ignored +(default `false`). !!! example "File" ```yaml @@ -274,7 +284,7 @@ You can configure more finely the way to analyze the image of your pods through |-------------------------------|---------------|---------------| | `diun.enable` | | Set to true to enable image analysis of this pod | | `diun.regopt` | | [Registry options](../config/regopts.md) name to use | -| `diun.watch_repo` | `false` | Watch all tags of this pod image | +| `diun.watch_repo` | `false` | Watch all tags of this pod image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them | | `diun.include_tags` | | Semi-colon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semi-colon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | diff --git a/docs/providers/swarm.md b/docs/providers/swarm.md index 76ba497b..f1629f50 100644 --- a/docs/providers/swarm.md +++ b/docs/providers/swarm.md @@ -2,13 +2,15 @@ ## About -The Swarm provider allows you to analyze the services of your Swarm cluster to extract images found and check for updates on the registry. +The Swarm provider allows you to analyze the services of your Swarm cluster to extract images found and check for +updates on the registry. ## Quick start In this section we quickly go over a basic stack using your local swarm cluster. -Here we use our local Swarm provider with a minimum configuration to analyze labeled containers (watch by default disabled). +Here we use our local Swarm provider with a minimum configuration to analyze labeled containers (watch by default +disabled). Now let's create a simple stack for Diun: @@ -56,7 +58,8 @@ services: - "diun.watch_repo=true" ``` -As an example we use [nginx](https://hub.docker.com/_/nginx/) Docker image. A few [labels](#docker-labels) are added to configure the image analysis of this service for Diun. We can now start these 2 stacks: +As an example we use [nginx](https://hub.docker.com/_/nginx/) Docker image. A few [labels](#docker-labels) are added +to configure the image analysis of this service for Diun. We can now start these 2 stacks: ``` docker stack deploy -c diun.yml diun @@ -175,7 +178,7 @@ You can configure more finely the way to analyze the image of your service throu |-------------------------------|---------------|---------------| | `diun.enable` | | Set to true to enable image analysis of this service | | `diun.regopt` | | [Registry options](../config/regopts.md) name to use | -| `diun.watch_repo` | `false` | Watch all tags of this service image | +| `diun.watch_repo` | `false` | Watch all tags of this service image ([be careful](../faq.md#docker-hub-rate-limits) with this setting) | | `diun.max_tags` | `0` | Maximum number of tags to watch if `diun.watch_repo` enabled. `0` means all of them | | `diun.include_tags` | | Semi-colon separated list of regular expressions to include tags. Can be useful if you enable `diun.watch_repo` | | `diun.exclude_tags` | | Semi-colon separated list of regular expressions to exclude tags. Can be useful if you enable `diun.watch_repo` | diff --git a/docs/user-guides/docker-file-providers.md b/docs/user-guides/docker-file-providers.md index 20455ec5..2ab4de16 100644 --- a/docs/user-guides/docker-file-providers.md +++ b/docs/user-guides/docker-file-providers.md @@ -35,15 +35,13 @@ services: ```yaml # /custom-images.yml - name: ghcr.io/crazy-max/diun - watch_repo: true - name: alpine - watch_repo: true - name: debian:stretch-slim - name: nginx:stable-alpine - name: traefik watch_repo: true include_tags: - - ^(0|[1-9]\d*)\..*-alpine + - ^(0|[1-9]\d*)\..* ``` Here we use a minimal configuration to analyze **all running containers** (watch by default enabled) of diff --git a/mkdocs.yml b/mkdocs.yml index 31410238..52960dbc 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -88,7 +88,6 @@ nav: - From Docker image: install/docker.md - From binary: install/binary.md - Linux service: install/linux-service.md - - Podman: install/podman.md - Usage: - Command line: usage/cli.md - Basic example: usage/basic-example.md