Release 2.7.0 - See CHANGELOG.md

Fixed #62

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,42 @@
+---
+name: Bug report
+about: If something isn't working right..
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+### Summary
+
+<!-- Summarize the bug encountered -->
+
+
+### Steps to reproduce
+
+<!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. -->
+
+
+### What is the expected *correct* behavior?
+
+<!-- Describe what should be seen instead. -->
+
+
+### Relevant logs and/or screenshots
+
+<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise. -->
+
+### Environment
+<!--Your Configuration (please complete the following information): -->
+
+- Image version / tag:  
+- Host OS:
+
+<details>
+<summary>Any logs | docker-compose.yml</summary>
+</details>
+
+<!-- Include anything additional -->
+    
+### Possible fixes
+<!-- If you can, provide details to the root cause that might be responsible for the problem. -->

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Suggest an idea or feature
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+---
+name: Feature Request
+about: Suggest an idea for this project
+
+---
+
+**Description of the feature**
+<!-- A clear description of the feature you'd like implemented  -->
+
+**Benftits of feature**
+<!-- Explain the measurable benefits this feature would achieve. -->
+
+**Additional context**
+<!--Add any other context or screenshots about the feature request here. -->

.github/workflows/main.yml

@@ -1,7 +1,7 @@
 ### Application Level Image CI
 ### Dave Conroy <dave at tiredofit dot ca>
 
-name: 'Build Images'
+name: 'build'
 
 on:
   push:

CHANGELOG.md

@@ -1,3 +1,15 @@
+## 2.7.0 2021-06-17 <dave at tiredofit dot ca>
+
+   ### Added
+      - MongoDB Authentication Database support (DB_AUTH)
+
+
+## 2.6.1 2021-06-08 <jwillmer@github>
+
+   ### Changed
+      - Fix for Issue #14 - SPLIT_DB=TRUE was not working for Postgres DB server
+
+
 ## 2.6.0 2021-02-19 <tpansino@github>
 
    ### Added

Dockerfile

@@ -63,5 +63,11 @@ RUN set -ex && \
     rm -rf /usr/src/* && \
     rm -rf /tmp/* /var/cache/apk/*
 
+RUN apk add --no-cache \
+               python3 \
+               py3-pip \
+          && pip3 install --upgrade pip \
+          && pip3 install awscli
+
 ### S6 Setup
     ADD install  /

README.md

@@ -1,13 +1,16 @@
-# hub.docker.com/r/tiredofit/db-backup
+# github.com/tiredofit/docker-db-backup
 
-[![Build Status](https://img.shields.io/docker/build/tiredofit/db-backup.svg)](https://hub.docker.com/r/tiredofit/db-backup)
-[![Docker Pulls](https://img.shields.io/docker/pulls/tiredofit/db-backup.svg)](https://hub.docker.com/r/tiredofit/db-backup)
-[![Docker Stars](https://img.shields.io/docker/stars/tiredofit/db-backup.svg)](https://hub.docker.com/r/tiredofit/db-backup)
-[![Docker Layers](https://images.microbadger.com/badges/image/tiredofit/db-backup.svg)](https://microbadger.com/images/tiredofit/db-backup)
+[![GitHub release](https://img.shields.io/github/v/tag/tiredofit/docker-db-backup?style=flat-square)](https://github.com/tiredofit/docker-db-backup/releases/latest)
+[![Build Status](https://img.shields.io/github/workflow/status/tiredofit/docker-db-backup/build?style=flat-square)](https://github.com/tiredofit/docker-db-backup/actions?query=workflow%3Abuild)
+[![Docker Stars](https://img.shields.io/docker/stars/tiredofit/db-backup.svg?style=flat-square&logo=docker)](https://hub.docker.com/r/tiredofit/db-backup/)
+[![Docker Pulls](https://img.shields.io/docker/pulls/tiredofit/db-backup.svg?style=flat-square&logo=docker)](https://hub.docker.com/r/tiredofit/db-backup/)
+[![Become a sponsor](https://img.shields.io/badge/sponsor-tiredofit-181717.svg?logo=github&style=flat-square)](https://github.com/sponsors/tiredofit)
+[![Paypal Donate](https://img.shields.io/badge/donate-paypal-00457c.svg?logo=paypal&style=flat-square)](https://www.paypal.me/tiredofit)
 
-## Introduction
+* * *
+## About
 
-This will build a container for backing up multiple type of DB Servers
+This will build a container for backing up multiple types of DB Servers
 
 Currently backs up CouchDB, InfluxDB, MySQL, MongoDB, Postgres, Redis servers.
 
@@ -22,53 +25,61 @@ Currently backs up CouchDB, InfluxDB, MySQL, MongoDB, Postgres, Redis servers.
 * select when to start the first dump, whether time of day or relative to container start time
 * Execute script after backup for monitoring/alerting purposes
 
-* This Container uses a [customized Alpine Linux base](https://hub.docker.com/r/tiredofit/alpine) which includes [s6 overlay](https://github.com/just-containers/s6-overlay) enabled for PID 1 Init capabilities, [zabbix-agent](https://zabbix.org) for individual container monitoring, Cron also installed along with other tools (bash,curl, less, logrotate, nano, vim) for easier management. It also supports sending to external SMTP servers.
-
-[Changelog](CHANGELOG.md)
-
-## Authors
+## Maintainer
 
 - [Dave Conroy](https://github.com/tiredofit)
 
 ## Table of Contents
 
-- [hub.docker.com/r/tiredofit/db-backup](#hubdockercomrtiredofitdb-backup)
-  - [Introduction](#introduction)
-  - [Authors](#authors)
-  - [Table of Contents](#table-of-contents)
-  - [Prerequisites](#prerequisites)
-  - [Installation](#installation)
-    - [Quick Start](#quick-start)
-  - [Configuration](#configuration)
-    - [Data-Volumes](#data-volumes)
-    - [Environment Variables](#environment-variables)
-  - [Maintenance](#maintenance)
-    - [Shell Access](#shell-access)
-      - [Custom Scripts](#custom-scripts)
-  - [Example Post Script](#example-post-script)
-  - [$1=EXIT_CODE (After running backup routine)](#1exit_code-after-running-backup-routine)
-  - [$2=DB_TYPE (Type of Backup)](#2db_type-type-of-backup)
-  - [$3=DB_HOST (Backup Host)](#3db_host-backup-host)
-  - [#4=DB_NAME (Name of Database backed up](#4db_name-name-of-database-backed-up)
-  - [$5=DATE (Date of Backup)](#5date-date-of-backup)
-  - [$6=TIME (Time of Backup)](#6time-time-of-backup)
-  - [$7=BACKUP_FILENAME (Filename of Backup)](#7backup_filename-filename-of-backup)
-  - [$8=FILESIZE (Filesize of backup)](#8filesize-filesize-of-backup)
-  - [$9=MD5_RESULT (MD5Sum if enabled)](#9md5_result-md5sum-if-enabled)
+- [About](#about)
+- [Maintainer](#maintainer)
+- [Table of Contents](#table-of-contents)
+- [Prerequisites and Assumptions](#prerequisites-and-assumptions)
+- [Installation](#installation)
+  - [Build from Source](#build-from-source)
+  - [Prebuilt Images](#prebuilt-images)
+- [Configuration](#configuration)
+  - [Quick Start](#quick-start)
+  - [Persistent Storage](#persistent-storage)
+  - [Environment Variables](#environment-variables)
+    - [Base Images used](#base-images-used)
+    - [Backing Up to S3 Compatible Services](#backing-up-to-s3-compatible-services)
+- [Maintenance](#maintenance)
+  - [Shell Access](#shell-access)
+  - [Manual Backups](#manual-backups)
+  - [Custom Scripts](#custom-scripts)
+- [Support](#support)
+  - [Usage](#usage)
+  - [Bugfixes](#bugfixes)
+  - [Feature Requests](#feature-requests)
+  - [Updates](#updates)
+- [License](#license)
 
-## Prerequisites
+## Prerequisites and Assumptions
 
 You must have a working DB server or container available for this to work properly, it does not provide server functionality!
 
 ## Installation
 
-Automated builds of the image are available on [Docker Hub](https://hub.docker.com/r/tiredofit/db-backup) and is the recommended
-method of installation.
+### Build from Source
+Clone this repository and build the image with `docker build -t (imagename) .`
+
+### Prebuilt Images
+Builds of the image are available on [Docker Hub](https://hub.docker.com/r/tiredofit/db-backup) and is the recommended method of installation.
 
 ```bash
-docker pull tiredofit/db-backup:latest
+docker pull tiredofit/db-backup:(imagetag)
 ```
 
+The following image tags are available along with their tagged release based on what's written in the [Changelog](CHANGELOG.md):
+
+| Container OS | Tag       |
+| ------------ | --------- |
+| Alpine       | `:latest` |
+
+
+## Configuration
+
 ### Quick Start
 
 * The quickest way to get started is using [docker-compose](https://docs.docker.com/compose/). See the examples folder for a working [docker-compose.yml](examples/docker-compose.yml) that can be modified for development or production use.
@@ -77,10 +88,7 @@ docker pull tiredofit/db-backup:latest
 * Map [persistent storage](#data-volumes) for access to configuration and data files for backup.
 
 > **NOTE**: If you are using this with a docker-compose file along with a seperate SQL container, take care not to set the variables to backup immediately, more so have it delay execution for a minute, otherwise you will get a failed first backup.
-
-## Configuration
-
-### Data-Volumes
+### Persistent Storage
 
 The following directories are used for configuration and can be mapped for persistent storage.
 
@@ -88,16 +96,25 @@ The following directories are used for configuration and can be mapped for persi
 | ------------------------ | ---------------------------------------------------------------------------------- |
 | `/backup`                | Backups                                                                            |
 | `/assets/custom-scripts` | *Optional* Put custom scripts in this directory to execute after backup operations |
-
 ### Environment Variables
 
-Along with the Environment Variables from the [Base image](https://hub.docker.com/r/tiredofit/alpine), below is the complete list of available options that can be used to customize your installation.
+#### Base Images used
+
+This image relies on an [Alpine Linux](https://hub.docker.com/r/tiredofit/alpine) base image that relies on an [init system](https://github.com/just-containers/s6-overlay) for added capabilities. Outgoing SMTP capabilities are handlded via `msmtp`. Individual container performance monitoring is performed by [zabbix-agent](https://zabbix.org). Additional tools include: `bash`,`curl`,`less`,`logrotate`, `nano`,`vim`.
+
+Be sure to view the following repositories to understand all the customizable options:
+
+| Image                                                  | Description                            |
+| ------------------------------------------------------ | -------------------------------------- |
+| [OS Base](https://github.com/tiredofit/docker-alpine/) | Customized Image based on Alpine Linux |
+
 
 | Parameter              | Description                                                                                                                                                                                        |
 | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `BACKUP_LOCATION`      | Backup to `FILESYSTEM` or `S3` compatible services like S3, Minio, Wasabi - Default `FILESYSTEM`                                                                                                   |
 | `COMPRESSION`          | Use either Gzip `GZ`, Bzip2 `BZ`, XZip `XZ`, ZSTD `ZSTD` or none `NONE` - Default `GZ`                                                                                                             |
 | `COMPRESSION_LEVEL`    | Numberical value of what level of compression to use, most allow `1` to `9` except for `ZSTD` which allows for `1` to `19` - Default `3`                                                           |
+| `DB_AUTH`              | (Mongo Only - Optional) Authentication Database |
 | `DB_TYPE`              | Type of DB Server to backup `couch` `influx` `mysql` `pgsql` `mongo` `redis` `sqlite3`                                                                                                             |
 | `DB_HOST`              | Server Hostname e.g. `mariadb`. For `sqlite3`, full path to DB file e.g. `/backup/db.sqlite3`                                                                                                      |
 | `DB_NAME`              | Schema Name e.g. `database`                                                                                                                                                                        |
@@ -118,33 +135,36 @@ Along with the Environment Variables from the [Base image](https://hub.docker.co
 
 When using compression with MongoDB, only `GZ` compression is possible.
 
-**Backing Up to S3 Compatible Services**
+#### Backing Up to S3 Compatible Services
 
 If `BACKUP_LOCATION` = `S3` then the following options are used.
 
 | Parameter       | Description                                                                             |
 | --------------- | --------------------------------------------------------------------------------------- |
 | `S3_BUCKET`     | S3 Bucket name e.g. 'mybucket'                                                          |
-| `S3_HOSTNAME`   | Hostname of S3 Server e.g "s3.amazonaws.com" - You can also include a port if necessary |
+| `S3_HOST`       | Hostname of S3 Server e.g "s3.amazonaws.com" - You can also include a port if necessary |
 | `S3_KEY_ID`     | S3 Key ID                                                                               |
 | `S3_KEY_SECRET` | S3 Key Secret                                                                           |
 | `S3_PATH`       | S3 Pathname to save to e.g. '`backup`'                                                  |
 | `S3_PROTOCOL`   | Use either `http` or `https` to access service - Default `https`                        |
 | `S3_URI_STYLE`  | Choose either `VIRTUALHOST` or `PATH` style - Default `VIRTUALHOST`                     |
 
+
 ## Maintenance
 
-Manual Backups can be performed by entering the container and typing `backup-now`
 
 ### Shell Access
 
 For debugging and maintenance purposes you may want access the containers shell.
 
-```bash
-docker exec -it (whatever your container name is e.g.) db-backup bash
-```
+``bash
+docker exec -it (whatever your container name is) bash
+``
+### Manual Backups
+Manual Backups can be performed by entering the container and typing `backup-now`
 
-#### Custom Scripts
+
+### Custom Scripts
 
 If you want to execute a custom script at the end of backup, you can drop bash scripts with the extension of `.sh` in this directory. See the following example to utilize:
 
@@ -152,16 +172,16 @@ If you want to execute a custom script at the end of backup, you can drop bash s
 $ cat post-script.sh
 ##!/bin/bash
 
-## Example Post Script
-## $1=EXIT_CODE (After running backup routine)
-## $2=DB_TYPE (Type of Backup)
-## $3=DB_HOST (Backup Host)
-## #4=DB_NAME (Name of Database backed up
-## $5=DATE (Date of Backup)
-## $6=TIME (Time of Backup)
-## $7=BACKUP_FILENAME (Filename of Backup)
-## $8=FILESIZE (Filesize of backup)
-## $9=MD5_RESULT (MD5Sum if enabled)
+# #### Example Post Script
+# #### $1=EXIT_CODE (After running backup routine)
+# #### $2=DB_TYPE (Type of Backup)
+# #### $3=DB_HOST (Backup Host)
+# #### #4=DB_NAME (Name of Database backed up
+# #### $5=DATE (Date of Backup)
+# ####  $6=TIME (Time of Backup)
+# ####  $7=BACKUP_FILENAME (Filename of Backup)
+# ####  $8=FILESIZE (Filesize of backup)
+# ####  $9=MD5_RESULT (MD5Sum if enabled)
 
 echo "${1} ${2} Backup Completed on ${3} for ${4} on ${5} ${6}. Filename: ${7} Size: ${8} bytes MD5: ${9}"
 ````
@@ -171,3 +191,23 @@ Outputs the following on the console:
 `0 mysql Backup Completed on example-db for example on 2020-04-22 05:19:10. Filename: mysql_example_example-db_20200422-051910.sql.bz2 Size: 7795 bytes MD5: 952fbaafa30437494fdf3989a662cd40`
 
 If you wish to change the size value from bytes to megabytes set environment variable `SIZE_VALUE=megabytes`
+
+## Support
+
+These images were built to serve a specific need in a production environment and gradually have had more functionality added based on requests from the community.
+### Usage
+- The [Discussions board](../../discussions) is a great place for working with the community on tips and tricks of using this image.
+- Consider [sponsoring me](https://github.com/sponsors/tiredofit) personalized support.
+### Bugfixes
+- Please, submit a [Bug Report](issues/new) if something isn't working as expected. I'll do my best to issue a fix in short order.
+
+### Feature Requests
+- Feel free to submit a feature request, however there is no guarantee that it will be added, or at what timeline.
+- Consider [sponsoring me](https://github.com/sponsors/tiredofit) regarding development of features.
+
+### Updates
+- Best effort to track upstream changes, More priority if I am actively using the image in a production environment.
+- Consider [sponsoring me](https://github.com/sponsors/tiredofit) for up to date releases.
+
+## License
+MIT. See [LICENSE](LICENSE) for more details.

install/etc/services.available/10-db-backup/run

@@ -35,7 +35,6 @@ case "$dbtype" in
         dbport=${DB_PORT:-27017}
         [[ ( -n "${DB_USER}" ) || ( -n "${DB_USER_FILE}" ) ]] && file_env 'DB_USER'
         [[ ( -n "${DB_PASS}" ) || ( -n "${DB_PASS_FILE}" ) ]] && file_env 'DB_PASS'
-    ;;
     "mysql" | "MYSQL" | "mariadb" | "MARIADB")
         dbtype=mysql
         dbport=${DB_PORT:-3306}
@@ -118,6 +117,7 @@ case "$dbtype" in
         [[ ( -n "${DB_USER}" ) ]] && MONGO_USER_STR=" --username ${dbuser}"
         [[ ( -n "${DB_PASS}" ) ]] && MONGO_PASS_STR=" --password ${dbpass}"
         [[ ( -n "${DB_NAME}" ) ]] && MONGO_DB_STR=" --db ${dbname}"
+        [[ ( -n "${DB_AUTH}" ) ]] && MONGO_AUTH_STR=" --authenticationDatabase ${DB_AUTH}"
     ;;
     "mysql" )
         [[ ( -n "${DB_PASS}" ) ]] && export MYSQL_PWD=${dbpass}
@@ -164,7 +164,7 @@ backup_mongo() {
         target=${dbtype}_${dbname}_${dbhost}_${now}.archivegz
         mongo_compression="--gzip"
     fi
-    mongodump --archive=${tmpdir}/${target} ${mongo_compression} --host ${dbhost} --port ${dbport} ${MONGO_USER_STR}${MONGO_PASS_STR}${MONGO_DB_STR} ${EXTRA_OPTS}
+    mongodump --archive=${tmpdir}/${target} ${mongo_compression} --host ${dbhost} --port ${dbport} ${MONGO_USER_STR}${MONGO_PASS_STR}${MONGO_AUTH_STR}${MONGO_DB_STR} ${EXTRA_OPTS}
     exit_code=$?
     cd ${tmpdir}
     generate_md5
@@ -203,7 +203,9 @@ backup_mysql() {
 backup_pgsql() {
   if var_true $SPLIT_DB ; then
       export PGPASSWORD=${dbpass}
-      DATABASES=$(psql -h $dbhost -U $dbuser -p ${dbport} -c 'COPY (SELECT datname FROM pg_database WHERE datistemplate = false) TO STDOUT;' )
+      authdb=${DB_USER}
+      [ -n "${DB_NAME}" ] && authdb=${DB_NAME}
+      DATABASES=$(psql -h $dbhost -U $dbuser -p ${dbport} -d ${authdb} -c 'COPY (SELECT datname FROM pg_database WHERE datistemplate = false) TO STDOUT;' )
             for db in $DATABASES; do
                 print_info "Dumping database: $db"
                 target=pgsql_${db}_${dbhost}_${now}.sql
@@ -213,13 +215,13 @@ backup_pgsql() {
                 generate_md5
                 move_backup
             done
-        else
-            export PGPASSWORD=${dbpass}
-            compression
-            pg_dump -h ${dbhost} -U ${dbuser} -p ${dbport} ${dbname} ${EXTRA_OPTS} | $dumpoutput > ${tmpdir}/${target}
-            exit_code=$?
-            generate_md5
-            move_backup
+  else
+      export PGPASSWORD=${dbpass}
+      compression
+      pg_dump -h ${dbhost} -U ${dbuser} -p ${dbport} ${dbname} ${EXTRA_OPTS} | $dumpoutput > ${tmpdir}/${target}
+      exit_code=$?
+      generate_md5
+      move_backup
   fi
 }
 
@@ -411,34 +413,10 @@ move_backup() {
             mv ${tmpdir}/"${target}" "${DB_DUMP_TARGET}"/"${target}"
         ;;
         "S3" | "s3" | "MINIO" | "minio" )
-            s3_content_type="application/octet-stream"
-            if [ "$S3_URI_STYLE" = "VIRTUALHOST" ] || [ "$S3_URI_STYLE" = "VHOST" ] || [ "$S3_URI_STYLE" = "virtualhost" ] || [ "$S3_URI_STYLE" = "vhost" ] ; then
-                s3_url="${S3_BUCKET}.${S3_HOST}"
-            else
-                s3_url="${S3_HOST}/${S3_BUCKET}"
-            fi
-
-            if var_true "$MD5" ; then
-                s3_date="$(LC_ALL=C date -u +"%a, %d %b %Y %X %z")"
-                s3_md5="$(libressl md5 -binary < "${tmpdir}/${target}.md5" | base64)"
-                sig="$(printf "PUT\n$s3_md5\n${s3_content_type}\n$s3_date\n/$S3_BUCKET/$S3_PATH/${target}.md5" | libressl sha1 -binary -hmac "${S3_KEY_SECRET}" | base64)"
-                print_debug "Uploading ${target}.md5 to S3"
-                curl -T "${tmpdir}/${target}.md5" "${S3_PROTOCOL}"://"${s3_url}"/"${S3_PATH}"/"${target}".md5 \
-                     -H "Date: $date" \
-                     -H "Authorization: AWS ${S3_KEY_ID}:$sig" \
-                     -H "Content-Type: ${s3_content_type}" \
-                     -H "Content-MD5: ${s3_md5}"
-            fi
-
-            s3_date="$(LC_ALL=C date -u +"%a, %d %b %Y %X %z")"
-            s3_md5="$(libressl md5 -binary < "${tmpdir}/${target}" | base64)"
-            sig="$(printf "PUT\n$s3_md5\n${s3_content_type}\n$s3_date\n/$S3_BUCKET/$S3_PATH/${target}" | libressl sha1 -binary -hmac "${S3_KEY_SECRET}" | base64)"
-            print_debug "Uploading ${target} to S3"
-            curl -T ${tmpdir}/"${target}" "${S3_PROTOCOL}"://"${s3_url}"/"${S3_PATH}"/"${target}" \
-                 -H "Date: $s3_date" \
-                 -H "Authorization: AWS ${S3_KEY_ID}:$sig" \
-                 -H "Content-Type: ${s3_content_type}" \
-                 -H "Content-MD5: ${s3_md5}"
+            export AWS_ACCESS_KEY_ID=${S3_KEY_ID}
+            export AWS_SECRET_ACCESS_KEY=${S3_KEY_SECRET}
+            export AWS_DEFAULT_REGION=ap-northeast-2
+            aws s3 cp ${tmpdir}/${target} s3://${S3_BUCKET}/${S3_PATH}/${target}
 
             rm -rf ${tmpdir}/*.md5
             rm -rf ${tmpdir}/"${target}"