mirrors/docker-db-backup
1
0
Fork 0
mirror of https://github.com/tiredofit/docker-db-backup.git synced 2025-12-21 21:33:28 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update README.md with new options

Browse Source
This commit is contained in:
Dave Conroy
2023-11-03 14:59:44 -07:00
parent 2b10a0b679
commit 96f9825e19
3 changed files with 615 additions and 195 deletions
Download Patch File Download Diff File Expand all files Collapse all files

789
README.md
View File

@@ -13,21 +13,22 @@
This will build a container for backing up multiple types of DB Servers
Currently backs up CouchDB, InfluxDB, MySQL, Microsoft SQL, MongoDB, Postgres, Redis servers.
Backs up CouchDB, InfluxDB, MySQL, Microsoft SQL, MongoDB, Postgres, Redis servers.
- dump to local filesystem or backup to S3 Compatible services, and Azure.
- select database user and password
- backup all databases, single, or multiple databases
- multiple backup job support
- selectable when to start the first dump, whether time of day or relative to container start time
- selectable interval
- selectable omit scheduling during periods of time
- selectable database user and password
- selectable cleanup and archive capabilities
- selectable database name support - all databases, single, or multiple databases
- backup all to separate files or one singular file
- choose to have an MD5 or SHA1 sum after backup for verification
- delete old backups after specific amount of time
- choose compression type (none, gz, bz, xz, zstd)
- connect to any container running on the same system
- Script to perform restores
- Zabbix Monitoring capabilities
- select how often to run a dump
- select when to start the first dump, whether time of day or relative to container start time
- Execute script after backup for monitoring/alerting purposes
- checksum support choose to have an MD5 or SHA1 hash generated after backup for verification
- compression support (none, gz, bz, xz, zstd)
- Zabbix Metrics support
- Hooks to execute pre and post backup job for customization purposes
- Companion script to aid in restores
## Maintainer
@@ -49,20 +50,56 @@ Currently backs up CouchDB, InfluxDB, MySQL, Microsoft SQL, MongoDB, Postgres, R
- [Environment Variables](#environment-variables)
- [Base Images used](#base-images-used)
- [Container Options](#container-options)
- [Database Specific Options](#database-specific-options)
- [For Influx DB2](#for-influx-db2)
- [Job Defaults](#job-defaults)
- [Compression Options](#compression-options)
- [Scheduling Options](#scheduling-options)
- [Backup Options](#backup-options)
- [Backing Up to S3 Compatible Services](#backing-up-to-s3-compatible-services)
- [Upload to a Azure storage account by `blobxfer`](#upload-to-a-azure-storage-account-by-blobxfer)
- [Maintenance](#maintenance)
- [Shell Access](#shell-access)
- [Manual Backups](#manual-backups)
- [Restoring Databases](#restoring-databases)
- [Custom Scripts](#custom-scripts)
- [Default Database Options](#default-database-options)
- [CouchDB](#couchdb)
- [InfluxDB](#influxdb)
- [MariaDB/MySQL](#mariadbmysql)
- [Microsoft SQL](#microsoft-sql)
- [MongoDB](#mongodb)
- [Postgresql](#postgresql)
- [Redis](#redis)
- [Default Storage Options](#default-storage-options)
- [Filesystem](#filesystem)
- [S3](#s3)
- [Azure](#azure)
- [Hooks](#hooks)
- [Path Options](#path-options)
- [Pre Backup](#pre-backup)
- [Post backup](#post-backup)
- [Job Backup Options](#job-backup-options)
- [Compression Options](#compression-options-1)
- [Scheduling Options](#scheduling-options-1)
- [Specific Database Options](#specific-database-options)
- [CouchDB](#couchdb-1)
- [InfluxDB](#influxdb-1)
- [MariaDB/MySQL](#mariadbmysql-1)
- [Microsoft SQL](#microsoft-sql-1)
- [MongoDB](#mongodb-1)
- [Postgresql](#postgresql-1)
- [Redis](#redis-1)
- [SQLite](#sqlite)
- [Specific Storage Options](#specific-storage-options)
- [Filesystem](#filesystem-1)
- [S3](#s3-1)
- [Azure](#azure-1)
- [Hooks](#hooks-1)
- [Path Options](#path-options-1)
- [Pre Backup](#pre-backup-1)
- [Post backup](#post-backup-1)
- [Notifications](#notifications)
- [Custom Notifications](#custom-notifications)
- [Email Notifications](#email-notifications)
- [Matrix Notifications](#matrix-notifications)
- [Mattermost Notifications](#mattermost-notifications)
- [Rocketchat Notifications](#rocketchat-notifications)
- [Maintenance](#maintenance)
- [Shell Access](#shell-access)
- [Manual Backups](#manual-backups)
- [Manual Backups](#manual-backups-1)
- [Restoring Databases](#restoring-databases)
- [Support](#support)
- [Usage](#usage)
- [Bugfixes](#bugfixes)
@@ -70,8 +107,6 @@ Currently backs up CouchDB, InfluxDB, MySQL, Microsoft SQL, MongoDB, Postgres, R
- [Updates](#updates)
- [License](#license)
> **NOTE**: If you are using this with a docker-compose file along with a seperate SQL container, take care not to set the variables to backup immediately, more so have it delay execution for a minute, otherwise you will get a failed first backup.
## Prerequisites and Assumptions
- You must have a working connection to one of the supported DB Servers and appropriate credentials
@@ -110,11 +145,10 @@ Images are built primarily for `amd64` architecture, and may also include builds
### Quick Start
- The quickest way to get started is using [docker-compose](https://docs.docker.com/compose/). See the examples folder for a working [docker-compose.yml](examples/docker-compose.yml) that can be modified for development or production use.
- The quickest way to get started is using [docker-compose](https://docs.docker.com/compose/). See the examples folder for a series of example compose.yml that can be modified for development or production use.
- Set various [environment variables](#environment-variables) to understand the capabilities of this image.
- Map [persistent storage](#data-volumes) for access to configuration and data files for backup.
- Make [networking ports](#networking) available for public access if necessary
### Persistent Storage
@@ -140,118 +174,570 @@ Be sure to view the following repositories to understand all the customizable op
#### Container Options
| Parameter | Description | Default |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------- | --------------- |
| `BACKUP_LOCATION` | Backup to `FILESYSTEM`, `blobxfer` or `S3` compatible services like S3, Minio, Wasabi | `FILESYSTEM` |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | --------------- |
| `MODE` | `AUTO` mode to use internal scheduling routines or `MANUAL` to simply use this as manual backups only executed by your own means | `AUTO` |
| `MANUAL_RUN_FOREVER` | `TRUE` or `FALSE` if you wish to try to make the container exit after the backup | `TRUE` |
| `DEBUG_MODE` | If set to `true`, print copious shell script messages to the container log. Otherwise only basic messages are printed. | `FALSE` |
| `LOG_PATH` | Path to log files | `/logs` |
| `LOG_FILE` | Logfile name | `dbbackup.log` |
| `CREATE_LATEST_SYMLINK` | Create a symbolic link pointing to last backup in this format: `latest-(DB_TYPE)-(DB_NAME)-(DB_HOST)` | `TRUE` |
| `PRE_SCRIPT` | Fill this variable in with a command to execute pre backing up | |
| `POST_SCRIPT` | Fill this variable in with a command to execute post backing up | |
| `USER_DBBACKUP` | The uid that the image should read and write files as (username is `dbbackup`) | `10000` |
| `GROUP_DBBACKUP` | The gid that the image should read and write files as (groupname is `dbbackup`) | `10000` |
| `SPLIT_DB` | For each backup, create a new archive. `TRUE` or `FALSE` (MySQL and Postgresql Only) | `TRUE` |
| `TEMP_LOCATION` | Perform Backups and Compression in this temporary directory | `/tmp/backups/` |
| `LOG_PATH` | Path to log files | `/logs` |
| `TEMP_PATH` | Perform Backups and Compression in this temporary directory | `/tmp/backups/` |
| `MANUAL_RUN_FOREVER` | `TRUE` or `FALSE` if you wish to try to make the container exit after the backup | `TRUE` |
| `DEBUG_MODE` | If set to `true`, print copious shell script messages to the container log. Otherwise only basic messages are printed. | `FALSE` |
| `BACKUP_JOB_CONCURRENCY` | How many backup jobs to run concurrently | `1` |
### Database Specific Options
#### Job Defaults
If these are set and no other defaults or variables are set explicitly, they will be added to any of the backup jobs.
| Parameter | Description | Default | `_FILE` |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DB_AUTH` | (Mongo/PGSQL Only - Optional) Authentication Database | | |
| `DB_TYPE` | Type of DB Server to backup `couch` `influx` `mysql` `mssql` `pgsql` `mongo` `redis` `sqlite3` | | |
| `DB_HOST` | Server Hostname e.g. `mariadb`. For `sqlite3`, full path to DB file e.g. `/backup/db.sqlite3` | | x |
| `DB_NAME` | Schema Name e.g. `database` or `ALL` to backup all databases the user has access to. Backup multiple by separating with commas eg `db1,db2` | | x |
| `DB_NAME_EXCLUDE` | If using `ALL` - use this as to exclude databases separated via commas from being backed up | | x |
| `DB_USER` | username for the database(s) - Can use `root` for MySQL | | x |
| `DB_PASS` | (optional if DB doesn't require it) password for the database | | x |
| `DB_PORT` | (optional) Set port to connect to DB_HOST. Defaults are provided | varies | x |
| `INFLUX_VERSION` | What Version of Influx are you backing up from `1`.x or `2` series - AMD64 and ARM64 only for `2` | | |
| `MONGO_CUSTOM_URI` | If you wish to override the MongoDB Connection string enter it here e.g. `mongodb+srv://username:password@cluster.id.mongodb.net` | | x |
| | This environment variable will be parsed and populate the `DB_NAME` and `DB_HOST` variables to properly build your backup filenames. You can override them by making your own entries | | |
| Variable | Description | Default |
| --------------------------------- | ------------------------------------------------------------------------------------- | ------------ |
| `DEFAULT_BACKUP_LOCATION` | Backup to `FILESYSTEM`, `blobxfer` or `S3` compatible services like S3, Minio, Wasabi | `FILESYSTEM` |
| `DEFAULT_CHECKSUM` | Either `MD5` or `SHA1` or `NONE` | `MD5` |
| `DEFAULT_LOG_LEVEL` | Log output on screen and in files `INFO` `NOTICE` `ERROR` `WARN` `DEBUG` | `notice` |
| `DEFAULT_SKIP_AVAILABILITY_CHECK` | Before backing up - skip connectivity check | `FALSE` |
#### For Influx DB2
##### Compression Options
Your Organization will be mapped to `DB_USER` and your root token will need to be mapped to `DB_PASS`. You may use `DB_NAME=ALL` to backup the entire set of databases. For `DB_HOST` use syntax of `http(s)://db-name`
| Variable | Description | Default |
| -------------------------------------- | ---------------------------------------------------------------------------------------------- | -------------- |
| `DEFAULT_COMPRESSION` | Use either Gzip `GZ`, Bzip2 `BZ`, XZip `XZ`, ZSTD `ZSTD` or none `NONE` | `ZSTD` |
| `DEFAULT_COMPRESSION_LEVEL` | Numerical value of what level of compression to use, most allow `1` to `9` | `3` |
| | except for `ZSTD` which allows for `1` to `19` | |
| `DEFAULT_GZ_RSYNCABLE` | Use `--rsyncable` (gzip only) for faster rsync transfers and incremental backup deduplication. | `FALSE` |
| `DEFAULT_ENABLE_PARALLEL_COMPRESSION` | Use multiple cores when compressing backups `TRUE` or `FALSE` | `TRUE` |
| `DEFAULT_PARALLEL_COMPRESSION_THREADS` | Maximum amount of threads to use when compressing - Integer value e.g. `8` | `autodetected` |
### Scheduling Options
##### Scheduling Options
| Parameter | Description | Default |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- |
| `DB_DUMP_FREQ` | How often to do a dump, in minutes after the first backup. Defaults to 1440 minutes, or once per day. | `1440` |
| `DB_DUMP_BEGIN` | What time to do the first dump. Defaults to immediate. Must be in one of two formats | |
| Variable | Description | Default |
| ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- |
| `DEFAULT_BACKUP_INTERVAL` | How often to do a backup, in minutes after the first backup. Defaults to 1440 minutes, or once per day. | `1440` |
| `DEFAULT_BACKUP_BEGIN` | What time to do the initial backup. Defaults to immediate. (`+1`) | `+0` |
| | Must be in one of two formats: | |
| | Absolute HHMM, e.g. `2330` or `0415` | |
| | Relative +MM, i.e. how many minutes after starting the container, e.g. `+0` (immediate), `+10` (in 10 minutes), or `+90` in an hour and a half | |
| `DB_DUMP_TARGET` | Directory where the database dumps are kept. | `${DB_DUMP_TARGET}/archive/` |
| `DB_DUMP_TARGET_PERMISSION` | Directory and File permissions to apply to files. | `700` |
| `DB_DUMP_TARGET_ARCHIVE` | Optional Directory where the database dumps archives are kept. | |
| `DB_CLEANUP_TIME` | Value in minutes to delete old backups (only fired when dump frequency fires). 1440 would delete anything above 1 day old. You don't need to set this variable if you want to hold onto everything. | `FALSE` |
| `DB_ARCHIVE_TIME` | Value in minutes to move all files files older than (x) from `DB_DUMP_TARGET` to `DB_DUMP_TARGET_ARCHIVE` - which is useful when pairing against an external backup system. | |
| `DEFAULT_CLEANUP_TIME` | Value in minutes to delete old backups (only fired when backup interval executes) | `FALSE` |
| | 1440 would delete anything above 1 day old. You don't need to set this variable if you want to hold onto everything. | |
| `DEFAULT_ARCHIVE_TIME` | Value in minutes to move all files files older than (x) from | `DEFAULT_BACKUP_BLACKOUT_BEGIN` | Use `HHMM` notation to start a blackout period where no backups occur eg `0420` | |
| `DEFAULT_BACKUP_BLACKOUT_END` | Use `HHMM` notation to set the end period where no backups occur eg `0430` | |
- You may need to wrap your `DB_DUMP_BEGIN` value in quotes for it to properly parse. There have been reports of backups that start with a `0` get converted into a different format which will not allow the timer to start at the correct time.
> You may need to wrap your `DEFAULT_BACKUP_BEGIN` value in quotes for it to properly parse. There have been reports of backups that start with a `0` get converted into a different format which will not allow the timer to start at the correct time.
### Backup Options
##### Default Database Options
###### CouchDB
| Variable | Description | Default | `_FILE` |
| -------------- | ------------ | ------- | ------- |
| `DEFAULT_PORT` | CouchDB Port | `5984` | x |
###### InfluxDB
| Variable | Description | Default | `_FILE` |
| ------------------------ | ------------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DEFAULT_PORT` | InfluxDB Port | | x |
| | Version 1.x | `8088` | |
| | Version 2.x | `8086` | |
| `DEFAULT_INFLUX_VERSION` | What Version of Influx are you backing up from `1`.x or `2` series - amd64 and aarch/armv8 only for `2` | `2` | |
###### MariaDB/MySQL
| Variable | Description | Default | `_FILE` |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------- | ------- |
| `DEFAULT_PORT` | MariaDB Port | `3306` | x |
| `DEFAULT_EXTRA_BACKUP_OPTS` | Pass extra arguments to the backup command only, add them here e.g. `--extra-command` | | |
| `DEFAULT_EXTRA_ENUMERATION_OPTS` | Pass extra arguments to the database enumeration command only, add them here e.g. `--extra-command` | | |
| `DEFAULT_EXTRA_OPTS` | Pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | | |
| `DEFAULT_MYSQL_MAX_ALLOWED_PACKET` | Max allowed packet if backing up MySQL / MariaDB | `512M` | |
| `DEFAULT_MYSQL_SINGLE_TRANSACTION` | Backup in a single transaction with MySQL / MariaDB | `TRUE` | |
| `DEFAULT_MYSQL_STORED_PROCEDURES` | Backup stored procedures with MySQL / MariaDB | `TRUE` | |
| `DEFAULT_MYSQL_ENABLE_TLS` | Enable TLS functionality for MySQL/MariaDB client | `FALSE` | |
| `DEFAULT_MYSQL_TLS_VERIFY` | (optional) If using TLS (by means of MYSQL_TLS_* variables) verify remote host | `FALSE` | |
| `DEFAULT_MYSQL_TLS_VERSION` | What TLS `v1.1` `v1.2` `v1.3` version to utilize | `TLSv1.1,TLSv1.2,TLSv1.3` | |
| `DEFAULT_MYSQL_TLS_CA_FILE` | Filename to load custom CA certificate for connecting via TLS | `/etc/ssl/cert.pem` | x |
| `DEFAULT_MYSQL_TLS_CERT_FILE` | Filename to load client certificate for connecting via TLS | | x |
| `DEFAULT_MYSQL_TLS_KEY_FILE` | Filename to load client key for connecting via TLS | | x |
###### Microsoft SQL
| Variable | Description | Default | `_FILE` |
| -------------- | ------------------ | ------- | ------- |
| `DEFAULT_PORT` | Microsoft SQL Port | `1433` | x |
###### MongoDB
| Variable | Description | Default | `_FILE` |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------ | ------- | ------- |
| `DEFAULT_AUTH` | (Optional) Authentication Database | | x |
| `DEFAULT_PORT` | MongoDB Port | `27017` | x |
| `MONGO_CUSTOM_URI` | If you wish to override the MongoDB Connection string enter it here e.g. `mongodb+srv://username:password@cluster.id.mongodb.net` | | x |
| | This environment variable will be parsed and populate the `DB_NAME` and `DB_HOST` variables to properly build your backup filenames. | | |
| | You can override them by making your own entries | | |
###### Postgresql
| Variable | Description | Default | `_FILE` |
| -------------------------------- | --------------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DEFAULT_AUTH` | (Optional) Authentication Database | | x |
| `DEFAULT_EXTRA_BACKUP_OPTS` | Pass extra arguments to the backup command only, add them here e.g. `--extra-command` | | |
| `DEFAULT_EXTRA_ENUMERATION_OPTS` | Pass extra arguments to the database enumeration command only, add them here e.g. `--extra-command` | | |
| `DEFAULT_EXTRA_OPTS` | Pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | | |
| `DEFAULT_PORT` | PostgreSQL Port | `5432` | x |
###### Redis
| Variable | Description | Default | `_FILE` |
| -------------------------------- | --------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DEFAULT_PORT` | Default Redis Port | `6379` | x |
| `DEFAULT_EXTRA_ENUMERATION_OPTS` | Pass extra arguments to the database enumeration command only, add them here e.g. `--extra-command` | | |
##### Default Storage Options
Options that are related to the value of `DEFAULT_BACKUP_LOCATION`
###### Filesystem
If `DEFAULT_BACKUP_LOCTION` = `FILESYSTEM` then the following options are used:
| Variable | Description | Default |
| --------------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `DEFAULT_CREATE_LATEST_SYMLINK` | Create a symbolic link pointing to last backup in this format: `latest-(DB_TYPE)-(DB_NAME)-(DB_HOST)` | `TRUE` |
| `DEFAULT_FILESYSTEM_PATH` | Directory where the database dumps are kept. | `/backup` |
| `DEFAULT_FILESYSTEM_ARCHIVE_PATH` | Optional Directory where the database dumps archives are kept | `${DEFAULT_FILESYSTEM_PATH}/archive/` |
| `DEFAULT_FILESYSTEM_PERMISSION` | Directory and File permissions to apply to files. | `700` |
###### S3
If `DEFAULT_BACKUP_LOCATION` = `S3` then the following options are used:
| Parameter | Description | Default | `_FILE` |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ------- |
| `COMPRESSION` | Use either Gzip `GZ`, Bzip2 `BZ`, XZip `XZ`, ZSTD `ZSTD` or none `NONE` | `ZSTD` | |
| `COMPRESSION_LEVEL` | Numerical value of what level of compression to use, most allow `1` to `9` except for `ZSTD` which allows for `1` to `19` - | `3` | |
| `ENABLE_PARALLEL_COMPRESSION` | Use multiple cores when compressing backups `TRUE` or `FALSE` | `TRUE` | |
| `PARALLEL_COMPRESSION_THREADS` | Maximum amount of threads to use when compressing - Integer value e.g. `8` | `autodetected` | |
| `GZ_RSYNCABLE` | Use `--rsyncable` (gzip only) for faster rsync transfers and incremental backup deduplication. e.g. `TRUE` | `FALSE` | |
| `ENABLE_CHECKSUM` | Generate either a MD5 or SHA1 in Directory, `TRUE` or `FALSE` | `TRUE` | |
| `CHECKSUM` | Either `MD5` or `SHA1` | `MD5` | |
| `EXTRA_OPTS` | If you need to pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | | |
| `EXTRA_DUMP_OPTS` | If you need to pass extra arguments to the backup command only, add them here e.g. `--extra-command` | | |
| `EXTRA_ENUMERATION_OPTS` | If you need to pass extra arguments to the database enumeration command only, add them here e.g. `--extra-command` | | |
| `MYSQL_MAX_ALLOWED_PACKET` | Max allowed packet if backing up MySQL / MariaDB | `512M` | |
| `MYSQL_SINGLE_TRANSACTION` | Backup in a single transaction with MySQL / MariaDB | `TRUE` | |
| `MYSQL_STORED_PROCEDURES` | Backup stored procedures with MySQL / MariaDB | `TRUE` | |
| `MYSQL_ENABLE_TLS` | Enable TLS functionality for MySQL client | `FALSE` | |
| `MYSQL_TLS_VERIFY` | (optional) If using TLS (by means of MYSQL_TLS_* variables) verify remote host | `FALSE` | |
| `MYSQL_TLS_VERSION` | What TLS `v1.1` `v1.2` `v1.3` version to utilize | `TLSv1.1,TLSv1.2,TLSv1.3` | |
| `MYSQL_TLS_CA_FILE` | Filename to load custom CA certificate for connecting via TLS | `/etc/ssl/cert.pem` | x |
| `MYSQL_TLS_CERT_FILE` | Filename to load client certificate for connecting via TLS | | x |
| `MYSQL_TLS_KEY_FILE` | Filename to load client key for connecting via TLS | | x |
- When using compression with MongoDB, only `GZ` compression is possible.
#### Backing Up to S3 Compatible Services
If `BACKUP_LOCATION` = `S3` then the following options are used.
| Parameter | Description | Default | `_FILE` |
| --------------------- | ----------------------------------------------------------------------------------------- | ------- | ------- |
| `S3_BUCKET` | S3 Bucket name e.g. `mybucket` | | x |
| `S3_KEY_ID` | S3 Key ID (Optional) | | x |
| `S3_KEY_SECRET` | S3 Key Secret (Optional) | | x |
| `S3_PATH` | S3 Pathname to save to (must NOT end in a trailing slash e.g. '`backup`') | | x |
| `S3_REGION` | Define region in which bucket is defined. Example: `ap-northeast-2` | | x |
| `S3_HOST` | Hostname (and port) of S3-compatible service, e.g. `minio:8080`. Defaults to AWS. | | x |
| `S3_PROTOCOL` | Protocol to connect to `S3_HOST`. Either `http` or `https`. Defaults to `https`. | `https` | x |
| `S3_EXTRA_OPTS` | Add any extra options to the end of the `aws-cli` process execution | | x |
| `S3_CERT_CA_FILE` | Map a volume and point to your custom CA Bundle for verification e.g. `/certs/bundle.pem` | | x |
| ----------------------------- | ----------------------------------------------------------------------------------------- | ------- | ------- |
| `DEFAULT_S3_BUCKET` | S3 Bucket name e.g. `mybucket` | | x |
| `DEFAULT_S3_KEY_ID` | S3 Key ID (Optional) | | x |
| `DEFAULT_S3_KEY_SECRET` | S3 Key Secret (Optional) | | x |
| `DEFAULT_S3_PATH` | S3 Pathname to save to (must NOT end in a trailing slash e.g. '`backup`') | | x |
| `DEFAULT_S3_REGION` | Define region in which bucket is defined. Example: `ap-northeast-2` | | x |
| `DEFAULT_S3_HOST` | Hostname (and port) of S3-compatible service, e.g. `minio:8080`. Defaults to AWS. | | x |
| `DEFAULT_S3_PROTOCOL` | Protocol to connect to `DEFAULT_S3_HOST`. Either `http` or `https`. Defaults to `https`. | `https` | x |
| `DEFAULT_S3_EXTRA_OPTS` | Add any extra options to the end of the `aws-cli` process execution | | x |
| `DEFAULT_S3_CERT_CA_FILE` | Map a volume and point to your custom CA Bundle for verification e.g. `/certs/bundle.pem` | | x |
| _*OR*_ | | | |
| `S3_CERT_SKIP_VERIFY` | Skip verifying self signed certificates when connecting | `TRUE` | |
| `DEFAULT_S3_CERT_SKIP_VERIFY` | Skip verifying self signed certificates when connecting | `TRUE` | |
- When `S3_KEY_ID` and/or `S3_KEY_SECRET` is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.
- When `DEFAULT_S3_KEY_ID` and/or `DEFAULT_S3_KEY_SECRET` is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.
#### Upload to a Azure storage account by `blobxfer`
###### Azure
Support to upload backup files with [blobxfer](https://github.com/Azure/blobxfer) to the Azure file share storage.
If `BACKUP_LOCATION` = `blobxfer` then the following options are used.
If `DEFAULT_BACKUP_LOCATION` = `blobxfer` then the following options are used:.
| Parameter | Description | Default | `_FILE` |
| ------------------------------ | ------------------------------------------- | ------------------- | ------- |
| `BLOBXFER_STORAGE_ACCOUNT` | Microsoft Azure Cloud storage account name. | | x |
| `BLOBXFER_STORAGE_ACCOUNT_KEY` | Microsoft Azure Cloud storage account key. | | x |
| `BLOBXFER_REMOTE_PATH` | Remote Azure path | `/docker-db-backup` | x |
| -------------------------------------- | ------------------------------------------- | ------------------- | ------- |
| `DEFAULT_BLOBXFER_STORAGE_ACCOUNT` | Microsoft Azure Cloud storage account name. | | x |
| `DEFAULT_BLOBXFER_STORAGE_ACCOUNT_KEY` | Microsoft Azure Cloud storage account key. | | x |
| `DEFAULT_BLOBXFER_REMOTE_PATH` | Remote Azure path | `/docker-db-backup` | x |
> This service uploads files from backup targed directory `DB_DUMP_TARGET`.
> If the a cleanup configuration in `DB_CLEANUP_TIME` is defined, the remote directory on Azure storage will also be cleaned automatically.
> This service uploads files from backup targed directory `DEFAULT_FILESYSTEM_PATH`.
> If the a cleanup configuration in `DEFAULT_CLEANUP_TIME` is defined, the remote directory on Azure storage will also be cleaned automatically.
##### Hooks
###### Path Options
| Parameter | Description | Default |
| ------------------------------ | --------------------------------------------------------------------------- | ----------------------- |
| `DEFAULT_SCRIPT_LOCATION_PRE` | Location on filesystem inside container to execute bash scripts pre backup | `/assets/scripts/pre/` |
| `DEFAULT_SCRIPT_LOCATION_POST` | Location on filesystem inside container to execute bash scripts post backup | `/assets/scripts/post/` |
| `DEFAULT_PRE_SCRIPT` | Fill this variable in with a command to execute pre backing up | |
| `DEFAULT_POST_SCRIPT` | Fill this variable in with a command to execute post backing up | |
###### Pre Backup
If you want to execute a custom script before a backup starts, you can drop bash scripts with the extension of `.sh` in the location defined in `DB01_SCRIPT_LOCATION_PRE`. See the following example to utilize:
```bash
$ cat pre-script.sh
##!/bin/bash
# #### Example Pre Script
# #### $1=DBXX_TYPE (Type of Backup)
# #### $2=DBXX_HOST (Backup Host)
# #### $3=DBXX_NAME (Name of Database backed up
# #### $4=BACKUP START TIME (Seconds since Epoch)
# #### $5=BACKUP FILENAME (Filename)
echo "${1} Backup Starting on ${2} for ${3} at ${4}. Filename: ${5}"
```
## script DBXX_TYPE DBXX_HOST DBXX_NAME STARTEPOCH BACKUP_FILENAME
${f} "${backup_job_db_type}" "${backup_job_db_host}" "${backup_job_db_name}" "${backup_routines_start_time}" "${target}"
Outputs the following on the console:
`mysql Backup Starting on example-db for example at 1647370800. Filename: mysql_example_example-db_202200315-000000.sql.bz2
###### Post backup
If you want to execute a custom script at the end of a backup, you can drop bash scripts with the extension of `.sh` in the location defined in `DB01_SCRIPT_LOCATION_POST`. Also to support legacy users `/assets/custom-scripts` is also scanned and executed.See the following example to utilize:
```bash
$ cat post-script.sh
##!/bin/bash
# #### Example Post Script
# #### $1=EXIT_CODE (After running backup routine)
# #### $2=DBXX_TYPE (Type of Backup)
# #### $3=DBXX_HOST (Backup Host)
# #### #4=DBXX_NAME (Name of Database backed up
# #### $5=BACKUP START TIME (Seconds since Epoch)
# #### $6=BACKUP FINISH TIME (Seconds since Epoch)
# #### $7=BACKUP TOTAL TIME (Seconds between Start and Finish)
# #### $8=BACKUP FILENAME (Filename)
# #### $9=BACKUP FILESIZE
# #### $10=HASH (If CHECKSUM enabled)
# #### $11=MOVE_EXIT_CODE
echo "${1} ${2} Backup Completed on ${3} for ${4} on ${5} ending ${6} for a duration of ${7} seconds. Filename: ${8} Size: ${9} bytes MD5: ${10}"
```
## script EXIT_CODE DB_TYPE DB_HOST DB_NAME STARTEPOCH FINISHEPOCH DURATIONEPOCH BACKUP_FILENAME FILESIZE CHECKSUMVALUE
${f} "${exit_code}" "${dbtype}" "${backup_job_db_host}" "${backup_job_db_name}" "${backup_routines_start_time}" "${backup_routines_finish_time}" "${backup_routines_total_time}" "${target}" "${FILESIZE}" "${checksum_value}" "${move_exit_code}
Outputs the following on the console:
`0 mysql Backup Completed on example-db for example on 1647370800 ending 1647370920 for a duration of 120 seconds. Filename: mysql_example_example-db_202200315-000000.sql.bz2 Size: 7795 bytes Hash: 952fbaafa30437494fdf3989a662cd40 0`
If you wish to change the size value from bytes to megabytes set environment variable `DB01_SIZE_VALUE=megabytes`
You must make your scripts executable otherwise there is an internal check that will skip trying to run it otherwise.
If for some reason your filesystem or host is not detecting it right, use the environment variable `DB01_POST_SCRIPT_SKIP_X_VERIFY=TRUE` to bypass.
#### Job Backup Options
If `DEFAULT_` variables are set and you do not wish for the settings to carry over into your jobs, you can set the appropriate environment variable with the value of `unset`.
Otherwise, override them per backup job. Additional backup jobs can be scheduled by using `DB02_`,`DB03_`,`DB04_` ... prefixes. See [Specific Database Options](#specific-database-options) which may overrule this list.
| Parameter | Description | Default | `_FILE` |
| ----------- | ---------------------------------------------------------------------------------------------- | ------- | ------- |
| `DB01_TYPE` | Type of DB Server to backup `couch` `influx` `mysql` `mssql` `pgsql` `mongo` `redis` `sqlite3` | | |
| `DB01_HOST` | Server Hostname e.g. `mariadb`. For `sqlite3`, full path to DB file e.g. `/backup/db.sqlite3` | | x |
| `DB01_NAME` | Schema Name e.g. `database` | | x |
| `DB01_USER` | username for the database(s) - Can use `root` for MySQL | | x |
| `DB01_PASS` | (optional if DB doesn't require it) password for the database | | x |
| | |
| Variable | Description | Default |
| ------------------------------ | --------------------------------------------------------------------------------------------------------- | ------------ |
| `DB01_BACKUP_LOCATION` | Backup to `FILESYSTEM`, `blobxfer` or `S3` compatible services like S3, Minio, Wasabi | `FILESYSTEM` |
| `DB01_CHECKSUM` | Either `MD5` or `SHA1` or `NONE` | `MD5` |
| `DB01_EXTRA_BACKUP_OPTS` | Pass extra arguments to the backup command only, add them here e.g. `--extra-command` | |
| `DB01_EXTRA_ENUMERATION_OPTS` | Pass extra arguments to the database enumeration command only, add them here e.g. `--extra-command` | |
| `DB01_EXTRA_OPTS` | Pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | |
| `DB01_LOG_LEVEL` | Log output on screen and in files `INFO` `NOTICE` `ERROR` `WARN` `DEBUG` | `debug` |
| `DB01_SKIP_AVAILABILITY_CHECK` | Before backing up - skip connectivity check | `FALSE` |
##### Compression Options
| Variable | Description | Default |
| ----------------------------------- | ---------------------------------------------------------------------------------------------- | -------------- |
| `DB01_COMPRESSION` | Use either Gzip `GZ`, Bzip2 `BZ`, XZip `XZ`, ZSTD `ZSTD` or none `NONE` | `ZSTD` |
| `DB01_COMPRESSION_LEVEL` | Numerical value of what level of compression to use, most allow `1` to `9` | `3` |
| | except for `ZSTD` which allows for `1` to `19` | |
| `DB01_GZ_RSYNCABLE` | Use `--rsyncable` (gzip only) for faster rsync transfers and incremental backup deduplication. | `FALSE` |
| `DB01_ENABLE_PARALLEL_COMPRESSION` | Use multiple cores when compressing backups `TRUE` or `FALSE` | `TRUE` |
| `DB01_PARALLEL_COMPRESSION_THREADS` | Maximum amount of threads to use when compressing - Integer value e.g. `8` | `autodetected` |
##### Scheduling Options
| Variable | Description | Default |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `DB01_BACKUP_INTERVAL` | How often to do a backup, in minutes after the first backup. Defaults to 1440 minutes, or once per day. | `1440` |
| `DB01_BACKUP_BEGIN` | What time to do the initial backup. Defaults to immediate. (`+1`) | `+0` |
| | Must be in one of two formats: | |
| | Absolute HHMM, e.g. `2330` or `0415` | |
| | Relative +MM, i.e. how many minutes after starting the container, e.g. `+0` (immediate), `+10` (in 10 minutes), or `+90` in an hour and a half | |
| `DB01_CLEANUP_TIME` | Value in minutes to delete old backups (only fired when backup interval executes) | `FALSE` |
| | 1440 would delete anything above 1 day old. You don't need to set this variable if you want to hold onto everything. | |
| `DB01_ARCHIVE_TIME` | Value in minutes to move all files files older than (x) from `DB01_BACKUP_FILESYSTEM_PATH` | |
| | to `DB01_BACKUP_FILESYSTEM_ARCHIVE_PATH` - which is useful when pairing against an external backup system. | |
| `DB01_BACKUP_BLACKOUT_BEGIN` | Use `HHMM` notation to start a blackout period where no backups occur eg `0420` | |
| `DB01_BACKUP_BLACKOUT_END` | Use `HHMM` notation to set the end period where no backups occur eg `0430` | |
##### Specific Database Options
###### CouchDB
| Variable | Description | Default | `_FILE` |
| ----------- | ------------ | ------- | ------- |
| `DB01_PORT` | CouchDB Port | `5984` | x |
###### InfluxDB
| Variable | Description | Default | `_FILE` |
| --------------------- | ------------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DB01_PORT` | InfluxDB Port | | x |
| | Version 1.x | `8088` | |
| | Version 2.x | `8086` | |
| `DB01_INFLUX_VERSION` | What Version of Influx are you backing up from `1`.x or `2` series - amd64 and aarch/armv8 only for `2` | `2` | |
> Your Organization will be mapped to `DB_USER` and your root token will need to be mapped to `DB_PASS`.
> You may use `DB_NAME=ALL` to backup the entire set of databases.
> For `DB_HOST` use syntax of `http(s)://db-name`
###### MariaDB/MySQL
| Variable | Description | Default | `_FILE` |
| ------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------- | ------- |
| `DB01_EXTRA_OPTS` | Pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | |
| `DB01_EXTRA_BACKUP_OPTS` | Pass extra arguments to the backup command only, add them here e.g. `--extra-command` | |
| `DB01_EXTRA_ENUMERATION_OPTS` | Pass extra arguments to the database enumeration command only, add them here e.g. `--extra-command` | |
| `DB01_NAME` | Schema Name e.g. `database` or `ALL` to backup all databases the user has access to. | | |
| | Backup multiple by separating with commas eg `db1,db2` | | x |
| `DB01_NAME_EXCLUDE` | If using `ALL` - use this as to exclude databases separated via commas from being backed up | | x |
| `DB01_PORT` | MariaDB Port | `3306` | x |
| `DB01_MYSQL_MAX_ALLOWED_PACKET` | Max allowed packet if backing up MySQL / MariaDB | `512M` | |
| `DB01_MYSQL_SINGLE_TRANSACTION` | Backup in a single transaction with MySQL / MariaDB | `TRUE` | |
| `DB01_MYSQL_STORED_PROCEDURES` | Backup stored procedures with MySQL / MariaDB | `TRUE` | |
| `DB01_MYSQL_ENABLE_TLS` | Enable TLS functionality for MySQL/MariaDB client | `FALSE` | |
| `DB01_MYSQL_TLS_VERIFY` | (optional) If using TLS (by means of MYSQL_TLS_* variables) verify remote host | `FALSE` | |
| `DB01_MYSQL_TLS_VERSION` | What TLS `v1.1` `v1.2` `v1.3` version to utilize | `TLSv1.1,TLSv1.2,TLSv1.3` | |
| `DB01_MYSQL_TLS_CA_FILE` | Filename to load custom CA certificate for connecting via TLS | `/etc/ssl/cert.pem` | x |
| `DB01_MYSQL_TLS_CERT_FILE` | Filename to load client certificate for connecting via TLS | | x |
| `DB01_MYSQL_TLS_KEY_FILE` | Filename to load client key for connecting via TLS | | x |
###### Microsoft SQL
| Variable | Description | Default | `_FILE` |
| ----------- | ------------------ | ------- | ------- |
| `DB01_PORT` | Microsoft SQL Port | `1433` | x |
###### MongoDB
| Variable | Description | Default | `_FILE` |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ------- | ------- |
| `DB01_AUTH` | (Optional) Authentication Database | | |
| `DB01_PORT` | MongoDB Port | `27017` | x |
| `DB01_MONGO_CUSTOM_URI` | If you wish to override the MongoDB Connection string enter it here e.g. `mongodb+srv://username:password@cluster.id.mongodb.net` | | x |
| | This environment variable will be parsed and populate the `DB_NAME` and `DB_HOST` variables to properly build your backup filenames. | | |
| | You can override them by making your own entries | | |
###### Postgresql
| Variable | Description | Default | `_FILE` |
| ----------------------------- | --------------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DB01_AUTH` | (Optional) Authentication Database | | |
| `DB01_EXTRA_OPTS` | Pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | |
| `DB01_EXTRA_BACKUP_OPTS` | Pass extra arguments to the backup command only, add them here e.g. `--extra-command` | |
| `DB01_EXTRA_ENUMERATION_OPTS` | Pass extra arguments to the database enumeration command only, add them here e.g. `--extra-command` | |
| `DB01_NAME` | Schema Name e.g. `database` or `ALL` to backup all databases the user has access to. | | |
| | Backup multiple by separating with commas eg `db1,db2` | | x |
| `DB01_PORT` | PostgreSQL Port | `5432` | x |
###### Redis
| Variable | Description | Default | `_FILE` |
| ------------------------ | --------------------------------------------------------------------------------------------------------- | ------- | ------- |
| `DB01_EXTRA_OPTS` | Pass extra arguments to the backup and database enumeration command, add them here e.g. `--extra-command` | |
| `DB01_EXTRA_BACKUP_OPTS` | Pass extra arguments to the backup command only, add them here e.g. `--extra-command` | |
| `DB01_PORT` | Redis Port | `6379` | x |
###### SQLite
| Variable | Description | Default | `_FILE` |
| ----------- | -------------------------------------------------------- | ------- | ------- |
| `DB01_HOST` | Enter the full path to DB file e.g. `/backup/db.sqlite3` | | x |
##### Specific Storage Options
Options that are related to the value of `DB01_BACKUP_LOCATION`
###### Filesystem
If `DB01_BACKUP_LOCTION` = `FILESYSTEM` then the following options are used:
| Variable | Description | Default |
| ------------------------------ | ----------------------------------------------------------------------------------------------------- | --------------------------------- |
| `DB01_CREATE_LATEST_SYMLINK` | Create a symbolic link pointing to last backup in this format: `latest-(DB_TYPE)-(DB_NAME)-(DB_HOST)` | `TRUE` |
| `DB01_FILESYSTEM_PATH` | Directory where the database dumps are kept. | `/backup` |
| `DB01_FILESYSTEM_ARCHIVE_PATH` | Optional Directory where the database dumps archives are kept | `${DB01_FILESYSTEM_PATH/archive/` |
| `DB01_FILESYSTEM_PERMISSION` | Directory and File permissions to apply to files. | `700` |
###### S3
If `DB01_BACKUP_LOCATION` = `S3` then the following options are used:
| Parameter | Description | Default | `_FILE` |
| -------------------------- | ----------------------------------------------------------------------------------------- | ------- | ------- |
| `DB01_S3_BUCKET` | S3 Bucket name e.g. `mybucket` | | x |
| `DB01_S3_KEY_ID` | S3 Key ID (Optional) | | x |
| `DB01_S3_KEY_SECRET` | S3 Key Secret (Optional) | | x |
| `DB01_S3_PATH` | S3 Pathname to save to (must NOT end in a trailing slash e.g. '`backup`') | | x |
| `DB01_S3_REGION` | Define region in which bucket is defined. Example: `ap-northeast-2` | | x |
| `DB01_S3_HOST` | Hostname (and port) of S3-compatible service, e.g. `minio:8080`. Defaults to AWS. | | x |
| `DB01_S3_PROTOCOL` | Protocol to connect to `DB01_S3_HOST`. Either `http` or `https`. Defaults to `https`. | `https` | x |
| `DB01_S3_EXTRA_OPTS` | Add any extra options to the end of the `aws-cli` process execution | | x |
| `DB01_S3_CERT_CA_FILE` | Map a volume and point to your custom CA Bundle for verification e.g. `/certs/bundle.pem` | | x |
| _*OR*_ | | | |
| `DB01_S3_CERT_SKIP_VERIFY` | Skip verifying self signed certificates when connecting | `TRUE` | |
> When `DB01_S3_KEY_ID` and/or `DB01_S3_KEY_SECRET` is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.
###### Azure
If `DB01_BACKUP_LOCATION` = `blobxfer` then the following options are used:.
| Parameter | Description | Default | `_FILE` |
| ----------------------------------- | ------------------------------------------- | ------------------- | ------- |
| `DB01_BLOBXFER_STORAGE_ACCOUNT` | Microsoft Azure Cloud storage account name. | | x |
| `DB01_BLOBXFER_STORAGE_ACCOUNT_KEY` | Microsoft Azure Cloud storage account key. | | x |
| `DB01_BLOBXFER_REMOTE_PATH` | Remote Azure path | `/docker-db-backup` | x |
> This service uploads files from backup target directory `DB01_BACKUP_FILESYSTEM_PATH`.
> If the a cleanup configuration in `DB01_CLEANUP_TIME` is defined, the remote directory on Azure storage will also be cleaned automatically.
##### Hooks
###### Path Options
| Parameter | Description | Default |
| --------------------------- | --------------------------------------------------------------------------- | ----------------------- |
| `DB01_SCRIPT_LOCATION_PRE` | Location on filesystem inside container to execute bash scripts pre backup | `/assets/scripts/pre/` |
| `DB01_SCRIPT_LOCATION_POST` | Location on filesystem inside container to execute bash scripts post backup | `/assets/scripts/post/` |
| `DB01_PRE_SCRIPT` | Fill this variable in with a command to execute pre backing up | |
| `DB01_POST_SCRIPT` | Fill this variable in with a command to execute post backing up | |
###### Pre Backup
If you want to execute a custom script before a backup starts, you can drop bash scripts with the extension of `.sh` in the location defined in `DB01_SCRIPT_LOCATION_PRE`. See the following example to utilize:
```bash
$ cat pre-script.sh
##!/bin/bash
# #### Example Pre Script
# #### $1=DB01_TYPE (Type of Backup)
# #### $2=DB01_HOST (Backup Host)
# #### $3=DB01_NAME (Name of Database backed up
# #### $4=BACKUP START TIME (Seconds since Epoch)
# #### $5=BACKUP FILENAME (Filename)
echo "${1} Backup Starting on ${2} for ${3} at ${4}. Filename: ${5}"
```
## script DB01_TYPE DB01_HOST DB01_NAME STARTEPOCH BACKUP_FILENAME
${f} "${backup_job_db_type}" "${backup_job_db_host}" "${backup_job_db_name}" "${backup_routines_start_time}" "${target}"
Outputs the following on the console:
`mysql Backup Starting on example-db for example at 1647370800. Filename: mysql_example_example-db_202200315-000000.sql.bz2
###### Post backup
If you want to execute a custom script at the end of a backup, you can drop bash scripts with the extension of `.sh` in the location defined in `DB01_SCRIPT_LOCATION_POST`. Also to support legacy users `/assets/custom-scripts` is also scanned and executed.See the following example to utilize:
```bash
$ cat post-script.sh
##!/bin/bash
# #### Example Post Script
# #### $1=EXIT_CODE (After running backup routine)
# #### $2=DB_TYPE (Type of Backup)
# #### $3=DB_HOST (Backup Host)
# #### #4=DB_NAME (Name of Database backed up
# #### $5=BACKUP START TIME (Seconds since Epoch)
# #### $6=BACKUP FINISH TIME (Seconds since Epoch)
# #### $7=BACKUP TOTAL TIME (Seconds between Start and Finish)
# #### $8=BACKUP FILENAME (Filename)
# #### $9=BACKUP FILESIZE
# #### $10=HASH (If CHECKSUM enabled)
# #### $11=MOVE_EXIT_CODE
echo "${1} ${2} Backup Completed on ${3} for ${4} on ${5} ending ${6} for a duration of ${7} seconds. Filename: ${8} Size: ${9} bytes MD5: ${10}"
```
## script EXIT_CODE DB_TYPE DB_HOST DB_NAME STARTEPOCH FINISHEPOCH DURATIONEPOCH BACKUP_FILENAME FILESIZE CHECKSUMVALUE
${f} "${exit_code}" "${dbtype}" "${dbhost}" "${dbname}" "${backup_routines_start_time}" "${backup_routines_finish_time}" "${backup_routines_total_time}" "${target}" "${FILESIZE}" "${checksum_value}" "${move_exit_code}
Outputs the following on the console:
`0 mysql Backup Completed on example-db for example on 1647370800 ending 1647370920 for a duration of 120 seconds. Filename: mysql_example_example-db_202200315-000000.sql.bz2 Size: 7795 bytes Hash: 952fbaafa30437494fdf3989a662cd40 0`
If you wish to change the size value from bytes to megabytes set environment variable `DB01_SIZE_VALUE=megabytes`
You must make your scripts executable otherwise there is an internal check that will skip trying to run it otherwise.
If for some reason your filesystem or host is not detecting it right, use the environment variable `DB01_POST_SCRIPT_SKIP_X_VERIFY=TRUE` to bypass.
#### Notifications
This image has capabilities on sending notifications via a handful of services when a backup job fails. This is a global option that cannot be individually set per backup job.
| Parameter | Description | Default |
| ---------------------- | --------------------------------------------------------------------------------- | ------- |
| `ENABLE_NOTIFICATIONS` | Enable Notifications | `FALSE` |
| `NOTIFICATION_TYPE` | `CUSTOM` `EMAIL` `MATRIX` `MATTERMOST` `ROCKETCHAT` - Seperate Multiple by commas | |
##### Custom Notifications
The following is sent to the custom script. Use how you wish:
````
$1 unix timestamp
$2 logfile
$3 errorcode
$4 subject
$5 body/error message
````
| Parameter | Description | Default |
| ---------------------------- | ------------------------------------------------------- | ------- |
| `NOTIFICATION_CUSTOM_SCRIPT` | Path and name of custom script to execute notification. | |
##### Email Notifications
| Parameter | Description | Default | `_FILE` |
| ----------- | ----------------------------------------------------------------------------------------- | ------- | ------- |
| `MAIL_FROM` | What email address to send mail from for errors | | |
| `MAIL_TO` | What email address to send mail to for errors. Send to multiple by seperating with comma. | | |
| `SMTP_HOST` | What SMTP server to use for sending mail | | x |
| `SMTP_PORT` | What SMTP port to use for sending mail | | x |
##### Matrix Notifications
Fetch a `MATRIX_ACCESS_TOKEN`:
````
curl -XPOST -d '{"type":"m.login.password", "user":"myuserid", "password":"mypass"}' "https://matrix.org/_matrix/client/r0/login"
````
Copy the JSON response `access_token` that will look something like this:
````
{"access_token":"MDAxO...blahblah","refresh_token":"MDAxO...blahblah","home_server":"matrix.org","user_id":"@myuserid:matrix.org"}
````
| Parameter | Description | Default | `_FILE` |
| --------------------- | ---------------------------------------------------------------------------------------- | ------- | ------- |
| `MATRIX_HOST` | URL (https://matrix.example.com) of Matrix Homeserver | | x |
| `MATRIX_ROOM` | Room ID eg `\!abcdef:example.com` to send to. Send to multiple by seperating with comma. | | x |
| `MATRIX_ACCESS_TOKEN` | Access token of user authorized to send to room | | x |
##### Mattermost Notifications
| Parameter | Description | Default | `_FILE` |
| ------------------------ | -------------------------------------------------------------------------------------------- | ------- | ------- |
| `MATTERMOST_WEBHOOK_URL` | Full URL to send webhook notifications to | | x |
| `MATTERMOST_RECIPIENT` | Channel or User to send Webhook notifications to. Send to multiple by seperating with comma. | | x |
| `MATTERMOST_USERNAME` | Username to send as eg `tiredofit` | | x |
##### Rocketchat Notifications
| Parameter | Description | Default | `_FILE` |
| ------------------------ | -------------------------------------------------------------------------------------------- | ------- | ------- |
| `ROCKETCHAT_WEBHOOK_URL` | Full URL to send webhook notifications to | | x |
| `ROCKETCHAT_RECIPIENT` | Channel or User to send Webhook notifications to. Send to multiple by seperating with comma. | | x |
| `ROCKETCHAT_USERNAME` | Username to send as eg `tiredofit` | | x |
## Maintenance
@@ -269,6 +755,9 @@ Manual Backups can be performed by entering the container and typing `backup-now
- Recently there was a request to have the container work with Kubernetes cron scheduling. This can theoretically be accomplished by setting the container `MODE=MANUAL` and then setting `MANUAL_RUN_FOREVER=FALSE` - You would also want to disable a few features from the upstream base images specifically `CONTAINER_ENABLE_SCHEDULING` and `CONTAINER_ENABLE_MONITORING`. This should allow the container to start, execute a backup by executing and then exit cleanly. An alternative way to running the script is to execute `/etc/services.available/10-db-backup/run`.
### Manual Backups
Manual Backups can be performed by entering the container and typing `backup-now`. This will execute all the backup tasks that are scheduled by means of the `BACKUPXX_` variables. Alternatively if you wanted to execute a job on its own you could simply type `backup01-now` (or whatever your number would be). There is no concurrency, and jobs will be executed sequentially.
### Restoring Databases
Entering in the container and executing `restore` will execute a menu based script to restore your backups - MariaDB, Postgres, and Mongo supported.
@@ -292,75 +781,7 @@ The script can also be executed skipping the interactive mode by using the follo
If you only enter some of the arguments you will be prompted to fill them in.
### Custom Scripts
#### Path Options
| Parameter | Description | Default |
| ---------------------- | --------------------------------------------------------------------------- | ----------------------- |
| `SCRIPT_LOCATION_PRE` | Location on filesystem inside container to execute bash scripts pre backup | `/assets/scripts/pre/` |
| `SCRIPT_LOCATION_POST` | Location on filesystem inside container to execute bash scripts post backup | `/assets/scripts/post/` |
#### Pre Backup
If you want to execute a custom script before a backup starts, you can drop bash scripts with the extension of `.sh` in the location defined in `SCRIPT_LOCATION_PRE`. See the following example to utilize:
```bash
$ cat pre-script.sh
##!/bin/bash
# #### Example Pre Script
# #### $1=DB_TYPE (Type of Backup)
# #### $2=DB_HOST (Backup Host)
# #### $3=DB_NAME (Name of Database backed up
# #### $4=BACKUP START TIME (Seconds since Epoch)ff
# #### $5=BACKUP FILENAME (Filename)
echo "${1} Backup Starting on ${2} for ${3} at ${4}. Filename: ${5}"
```
## script DB_TYPE DB_HOST DB_NAME STARTEPOCH BACKUP_FILENAME
${f} "${dbtype}" "${dbhost}" "${dbname}" "${backup_start_time}" "${target}"
Outputs the following on the console:
`mysql Backup Starting on example-db for example at 1647370800. Filename: mysql_example_example-db_202200315-000000.sql.bz2
#### Post backup
If you want to execute a custom script at the end of a backup, you can drop bash scripts with the extension of `.sh` in the location defined in `SCRIPT_LOCATION_POST`. Also to support legacy users `/assets/custom-scripts` is also scanned and executed.See the following example to utilize:
```bash
$ cat post-script.sh
##!/bin/bash
# #### Example Post Script
# #### $1=EXIT_CODE (After running backup routine)
# #### $2=DB_TYPE (Type of Backup)
# #### $3=DB_HOST (Backup Host)
# #### #4=DB_NAME (Name of Database backed up
# #### $5=BACKUP START TIME (Seconds since Epoch)
# #### $6=BACKUP FINISH TIME (Seconds since Epoch)
# #### $7=BACKUP TOTAL TIME (Seconds between Start and Finish)
# #### $8=BACKUP FILENAME (Filename)
# #### $9=BACKUP FILESIZE
# #### $10=HASH (If CHECKSUM enabled)
# #### $11=MOVE_EXIT_CODE
echo "${1} ${2} Backup Completed on ${3} for ${4} on ${5} ending ${6} for a duration of ${7} seconds. Filename: ${8} Size: ${9} bytes MD5: ${10}"
```
## script EXIT_CODE DB_TYPE DB_HOST DB_NAME STARTEPOCH FINISHEPOCH DURATIONEPOCH BACKUP_FILENAME FILESIZE CHECKSUMVALUE
${f} "${exit_code}" "${dbtype}" "${dbhost}" "${dbname}" "${backup_start_timme}" "${backup_finish_time}" "${backup_total_time}" "${target}" "${FILESIZE}" "${checksum_value}" "${move_exit_code}
Outputs the following on the console:
`0 mysql Backup Completed on example-db for example on 1647370800 ending 1647370920 for a duration of 120 seconds. Filename: mysql_example_example-db_202200315-000000.sql.bz2 Size: 7795 bytes Hash: 952fbaafa30437494fdf3989a662cd40 0`
If you wish to change the size value from bytes to megabytes set environment variable `SIZE_VALUE=megabytes`
You must make your scripts executable otherwise there is an internal check that will skip trying to run it otherwise.
If for some reason your filesystem or host is not detecting it right, use the environment variable `POST_SCRIPT_SKIP_X_VERIFY=TRUE` to bypass.
## Support

1
install/assets/defaults/10-db-backup
View File

@@ -13,7 +13,6 @@ DB_DUMP_TARGET_PERMISSION=${DB_DUMP_TARGET_PERMISSION:-"700"}
DB_DUMP_TARGET_ARCHIVE=${DB_DUMP_TARGET_ARCHIVE:-"${DB_DUMP_TARGET}/archive/"}
ENABLE_CHECKSUM=${ENABLE_CHECKSUM:-"TRUE"}
ENABLE_PARALLEL_COMPRESSION=${ENABLE_PARALLEL_COMPRESSION:-"TRUE"}
LOG_FILE=${LOG_FILE:-"dbbackup.log"}
LOG_LEVEL=${LOG_LEVEL:-"debug"}
LOG_PATH=${LOG_PATH:-"/logs"}
MANUAL_RUN_FOREVER=${MANUAL_RUN_FOREVER:-"TRUE"}

12
install/assets/functions/10-db-backup
View File

@@ -19,7 +19,7 @@ bootstrap_filesystem() {
if [ "$(stat -c %U "${LOG_PATH}")" != "dbbackup" ] ; then chown dbbackup:dbbackup "${LOG_PATH}" ; fi
if [ "$(stat -c %a "${LOG_PATH}")" != "755" ] ; then chmod -R 755 "${LOG_PATH}" ; fi
create_logrotate dbbackup "${LOG_PATH}"/"${LOG_FILE}" none dbbackup dbbackup
echo "0 0 * * * /usr/sbin/logrotate_dbbackup >/dev/null 2>&1" > /assets/cron/dbbackup_logrotate
if [ ! -d "${TEMP_LOCATION}" ]; then
mkdir -p "${TEMP_LOCATION}"
@@ -963,7 +963,7 @@ write_log() {
case "${LOG_LEVEL,,}" in
"debug" )
print_debug "$@"
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [debug] $@" | run_as_user tee -a "${LOG_PATH}"/"${LOG_FILE}" > /dev/null
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [debug] $@" | run_as_user tee -a "${LOG_PATH}"/"${DB_TYPE}"_"${DB_NAME}"_"${DB_HOST}"_"${now}".log > /dev/null
;;
esac
;;
@@ -972,21 +972,21 @@ write_log() {
case "${LOG_LEVEL,,}" in
"debug" | "notice" | "warn" | "error")
print_error "$@"
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [error] $@" | run_as_user tee -a "${LOG_PATH}"/"${LOG_FILE}" > /dev/null
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [error] $@" | run_as_user tee -a "${LOG_PATH}"/"${DB_TYPE}"_"${DB_NAME}"_"${DB_HOST}"_"${now}".log > /dev/null
;;
esac
;;
info )
shift 1
print_info "$@"
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [info] $@" | run_as_user tee -a "${LOG_PATH}"/"${LOG_FILE}" > /dev/null
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [info] $@" | run_as_user tee -a "${LOG_PATH}"/"${DB_TYPE}"_"${DB_NAME}"_"${DB_HOST}"_"${now}".log > /dev/null
;;
notice )
shift 1
case "${LOG_LEVEL,,}" in
"debug" | "notice" )
print_notice "$@"
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [notice] $@" | run_as_user tee -a "${LOG_PATH}"/"${LOG_FILE}" > /dev/null
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [notice] $@" | run_as_user tee -a "${LOG_PATH}"/"${DB_TYPE}"_"${DB_NAME}"_"${DB_HOST}"_"${now}".log > /dev/null
;;
esac
;;
@@ -995,7 +995,7 @@ write_log() {
case "${LOG_LEVEL,,}" in
"debug" | "notice" | "warn" )
print_warn "$@"
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [warn] $@" | run_as_user tee -a "${LOG_PATH}"/"${LOG_FILE}" > /dev/null
echo "$(date +'%Y-%m-%d %H:%M:%S %Z') [warn] $@" | run_as_user tee -a "${LOG_PATH}"/"${DB_TYPE}"_"${DB_NAME}"_"${DB_HOST}"_"${now}".log > /dev/null
;;
esac
;;