Add support for backing up / importing all Postgres databases/users

This commit is contained in:
Slavi Pantaleev 2019-10-05 11:28:26 +03:00
parent 2f9a197d30
commit 29526e7bb1
5 changed files with 93 additions and 28 deletions

View File

@ -1,3 +1,17 @@
# 2019-10-05
## Improved Postgres upgrading/importing
Postgres [upgrading](docs/maintenance-postgres.md#upgrading-postgresql) and [importing](docs/importing-postgres.md) have been improved to add support for multiple databases and roles.
Previously, the playbook would only take care of the `homeserver` database and `synapse` user.
We now back up and restore all databases and users on the Postgres server.
For now, the playbook only uses that one database (`homeserver`) and that one single user (`synapse`), so it's all the same.
However, in the future, additional components besides Synapse may also make use the Postgres database server.
One such example is the [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) bridge, which strongly encourages use of Postgres in its v1.0 release. We are yet to upgrade to it.
# 2019-10-04 # 2019-10-04
## Postgres 12 support ## Postgres 12 support

View File

@ -12,6 +12,8 @@ If your database name differs, be sure to change `matrix_postgres_db_name` to yo
The playbook supports importing Postgres dump files in **text** (e.g. `pg_dump > dump.sql`) or **gzipped** formats (e.g. `pg_dump | gzip -c > dump.sql.gz`). The playbook supports importing Postgres dump files in **text** (e.g. `pg_dump > dump.sql`) or **gzipped** formats (e.g. `pg_dump | gzip -c > dump.sql.gz`).
Importing multiple databases (as dumped by `pg_dumpall`) is also supported.
Before doing the actual import, **you need to upload your Postgres dump file to the server** (any path is okay). Before doing the actual import, **you need to upload your Postgres dump file to the server** (any path is okay).

View File

@ -40,16 +40,18 @@ To make a back up of the current PostgreSQL database, make sure it's running and
```bash ```bash
docker run \ docker run \
--rm \ --rm \
--network matrix \ --network=matrix \
--env-file=/matrix/postgres/env-postgres-psql \ --env-file=/matrix/postgres/env-postgres-psql \
postgres:12.0-alpine \ postgres:12.0-alpine \
pg_dump -h matrix-postgres \ pg_dumpall -h matrix-postgres \
| gzip -c \ | gzip -c \
> /postgres.sql.gz > /postgres.sql.gz
``` ```
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above command will not work, because the credentials file (`/matrix/postgres/env-postgres-psql`) is not available. If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above command will not work, because the credentials file (`/matrix/postgres/env-postgres-psql`) is not available.
Restoring a backup made this way can be done by [importing it](importing-postgres.md).
## Upgrading PostgreSQL ## Upgrading PostgreSQL
@ -64,7 +66,7 @@ This playbook can upgrade your existing Postgres setup with the following comman
ansible-playbook -i inventory/hosts setup.yml --tags=upgrade-postgres ansible-playbook -i inventory/hosts setup.yml --tags=upgrade-postgres
**The old Postgres data directory is backed up** automatically, by renaming to `/matrix/postgres-auto-upgrade-backup`. **The old Postgres data directory is backed up** automatically, by renaming it to `/matrix/postgres-auto-upgrade-backup`.
To rename to a different path, pass some extra flags to the command above, like this: `--extra-vars="postgres_auto_upgrade_backup_data_path=/another/disk/matrix-postgres-before-upgrade"` To rename to a different path, pass some extra flags to the command above, like this: `--extra-vars="postgres_auto_upgrade_backup_data_path=/another/disk/matrix-postgres-before-upgrade"`
The auto-upgrade-backup directory stays around forever, until you **manually decide to delete it**. The auto-upgrade-backup directory stays around forever, until you **manually decide to delete it**.
@ -72,5 +74,4 @@ The auto-upgrade-backup directory stays around forever, until you **manually dec
As part of the upgrade, the database is dumped to `/tmp`, an upgraded and empty Postgres server is started, and then the dump is restored into the new server. As part of the upgrade, the database is dumped to `/tmp`, an upgraded and empty Postgres server is started, and then the dump is restored into the new server.
To use a different directory for the dump, pass some extra flags to the command above, like this: `--extra-vars="postgres_dump_dir=/directory/to/dump/here"` To use a different directory for the dump, pass some extra flags to the command above, like this: `--extra-vars="postgres_dump_dir=/directory/to/dump/here"`
**ONLY one database is migrated** (the one specified in `matrix_postgres_db_name`, named `homeserver` by default). **All databases, roles, etc. on the Postgres server are migrated**.
If you've created other databases in that database instance (something this playbook never does and never advises), data will be lost.

View File

@ -56,6 +56,10 @@
msg: "Could not find existing Postgres installation" msg: "Could not find existing Postgres installation"
when: "not matrix_postgres_detected_existing|bool" when: "not matrix_postgres_detected_existing|bool"
# Starting the database container had automatically created the default
# role (`matrix_postgres_connection_username`) and database (`matrix_postgres_db_name`).
# The dump most likely contains those same entries and would try to re-create them, leading to errors.
# We need to skip over those lines.
- name: Generate Postgres database import command - name: Generate Postgres database import command
set_fact: set_fact:
matrix_postgres_import_command: >- matrix_postgres_import_command: >-
@ -67,17 +71,26 @@
-v {{ server_path_postgres_dump }}:/{{ server_path_postgres_dump|basename }}:ro -v {{ server_path_postgres_dump }}:/{{ server_path_postgres_dump|basename }}:ro
--entrypoint=/bin/sh --entrypoint=/bin/sh
{{ matrix_postgres_docker_image_latest }} {{ matrix_postgres_docker_image_latest }}
-c 'cat /{{ server_path_postgres_dump|basename }} | -c "cat /{{ server_path_postgres_dump|basename }} |
{{ 'gunzip |' if server_path_postgres_dump.endswith('.gz') else '' }} {{ 'gunzip |' if server_path_postgres_dump.endswith('.gz') else '' }}
psql -v ON_ERROR_STOP=1 -h matrix-postgres' grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' |
grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' |
psql -v ON_ERROR_STOP=1 -h matrix-postgres"
# This is a hack.
# See: https://ansibledaily.com/print-to-standard-output-without-escaping/
#
# We want to run `debug: msg=".."`, but that dumps it as JSON and escapes double quotes within it,
# which ruins the command (`matrix_postgres_import_command`)
- name: Note about Postgres importing alternative - name: Note about Postgres importing alternative
debug: set_fact:
msg: >- dummy: true
Importing Postgres database using the following command: `{{ matrix_postgres_import_command }}`. with_items:
If this crashes, you can stop Postgres (`systemctl stop matrix-postgres`), - >-
delete its existing data (`rm -rf {{ matrix_postgres_data_path }}/*`), start it again (`systemctl start matrix-postgres`) Importing Postgres database using the following command: `{{ matrix_postgres_import_command }}`.
and manually run the above import command directly on the server. If this crashes, you can stop Postgres (`systemctl stop matrix-postgres`),
delete its existing data (`rm -rf {{ matrix_postgres_data_path }}/*`), start it again (`systemctl start matrix-postgres`)
and manually run the above import command directly on the server.
- name: Perform Postgres database import - name: Perform Postgres database import
command: "{{ matrix_postgres_import_command }}" command: "{{ matrix_postgres_import_command }}"

View File

@ -67,14 +67,21 @@
delegate_to: 127.0.0.1 delegate_to: 127.0.0.1
become: false become: false
# We dump all databases, roles, etc.
#
# Because we'll be importing into a new container which initializes the default
# role (`matrix_postgres_connection_username`) and database (`matrix_postgres_db_name`) by itself on startup,
# we need to remove these from the dump, or we'll get errors saying these already exist.
- name: Perform Postgres database dump - name: Perform Postgres database dump
command: | command: >-
/usr/bin/docker run --rm --name matrix-postgres-dump \ /usr/bin/docker run --rm --name matrix-postgres-dump
--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }}
--network={{ matrix_docker_network }} \ --network={{ matrix_docker_network }}
--env-file={{ matrix_postgres_base_path }}/env-postgres-psql \ --env-file={{ matrix_postgres_base_path }}/env-postgres-psql
-v {{ postgres_dump_dir }}:/out \ --entrypoint=/bin/sh
{{ matrix_postgres_detected_version_corresponding_docker_image }} pg_dump -h matrix-postgres {{ matrix_postgres_db_name }} -f /out/{{ postgres_dump_name }} -v {{ postgres_dump_dir }}:/out
{{ matrix_postgres_detected_version_corresponding_docker_image }}
-c "pg_dumpall -h matrix-postgres > /out/{{ postgres_dump_name }}"
- name: Ensure matrix-postgres is stopped - name: Ensure matrix-postgres is stopped
service: service:
@ -102,15 +109,43 @@
delegate_to: 127.0.0.1 delegate_to: 127.0.0.1
become: false become: false
# Starting the database container had automatically created the default
# role (`matrix_postgres_connection_username`) and database (`matrix_postgres_db_name`).
# The dump most likely contains those same entries and would try to re-create them, leading to errors.
# We need to skip over those lines.
- name: Generate Postgres database import command
set_fact:
matrix_postgres_import_command: >-
/usr/bin/docker run --rm --name matrix-postgres-import
--user={{ matrix_user_uid }}:{{ matrix_user_gid }}
--cap-drop=ALL
--network={{ matrix_docker_network }}
--env-file={{ matrix_postgres_base_path }}/env-postgres-psql
--entrypoint=/bin/sh
-v {{ postgres_dump_dir }}:/in:ro
{{ matrix_postgres_docker_image_latest }}
-c "cat /in/{{ postgres_dump_name }} |
grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' |
grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' |
psql -v ON_ERROR_STOP=1 -h matrix-postgres"
# This is a hack.
# See: https://ansibledaily.com/print-to-standard-output-without-escaping/
#
# We want to run `debug: msg=".."`, but that dumps it as JSON and escapes double quotes within it,
# which ruins the command (`matrix_postgres_import_command`)
- name: Note about Postgres importing
set_fact:
dummy: true
with_items:
- >-
Importing Postgres database using the following command: `{{ matrix_postgres_import_command }}`.
If this crashes, you can stop Postgres (`systemctl stop matrix-postgres`),
delete the new database data (`rm -rf {{ matrix_postgres_data_path }}`)
and restore the automatically-made backup (`mv {{ postgres_auto_upgrade_backup_data_path }} {{ matrix_postgres_data_path }}`).
- name: Perform Postgres database import - name: Perform Postgres database import
command: | command: "{{ matrix_postgres_import_command }}"
/usr/bin/docker run --rm --name matrix-postgres-import \
--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
--cap-drop=ALL \
--network={{ matrix_docker_network }} \
--env-file={{ matrix_postgres_base_path }}/env-postgres-psql \
-v {{ postgres_dump_dir }}:/in:ro \
{{ matrix_postgres_docker_image_latest }} psql -v ON_ERROR_STOP=1 -h matrix-postgres -f /in/{{ postgres_dump_name }}
- name: Delete Postgres database dump file - name: Delete Postgres database dump file
file: file: