WAL-G

  • 是什么?
    • PostgreSQL 备份、恢复工具
    • WAL-E 后继 - Golang 重写 Python - 性能 x4
    • 支持 PostgreSQL, MySQL/MariaDB, Mongo
    • 支持 S3, GCS, Azure, Swift, 本地, SSH 存储
  • wal-g/wal-g
  • WAL-E 不支持关闭 S3 SSE - SelfHost 常见不便于使用
  • 参考
envdesc
WALG_S3_PREFIXs3://bucket/path/to/folder
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEYAWS_SESSION_TOKEN,~/.aws/credentials,AWS_PROFILE
AWS_REGIONus-west-2
AWS_ENDPOINThttp://s3-like-service:9000
AWS_S3_FORCE_PATH_STYLEfalse
http://BUCKET.s3.amazonaws.com/KEY -> http://s3.amazonaws.com/BUCKET/KEY
WALG_FILE_PREFIX/tmp/wal-g-test-data
WALG_SSH_PREFIXssh://localhost/walg-folder
SSH_PORT
SSH_USERNAME
SSH_PASSWORD
TOTAL_BG_UPLOADED_LIMIT32
WALG_S3_STORAGE_CLASSSTANDARD
WALG_S3_SSEfalse
WALG_S3_SSE_KMS_ID
WALG_COMPRESSION_METHODlz4,lzma,brotli
S3_USE_LIST_OBJECTS_V1false
# Minio
AWS_ACCESS_KEY_ID: '<minio-key>'
AWS_SECRET_ACCESS_KEY: '<minio-secret>'
WALG_S3_PREFIX: 's3://my-minio-bucket/sub-dir'
AWS_ENDPOINT: 'http://minio:9000'
AWS_S3_FORCE_PATH_STYLE: 'true'
AWS_REGION: us-east-1
WALG_S3_CA_CERT_FILE: '/path/to/custom/ca/file'
docker pull bitnami/wal-g
wal-g backup-list
namelast_modifiedwal_segment_backup_startstart_timefinish_timehostnamedata_dirpg_versionstart_lsnfinish_lsnis_permanent
base_0000000200000000000000052021-03-09T08:07:49Z000000020000000000000005Tuesday, 09-Mar-21 08:07:43 UTCTuesday, 09-Mar-21 08:07:49 UTCacid-demo/home/postgres/pgdata/pgroot/data1300028388612083927800false
base_0000000300000000000000082021-03-09T08:14:48Z000000030000000000000008Tuesday, 09-Mar-21 08:14:42 UTCTuesday, 09-Mar-21 08:14:48 UTCacid-demo/home/postgres/pgdata/pgroot/data130002134217768134250592false

目录结构

  • /
    • basebackup_005/
      • base_000000020000000000000005_backup_stop_sentinel.json
      • base_000000020000000000000005/
        • metadata.json
        • tar_partitions/
          • pg_control.tar.lz4
          • part_001.tar.lz4
    • wal_005/
      • 00000002.history.lz4
      • 000000020000000000000005.lz4
      • 000000020000000000000007.partial.lz4
      • 000000020000000000000005.00000028.backup.lz4

操作#

# Push base backup
wal-g backup-push /data
# Push WAL 片段
wal-g wal-push /data

postgresql.conf

wal_level = archive # hot_standby and logical in 9.x is also acceptable
archive_mode = on
archive_command = 'envdir /etc/wal-e.d/env wal-e wal-push %p'
archive_timeout = 60
wal-g backup-fetch /data LATEST
wal-g backup-fetch /data base_LONGWALNUMBER_POSITION_NUMBER

recovery.conf

# https://www.postgresql.org/docs/11/recovery-config.html
restore_command = 'envdir /etc/wal-e.d/env wal-e wal-fetch %f %p'
standby_mode = on
# 提升 standby
# https://www.postgresql.org/docs/current/app-pg-ctl.html
pg_ctl promote