Not like the usual multi-node Postgres replication cluster, when managed by Patroni, all failovers are mechanically executed. Nevertheless, this isn’t the case when coping with inter-datacentre failovers when as an example a standby datacentre should take over from a failed major.
The next describes the mechanisms required to carry out such a process when the case arises.
Herein two mechanism are described, each of which administers the clusters’ DCS:
- Failover execution by way of patronictl
- Failover execution by way of Patroni’s REST API
That is the bottom command allowing modifications to the DCs.
|
patronictl –c /and many others/patroni/config.yml edit–config —assist |
|
Utilization: patronictl edit–config [OPTIONS] [CLUSTER_NAME]
Edit cluster configuration
Choices: —group INTEGER Citus group –q, —quiet Do not present modifications –s, —set TEXT Set particular configuration worth. Can be specified a number of occasions –p, —pg TEXT Set particular PostgreSQL parameter worth. Shorthand for –s postgresql.parameters. Can be specified a number of occasions —apply TEXT Apply configuration from file. Use – for stdin. —substitute TEXT Apply configuration from file, changing present configuration. Use – for stdin. —power Do not ask for affirmation at any level —assist Present this message and exit. |
Patroni’s REST API is integral to Patroni’s chief election course of for operations like failovers, switchovers, reinitializations, restarts, and reloads. It can be employed by load balancers akin to HAProxy for HTTP well being checks as effectively for monitoring too.
Usually the CLI curl and jq are used when performing operations with the REST API.
For the needs of documentation, assume the next: Two datacentres comprising a 3 node replication cluster in every datacentre. Frankfurt is presently configured because the PRIMARY and AMSTERDAM the STANDBY datacentre respectively.
|
+ Cluster: Frankfurt (7546601820334116441) ———+——+—————–+ | Member | Host | Function | State | TL | Lag in MB | +————–+————————+————–+—————–+——+—————–+ | fradb1v | 10.108.134.168 | Chief | working | 2 | | | fradb2v | 10.108.134.113 | Duplicate | streaming | 2 | 0 | | fradb3v | 10.108.134.150 | Duplicate | streaming | 2 | 0 | +————–+————————+————–+—————–+——+—————–+ |
|
+ Cluster: Amsterdam (7546601820334116441) –+—————–+——+—————–+ | Member | Host | Function | State | TL | Lag in MB | +————–+————————+————————+—————–+——+—————–+ | amsdb1v | 10.108.134.211 | Standby Chief | streaming | 2 | | | amsdb2v | 10.108.134.235 | Duplicate | streaming | 2 | 0 | | amsdb3v | 10.108.134.29 | Duplicate | streaming | 2 | 0 | +————–+————————+————————+—————–+——+—————–+ |
Two steps are required to advertise the standby chief:
- Promote the present standby chief
- Drop the slot utilized by the remotely linked major
Model 1: Utilizing patronictl
Promote Standby Chief
The one distinction between these two variants is that one is interactive and the opposite is just not.
Variation A
|
# interactive; asks earlier than commiting patronictl –c /and many others/patroni/config.yml edit–config —set standby_cluster=null |
Variation B
|
# pressured execution patronictl –c /and many others/patroni/config.yml edit–config —set standby_cluster=null —power |
Create New Slot
This might be thought of an elective exercise the place a replication slot is explicitly created with the understanding {that a} new Standby datacentre is to be provisioned i.e. the failed major datacentre.
|
patronictl –c /and many others/patroni/config.yml edit–config —set slots.standby_cluster.kind=bodily —power |
Consideration: This and the earlier command could be mixed as a single command. For less than documentation functions had been they break up into two separate ones.
Model 2: Utilizing REST API
The good thing about this methodology is that it may be executed from any host that may entry any of the cluster’s port 8008 which is managed by Patroni.
Promote Standby Chief
The host is promoted, restoration is accomplished and turns into read-write.
|
curl –s –XPATCH –d ‘{“standby_cluster”:null}’ http://localhost:8008/config | jq . |
Create New Slot
As per the beforehand demonstrated instance, a brand new slot is created on the brand new Chief for the needs of replicating to a Standby Chief.
|
curl –s –XPATCH –d ‘{ “slots”:{“standby_cluster”:{“kind”: “bodily”}} }’ http://localhost:8008/config | jq . |
The next directions are considerably much like the earlier ones. Execute the next in Patroni cluster “Frankfurt”.
Model 1: patronictl
Provision New Standby Chief
|
patronictl –c /and many others/patroni/config.yml edit–config —set standby_cluster.host=‘amsdb1v,amsdb2v,amsdb3v’ —set standby_cluster.port=5432 —set standby_cluster.primary_slot_name=standby_cluster —set standby_cluster.create_replica_methods=basebackup —power |
Drop Previous Slot
Until there’s an acute purpose requiring the slot to stay on the newly promoted Chief it needs to be eliminated.
|
patronictl –c /and many others/patroni/config.yml edit–config —set slots=null —power |
Consideration: trying to take away the slot utilizing the postgres operate referred to as pg_drop_replication_slot will fail as a result of patroni will merely put it again.
Model 2: REST API
Provision New Standby Chief
|
curl –s –XPATCH –d ‘{ “standby_cluster”: { “host”: “amsdb1v,amsdb2v,amsdb3v”, “port”: 5432, “primary_slot_name”: “standby_cluster”, “create_replica_methods”: [ “basebackup” ] } }’ http://localhost:8008/config | jq . |
Drop Previous Slot
|
curl –s –XPATCH –d ‘{“slots”:null}’ http://localhost:8008/config | jq . |
- Bear in mind, you might be administering two distinct replication clusters i.e. Major and Standby.
- Utilizing patronictl requires entry to one of many hosts in every respective cluster.
- Utilizing the REST API
- presumes Patroni’s port 8008 could be reached when utilizing command line interfaces akin to curl.
- As a result of entry to the port is required it presents a possible safety danger subsequently both TLS with a password is used or the firewall rule have to be configured accordingly.
- It goes with out saying … be careful for break up mind eventualities
Lastly: right here’s a earlier weblog on Patroni catastrophe restoration which additionally explains the essential pondering behind the know-how.
