patroni.ctl module

Implement patronictl: a command-line application which utilises the REST API to perform cluster operations.

var CONFIG_DIR_PATH

path to Patroni configuration directory as per click.get_app_dir() output.

var CONFIG_FILE_PATH

default path to patronictl.yaml configuration file.

var DCS_DEFAULTS

auxiliary dictionary to build the DCS section of the configuration file. Mainly used to help parsing --dcs-url command-line option of patronictl.

Note

Most of the patronictl commands (restart/reinit/pause/resume/show-config/edit-config and similar) require the group argument and work only for that specific Citus group. If not specified in the command line the group might be taken from the configuration file. If it is also missing in the configuration file we assume that this is just a normal Patroni cluster (not Citus).

exception patroni.ctl.PatroniCtlException(message: str)

Bases: click.exceptions.ClickException

Raised upon issues faced by patronictl utility.

class patroni.ctl.PatronictlPrettyTable(header: str, *args: Any, **kwargs: Any)

Bases: prettytable.prettytable.PrettyTable

Utilitary class to print pretty tables.

Extend PrettyTable to make it print custom information in the header line. The idea is to print a header line like this:

` + Cluster: batman --------+--------+---------+----+-----------+ `

Instead of the default header line which would contain only dash and plus characters.

patroni.ctl.apply_config_changes(before_editing: str, data: Dict[str, Any], kvpairs: List[str]) Tuple[str, Dict[str, Any]]

Apply config changes specified as a list of key-value pairs.

Keys are interpreted as dotted paths into the configuration data structure. Except for paths beginning with postgresql.parameters where rest of the path is used directly to allow for PostgreSQL GUCs containing dots. Values are interpreted as YAML values.

Parameters
  • before_editing – human representation before editing.

  • data – configuration data structure.

  • kvpairs – list of strings containing key value pairs separated by =.

Returns

tuple of human-readable, parsed data structure after changes.

Raises

PatroniCtlException: if any entry in kvpairs is None or not in the expected format.

patroni.ctl.apply_yaml_file(data: Dict[str, Any], filename: str) Tuple[str, Dict[str, Any]]

Apply changes from a YAML file to configuration.

Parameters
  • data – configuration data structure.

  • filename – name of the YAML file, - is taken to mean standard input.

Returns

tuple of human-readable and parsed data structure after changes.

patroni.ctl.check_response(response: urllib3.response.HTTPResponse, member_name: str, action_name: str, silent_success: bool = False) bool

Check an HTTP response and print a status message.

Parameters
  • response – the response to be checked.

  • member_name – name of the member associated with the response.

  • action_name – action associated with the response.

  • silent_success – if a status message should be skipped upon a successful response.

Returns

True if the response indicates a sucessful operation (HTTP status < 400), False otherwise.

patroni.ctl.confirm_members_action(members: List[patroni.dcs.Member], force: bool, action: str, scheduled_at: Optional[datetime.datetime] = None) None

Ask for confirmation if action should be taken by members.

Parameters
  • members – list of member which will take the action.

  • force – if True skip the confirmation prompt and allow the action to proceed.

  • action

    the action that is being processed, one among:

    • reload: reload PostgreSQL configuration; or

    • restart: restart PostgreSQL; or

    • reinitialize: reinitialize PostgreSQL data directory; or

    • flush: discard scheduled actions.

  • scheduled_at – timestamp at which the action should be scheduled to. If None action is taken immediately.

Raises

PatroniCtlException: if the user aborted the action.

patroni.ctl.format_config_for_editing(data: Any, default_flow_style: bool = False) str

Format configuration as YAML for human consumption.

Parameters
  • data – configuration as nested dictionaries.

  • default_flow_style – passed down as default_flow_style argument of yaml.safe_dump().

Returns

unicode YAML of the configuration.

patroni.ctl.format_pg_version(version: int) str

Format Postgres version for human consumption.

Parameters

version – Postgres version represented as an integer.

Returns

Postgres version represented as a human-readable string.

Example
>>> format_pg_version(90624)
'9.6.24'
>>> format_pg_version(100000)
'10.0'
>>> format_pg_version(140008)
'14.8'
patroni.ctl.generate_topology(level: int, member: Dict[str, Any], topology: Dict[str, List[Dict[str, Any]]]) Iterator[Dict[str, Any]]

Recursively yield members with their names adjusted according to their level in the cluster topology.

Note

The idea is to get a tree view of the members when printing their names. For example, suppose you have a cascading replication composed of 3 nodes, say postgresql0, postgresql1, and postgresql2. This function would adjust their names to be like this:

  • 'postgresql0' -> 'postgresql0'

  • 'postgresql1' -> '+ postgresql1'

  • 'postgresql2' -> '  + postgresql2'

So, if you ever print their names line by line, you would see something like this:

postgresql0
+ postgresql1
  + postgresql2
Parameters
  • level – the current level being inspected in the topology.

  • member

    information about the current member being inspected in level of topology. Should countain at least this key: * name: name of the node, according to name configuration;

    But may contain others, which although ignored by this function, will be yielded as part of the resulting object. The value of key name is changed as explained in the note.

  • topology – each key is the name of a node which has at least one replica attached to it. The corresponding value is a list of the attached replicas, each of them with the same structure described for member.

Yields

the current member with its name changed. Besides that reyield values from recursive calls.

patroni.ctl.get_all_members(obj: Dict[str, Any], cluster: patroni.dcs.Cluster, group: Optional[int], role: str = 'leader') Iterator[patroni.dcs.Member]

Get all cluster members that have the given role.

Parameters
  • obj – the Patroni configuration.

  • cluster – the Patroni cluster.

  • group – filter which Citus group we should get members from. If None get from all groups.

  • role

    role to filter members. Can be one among:

    • primary or master: the primary PostgreSQL instance;

    • replica or standby: a standby PostgreSQL instance;

    • leader: the leader of a Patroni cluster. Can also be used to get the leader of a Patroni standby cluster;

    • standby-leader: the leader of a Patroni standby cluster;

    • any: matches any node independent of its role.

Yields

members that have the given role.

patroni.ctl.get_all_members_leader_first(cluster: patroni.dcs.Cluster) Iterator[patroni.dcs.Member]

Get all cluster members, with the cluster leader being yielded first.

Note

Only yield members that have a restapi.connect_address configured.

Yields

all cluster members, with the leader first.

patroni.ctl.get_any_member(obj: Dict[str, Any], cluster: patroni.dcs.Cluster, group: Optional[int], role: Optional[str] = None, member: Optional[str] = None) Optional[patroni.dcs.Member]

Get the first found cluster member that has the given role.

Parameters
  • obj – the Patroni configuration.

  • cluster – the Patroni cluster.

  • group – filter which Citus group we should get members from. If None get from all groups.

  • role – role to filter members. See get_all_members() for available options.

  • member – if specified, then besides having the given role, the cluster member’s name should be member.

Returns

the first found cluster member that has the given role.

Raises

PatroniCtlException: if both role and member are provided.

patroni.ctl.get_cluster_service_info(cluster: Dict[str, Any]) List[str]

Get complementary information about the cluster.

Parameters

cluster – a Patroni cluster represented as an object created through cluster_as_json().

Returns

a list of 0 or more informational messages. They can be about:

  • Cluster in maintenance mode;

  • Scheduled switchovers.

patroni.ctl.get_cursor(obj: Dict[str, Any], cluster: patroni.dcs.Cluster, group: Optional[int], connect_parameters: Dict[str, Any], role: Optional[str] = None, member_name: Optional[str] = None) Optional[Union[cursor, Cursor[Any]]]

Get a cursor object to execute queries against a member that has the given role or member_name.

Note

Besides what is passed through connect_parameters, this function also sets the following parameters:
  • fallback_application_name: as Patroni ctl;

  • connect_timeout: as 5.

Parameters
  • obj – the Patroni configuration.

  • cluster – the Patroni cluster.

  • group – filter which Citus group we should get members to create a cursor against. If None consider members from all groups.

  • connect_parameters – database connection parameters.

  • role – role to filter members. See get_all_members() for available options.

  • member_name – if specified, then besides having the given role, the cluster member’s name should be member_name.

Returns

a cursor object to execute queries against the database. Can be either:

  • A psycopg.Cursor if using psycopg; or

  • A psycopg2.extensions.cursor if using psycopg2;

  • None if not able to get a cursor that attendees role and member_name.

patroni.ctl.get_dcs(config: Dict[str, Any], scope: str, group: Optional[int]) patroni.dcs.AbstractDCS

Get the DCS object.

Parameters
  • config – Patroni configuration.

  • scope – cluster name.

  • group – if group is defined, use it to select which alternative Citus group this DCS refers to. If group is None and a Citus configuration exists, assume this is the coordinator. Coordinator has the group 0. Refer to the module note for more details.

Returns

a subclass of AbstractDCS, according to the DCS technology that is configured.

Raises

PatroniCtlException: if not suitable DCS configuration could be found.

patroni.ctl.get_members(obj: Dict[str, Any], cluster: patroni.dcs.Cluster, cluster_name: str, member_names: List[str], role: str, force: bool, action: str, ask_confirmation: bool = True, group: Optional[int] = None) List[patroni.dcs.Member]

Get the list of members based on the given filters.

Note

Contain some filtering and checks processing that are common to several actions that are exposed by patronictl, like:

  • Get members of cluster that respect the given member_names, role, and group;

  • Bypass confirmations;

  • Prompt user for information that has not been passed through the command-line options;

  • etc.

Designed to handle both attended and unattended patronictl commands execution that need to retrieve and validate the members before doing anything.

In the very end may call confirm_members_action() to ask if the user would like to proceed with action over the retrieved members. That won’t actually perform the action, but it works as the “last confirmation” before the action is processed by the caller method.

Additional checks can also be implemented in the caller method, in which case you might want to pass ask_confirmation=False, and later call confirm_members_action() manually in the caller method. That way the workflow won’t look broken to the user that is interacting with patronictl.

Parameters
  • obj – Patroni configuration.

  • cluster – Patroni cluster.

  • cluster_name – name of the Patroni cluster.

  • member_names – used to filter which members should take the action based on their names. Each item is the name of a Patroni member, as per name configuration. If member_names is an empty tuple no filters are applied based on names.

  • role – used to filter which members should take the action based on their role. See get_all_members() for available options.

  • force – if True, then it won’t ask for confirmations at any point nor prompt the user to select values for options that were not specified through the command-line.

  • action

    the action that is being processed, one among:

    • reload: reload PostgreSQL configuration; or

    • restart: restart PostgreSQL; or

    • reinitialize: reinitialize PostgreSQL data directory; or

    • flush: discard scheduled actions.

  • ask_confirmation – if False, then it won’t ask for the final confirmation regarding the action before returning the list of members. Usually useful as False if you want to perform additional checks in the caller method besides the checks that are performed through this generic method.

  • group – filter which Citus group we should get members from. If None consider members from all groups.

Returns

a list of members that respect the given filters.

Raises
PatroniCtlException: if
  • Cluster does not have members that match the given role; or

  • Cluster does not have members that match the given member_names; or

  • No member with given role is found among the specified member_names.

patroni.ctl.invoke_editor(before_editing: str, cluster_name: str) Tuple[str, Dict[str, Any]]

Start editor command to edit configuration in human readable format.

Note

Requires an editor program, and uses first found among:
  • Program given by EDITOR environemnt variable; or

  • editor; or

  • vi.

Parameters
  • before_editing – human representation before editing.

  • cluster_name – name of the Patroni cluster.

Returns

tuple of human-readable, parsed data structure after changes.

Raises
PatroniCtlException: if
  • No suitable editor can be found; or

  • Editor call exits with unexpected return code.

patroni.ctl.load_config(path: str, dcs_url: Optional[str]) Dict[str, Any]

Load configuration file from path and optionally override its DCS configuration with dcs_url.

Parameters
  • path – path to the configuration file.

  • dcs_url – the DCS URL in the format DCS://HOST:PORT, e.g. etcd3://random.com:2399. If given override whatever DCS is set in the configuration file.

Returns

a dictionary representing the configuration.

Raises

PatroniCtlException: if path does not exist or is not readable.

patroni.ctl.output_members(obj: Dict[str, Any], cluster: patroni.dcs.Cluster, name: str, extended: bool = False, fmt: str = 'pretty', group: Optional[int] = None) None

Print information about the Patroni cluster and its members.

Information is printed to console through print_output(), and contains:

  • Cluster: name of the Patroni cluster, as per scope configuration;

  • Member: name of the Patroni node, as per name configuration;

  • Host: hostname (or IP) and port, as per postgresql.listen configuration;

  • Role: Leader, Standby Leader, Sync Standby or Replica;

  • State: stopping, stopped, stop failed, crashed, running, starting, start failed, restarting, restart failed, initializing new cluster, initdb failed, running custom bootstrap script, custom bootstrap failed, creating replica, streaming, in archive recovery, and so on;

  • TL: current timeline in Postgres; Lag in MB: replication lag.

Besides that it may also have:
  • Group: Citus group ID – showed only if Citus is enabled.

  • Pending restart: if the node is pending a restart – showed only if extended;

  • Scheduled restart: timestamp for scheduled restart, if any – showed only if extended;

  • Tags: node tags, if any – showed only if extended.

The 3 extended columns are always included if extended, even if the member has no value for a given column. If not extended, these columns may still be shown if any of the members has any information for them.

Parameters
  • obj – Patroni configuration.

  • cluster – Patroni cluster.

  • name – name of the Patroni cluster.

  • extended – if extended information (pending restarts, scheduled restarts, node tags) should be printed, if available.

  • fmt – the output table printing format. See print_output() for available options. If fmt is neither topology nor pretty, then complementary information gathered through get_cluster_service_info() is not printed.

  • group – filter which Citus group we should get members from. If None get from all groups.

patroni.ctl.parse_dcs(dcs: Optional[str]) Optional[Dict[str, Any]]

Parse a DCS URL.

Parameters

dcs

the DCS URL in the format DCS://HOST:PORT. DCS can be one among:

  • consul

  • etcd

  • etcd3

  • exhibitor

  • zookeeper

If DCS is not specified, assume etcd by default. If HOST is not specified, assume localhost by default. If PORT is not specified, assume the default port of the given DCS.

Returns

None if dcs is None, otherwise a dictionary. The dictionary represents dcs as if it were parsed from the Patroni configuration file.

Raises

PatroniCtlException: if the DCS name in dcs is not valid.

Example
>>> parse_dcs('')
{'etcd': {'host': 'localhost:2379'}}
>>> parse_dcs('etcd://:2399')
{'etcd': {'host': 'localhost:2399'}}
>>> parse_dcs('etcd://test')
{'etcd': {'host': 'test:2379'}}
>>> parse_dcs('etcd3://random.com:2399')
{'etcd3': {'host': 'random.com:2399'}}
patroni.ctl.parse_scheduled(scheduled: Optional[str]) Optional[datetime.datetime]

Parse a string scheduled timestamp as a datetime object.

Parameters

scheduled – string representation of the timestamp. May also be now.

Returns

the corresponding datetime object, if scheduled is not now, otherwise None.

Raises

PatroniCtlException: if unable to parse scheduled from str to datetime.

Example
>>> parse_scheduled(None) is None
True
>>> parse_scheduled('now') is None
True
>>> parse_scheduled('2023-05-29T04:32:31')
datetime.datetime(2023, 5, 29, 4, 32, 31, tzinfo=tzlocal())
>>> parse_scheduled('2023-05-29T04:32:31-3')
datetime.datetime(2023, 5, 29, 4, 32, 31, tzinfo=tzoffset(None, -10800))
patroni.ctl.print_output(columns: Optional[List[str]], rows: List[List[Any]], alignment: Optional[Dict[str, str]] = None, fmt: str = 'pretty', header: str = '', delimiter: str = '\t') None

Print tabular information.

Parameters
  • columns – list of column names.

  • rows – list of rows. Each item is a list of values for the columns.

  • alignment

    alignment to be applied to column values. Each key is the name of a column to be aligned, and the corresponding value can be one among:

    • l: left-aligned

    • c: center-aligned

    • r: right-aligned

    A key in the dictionary is only required for a column that needs a specific alignment. Only apply when fmt is either pretty or topology.

  • fmt

    the printing format. Can be one among:

    • json: to print as a JSON string – array of objects;

    • yaml or yml: to print as a YAML string;

    • tsv: to print a table of separated values, by default by tab;

    • pretty: to print a pretty table;

    • topology: similar to pretty, but with a topology view when printing cluster members.

  • header – a string to be included in the first line of the table header, typically the cluster name. Only apply when fmt is either pretty or topology.

  • delimiter – the character to be used as delimiter when fmt is tsv.

patroni.ctl.query_member(obj: Dict[str, Any], cluster: patroni.dcs.Cluster, group: Optional[int], cursor: Optional[Union[cursor, Cursor[Any]]], member: Optional[str], role: Optional[str], command: str, connect_parameters: Dict[str, Any]) Tuple[List[List[Any]], Optional[List[Any]]]

Execute SQL command against a member.

Parameters
  • obj – Patroni configuration.

  • cluster – the Patroni cluster.

  • group – filter which Citus group we should get members from to perform the query. Refer to the module note for more details.

  • cursor – cursor through which command is executed. If None a new cursor is instantiated through get_cursor().

  • member – filter which member to create a cursor against based on its name, if cursor is None.

  • role – filter which member to create a cursor against based on their role, if cursor is None. See get_all_members() for available options.

  • command – SQL command to be executed.

  • connect_parameters – connection parameters to be passed down to get_cursor(), if cursor is None.

Returns

a tuple composed of two items:

  • List of rows returned by the executed command;

  • List of columns related to the rows returned by the executed command.

If an error occurs while executing command, then returns the following values in the tuple:

  • List with 2 items:

    • Current timestamp;

    • Error message.

  • None.

patroni.ctl.request_patroni(member: patroni.dcs.Member, method: str = 'GET', endpoint: Optional[str] = None, data: Optional[Any] = None) urllib3.response.HTTPResponse

Perform a request to Patroni REST API.

Parameters
  • member – DCS member, used to get the base URL of its REST API server.

  • method – HTTP method to be used, e.g. GET.

  • endpoint – URL path of the request, e.g. patroni.

  • data – anything to be used as the request body.

Returns

the response for the request.

patroni.ctl.show_diff(before_editing: str, after_editing: str) None

Show a diff between two strings.

Inputs are expected to be unicode strings.

If the output is to a tty the diff will be colored.

Note

If tty it requires a pager program, and uses first found among:
  • Program given by PAGER environment variable; or

  • less; or

  • more.

Parameters
  • before_editing – string to be compared with after_editing.

  • after_editing – string to be compared with before_editing.

Raises

PatroniCtlException: if no suitable pager can be found when printing diff output to a tty.

patroni.ctl.temporary_file(contents: bytes, suffix: str = '', prefix: str = 'tmp') Iterator[str]

Create a temporary file with specified contents that persists for the context.

Parameters
  • contents – binary string that will be written to the file.

  • prefix – will be prefixed to the filename.

  • suffix – will be appended to the filename.

Yields

path of the created file.

patroni.ctl.timestamp(precision: int = 6) str

Get current timestamp with given precision as a string.

Parameters

precision – Amount of digits to be present in the precision.

Returns

the current timestamp with given precision.

patroni.ctl.toggle_pause(config: Dict[str, Any], cluster_name: str, group: Optional[int], paused: bool, wait: bool) None

Toggle the pause state in the cluster members.

Parameters
  • config – Patroni configuration.

  • cluster_name – name of the Patroni cluster.

  • group – filter which Citus group we should toggle the pause state of. Refer to the module note for more details.

  • paused – the desired state for pause in all nodes.

  • waitTrue if it should block until the operation is finished or false for returning immediately.

Raises
PatroniCtlException: if
  • pause state is already paused; or

  • cluster contains no accessible members.

patroni.ctl.topology_sort(members: List[Dict[str, Any]]) Iterator[Dict[str, Any]]

Sort members according to their level in the replication topology tree.

Parameters

members

list of members in the cluster. Each item should countain at least these keys:

  • name: name of the node, according to name configuration;

  • role: leader, standby_leader or replica.

Cascading replicas are identified through tags -> replicatefrom value – if that is set, and they are in fact attached to another replica.

Besides name, role and tags keys, it may contain other keys, which although ignored by this function, will be yielded as part of the resulting object. The value of key name is changed through generate_topology().

Yields

members sorted by level in the topology, and with a new name value according to their level in the topology.

patroni.ctl.wait_until_pause_is_applied(dcs: patroni.dcs.AbstractDCS, paused: bool, old_cluster: patroni.dcs.Cluster) None

Wait for all members in the cluster to have pause state set to paused.

Parameters
  • dcs – DCS object from where to get fresh cluster information.

  • paused – the desired state for pause in all nodes.

  • old_cluster – original cluster information before pause or unpause has been requested. Used to report which nodes are still pending to have pause equal paused at a given point in time.

patroni.ctl.watching(w: bool, watch: Optional[int], max_count: Optional[int] = None, clear: bool = True) Iterator[int]

Yield a value every x seconds.

Used to run a command with a watch-based aproach.

Parameters
  • w – if True and watch is None, then watch assumes the value 2.

  • watch – amount of seconds to wait before yielding another value.

  • max_count – maximum number of yielded values. If None keep yielding values indefinitely.

  • clear – if the screen should be cleared out at each iteration.

Yields

0 each time watch seconds have passed.

Example
>>> len(list(watching(True, 1, 0)))
1
>>> len(list(watching(True, 1, 1)))
2
>>> len(list(watching(True, None, 0)))
1