This document describes how mysqld upgrades its data and what MOCO has to do about it.
- MOCO implementation
- User's responsibility
Beginning with 8.0.16,
mysqld can update all data that need to be updated when it starts running.
This means that MOCO needs nothing to do with MySQL data.
One thing that we should care about is that the update process may take a long time.
The startup probe of
mysqld container should be configured to wait for
complete updating data.
MySQL 8.0 does not support any kind of downgrading.
Internally, MySQL has a version called "data dictionary (DD) version". If two MySQL versions have the same DD version, they are considered to have data compatibility.
Nevertheless, DD versions do change from time to time between revisions of MySQL 8.0. Therefore, the simplest way to avoid DD version mismatch is to not downgrade MySQL.
Upgrading a replication setup
In a nutshell, replica MySQL instances should be the same or newer than the source MySQL instance.
When the Pod template of a StatefulSet is updated, Kubernetes updates the Pods.
With the default update strategy
RollingUpdate, the Pods are updated one by one
from the largest ordinal to the smallest.
StatefulSet controller keeps the old Pod template until it completes the rolling update. If a Pod that is not being updated are deleted, StatefulSet controller restores the Pod from the old template.
This means that, if the cluster is Healthy, MySQL is assured to be updated one by one from the instance of the largest ordinal to the smallest.
MOCO switches the primary instance when the Pod of the instance is being deleted.
clustering.md for details.
With the preconditions listed above, MOCO can upgrade
mysqld in MySQLCluster safely
.spec.updateStrategyfield in StatefulSet to
- Choose the lowest ordinal Pod as the next primary upon a switchover.
- Configure the startup probe of
mysqldcontainer to wait long enough.
- By default, MOCO configures the probe to wait up to one hour.
- Users can adjust the duration for each MySQLCluster.
Suppose that we are updating a three-instance cluster.
mysqld instances in the cluster have ordinals 0, 1, and 2, and the
current primary instance is instance 1.
After MOCO updates the Pod template of the StatefulSet created for the cluster, Kubernetes start re-creating Pods starting from instance 2.
Instance 2 is a replica and therefore is safe for an update.
Next to instance 2, the instance 1 Pod is deleted. The deletion triggers
an automatic switchover so that MOCO changes the primary to the instance 0
because it has the lowest ordinal. Because instance 0 is running an old
mysqld, the preconditions are kept.
Finally, instance 0 is re-created in the same way. This time, MOCO switches the primary to instance 1. Since both instance 1 and 2 has been updated and instance 0 is being deleted, the preconditions are kept.
If an instance is down during an upgrade, MOCO may choose an already updated instance as the new primary even though some instances are still running an old version.
If this happens, users may need to manually delete the old replica data and re-initialize the replica to restore the cluster health.
- Make sure that the cluster is healthy before upgrading
- Check and prepare your installation for upgrade
- Do not attempt to downgrade MySQL