Compatibility
This guideline will introduce how to investigate and manage the compatibility:
- between databend-query and databend-meta.
- between different versions of databend-meta.
Compatibility between databend-query and databend-meta
Identifying the versions
-
To find out the build version of databend-query and its compatible databend-meta version:
databend-query --cmd ver
# output:
version: 0.7.61-nightly
min-compatible-metasrv-version: 0.7.59Which means this build of databend-query(
0.7.61-nightly
) can talk to a databend-meta of at least version0.7.59
, inclusive. -
To find out the build version of databend-meta and its compatible databend-query version:
databend-meta --cmd ver
# output:
version: 0.7.61-nightly
min-compatible-client-version: 0.7.57Which means this build of databend-meta(
0.7.61-nightly
) can talk to a databend-query of at least version0.7.57
, inclusive.
Maintaining compatibility
A databend cluster has to be deployed with compatible versions of databend-query and databend-meta. A databend-query and databend-meta are compatible iff the following statements hold:
databend-query.version >= databend-meta.min-compatible-client-version
databend-bend.version >= databend-query.min-compatible-metasrv-version
If incompatible versions are deployed, an error InvalidArgument
will occur when databend-query tries to connect to databend-meta,
which can be found in databend-query log.
Then databend-query will stop working.
Compatibility verification protocol
Compatibility will be checked when a connection is established between meta-client(databend-query) and databend-meta, in a handshake
RPC.
The client C
(databend-query) and the server S
(databend-meta) maintains two semantic-versions:
C
maintains its own semver(C.ver
) and the minimal compatibleS
semver(C.min_srv_ver
).S
maintains its own semver(S.ver
) and the minimal compatibleS
semver(S.min_cli_ver
).
When handshaking:
C
sends its verC.ver
toS
,- When
S
receives handshake request,S
asserts thatC.ver >= S.min_cli_ver
. - Then
S
replies handshake-reply with itsS.ver
. - When
C
receives the reply,C
asserts thatS.ver >= C.min_srv_ver
.
Handshake succeeds if both of these two assertions hold.
E.g.:
S: (ver=3, min_cli_ver=1)
is compatible withC: (ver=3, min_srv_ver=2)
.S: (ver=4, min_cli_ver=4)
is NOT compatible withC: (ver=3, min_srv_ver=2)
. Because althoughS.ver(4) >= C.min_srv_ver(3)
holds, butC.ver(3) >= S.min_cli_ver(4)
does not hold.
C.ver: 1 3 4
C --------+-------------+------+------------>
^ .------' ^
| | |
'-------------. |
| | |
v | |
S ---------------+------+------+------------>
S.ver: 2 3 4
Compatibility status
The following is an illustration of the latest query-meta compatibility:
Meta\Query | [0.9.41, 1.1.34) | [1.1.34, 1.2.287) | [1.2.287, 1.2.361) | [1.2.361, +∞) |
---|---|---|---|---|
[0.8.30, 0.8.35) | ❌ | ❌ | ❌ | ❌ |
[0.8.35, 0.9.23) | ✅ | ❌ | ❌ | ❌ |
[0.9.23, 0.9.42) | ✅ | ❌ | ❌ | ❌ |
[0.9.42, 1.1.32) | ✅ | ❌ | ❌ | ❌ |
[1.1.32, 1.2.63) | ✅ | ✅ | ❌ | ❌ |
[1.2.63, 1.2.226) | ✅ | ✅ | ❌ | ❌ |
[1.2.226, 1.2.258) | ✅ | ✅ | ✅ | ❌ |
[1.2.258, +∞) | ✅ | ✅ | ✅ | ✅ |
History versions that are not included in the above chart:
- Query
[0.7.59, 0.8.80)
is compatible with Meta[0.8.30, 0.9.23)
. - Query
[0.8.80, 0.9.41)
is compatible with Meta[0.8.35, 0.9.42)
.

Compatibility between databend-query
Version Compatibility Matrix
Query version | Backward compatible with | Key Changes |
---|---|---|
[-∞, 1.2.307) | [-∞, 1.2.311) | Original format |
[1.2.307, 1.2.311) | [-∞, 1.2.311) | Added Role info with PB/JSON support |
[1.2.311, 1.2.709) | [1.2.307, +∞) | Role info serialized to PB only |
[1.2.709, +∞) | [1.2.709, +∞) | Important: Fuse storage path changed |
Important Changes & Upgrade Instructions
Version 1.2.307
- Support deserialize Role info with PB and JSON
- Only support serialize Role info to JSON
- Upgrade to this version first if you're on an earlier version
Version 1.2.311
- Only support serialize Role info to PB
- Upgrade to this version next after reaching 1.2.307
- Example upgrade path:
1.2.306 -> 1.2.307 -> 1.2.311 -> 1.2.312
Version 1.2.709
- Important Change: Fuse storage path modified
- ⚠️ Versions before 1.2.709 may not be able to read some data from versions 1.2.709+
- ⚠️ Recommendation: All nodes under the same tenant should be upgraded together
- Avoid mixing nodes with versions before and after 1.2.709 to prevent potential data access issues
Compatibility between databend-meta
Meta version | Backward compatible with |
---|---|
[0.9.41, 1.2.212) | [0.9.41, 1.2.212) |
[1.2.212, 1.2.479) | [0.9.41, 1.2.479) |
[1.2.479, 1.2.655) | [1.2.288, 1.2.655) |
[1.2.655, +∞) | [1.2.288, +∞) |
-
1.2.53
Incompatible, rolling upgrade is allowed without snapshot transmitting. Snapshot format changed thus during rolling upgrading, it requires all node data to be up-to-date, ensure there is no need to replicate with snapshot. -
1.2.163
Feature: gRPC API:kv_read_v1()
is added. For stream reading. -
1.2.212
2023-11-16 Feature: raft API:install_snapshot_v1()
. Compatible with old versions. Rolling upgrade is supported. In this version, databend-meta raft-server introduced a new APIinstall_snapshot_v1()
. The raft-client will try to use either this new API or the originalinstall_snapshot()
. -
1.2.479
2024-05-21 Remove:install_snapshot()
(v0) from client and server. Theinstall_snapshot_v1()
is the only API to install snapshot, and becomes REQUIRED for the client. -
1.2.528
2024-06-13 Remove on-disk data versionV001
. The first version usingV002
is1.2.53
, 2023-08-08. Therefore, since1.2.528
, the oldest compatible version is1.2.53
. Consequently, compatibility remains unchanged from this version onward. -
1.2.552
2024-07-02 Introduce on-diskV003
, usingrotbl
format snapshot, which is compatible withV002
. The oldest compatible version is1.2.288
(1.2.212~1.2.287
are removed). -
1.2.655
2024-11-11 Introduce on-diskV004
, using WAL based Raft log storage, which is compatible withV002
. The oldest compatible version is1.2.288
(1.2.212~1.2.287
are removed).
Compatibility of databend-meta on-disk data
The on-disk data of Databend-meta evolves over time while maintaining backward compatibility.
DataVersion | Databend-version | Min Compatible with |
---|---|---|
V004 | 1.2.655 | V002 |
V003 | 1.2.547 | V002 |
V002 | 1.2.53 | V001 |
V001 | 1.1.40 | V0 |
Identifying the versions
Upon startup, Databend-meta will display the on-disk data version:
For example, running databend-meta --single
produces:
Databend Metasrv
Version: v1.1.33-nightly-...
Working DataVersion: V0
On Disk Data:
Dir: ./.databend/meta
Version: version=V0, upgrading=None
Working DataVersion
denotes the version Databend-meta operates on.On Disk Data -- DataVersion
denotes the version of the on-disk data.
The Working DataVersion must be greater than or equal to the on-disk DataVersion; otherwise, it will panic.
The on-disk DataVersion must be compatible with the current Databend-meta version. If not, the system will prompt the user to downgrade Databend-meta and quit with a panic.
Automatic upgrade
When databend-meta
starting up, the on-disk is upgraded if it is compatible with the working DataVersion.
The upgrade progress will be printed to stderr
and to log file at INFO level, e.g.:
Upgrade on-disk data
From: V0(2023-04-21: compatible with openraft v07 and v08, using openraft::compat)
To: V001(2023-05-15: Get rid of compat, use only openraft v08 data types)
Begin upgrading: version: V0, upgrading: V001
Write header: version: V0, upgrading: V001
Upgraded 167 records
Finished upgrading: version: V001, upgrading: None
Write header: version: V001, upgrading: None
If databend-meta
crashes before upgrading finishes,
it will clear partially upgraded data and resume the upgrade when it starts up again.
Backup data compatibility
-
The exported backup data can only be imported with the same version of
databend-metactl
. -
The first line of the backup is the version, e.g.:
["header",{"DataHeader":{"key":"header","value":{"version":"V100","upgrading":null}}}]
-
NO automatic upgrade will be done when importing. Automatic upgrade will only be done when
databend-meta
is brought up.