v4.2.1_CE_BP7

版本信息

项目	描述
发布日期	2024-06-06
版本号	V4.2.1_CE_BP7
Commit 号	69b64b8
OBServer RPM 版本号	oceanbase-ce-4.2.1.7-107000162024060611

特性增强

• 操作系统适配支持

  新增 8U 操作系统适配支持。

• 旁路导入中间数据支持开启压缩
  旁路导入过程中，中间数据占用空间过大时，可能会导致磁盘空间满，进而导致导入失败。新版本增加旁路导入中间数据压缩功能，通过租户级配置项 _ob_ddl_temp_file_compress_func 控制是否开启及使用的压缩算法。

  默认为 NONE，表示不开启压缩，适用于磁盘空间足够，希望获得更高导入性能的场景。根据业务需要可修改为 ZSTD 或 LZ4，来指定对应的压缩算法开启中间数据压缩，或设置为 AUTO 自适应开启压缩。

• 备份恢复支持 S3/OBS/GCS

  新增支持 S3 协议，可将 S3/OBS/GCS 作为日志归档和数据备份的目的端，同时支持使用 S3/OBS/GCS 上的备份数据进行物理恢复。

• SET/PIECE 级物理恢复

  实际业务中存在二次备份场景，需把数据备份集或归档日志手动搬迁到新的路径。新版本增加了 SET/PIECE 级物理恢复功能，提供 ADD RESTORE SOURCE 命令来加载新路径的数据备份集 SET 或日志归档 PIECE，支持按需恢复到指定时间。更多信息请参见 执行指定路径的恢复。

• 迁移复制源端选择优化

  新版本将各 Server 按照地域信息划分为同 IDC、同 Region 不同 IDC、跨 Region 三种区域关系，同时提供 choose_migration_source_policy 配置项，用于迁移复制场景下，指定源端的选择模式，以便优先考虑地理位置就近因素以及 Follower 副本来提升迁移效率、降低 Leader 压力。更多信息请参见 choose_migration_source_policy。

• 物理恢复进度统计

  为了用户在使用物理恢复功能时可以了解恢复任务的运行状态和进度，并预估完成时间，新版本增加了物理恢复进度统计功能。用户可通过 CDB/DBA_OB_RESTORE_PROGRESS 视图实时查看恢复进度，获得更好的使用体验。更多信息请参见 查看物理恢复进度。

• 大规格租户事务数据表转储调度优化

  优化大规格租户下事务数据表转储不及时的问题，加快事务数据表的转储调度，进一步优化 Buffer 表性能。

• 全局 CPU 前后台任务隔离

  在之前版本，OceanBase 数据库已实现通过租户 Unit 规格来配置租户间的 CPU 资源隔离，提供 DBMS_RESOURCE_MANAGER 系统包来配置租户内的 CPU 资源隔离。新版本支持全局 CPU 前后台任务隔离，可以在整体层面上限制后台任务的可用资源，相对租户内使用 DBMS_RESOURCE_MANAGER 单独配置更加方便易用。更多信息请参见 使用全局 CPU 资源的前后台隔离。

• 新增响应时间统计直方图

  在之前版本，OceanBase 数据库已支持基于不同 SQL 类别的平均响应时间/最大响应时间指标，但缺乏更细粒度的反映某个分位 SQL 执行性能的指标展示。新版本增加响应时间直方图功能，可基于 [G]V$OB_QUERY_RESPONSE_TIME_HISTOGRAM 系统视图计算和监控不同类型 SQL 的 P90/P95 之类的响应时间统计。更多信息请参见 GV$OB_QUERY_RESPONSE_TIME_HISTOGRAM、V$OB_QUERY_RESPONSE_TIME_HISTOGRAM。

• 分区均衡策略优化

  新版本优化了分区均衡策略，支持在创建分区表时连续分区打散。当用户表存在 longtext/lob 列或局部索引时，分区磁盘均衡也会计算关联表，使磁盘使用更加均衡。更多信息请参见 租户内均衡。

• OBCDC 启动加速

  新版本优化了 OBCDC 启动性能，协助提升下游 Binlog 工具的同步性能。

• JSON 可嵌套层数扩展

  历史版本限制 JSON 文档的可嵌套最大深度为 100，新版本提供 json_document_max_depth 配置项，允许用户根据需求调整嵌套深度。对于超过 100 的 JSON 嵌套层数需求，用户可适当调大该配置。更多信息请参见 json_document_max_depth.

• 支持 HBase PageFilter 功能

  OB-HBase 模型下, 支持使用 HBase 中的 PageFilter 对 Row 进行限制, 返回指定数量的 Row。

• 备份性能优化

  OceanBase 数据库在备份过程中，通过连续性校验确保基线版本大于转储版本来保证数据完整。然而，连续性检查涉及读取备份介质上的数据并执行带锁操作，因此会影响备份性能。新版本优化了备份过程中连续性校验操作带来的性能开销，支持通过 ha_low_thread_score 控制备份使用的线程数，从而有效提高备份性能。更多信息请参见 ha_low_thread_score。

产品行为变更

• 优化器版本调整默认值为 "4.2.1.7"（optimizer_features_enable）

  V4.2.1 BP7 开始，新建的集群可以使用新版优化器能力，但从 V4.2.1 BP6 及以下版本升级到 V4.2.1 BP7 依然会保留之前的系统参数配置值。V4.2.1 BP6 及以下版本，默认值都为空，表示使用 V4.2.1.0 的优化器能力。

• 升级合并限制放开

  从 OceanBase 数据库 V4.2 的各版本升级到 V4.2.1 BP7 时，升级过程中放开了合并的限制。

配置项变更

配置项	变更类型	描述
choose_migration_source_policy	新增	新增租户级配置项，用于控制迁移源端的选择策略。提供 2 种选择： idc：在同 IDC 的机器中优先选择 Follower 副本作为源端。若仅有 Leader 副本，则选择 Leader 副本。 region：在同 Region 的机器中优先选择 Follower 副本作为源端。若仅有 Leader 副本，则选择 Leader 副本。 默认策略为 idc。
json_document_max_depth	新增	新增租户级配置项，用于设置 JSON 文档中允许的最大嵌套层数，默认为 100，可修改嵌套层数上限至 1024。
log_storage_warning_trigger_percentage	新增	新增集群级配置项，用于设置触发日志盘故障的写入性能百分比阈值。 默认为 0，表示如果日志盘写盘延迟大于配置项 log_storage_warning_tolerance_time 的值，则认为日志盘故障。 若设置值大于 0，表示如果当前的日志盘写入性能下跌到基准性能乘以百分之 log_storage_warning_trigger_percentage 以下，并且性能下降持续 log_storage_warning_tolerance_time 秒，则认为日志盘故障。 建议设置不超过 10。
sql_plan_management_mode	新增	新增租户级配置项，用于控制是否开启 SPM 功能。默认为 Disable，表示关闭 SPM 功能。 此配置项与系统变量 optimizer_use_sql_plan_baselines 都可以控制 SPM 功能的开启，两者任一打开都可以开启 SPM 功能。推荐使用此配置项开启 SPM，可以避免重启业务应用或 ODP 才能使 SPM 生效的步骤。

系统变量变更

系统变量	变更类型	描述
optimizer_features_enable	默认值调整	V4.2.1 BP6 及之前的 V4.2.1 BP 版本，默认值为空，表示使用 V4.2.1.0 的优化器能力。 V4.2.1 BP7 版本调整默认值为 4.2.1.7，表示 V4.2.1 BP7 开始，新建的集群可以使用新版优化器能力，而从 V4.2.1 BP6 及以下版本升级依然使用升级前指定的优化器版本。

视图变更

视图	变更类型	描述
CDB/DBA_OB_RESTORE_PROGRESS	新增列	新增 RECOVER_SCN、RECOVER_SCN_DISPLAY、RECOVER_PROGRESS、TABLET_COUNT、FINISH_TABLET_COUNT、RESTORE_PROGRESS 6 列，用于展示物理恢复进度。
[G]V$OB_QUERY_RESPONSE_TIME_HISTOGRAM	新增	各租户新增视图，用于展示不同类别的 SQL 响应时间直方图信息。
CDB_TAB_COL_STATISTICS	视图内容变更	新版本不再展示索引表相关信息。

缺陷修复

• 修复通过 DDL 在表尾部加多个列 (>=2) 后，若 select count (指定列) from table [where]; 中指定列不是最后加的列，如果有转储数据，查询结果可能出现比预期多的问题。 #1952

• 修复事务第一条语句锁冲突后自动结束事务的问题。

• 修复并发 Truncate Table 可能卡住的问题。

• 增加重启时对 Slog 盘空间的检查。

• 修复登录备租户偶现密码错误的问题。

• 修复 open cursor 失败重试，导致 PlTemp 内存膨胀的问题。

• 修复特定下压场景下可能出现瞬时查询多行的问题。

• 修复大事务锁冲突检查可能导致 SQL 一直重试的问题。

• 修复混合直方图收集导致数据盘出现暴涨的问题。

• 修复索引携带特殊字符，导致 Outline 绑定不上的问题。

• 修复缩容场景下计划生成报错导致估行出错，计划走偏的问题。

• 修复迁移复制过程中，有可能触发 QPS 或 TPS 下跌的问题。

• 修复使用 obloader 旁路导入可能导致 Core Dump 的问题。

• 修复 OBKV 执行 check_and_upsert 操作时，CDC 异常处理导致 OMS 误订到 Delete 操作的问题。

• 修复建索引产生超大临时文件导致报 4013 的问题。

• 修复连接数量过多导致递归爆栈引起 Core Dump 的问题。

• 修复备份过程中占用 MemTable 导致内存消耗大的问题。

• 修复旁路导入与 DDL 并发可能导致 Core 的问题。

• 修复归档模式为 optional，且归档介质出现长时间不可写入的情况下，归档状态不一致导致迁移任务失败的问题。

• 修复 V4.2.1 BP5 之前的版本升级至 V4.2.1 BP5 及之后版本期间，可能会出现迁移、复制、Rebuild 失败的问题。

v4.3.1_CE_BETA

版本信息

项目	描述
发布日期	2024-05-22
版本号	V4.3.1_CE_BETA
Commit 号	bad90e8
RPM 版本号	oceanbase-ce-4.3.1.0-100000032024051615
版本说明	Beta 版本解决了大部分缺陷，并趋于稳定。推荐测试环境使用。

版本概览

OceanBase V4.3.1 在 V4.3.0 基础上，新增全文索引特性提升文档检索效率，扩展实时物化视图、物化视图改写、主键物化视图等功能以满足多场景的分析需求。引入分区管理新机制，如分区交换、外表分区、MySQL 模式分区键数据类型扩展等，提升大规模数据的处理能力。多模特性（含 JSON、XML、GIS）功能升级，增加 JSON 多值索引、JSON 部分更新的能力支持，促进异构数据的迁移与融合。MySQL 兼容性持续增强，支持 Lateral Derived Tables、 MySQL 锁函数等，以便融入生态。提供增量旁路导入能力，提升了多次导入场景的入库性能，同时优化了多局部索引场景下的 DML 性能，提高了基础统计信息收集效率，并在行采样、小规格 TP 场景取得了显著的性能提升。新版本也进一步优化了资源使用，支持了 CLOG 日志缓存、SQL 临时结果压缩、系统日志压缩等特性。补充完善了 MySQL 权限体系、支持了操作系统配置检查，加固系统安全。一如既往地关注用户体验，增加资源规格估算能力，增强备份透明度，提供 IPV6 格式支持，赋能数据库管理与运维。

V4.3.1 定位为 Beta 版本，推荐实时数仓或联机交易业务测试使用，下半年将发布推荐用于生产的 GA 版本。

关键特性说明

内核增强

• MySQL 模式全文索引（Experimental）

  关系型数据库中通常会使用索引来加速基于精准值匹配的查询，但普通的 B-Tree 索引无法应用于涉及大量文本数据需要进行模糊检索的场景，这时只能通过全表扫描来对每一行数据进行模糊查询，文本较大、数据量较多的情况下性能往往不能满足要求。另外一些复杂的查询场景，如近似匹配、相关性排序等，也难以通过改写 SQL 支撑。

  为了解决上述问题，OceanBase 在 V4.3.1 支持了全文索引功能，通过预先处理文本内容，建立关键词索引，有效提升全文检索效率。该功能在 V4.3.1 定义为实验特性，后续版本会继续扩展功能并演进为生产可用特性。本期包含以下功能：

  • MySQL 模式下支持全文索引功能，兼容 MySQL 基本语法。
  • 建表时支持对 CHAR、VARCHAR 和 TEXT 列预建全文索引。
  • 适用于分区表。
  • 支持对主表创建多个全文索引。
  • 支持 SPACE（空格）、NGRAM 和 BENG（基础英文）三种内置分词器。
  • 支持一个 match against 对多个列进行全文检索。
  • 支持 NATURAL LANGUAGE MODE 模式。

• 物化视图增强

  OceanBase V4.3.0 支持了物化视图特性，通过预计算和存储视图的查询结果，减少实时计算来提升查询性能，简化复杂查询逻辑，支撑 AP 场景需求。在此基础上，V4.3.1 版本扩展支持了实时物化视图功能，提供基于物化视图和 MLOG 两部分数据的实时计算能力，适用于实时性要求较高的分析业务。新增物化视图主键约束，支持用户为物化视图指定主键，以此优化基于主键的单行查找、范围查询或关联场景性能。同时扩展支持了内连接场景物化视图的增量更新能力，提升部分场景的物化视图刷新性能。

  另外在 V4.3.0 版本使用物化视图时，需要将业务脚本中对原始表的访问计算手工替换为查询对应的物化视图，引入了人工改写成本。新版本支持了部分场景的物化视图改写能力，在系统变量 QUERY_REWRITE_ENABLED 设置为 True 的情况下，创建物化视图时可以通过指定 ENABLE QUERY REWRITE 来使用物化视图的自动改写能力，此时系统可以将对原始表的查询改写为针对物化视图的查询，以此减少业务改造量。

• 分区交换

  随着时间的推移，表中可能会累积大量的历史数据，这些数据可能不需要频繁访问，但需要保留以满足合规性或历史数据分析需求。为了提升查询性能，业务往往需要区分活跃和非活跃数据，并将非活跃数据进行归档。通过 SQL 迁移数据到新表可以解决这个场景需求，但大数据量情况下性能往往不优。因此，OceanBase V4.3.1 版本新增分区交换功能，通过修改数据字典中分区和表的定义，而无需进行数据的物理复制，可以将 A 表数据近乎瞬时地移动到 B 表某个分区下，极大地提升了数据迁移的性能。本期包含以下功能：

  • 支持分区表中的一个分区和一个普通表进行数据互换。
  • 支持一级分区表，分区类型为 Range（Range Columns）。
  • 支持二级分区表，一级分区类型无要求，二级分区类型为 Range（Range Columns）。
  • 支持 INCLUDING INDEXES 的行为，即分区交换时，对应局部索引也会参与交换，并在交换后保持可用状态。
  • 支持 WITHOUT VALIDATION 模式，需要用户保证数据符合分区键范围。

• 外部表分区

  OceanBase V4.2.0 开始支持外部表功能，局限于非分区表。当文件数量较多，但某次查询理论上只需要扫描一部分数据文件的场景，非分区外部表只能扫描全量文件，无法进行裁剪，性能较差。V4.3.1 版本新增外部表分区功能，支持类似普通表 list 分区的分区方式，并提供了自动/手动分区的两种语法。指定自动创建分区时，系统将按照分区键的定义方式，将文件按照分区进行分组；指定手动创建分区时，需要用户指定每个分区对应的数据文件子路径。这种情况下，外表查询可以根据分区条件实现分区裁剪，减少扫描的文件数量，显著提升查询性能。

• MySQL 模式 Range Columns 分区键类型扩展

  为了满足不同业务的分区要求，OceanBase V4.3.1 在兼容 MySQL Range Columns 分区键类型的基础上，扩展支持了 double、float、decimal、timestamp 等类型作为 Range Columns 分区的分区键。具体类型包括：

  • DECIMAL、DECIMAL[(M[,D])]
  • DEC、NUMERIC、FIXED
  • FLOAT[(M,D)]、FLOAT(p)
  • DOUBLE、DOUBLE[(M,D)]
  • DOUBLE PRECISION、REAL
  • TIMESTAMP

• PL 重新编译逻辑优化

  存储过程编译为共享库后，可以提供给多个线程使用。但如果发生依赖对象变更，共享库可能失效需要重新编译。V4.3.1 版本针对重新编译场景做了梳理细化，在临时表匹配、静态 SQL 依赖对象信息收集、表 DDL 变更等方面进行一系列逻辑优化，减少因 PL CACHE 缓存对象失效导致重新编译的场景。

• PL 编译结果落盘

  当前 PL Function/Procedure 编译后会加入到 PL Cache 中，当内存压力过大时可能会被淘汰，重启后会丢失编译结果，并且在分布式场景下也需要重新编译。在这些情况下，PL 无法命中 PL Cache，进而触发 LLVM 编译，导致消耗部分 CPU 资源。V4.3.1 开始，PL Function/Procedure 被加入系统表落盘保存，未触发 DDL 使 PL 缓存失效的情况下，无论重启还是分布式场景都可以只编译一次后复用缓存。

多模特性

• MySQL 模式 JSON 多值索引（Experimental）

  MySQL 8.0 版本开始支持多值索引特性，适用于 JSON 文档和其他集合数据类型。这使得用户可以通过在数组或者集合上构建索引，来实现元素的高效检索。OceanBase V4.3.1 在 MySQL 模式下兼容了 JSON 多值索引特性，允许在包含多个元素的 JSON 数组字段上创建高效的二级索引，增强了复杂 JSON 数据结构的查询能力，兼顾了数据模型的灵活性和数据查询的高性能。该功能在 V4.3.1 定义为实验特性，后续版本会继续扩展功能并演进为生产可用特性。本期包含以下功能：

  • 支持多值索引、复合多值索引的预创建。
  • 支持唯一、非唯一的多值索引。
  • 支持创建多值索引的 JSON 数组元素类型为 INT、UINT、DOUBLE、FLOAT、CHAR 等。
  • 适用于分区表。
  • 支持 MEMBER_OF()、JSON_CONTAINS() 和 JSON_OVERLAPS() 函数使用多值索引查询。

• MySQL JSON

  新增兼容 JSON_SCHEMA_VALID、JSON_SCHEMA_VALIDATION_REPORT 和 JSON_ARRAY_APPEND 表达式。

• MySQL JSON Partial Update

  一些用户会使用 JSON 文档存储业务数据，文档需要更新时，目前只能全量读取后全量更新，文档较大时性能无法满足预期。V4.3.1 版本新增支持 JSON Partial Update 功能，当用户通过特定表达式（json_set、json_replace 和 json_remove）更新 JSON 文档的部分字段时，可以只更新要修改的部分，无需全量覆盖更新，提升更新性能。该功能需要通过 log_row_value_options 配置项开启使用。

• MySQL GIS 增强

  OceanBase V4.1 版本支持了 MySQL GIS 数据类型及部分空间对象相关的表达式，V4.3.1 版本在此基础上进一步补齐空间数据存储和计算分析的能力，新增支持 MySQL 兼容的 ST_Crosses、ST_Overlaps、ST_Difference、ST_Union、ST_SymDifference、ST_Length、ST_Centroid 和 ST_AsGeoJSON 等表达式。另外 PG 作为 GIS 行业使用最广的数据库，提供了部分区别于 MySQL 的常用表达式，OceanBase 在新版本也进行了扩展补充，包括 _ST_Touches、_ST_Equals、_ST_MakeEnvelope、_ST_ClipByBox2D、_ST_GeometryType、_ST_IsCollection、_ST_NumInteriorRings、_ST_PointOnSurface、ST_AsMVTGeom 和 _ST_AsMVT 等，同时也支持了三维空间对象的存储能力。

• MySQL XML

  新增兼容 ExtractValue、UpdateXML 表达式。

MySQL 兼容

• Lateral Derived Tables

  Lateral Derived Tables 是一种特殊的 Derived Tables，可以引用同一 FROM 子句中前面表的列，这种用法下 SQL 更易读易写，往往也可以减少数据的重复扫描计算，提升 SQL 执行性能。V4.3.1 兼容了 MySQL 8.0 的用法。

• 通信协议完善

  兼容 MySQL COM_SET_OPTION 通信协议命令，用于指定 CONNECTION 级别选项，如 MYSQL_OPTION_MULTI_STATEMENTS_ON、MYSQL_OPTION_MULTI_STATEMENTS_OFF。兼容 MySQL AuthSwitchRequest 协议，用于支持认证方式切换，解决部分客户端版本不匹配导致认证出错的问题。

• Show Extended Tables/Columns/Index

  支持 MySQL 8.0 新增的 SHOW EXTENDED 语法，如 SHOW EXTENDED TABLES、SHOW EXTENDED COLUMNS 和 SHOW EXTENDED INDEX。

• MySQL 锁函数

  V4.3.1 兼容了 MySQL GET_LOCK、IS_FREE_LOCK、IS_USED_LOCK、RELEASE_ALL_LOCKS 和 RELEASE_LOCK 等锁相关的表达式。表达式可以用在 SQL 语句的多个地方，例如 SELECT 语句的 ORDER BY 和 HAVING 子语句，SELECT、DELETE 和 UPDATE 的 WHERE 条件中以及 SET 语句中。表达式的值可以是字面值、列值、NULL、变量、内部函数和操作符、可加载函数、存储函数。锁函数是一类内部函数，允许用户自定义锁，并进行使用。

• PS 协议支持存储过程出参

  MySQL 模式下，使用 PS 协议执行 CALL PROCEDURE 语句时，新增支持存储过程带出参的场景。

性能提升

• 增量旁路导入（Experimental）

  OceanBase V4.1.0 开始支持旁路导入特性，通过精简数据加载的执行路径，跳过 SQL、事务、memtable 等模块，直接将数据持久化为 SSTable 来显著提升数据导入效率。但在表数据需要多次导入的场景，每次导入都需要将表中已有数据重写一遍，影响了增量导入的性能。V4.3.1 针对性地优化了增量导入场景，增量旁路导入无需重写原有数据，只需处理新增数据，让多次导入像第一次导入一样具有高性能。支持在 LOAD DATA 和 INSERT INTO SELECT 语句中通过 /*+ direct(need_sort, max_errors_allowed, load_mode) */ Hint 指定是否使用增量旁路导入功能。不指定 load_mode 或 load_mode 为 full 时，表示使用原有的全量旁路导入方式；指定 load_mode 为 inc_replace 时使用增量旁路导入方式。该功能在 V4.3.1 定义为实验特性，后续版本会继续扩展功能并演进为生产可用特性。

• SELECT INTO OUTFILE 性能优化

  现有的 SELECT INTO OUTFILE 功能虽然支持并行读取数据表，但只能串行写入外部文件，存在数据导出的性能瓶颈。V4.3.1 增加并行导出能力，在 SELECT INTO OUTFILE 命令基础上增加了 SINGLE、MAX_FILE_SIZE 选项，用于控制数据写入外部文件的方式。SINGLE 选项可控制将数据导出到单个文件或多个文件，指定并行度大于 1 且 SINGLE = FALSE 时，可以将数据导出到多个文件，达到并行读并行写的效果；MAX_FILE_SIZE 选项可控制导出文件的大小。

• 多局部索引场景下 DML 性能优化

  当数据库表上存在索引时，DML 会因为需要同步更新索引而产生性能下降。在索引数量特别多时，性能下降尤其明显。新版本针对表上存在多个局部索引的场景，通过局部索引去表锁、非唯一索引不记录持锁者、非唯一索引不检查事务冲突、降低 DML 统计信息上报开销等等一系列系统优化，将局部索引维护开销降低了 45%，从而提升 DML 整体执行性能。

• 单日志流并行同步

  OceanBase V4.3.1 版本之前的日志同步模型为不同日志流并行同步和处理，单日志流基于 Pipeline 方式同步。目前在同城消费本地盘日志的场景下是可以满足性能要求的，但在备库异地部署、公有云读对象存储的场景下，性能相对会差一些。V4.3.1 版本实现了基于文件数据块的单日志流并行同步模型，显著提升同步性能并优化内存使用。

• 统计信息收集性能优化

  当前基础统计信息的收集依赖通过执行相关聚合函数的内部 SQL 完成，V4.3.0 开始支持了聚合函数下压到存储层，V4.3.1 进一步扩展了收集基础统计信息可以下压到存储层的聚合函数，避免投影数据到 SQL 层再做计算，基础统计信息的收集操作全部放到了存储层，提升了基础统计信息的收集效率。新版本收集性能整体上比 V4.3.0 提升 5% 左右。

• 行采样性能优化

  在处理全量数据开销太大时，往往可以通过小部分数据观察整体的数据分布，例如优化器通过采样来分析数据分布，辅助执行计划的生成。OceanBase sample 行采样当前会先应用 WHERE 过滤条件，然后过滤出的一行数据，判断是否进行采样。这相当于逐行扫描进行判断，把数据整体都探测一遍，十分耗时。V4.3.1 优化了行采样流程，先过滤数据再应用条件，省去部分数据的读取成本，同时针对列存也优化为只读取需要的列数据，显著提升采样性能。

• 小规格性能优化

  针对 4C/8C 小规格环境，V4.3.1 版本在后台线程使用、Location Cache 访问、读写主路径、系统调用等方面进行了一系列优化，OLTP 相关测试场景，性能相对 V4.3.0 提升 20%-30%。

可靠性提升

• 数据盘满停写

  数据盘使用量过高时，当前不会停写用户请求，一直到内存因为无法转储写满，或者 clog 盘写满后才会报错。这种情况下，需要通过紧急扩 clog 盘或租户内存恢复。新版本在内核层面增加数据盘满停写功能，当用户数据盘水位线达到 data_disk_write_limit_percentage 配置后，用户写入请求会报错。通过删表、transfer 或者扩磁盘方式降低数据盘使用比例后，用户写入操作可以自动恢复。

资源使用优化

• CLOG 日志缓存

  V4.x 在归档、回放等场景已经支持了 LogHotCache，用来缓存部分实时日志，一定程度上避免了日志读盘开销。但当日志写入速度比较快或多个 OBCDC 重复拉取相近日志时，还是无法避免大量的日志读盘，极端情况下会将日志盘带宽打满。同时，云上磁盘带宽往往有限的，我们需要通过降低日志读盘的开销来支撑更多的日志消费场景。因此，该版本新增 CLOG 日志缓存功能，支持消费者直接从日志缓存读取日志进行消费，没有命中日志缓存时，依然支持从磁盘读取日志，并将日志同步写入缓存中，避免重复读取磁盘，降低日志盘带宽使用。为了方便用户监控日志缓存命中情况，SYSSTAT 新增 50065、50066 和 120010 三个统计项，用于展示租户级别的日志缓存命中次数、未命中次数和 CLOG 缓存大小。

• 旁路导入资源控制强化

  OceanBase V4.x 版本支持的旁路导入功能显著提高了数据导入速率，但在用户设定较高并行度且并发执行的旁路导入任务较多时，资源缺少严格控制，容易造成较多线程和内存占用，影响其他任务的正常执行。V4.3.1 新增三个维度的旁路导入资源管理能力：

  • 新增任务级资源申请管理模块，根据导入任务的执行模式和分区数量，申请对应的线程和内存资源。
  • 新增租户级资源管理模块，管理租户用于旁路导入的线程和内存资源，以定时任务感知资源池变化，回收异常中断的导入任务申请的资源。
  • 新增 OBServer 级资源管理模块，记录节点级资源申请，并根据任务数动态伸缩旁路导入任务排序阶段的可用内存。

• 系统日志压缩

  业务流量过大时，系统日志刷新的会比较快，保留时间较短可能影响问题排查。新版本增加系统日志压缩功能，支持对 observer.log、rootservice.log、election.log 和 trace.log 等日志文件，每类超过 syslog_file_uncompressed_count 个数时，采用 syslog_compress_func 设置的压缩方法，对最早日志进行压缩。当日志使用总空间接近 syslog_disk_size 设置的磁盘空间上限时，开始删除最早生成的日志文件，回收空间。磁盘空间不变的情况下，开启 zstd 压缩后，预计可以存储不开启压缩时 20 倍的日志量。

• R 副本按 Region 级联

  OceanBase V4.2.0 开始支持 R 副本功能，服务于弱读、复制表等场景。一个 R 副本通过注册为 F 副本或其他 R 副本的下游来同步日志，当多个 R 副本和其上游被部署在不同的 Region 时，可能占用额外的跨 Region 网络带宽。V4.3.1 增加 R 副本对 Region 的感知能力，在日志同步时会尽量选择相同 Region 的其他副本作为上游，尽量避免跨 Region 的网络传输，节省跨 Region 带宽。

• SQL 临时结果压缩

  SQL 执行涉及的数据量过大的情况下，可能出现内存不足的问题，这时部分算子需要物化临时的中间结果。当物化的数据量过大，磁盘空间写满时，会导致 SQL 执行失败。V4.3.1 新增 SQL 临时结果压缩功能，允许通过租户级配置项 spill_compression_codec 或 SQL 级 Hint（如 /*+opt_param('spill_compression_codec', 'lz4') */）指定临时结果是否压缩及压缩算法，当指定临时结果压缩时可有效降低临时磁盘空间占用，以支撑更大运算量的查询任务。

安全强化

• MySQL PL 权限管理

  MySQL 模式下新增 PL 权限管理能力，支持 CREATE ROUTINE、EXECUTE 和 ALTER ROUTINE 等权限控制。增加 mysql.procs_priv 内部表，用于展示存储过程或函数授权信息。升级集群默认不开启，新建集群会自动开启。

• MySQL 角色

  OceanBase V4.3.1 兼容了 MySQL 8.0 的角色管理功能，通过角色来管理维护一组权限，可以更方便地对某...

v4.2.1_CE_BP6

Version information

Information	Description
Release date	May 15, 2024
Version	V4.2.1_CE_BP6
Commit number	38166dc
OBServer RPM version	oceanbase-ce-4.2.1.6-106000012024042515

Performance optimization

Optimization of show table status performance: In earlier versions, the performance of the show table status from ... like ... command is not ideal because it did not make use of indexes related to table_name. In the new version, significant improvement in query performance has been achieved for scenarios with single table filtering conditions.

Bug fixes

• Fixed the issue where after SET collation_connection = 'utf8mb4_unicode_ci' is executed, comparing/joining columns of string type in information_schema would cause an error Illegal mix of collations, which is not compatible with native MySQL.
• Fixed the issue where reading and writing LOB data during the upgrades while running both the old and new versions could lead to OBServer core dump.
• Fixed the memory leak issue caused by asynchronous query timeouts in OBKV.

版本信息

项目	描述
发布日期	2024-05-15
版本号	V4.2.1_CE_BP6
Commit 号	38166dc
OBServer RPM 版本号	oceanbase-ce-4.2.1.6-106000012024042515

性能优化

优化 show table status 性能： 老版本 show table status from ... like ... 场景性能较差，未利用 table_name 相关索引。新版本针对存在单表过滤条件的场景，显著提升查询性能。

缺陷修复

• 修复 SET collation_connection = 'utf8mb4_unicode_ci' 后，information_schema 下字符串类型的列比较/关联操作，会报错 Illegal mix of collations，和原生 MySQL 不兼容的问题。
• 修复升级混跑过程中读写 LOB 数据可能导致 OBServer Core 的问题。
• 修复 OBKV 异步查询超时后导致内存泄漏的问题。

v4.2.3_CE_BETA

Version information

Information	Description
Release date	April 29, 2024
Version	V4.2.3_CE_BETA
Commit number	c129d71
OBServer RPM version	oceanbase-ce-4.2.3.0-100000112024042411

Overview

OceanBase Database V4.2.3_CE_BETA is the latest version in the V4.2.x_CE series. It introduces a range of MySQL compatibility enhancements, including roles, column-level permissions, UNION DISTINCT in recursive common table expressions (RCTEs), along with several other small, but significant, functionality improvements. The new version improves the performance in various scenarios. Specifically, it improves the read/write calculation and OBKV processing efficiency for multi-model data types, extends the parallel execution (PX) capabilities for batch DML operations, improves the performance of log archiving-based and network-based physical standby databases, and resolves performance issues of auto-increment columns and sequences in ORDER mode. This version also supports Amazon Simple Storage Service (S3) and object storage services that are compatible with the S3 protocol, such as Huawei Object Storage Service (OBS) and Google Cloud Storage (GCS), as the backup destination for the backup and restore feature. In terms of resource optimization, this version optimizes the storage space for temporary results of DDL operations, improves the bypass import capabilities, and implements global CPU resource isolation in the foreground and background. Moreover, the new version implements multiple usability features expected by business, such as oblogminer for log analytics and misoperation identification, major compaction strategy selection for coping with performance issues of buffer tables, format outlines for fuzzy plan binding and throttling, and index usage monitoring for identifying useless indexes. It also provides the alert.log system log file with higher readability for the database administrator (DBA), user commands for log stream (LS) replica management, and the capability to show the physical restore progress. In the new version, more detailed PX diagnostic data and node-level analytics of ASH reports are provided to significantly improve the usability of the system.

This version is recommended for use in testing environments only and is not yet advised for deployment in production environments.

Key features

Kernel enhancements

• Enhancements in query parsing capabilities

  When an SQL statement contains an excessive number of IN lists, AND/OR, or UNION operators, it can generate extremely deep syntax trees, which in turn can cause issues like excessive memory consumption, stack shortages, and extended parsing time. In response, the new version has revamped the query parsing process for these extreme cases. This rewrite reduces the resource expenditure during the parsing phase and improves the stability of executing complex SQL statements, while also boosting SQL parsing speed. For example, in a query like select sum(c1) from t1 where c1 in (1, 2, 3, ...N) with N being 200,000, the parsing efficiency in V4.2.3_CE can be 20% to 30% faster than in V4.2.2_CE.

• Enhancements in query rewrite capabilities

  In V4.2.3_CE, the optimizer has improved the query rewriting logic in several respects, such as:

  • Expression canonicalization enhancement: In earlier versions, the resolver will canonicalize various expressions in the parsing phase. For example, it will rewrite the expression A and (A or B) as A. However, nonstandard expressions generated in the SQL rewrite phase will not be rewritten. The new version introduces the expression canonicalization process in the rewrite phase to simplify expressions to the maximum extent.

  • Rewrite of conditional aggregate functions: A conditional aggregate function does not aggregate a definite expression. Instead, it aggregates an expression selected based on the branch condition judgement result. For example, the aggregate functions in select c1, sum(case when c2 = 0 then c3 else 0 end),sum(case when...)...,... from t1 group by c1, are conditional aggregate functions. The new version supports rewriting a conditional aggregate function by pushing down the function to the selection objects of each branch to reduce the number of calculations for case when expressions and aggregate functions, thereby improving the execution performance in relevant scenarios.

  • Rewrite of queries with multiple MIN/MAX aggregate functions: For a scalar group by operator, if the query contains only MIN/MAX functions and indexes are created for the MIN/MAX columns, the query can be rewritten to order by xxx limit 1, to remove ORDER BY and push LIMIT 1 to the TABLE SCAN operator based on the index order, thereby reducing data reads and sorting. In earlier versions, an SQL query containing multiple MIN/MAX functions would result in a full table scan. However, the new version optimizes this by rewriting multiple MIN/MAX functions into separate subqueries. Each subquery directly obtains the maximum or minimum value based on the index. This avoids full table scan and sorting, thereby improving the query performance.

  • Rewrite of UNION operations on constants into a query on a VALUES table: In practice, a constant table is usually expressed by using multiple UNION ALL operators. The CPU consumption and memory consumption will be high when many subqueries are involved. In the new version, UNION ALL/UNION operations on constants are rewritten into a query on a VALUES table to remarkably reduce the resource consumption in the hard parsing phase.

  • Semi-join splitting: When an EXISTS or IN subquery involves a multi-table join without join conditions, the execution plan may produce a Cartesian product with a high overhead. In the new version, semi-join splitting is performed on tables without join conditions to avoid the overhead for producing a Cartesian product.

• Plan selection optimization

  OceanBase Database V4.2.2_CE implements new query range extraction logic, which extends predicate pushdown scenarios so that a more accurate query range is extracted for vector predicates. It also resolves the issue of memory amplification during query range extraction in some scenarios in earlier versions. On this basis, OceanBase Database V4.2.3_CE modifies the method of generating query ranges for not in expressions to optimize the range extraction performance. It supports range graph pruning to reduce the number of rows extracted in the query range, thereby decreasing memory consumption. It also allows you to specify an INDEX hint during query range extraction. For example, you can specify /*+index(t1 k1 1)*/ to extract the query range only on the first column in the k1 index. Moreover, the new version provides strategy optimization measures, such as rule-based path pruning for interesting ordering indexes and LIMIT recalculation, to resolve performance issues caused by a non-optimal plan selected in the order by limit scenario.

• Adaptive cost model

  In earlier versions of OceanBase Database, the cost model uses constant parameters measured by internal machines to represent hardware system statistics, and describes the execution overhead of each operator by using a series of formulas and constant parameters. However, in real business scenarios, different hardware environments may have different CPU clock frequencies, sequential or random read speeds, and NIC bandwidths, thereby resulting in cost estimation deviations. The optimizer cannot always generate optimal plans in different business environments because of these deviations. The new version optimizes the implementation of the cost model. It allows you to use the DBMS_STATS package to collect or set system statistics coefficients so that the cost model can automatically adapt to the hardware environment. It also provides the DBA_OB_AUX_STATISTICS view to display the system statistics coefficients of the current tenant.

• Change of the replicated table attribute

  Replicated tables are a special type of tables supported since OceanBase Database V4.2.0. A replicated table can read the latest data modifications from any healthy replica. To convert a replicated table into a normal table or convert a normal table into a replicated table, you must re-create the table and import data in versions earlier than V4.2.3_CE, which is time-consuming and complex. The new version allows you to use the ALTER TABLE statement to directly change the replicated table attribute, reducing the costs.

• Compatible version control for product behavior

  To reduce errors in using features of some earlier versions that are caused due to behavioral changes in the new version after an upgrade, OceanBase Database V4.2.3_CE supports compatible version control for product behavior. You can use system variables ob_compatibility_version and ob_security_version to respectively control whether normal behavioral changes (no errors -> errors) and security behavioral changes (with privileges -> without privileges) take effect, and the OceanBase Database version whose product behavior takes effect in a tenant after the changes. Valid values are released version numbers, such as 4.2.3.0. For example, assume that the product behavior of a feature is changed in V4.2.4. When the variable is set to 4.2.3.0, the product behavior in V4.2.3.0 takes effect. When the variable is set to 4.2.4.0, the product behavior in V4.2.4.0 takes effect. Generally, the version number that determines the product behavior does not automatically change after an upgrade. If you...

v4.2.1_CE_BP5

Version information

Information	Description
Release date	April 24, 2024
Version	V4.2.1_CE_BP5
Commit number	ec03c25
OBServer RPM version	oceanbase-ce-4.2.1.5-105000032024041915

Enhanced features

• Compatibility with MySQL

  • A SET statement that contains the SET NAMES statement can also set other variables.
  • The information_schema.events table of MySQL is supported. #1194

• Other optimizations

  • The [G]V$OB_SESSION view is provided to display session information and help reduce the time required in collecting processlist statistics.
  • The NETWORK_WAIT_TIME column is added to the GV$OB_SQL_AUDIT view to indicate the total amount of time spent on events of the Network class.
  • A switch is provided for controlling parallel execution of DDL statements.
  • The slog and sstable directories now can be stored in different paths on the disk.
  • The OUTROW storage performance of large object (LOB) data is improved.
  • The issue of PL cache failure is resolved.
  • Adaptive major compaction is optimized for buffer tables.
  • The statistics collection performance is improved.

Product behavioral changes

To ensure stability, the batch rescan optimization feature was disabled by default for the NESTED-LOOP JOIN (NLJ) and SUBPLAN FILTER (SPF) operators during cluster creation in OceanBase Database since V4.2.1_CE_BP2_HF1. This feature is optimized in V4.2.1_CE_BP5 and is now enabled by default. In other words, the batch rescan optimization feature is enabled by default for new tenants. In OceanBase Database of a version earlier than V4.2.1_CE_BP5, the feature is still disabled by default. You can manually enable the feature as needed.

Bug fixes

• Fixed the issue where the query result of an SQL statement in a standalone database is inconsistent with that in a multi-node cluster. #1845
• Improved the performance of hotspot functions in an ARM environment.
• Fixed the issue where requests are accumulated in the queue of the sys tenant because the slog disk is used up.
• Fixed the issue where an additional record is displayed in some views such as information_schema.tables and all_tables after a DDL operation is performed to convert a non-partitioned table into a partitioned table.
• Fixed the issue where in the case of frequent connection and disconnection attempts, Error 4016 is returned during connection establishment when the total number of established connections reaches the specified value, because the reference count is leaked.
• Fixed the issue where the OBServer node can be hung if the obstack command is automatically executed when requests are accumulated in the queue.
• Fixed the issue where Error 4016 is returned when you execute the SHOW TRACE statement for a distributed execution plan after end-to-end tracing is enabled.
• Fixed the issue where the performance deteriorates after SQL plan management (SPM) is enabled.
• Fixed the issue where a core dump occurs if the offset exceeds INT64_MAX when the LIMIT OFFSET syntax is used in the .NET driver.
• Fixed the issue where the memory of the CommonArray memory module leaks due to bypass import.
• Fixed the issue where the memory of the PlJit module leaks due to exceptions.
• Fixed the issue where a core dump occurs during parallel execution when memory allocation fails.
• Fixed the issue of memory bloat during parallel index creation in specific scenarios.
• Fixed the issue where Error 4344 may be returned when you back up a standby database in specific scenarios.
• Fixed the issue where creating a standby tenant in the standby cluster will fail if case sensitivity is enabled for table names of tenants in the primary cluster.
• Fixed the issue where the execution time is abnormal if you execute the create table if not exists statement to create an existing table immediately after the cluster is restarted.

Considerations

OceanBase Database V4.2.1_CE_BP5 fixes the issue where connection establishment fails because the reference count of session connections is leaked. We recommend that you upgrade OceanBase Database to V4.2.1_CE_BP5 as soon as possible.

版本信息

项目	描述
发布日期	2024-04-24
版本号	V4.2.1_CE_BP5
Commit 号	ec03c25
OBServer RPM 版本号	oceanbase-ce-4.2.1.5-105000032024041915

特性增强

• MySQL 兼容性

  • 支持在同一个 SET 语句中同时包含 SET NAMES 语句和使用 SET 设置其他变量。
  • 支持兼容 MySQL 的 information_schema.events 表结构。 #1194

• 其他优化

  • 新增 [G]V$OB_SESSION 视图展示会话信息，优化 Processlist 统计时间。
  • GV$OB_SQL_AUDIT 视图新增 NETWORK_WAIT_TIME 字段，用于展示所有 Network 类事件的总时间。
  • 新增 _parallel_ddl_control 参数用于控制是否在租户级别开启各类并行 DDL 的功能。
  • 支持将 slog 和 sstable 的目录放在不同的磁盘目录下。
  • 优化 LOB 类型数据外联存储（OUTROW）性能。
  • 优化 PL Cache 缓存失效的问题。
  • 优化 Buffer 表的自适应合并。
  • 优化统计信息收集的性能。

产品行为变更

出于稳定性因素的考虑，从 V4.2.1_CE_BP2_HF1 版本开始，新建集群时，Nested Loop join（NLJ）和 SUBPLAN FILTER（SPF）算子的 BATCH RESCAN 优化默认会被关闭。我们针对该功能进行了专项提升，在 V4.2.1_CE_BP5 版本重新打开该默认开关。V4.2.1_CE_BP5 版本开始，新建的租户默认保持 BATCH RESCAN 优化开启。升级前已存在的老租户会维持原来的配置，如需打开优化，需在对应的租户下执行 SET GLOBAL _nlj_batching_enabled = true 手动开启。

缺陷修复

• 修复单节点与多节点查询结果不一致的问题。#1845
• 优化 ARM 环境下的性能热点问题。
• 修复 slog 盘满导致系统租户队列积压的问题。
• 修复执行非分区表转分区表 DDL，会导致一些视图（如 information_schema.TABLES/ALL_TABLES）多显示一条记录的问题。
• 修复频繁建连接/断连接场景下，引用计数泄漏导致建连接总数达到一定量后，建连接报错 4016 的问题。
• 修复队列积压情况下自动 obstack 可能导致 observer hang 住的问题。
• 修复开启全链路追踪后分布式执行计划 show trace 报错 4016 的问题。
• 修复打开 SPM 后性能下降的问题。
• 修复 .NET 驱动中使用 limit offset 时遇到 int64_max 溢出导致的 core 问题。
• 修复旁路导入产生 CommonArray 内存模块泄漏的问题。
• 修复异常场景下 PlJit 模块内存泄漏问题。
• 修复分配内存失败场景下，并行执行 core 的问题。
• 修复并行执行创建索引特定场景下内存膨胀的问题。
• 修复特定场景下备库备份可能出现 4344 报错的问题。
• 修复主集群租户设置了区分表名大小写，备集群同步创建备租户会失败的问题。
• 修复集群重启后，立即执行 create table if not exists 创建已经存在的表，执行时间异常的问题。

注意事项

V4.2.1_CE_BP5 版本修复了集群 session 连接数引用计数泄漏导致不能新建连接的问题，请尽快升级到 V4.2.1_CE_BP5 版本。

v4.3.0_CE_BETA

Version information

Information	Description
Release date	March 28, 2024
Version	V4.3.0_CE_BETA
Commit number	0193a34
Release note	The Beta version resolved most of the issues and is becoming more and more stable. However, there may still be some minor issues or errors that need to be addressed in the final stable release, so we recommend that you use this version in a testing environment.

Overview

OceanBase Database V4.3.0 is released to accommodate typical analytical processing (AP) scenarios in addition to transaction processing (TP) and lightweight AP scenarios. It provides a columnar engine based on the log-structured merge-tree (LSM-tree) architecture to implement integrated row- and column-based data storage. Powered by a new vectorized engine that is based on column data format descriptions and a cost model that is based on columnar storage, this version supports efficient processing of wide tables. This greatly improves the query performance in AP scenarios while ensuring the performance in TP business scenarios. Moreover, the new version includes a materialized view feature, which pre-evaluates and stores view query results to enhance real-time query performance. You can use materialized views for quick report generation and data analytics. The kernel of this version supports online DDL and tenant cloning features, optimizes the parallel DML (PDML) and node restart performance, and improves the bypass import efficiency for data of large object (LOB) types. It also supports AWS Simple Storage Service (S3) as the backup and restore media, optimizes system resource usage, and provides features such as index usage monitoring and local import to improve the ease of use of the system. We recommend that you use this version in hybrid load scenarios such as complex analytics, real-time reports, real-time data warehousing, and online transactions.

Key features

Key AP features

• Columnar engine

  Columnar storage is crucial for AP databases in scenarios involving complex analytics or ad-hoc queries on a large amount of data. A columnar storage differs from a row-based storage in that it physically arranges data in tables based on columns. When data is stored in columnar storage, the engine can scan only the column data required for query evaluation without scanning entire rows in AP scenarios. This reduces the usage of I/O and memory resources and increases the evaluation speed. In addition, columnar storage naturally provides better data compression conditions to achieve a higher compression ratio, thereby reducing the storage space and network bandwidth required.

  However, common columnar engines are implemented generally based on the assumption that the data organized by column is static without massive random updates. In the case of massive random updates, system performance issues are unavoidable. The LSM-Tree architecture of OceanBase Database can resolve this problem by separately processing baseline data and incremental data. Therefore, OceanBase Database V4.3.0 supports the columnar engine based on the current architecture. It implements columnar storage and rowstores on the same OBServer node based on a single set of code and architecture, ensuring both TP and AP query performance.

  The columnar engine is optimized in terms of optimizer, executor, DDL processing, and transaction processing modules to facilitate AP business migration and improve ease of use in the new version. Specifically, a columnar storage-based new cost model and a vectorized engine are introduced, the query pushdown feature is extended and enhanced, and new features such as the Skip Index attribute, new column-based encoding algorithm, and adaptive compactions are provided.

  You can flexibly set a table in your business system as a row-based store table, columnar storage table, or row/column redundant table based on the load type.

• New vectorized engine

  OceanBase Database has implemented a vectorized engine based on uniform data descriptions in earlier versions, which obviously improves the performance in contrast to non-vectorized engines but is incompetent in deep AP scenarios. OceanBase Database V4.3.0 implements vectorized engine 2.0 that is based on column data format descriptions. This avoids memory use, serialization, and read/write overheads caused by ObDatum maintenance. Based on the column data format descriptions, OceanBase Database V4.3.0 also reimplements more than 10 common operators such as HashJoin, AGGR, HashGroupBy, and Exchange (DTL Shuffle), and about 20 MySQL expressions including relational operation, logical operation, and arithmetic operation expressions. Based on the new vectorized engine, OceanBase Database will implement more operators and expressions in later V4.3.x versions to achieve higher performance in AP scenarios.

• Materialized views

  The materialized view feature is introduced since OceanBase Database V4.3.0. This feature is a key feature for AP business scenarios. It pre-evaluates and stores the view query results to improve the query performance and simplify the query logic by reducing real-time evaluations. This feature applies to quick report generation and AP scenarios.

  A materialized view stores query result sets to improve the query performance and depends on the data in the base table. When data in the base table changes, the data in the materialized view must be updated accordingly to ensure synchronization. Therefore, this version introduces a materialized view refresh mechanism, which supports two strategies: complete refresh and fast refresh. In a complete refresh of a materialized view, the system re-executes the query statements corresponding to the materialized view and overwrites the original query result sets with new ones. Complete refreshes apply to scenarios with small amounts of data. In a fast refresh, the system needs to process only the data changed since the last refresh. To implement accurate fast refreshes, OceanBase Database provides the materialized view log feature, which is similar to Oracle Materialized View Log (MLOG). The incremental data updates in the base table are logged to ensure that materialized views can be fast refreshed. Fast refreshes apply to business scenarios with large amounts of data and frequent data changes.

Kernel enhancements

• Enhancement of the row-based cost estimation system

  As the OceanBase Database version evolves, more cost estimation methods are supported for optimizers. For row-based cost estimation by each operator, a variety of algorithms, such as storage-layer cost estimation, statistics cost estimation, dynamic sampling, and default statistics, are supported. However, no clear cost estimation strategies or control measures are available. OceanBase Database V4.3.0 restructures the row-based cost estimation system. Specifically, it prioritizes cost estimation strategies based on scenarios and provides methods such as hints and system variables for manually intervening in the selection of a cost estimation strategy. This version also enhances the predicate selectivity and number of distinct values (NDV) calculation framework to improve the accuracy of cost estimation by optimizers.

• Enhancement of the statistics feature

  OceanBase Database V4.3.0 improves the statistics feature in terms of functionality, statistics collection performance, compatibility, and ease of use. Specifically, this version restructures the offline statistics collection process to improve the statistics collection efficiency, and optimizes the statistics collection strategies. By default, OceanBase Database of this version automatically collects information about index histograms and uses derived statistics. This version ensures transaction-level consistency of statistics collected online. It is compatible with the DBMS_STATS.COPY_TABLE_STATS procedure of Oracle to copy statistics and compatible with the ANALYZE TABLE statement of MySQL to support more syntaxes. Moreover, this version provides a command to cancel statistics collection, supports statistics collection progress monitoring, and enhances the ease of maintenance. It also supports parallel deletion of statistics.

• Adaptive cost model

  In earlier versions of OceanBase Database, the cost model uses constant parameters evaluated by internal servers as hardware system statistics. It uses a series of formulas and constant parameters to describe the execution overhead of each operator. In actual business scenarios, different hardware environments can provide different CPU clock frequencies, sequential/random read speeds, and NIC bandwidths. The differences may contribute to cost estimation deviations. Due to the deviations, the optimizer cannot always generate the optimal execution plan in different business environments. This version optimizes the implementation of the cost model. The cost model can use the DBMS_STATS package to collect or set system statistics parameters to adapt to the hardware environment. The DBA_OB_AUX_STATISTICS view is provided to display the system statistics parameters of the current tenant.

• Fixing of session variables for function indexes

  When a function index is created on a table, a hidden virtual generated column is added to the table and defined as the index key of the function index. The values of the virtual generated column are stored in the index table. The results of some built-in system functions are affected by session variables. The evaluation result of a function varies based on the values of session variables, even if the input arguments are the same. When a function index or generated column is created in this version, the dependent sessi...

v4.2.2_CE_BP1

Version information

Information	Description
Release date	March 13, 2024
Version	V4.2.2_CE_BP1
Commit number	083a68a
OBServer RPM version	oceanbase-ce-4.2.2.1-101000012024030709

Overview

• This release addressed several internal code compatibility issues. To upgrade a cluster from V4.2.2_CE or V4.2.2_CE_HF1 to a version higher than V4.2.2_CE_BP1, you need to upgrade it to V4.2.2_CE_BP1 first.

Bug fixes

• Fixed the issue where during the current execution of transfer, medium compaction, and backup, if an exception occurs, such as OSS write failure, Error 4016 might be reported during physical restore.
• Fixed the issue that could cause core dump on OBServer nodes when creating and switching sessions frequently.
• Fixed the issue that could cause core dump on OBServer nodes due to memory allocation problems in parallel column deletion scenarios.

版本信息

项目	描述
发布日期	2024-03-13
版本号	V4.2.2_CE_BP1
Commit 号	083a68a
OBServer RPM 版本号	oceanbase-ce-4.2.2.1-101000012024030709

发版目的

• 修复若干内部代码兼容性问题，已经部署 V4.2.2_CE 或 V4.2.2_CE_HF1 版本的集群，升级更高版本时，需经停 V4.2.2_CE_BP1 版本。

缺陷修复

• 修复在 Transfer、Medium Compaction 和备份操作并发执行期间，遇到写入 OSS 失败等可重试错误的异常场景时，可能会导致物理恢复数据报错 4016 的问题。
• 修复多次创建并切换 Session 时，可能导致 OBServer Core 的问题。
• 修复在并行删除列场景下，可能由于内存分配问题导致 OBServer Core 的问题。

v4.2.1_CE_BP4

Version information

Information	Description
Release date	March 5, 2024
Version	V4.2.1_CE_BP4
Commit number	3246b00
OBServer RPM version	oceanbase-ce-4.2.1.4-104000052024022918

Enhanced features

• The overwrite feature is now supported for OBKV Batch Put. To use this feature, you must first upgrade your client to the latest version.
• The manual partition transfer now can be cancelled.
• The query performance is now improved for query statements that contain multiple MIN or MAX functions.
• The index status now can be determined based on the COMMENT field in the information_schema.STATISTICS view.

Product behavioral changes

• The TENANT keyword is changed from a required parameter to an optional one in the table-level restore statement.
• The max_syslog_file_count parameter is changed to control the total number of log files of all types.

Bug fixes

• Fixed the issue where the return result for a UNION ALL operation on TIMESTAMP data may be inaccurate.
• Fixed the issue where Error 4016 is returned for a rewrite failure that occurs due to a parsing error when SELECT and JOIN statements are both used in PS mode.
• Fixed the issue where the performance of the HBase PUT operation of OBKV is affected by the system variable binlog_row_image.
• Fixed the issue where the upgrade_post.py script fails to execute when an upgrade is performed after the enable_rebalance parameter is manually disabled.
• Fixed the issue where memory leaks may occur in the SeArray module when an SQL statement accesses multiple sequences.
• Fixed the issue where SQL execution is stuck due to a deadlock caused by insufficient RPC threads when multiple SQL statements are executed in distributed mode and resource isolation is enabled.
• Fixed the issue where some parameters may fail to be synchronized an ALTER SYSTEM SET statement is used to modify multiple parameters.
• Fixed the issue where a cluster with a large number of tablets becomes unavailable when it is restarted. This is because all tablets in virtual tables are scanned during the restart, which occupies threads and leads to accumulation in the queue.
• Fixed the issue where CPU utilization remains high in a no-load scenario due to conflicts between automatic cleanup tasks when there exists a very large amount of historical statistics. Such conflicts occur when tasks on the previous day are not completed within 24 hours but new tasks are triggered on the current day.
• Fixed the issue where Error 4013 may be returned due to insufficient memory of the sys tenant when the DROP DATABASE statement is executed in a cluster with small specifications.
• Fixed the issue where the import task times out and cannot normally exit due to incorrect stop judgment logic when a large amount of data is imported in bypass mode.
• Fixed the issue where a major compaction cannot be started due to frequent transfer retries during RootService scheduling.
• Fixed the issue where TPS jitter occurs during unit migration and replication in weak-consistency read scenarios.
• Fixed the issue where the query result is inconsistent with that in MySQL when a query statement contains a field whose name starts with a digit.

版本信息

项目	描述
发布日期	2024-03-05
版本号	V4.2.1_CE_BP4
Commit 号	3246b00
OBServer RPM 版本号	oceanbase-ce-4.2.1.4-104000052024022918

特性增强

• OBKV Batch Put 现支持覆盖写功能，使用该功能前需要升级到最新版本的客户端。
• 手动迁移分区功能支持 Cancel。
• 优化查询语句中包含多个 MIN 或 MAX 函数时的查询性能。
• 支持通过 information_schema.STATISTICS 的 COMMENT 字段来判断索引状态。

产品行为变更

• 表级恢复命令调整 TENANT 关键字为可选。
• max_syslog_file_count 调整为控制所有类型的 Log 总量。

缺陷修复

• 修复对 TIMESTAMP 类型的数据进行 Union All 操作时，返回结果可能不正确的问题。
• 修复 PS 模式下包含 Select 和 Join 语句时，可能存在解析问题导致改写无法正确处理报错 4016 的问题。
• 修复 OBKV HBase Put 接口性能问题，修复后性能不受 binlog_row_image 系统变量的影响。
• 修复用户手动关闭 enable_rebalance 参数后升级，执行 upgrade_post.py 脚本失败的问题。
• 修复 SQL 访问多个 Sequence 场景下，可能存在的 SeArray 模块内存泄漏的问题。
• 修复使用资源隔离的场景下，当存在多条 SQL 有分布式执行场景下，可能存在 RPC 线程不足引发死锁，导致 SQL 执行卡住的问题。
• 修复通过一条 ALTER SYSTEM SET 语句修改多个配置项时，可能出现部分配置项同步失败的问题。
• 修复 Tablet 数量过多场景下，重启集群时虚表扫描所有 Tablet 占住线程引发队列积压，导致集群不可用的问题。
• 修复当历史统计信息数据量非常大的情况下，当天的清理任务 24 小时没有执行完，第二天又发起了新的自动清理任务，出现并发冲突，导致空载场景下，CPU 利用率一直很高的问题。
• 修复小规格场景下，执行 DROP DATABASE 可能由于 sys 租户内存过小报错 4013 的问题。
• 修复旁路导入大量数据时，可能由于 Stop 判断逻辑问题导致超时无法正常退出的问题。
• 修复因 RS 调度中 Transfer 重试过于频繁，导致无法启动合并操作的问题。
• 修复弱读业务场景下，Unit 迁移复制过程中 TPS 抖动的问题。
• 修复查询中包含数字开头的字段名时，与 MySQL 表现不一致的问题。

v4.2.2_CE_HF1

Version information

Information	Description
Release date	February 28, 2024
Version	V4.2.2_CE_HF1
Commit number	c984fe7
OBServer RPM version	oceanbase-ce-4.2.2.0-100010012024022719

Bug fixes

Fixed the issue where DDL execution was stuck due to a schema refresh failure caused by system view changes. This issue occurred when the cluster was either directly upgraded from V4.1.0_CE_BETA or V4.1.0_CE to V4.2.2_CE, or when the cluster was first upgraded to V4.1.0_CE_BETA or V4.1.0_CE and then further upgraded to V4.2.2_CE.

版本信息

项目	描述
发布日期	2024-2-28
版本号	V4.2.2_CE_HF1
Commit 号	c984fe7
OBServer RPM 版本号	oceanbase-ce-4.2.2.0-100010012024022719

缺陷修复

修复部署或在升级过程中经停过 V4.1.0_CE_BETA 或 V4.1.0_CE 版本的集群，在升级到 V4.2.2_CE 版本后，由于系统视图变更引发 Schema 刷新失败，导致 DDL 执行卡住的问题。

v4.2.1_CE_BP3_HF2

Version information

Information	Description
Release date	February 5, 2024
Version	V4.2.1_CE_BP3_HF2
Commit number	73d0496
OBServer RPM version	oceanbase-ce-4.2.1.3-103020042024020317

Bug fixes and improvements

• Supports importing LOB data larger than 2 MB using the Java client.
• Supports cluster maintenance and management through the OBShell command-line tool, allowing you to start or stop OceanBase Database using systemd service.
• Fixed the issue where OBServer core dump could occur due to memory allocation problems in parallel column deletion scenarios.

Considerations

• When using the bypass import feature to import LOB data that exceeds 4 KB, a transfer event could result in import failure. Therefore, we recommend that you temporarily disable the transfer feature.

• When using the bypass import feature to import LOB data that exceeds 4 KB, if a leader switchover occurs during the import process, the system will attempt to perform the import operation again.

版本信息

项目	描述
发布日期	2024-02-05
版本号	V4.2.1_CE_BP3_HF2
Commit 号	73d0496
OBServer RPM 版本号	oceanbase-ce-4.2.1.3-103020042024020317

发版目的

• 支持 Java 客户端导入超过 2M 的 LOB 数据。
• 支持通过 OBShell 命令行工具对集群进行运维管理，支持通过 systemd service 的方式启停 OceanBase 数据库。
• 修复在并行删除列场景下，可能由于内存分配问题导致 OBServer Core 的问题。

注意事项

• 使用旁路导入功能导入超过 4 KB 的 LOB 数据时，如果发生 Transfer，则会导致导入失败，建议使用时临时关闭 Transfer 功能。

• 使用旁路导入功能导入超过 4 KB 的 LOB 数据时，如果导入过程中发生切主，则系统会进行重试。