use lambda instead of std::bind #1243
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. The lambda just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.
|
@hermanlee has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Nov 15, 2022
Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: #1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7
alanliang
pushed a commit
to alanliang/mysql-5.6
that referenced
this pull request
Mar 4, 2023
Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 GitHub Author: leipeng <peng@topling.cn> Test Plan: Imported from GitHub, without a `Test Plan:` line. Reviewers: mung Reviewed By: mung Subscribers: pgl, webscalesql-eng@fb.com Differential Revision: https://phabricator.intern.facebook.com/D40858532 Tags: mysql80, accept2ship
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 12, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 15, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 16, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 17, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 18, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 26, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7 --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 fbshipit-source-id: 3104275 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441 fbshipit-source-id: e0f9d5a
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 1, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7 --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 fbshipit-source-id: 3104275 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441 fbshipit-source-id: e0f9d5a
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 14, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7 --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 fbshipit-source-id: 3104275 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441 fbshipit-source-id: e0f9d5a
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 19, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7 --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 fbshipit-source-id: 3104275 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441 fbshipit-source-id: e0f9d5a
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 23, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee fbshipit-source-id: f7431f7 --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 fbshipit-source-id: 3104275 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441 fbshipit-source-id: e0f9d5a
hermanlee
pushed a commit
to hermanlee/mysql-5.6
that referenced
this pull request
Oct 3, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
hermanlee
pushed a commit
to hermanlee/mysql-5.6
that referenced
this pull request
Oct 18, 2023
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Apr 25, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 7, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 8, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 9, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 10, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 13, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 15, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441 ------------------------------------------------------------------------------ Add yields to Item_func::fix_fields Summary: When resolving an expression with many arguments (e.g. a long `IN (...)` list) we may not yield the scheduler in a timely fashion. This should resolve stalls like ``` W0127 03:04:04.517819 2608142 TpScheduler::checkStall: Conn state: acExitLevel=0 isGoverned=1 readmissionMode=0 admitCount=1 waitEvents=0xa7 acFilter=0x0 W0127 03:04:04.517819 2608142 TpScheduler::checkStall: Collect time 748751 nsec, symbolize time 6554425 nsec, digest dc6129f5efe09474cfacb020cdc3521b W0127 03:04:04.517819 2608142 TpScheduler::checkStall: Stack trace for 1 thread(s) [2608144 tp_conn-0]: W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 0000000002c134bd Item_func_in::resolve_type(THD*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 0000000002839275 Item_func::fix_fields(THD*, Item**) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 0000000002eca7d5 Item_func_in::fix_fields(THD*, Item**) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 000000000295fd80 Query_block::setup_conds(THD*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 0000000002ca4b77 Sql_cmd_delete::prepare_inner(THD*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 000000000282d295 Sql_cmd_dml::prepare(THD*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 0000000002827053 Sql_cmd_dml::execute(THD*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 000000000282177d mysql_execute_command(THD*, bool, unsigned long long*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 0000000002807d26 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 00000000027f8238 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 00000000027f6d20 do_command(THD*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 00000000062dedd7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 00000000062f01f6 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 000000000475447a pfs_spawn_thread(void*) [clone .__uniq.73903348312165179054628174044725776814] [clone .llvm.15823314186845019609] W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 000000000009abae start_thread W0127 03:04:04.517819 2608142 TpScheduler::checkStall: @ 000000000012d17b __clone3 ``` Differential Revision: D53241252 fbshipit-source-id: 7fee664
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 16, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 17, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 17, 2024
…read pool plugin Summary: 1. Change the code to admit new query through `thd_wait_begin/end` callbacks using new `THD_WAIT_ADMIT` wait type. 2. Move yield counting to the callbacks to unify the code between the server and the plugin. `yield_cond` exposes the yield condition to check once the yield counter says that yield is needed. 3. Expose all wait events through `admission_control_wait_events` var. This is especially needed for `USER_LOCK` which still controls innodb thread concurrency yield. 4. Copy and fix up admission control tests to the thread_pool suite. Only the var and table names are changed to refer to thread_pool plugin. The results have been diffed with original tests to make sure they are identical (except for var/table names). Reviewed By: lth Differential Revision: D27593084 --------------------------------------------------------------------------------------------- use lambda instead of std::bind (facebook#1243) Summary: std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor. This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed. Pull Request resolved: facebook#1243 Reviewed By: lth Differential Revision: D40858532 Pulled By: hermanlee --------------------------------------------------------------------------------------------- Add yields to hash join iterator Summary: Queries performing joins are lacking yields and monopolizing thread pool scheduler. Differential Revision: D39948376 ------------------------------------------------------------------------------ Add yield to union iterator Summary: This was detected by the stall detection (8.0.23 based callstack): ``` W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Scheduler 12 stalled by worker 0x7f6367d5d640 W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] Stack trace for 1 thread(s) [1422783 mysqld]: W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000001204d0 ppoll W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000289d67f vio_io_wait(Vio*, enum_vio_io_event, timeout_t) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002952296 vio_socket_io_wait(Vio*, enum_vio_io_event) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002385fb2 net_write_packet(NET*, unsigned char const*, unsigned long) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 00000000025b2c44 SELECT_LEX_UNIT::ExecuteIteratorQuery(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000258d2db Sql_cmd_dml::execute(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000254f179 mysql_execute_command(THD*, bool, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000002545936 dispatch_sql_command(THD*, Parser_state*, unsigned long long*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269d931 dispatch_command(THD*, COM_DATA const*, enum_server_command) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000269c3e0 do_command(THD*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c4ffa7 mysql::thread_pool::TpConnHandler::processEvent(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5395f mysql::thread_pool::TpTask::execute() W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000005c5c299 mysql::thread_pool::TpWorkerPool::processWorker(void*) W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 0000000004867855 pfs_spawn_thread(void*) [clone .llvm.11916604799980067416] W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000009ac0e start_thread W0205 14:38:40.101111 2319640 TpScheduler.cpp:431] @ 000000000012d1db __clone3 ``` Cleaned up existing yield code to use a common default yield condition. Reviewed By: preritj24 Differential Revision: D43071441
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
std::bind(yield_condition, table) will generate a functor which size is larger than std::function's local buf, thus std::function needs to new/delete memory to store the functor.
This PR use the lambda which just capture one pointer(table), which size can fit into std::function's local buf thus new/delete is not needed.