8.4.4 Internal Temporary Table Use in MySQLMySQL中内部临时表的使用

In some cases, the server creates internal temporary tables while processing statements. 在某些情况下,服务器在处理语句时创建内部临时表。Users have no direct control over when this occurs.用户无法直接控制何时发生这种情况。

The server creates temporary tables under conditions such as these:服务器在以下条件下创建临时表:

To determine whether a statement requires a temporary table, use EXPLAIN and check the Extra column to see whether it says Using temporary (see Section 8.8.1, “Optimizing Queries with EXPLAIN”). 要确定语句是否需要临时表,请使用EXPLAIN并检查Extra的列,以查看它是否表示使用临时表(请参阅第8.8.1节,“使用EXPLAIN优化查询”)。EXPLAIN does not necessarily say Using temporary for derived or materialized temporary tables. EXPLAIN不一定表示对派生的或物化的临时表使用临时表。For statements that use window functions, EXPLAIN with FORMAT=JSON always provides information about the windowing steps. 对于使用窗口函数的语句,使用FORMAT=JSONEXPLAIN始终提供有关窗口操作步骤的信息。If the windowing functions use temporary tables, it is indicated for each step.如果窗口功能使用临时表,则每个步骤都会显示临时表。

Some query conditions prevent the use of an in-memory temporary table, in which case the server uses an on-disk table instead:某些查询条件阻止使用内存中的临时表,在这种情况下,服务器将使用磁盘上的表:

The server does not use a temporary table for UNION statements that meet certain qualifications. 对于满足某些限定条件的UNION语句,服务器不使用临时表。Instead, it retains from temporary table creation only the data structures necessary to perform result column typecasting. 相反,它从临时表创建中只保留执行结果列类型转换所需的数据结构。The table is not fully instantiated and no rows are written to or read from it; rows are sent directly to the client. 该表没有完全实例化,没有写入或读取任何行;行直接发送到客户端。The result is reduced memory and disk requirements, and smaller delay before the first row is sent to the client because the server need not wait until the last query block is executed. 结果是减少了内存和磁盘需求,并且在将第一行发送到客户机之前的延迟更小,因为服务器不需要等到执行最后一个查询块。EXPLAIN and optimizer trace output reflects this execution strategy: The UNION RESULT query block is not present because that block corresponds to the part that reads from the temporary table.EXPLAIN和优化器跟踪输出反映了这种执行策略:联合结果查询块不存在,因为该块对应于从临时表读取的部分。

These conditions qualify a UNION for evaluation without a temporary table:这些条件使UNION有资格在没有临时表的情况下进行评估:

Internal Temporary Table Storage Engine内部临时表存储引擎

An internal temporary table can be held in memory and processed by the TempTable or MEMORY storage engine, or stored on disk by the InnoDB storage engine.内部临时表可以保存在内存中,并由TempTableMEMORY存储引擎处理,或者由InnoDB存储引擎存储在磁盘上。

Storage Engine for In-Memory Internal Temporary Tables内存内部临时表的存储引擎

The internal_tmp_mem_storage_engine session variable defines the storage engine for in-memory internal temporary tables. internal_tmp_mem_storage_engine会话变量定义内存内部临时表的存储引擎。Permitted values are TempTable (the default) and MEMORY.允许的值是TempTable(默认值)和MEMORY

The TempTable storage engine provides efficient storage for VARCHAR and VARBINARY columns, and other binary large object types as of MySQL 8.0.13.TempTable存储引擎为VARCHARVARBINARY列以及MySQL 8.0.13中的其他二进制大对象类型提供了高效的存储。

The temptable_max_ram variable defines the maximum amount of RAM that can be occupied by the TempTable storage engine before it starts allocating space from disk in the form memory-mapped temporary files or InnoDB on-disk internal temporary tables. temptable_max_ram变量定义了诱惑存储引擎在开始以内存映射临时文件或InnoDB on disk内部临时表的形式从磁盘分配空间之前可以占用的最大RAM量。The default temptable_max_ram setting is 1GiB. 默认temptable_max_ram设置为1GiB。The temptable_use_mmap variable (introduced in MySQL 8.0.16; deprecated in MySQL 8.0.26) controls whether the TempTable storage engine uses memory-mapped files or InnoDB on-disk internal temporary tables when the temptable_max_ram limit is exceeded. temptable_use_mmap变量(在MySQL 8.0.16中引入;MySQL 8.0.26中已弃用)控制当超过temptable_max_ram限制时,TempTable存储引擎是使用内存映射文件还是使用磁盘上的InnoDB内部临时表。The default setting is temptable_use_mmap=ON. 默认设置是temptable_use_mmap=ONThe temptable_max_mmap variable, introduced in MySQL 8.0.23, defines the maximum amount of memory the TempTable storage engine is permitted to allocate from memory-mapped files before it starts storing internal temporary table data to InnoDB on-disk internal temporary tables. MySQL 8.0.23中引入的temptable_max_mmap变量定义了TempTable存储引擎在开始将内部临时表数据存储到磁盘上的InnoDB内部临时表之前,允许从内存映射文件中分配的最大内存量。A temptable_max_mmap=0 setting disables allocation from memory-mapped files, effectively disabling their use, regardless of the temptable_use_mmap setting.temptable_max_mmap=0设置会禁用内存映射文件的分配,从而有效禁用它们的使用,而不管temptable_use_mmap设置如何。

Note注意

The temptable_use_mmap variable is deprecated as of MySQL 8.0.26; expect support for it to be removed in a future version of MySQL. 从MySQL 8.0.26开始,不推荐使用temptable_use_mmap变量;希望在MySQL的未来版本中删除对它的支持。Setting temptable_max_mmap=0 is equivalent to setting temptable_use_mmap=OFF.设置temptable_max_mmap=0相当于设置temptable_use_mmap=OFF

The temptable_max_ram setting does not account for the thread-local memory block allocated to each thread that uses the TempTable storage engine. temptable_max_ram设置不考虑分配给使用TempTable存储引擎的每个线程的线程本地内存块。The size of the thread-local memory block depends on the size of the thread's first memory allocation request. 线程本地内存块的大小取决于线程的第一个内存分配请求的大小。If the request is less than 1MB, which it is in most cases, the thread-local memory block size is 1MB. If the request is greater than 1MB, the thread-local memory block is approximately the same size as the initial memory request. 如果请求小于1MB,在大多数情况下,线程本地内存块大小为1MB。如果请求大于1MB,则线程本地内存块的大小与初始内存请求的大小大致相同。The thread-local memory block is held in thread-local storage until thread exit.线程本地内存块保存在线程本地存储器中,直到线程退出。

Use of memory-mapped temporary files by the TempTable storage engine as an overflow mechanism for internal temporary tables is governed by these rules:TempTable存储引擎将内存映射的临时文件用作内部临时表的溢出机制受以下规则控制:

  • Temporary files are created in the directory defined by the tmpdir variable.临时文件在tmpdir变量定义的目录中创建。

  • Temporary files are deleted immediately after they are created and opened, and therefore do not remain visible in the tmpdir directory. 临时文件在创建和打开后立即被删除,因此在tmpdir目录中不可见。The space occupied by temporary files is held by the operating system while temporary files are open. 临时文件占用的空间在临时文件打开时由操作系统保留。The space is reclaimed when temporary files are closed by the TempTable storage engine, or when the mysqld process is shut down.TempTable存储引擎关闭临时文件时,或者当mysqld进程关闭时,会回收空间。

  • Data is never moved between RAM and temporary files, within RAM, or between temporary files.数据永远不会在RAM和临时文件之间、RAM内或临时文件之间移动。

  • New data is stored in RAM if space becomes available within the limit defined by temptable_max_ram. 如果空间在temptable_max_ram定义的限制范围内可用,则新数据存储在RAM中。Otherwise, new data is stored in temporary files.否则,新数据将存储在临时文件中。

  • If space becomes available in RAM after some of the data for a table is written to temporary files, it is possible for the remaining table data to be stored in RAM.如果一个表的某些数据写入临时文件后,RAM中的空间变为可用,则剩余的表数据可能存储在RAM中。

If the TempTable storage engine is configured to use InnoDB on-disk internal temporary tables as the overflow mechanism (temptable_use_mmap=OFF or temptable_max_mmap=0), an in-memory table that exceeds the temptable_max_ram limit is converted to an InnoDB on-disk internal temporary table, and any rows belonging to that table are moved from memory to the InnoDB on-disk internal temporary table. 如果TempTable存储引擎配置为使用InnoDB磁盘上内部临时表作为溢出机制(temptable_use_mmap=OFFtemptable_max_mmap=0),则超过temptable_max_ram限制的内存中表将转换为InnoDB磁盘上内部临时表,并且属于该表的任何行都将从内存移动到磁盘上的InnoDB内部临时表。The internal_tmp_disk_storage_engine setting (removed in MySQL 8.0.16) has no affect on the TempTable storage engine overflow mechanism.internal_tmp_disk_storage_engine设置(在MySQL 8.0.16中删除)对TempTable存储引擎溢出机制没有影响。

Prior to MySQL 8.0.23, InnoDB on-disk internal temporary tables are recommended as the TempTable overflow mechanism if the TempTable storage engine often exceeds the temptable_max_ram limit and uses excessive space in the temporary directory for memory-mapped files. 在MySQL 8.0.23之前,如果TempTable存储引擎经常超过temptable_max_ram限制,并在临时目录中为内存映射文件使用过多空间,则建议使用InnoDB磁盘上内部临时表作为TempTable溢出机制。As of MySQL 8.0.23, the temptable_max_mmap variable defines a limit on the amount of memory the TempTable storage engine allocates from memory-mapped files, which addresses the risk of those files using too much space. 从MySQL 8.0.23开始,temptable_max_mmap变量定义了诱惑式存储引擎从内存映射文件分配的内存量的限制,这解决了这些文件使用过多空间的风险。Exceeding the temptable_max_ram limit typically occurs due to use of large internal temporary tables or extensive use of internal temporary tables. 由于使用大型内部临时表或大量使用内部临时表,通常会超过temptable_max_ram限制。InnoDB on-disk internal temporary tables are created in session temporary tablespaces, which reside in the data directory by default. InnoDB磁盘上的内部临时表是在会话临时表空间中创建的,默认情况下,会话临时表空间位于数据目录中。For more information, see Section 15.6.3.5, “Temporary Tablespaces”.有关更多信息,请参阅第15.6.3.5节,“临时表空间”

When using the MEMORY storage engine for in-memory temporary tables, MySQL automatically converts an in-memory temporary table to an on-disk table if it becomes too large. 当使用MEMORY存储引擎处理内存中的临时表时,如果内存中的临时表太大,MySQL会自动将其转换为磁盘上的临时表。The maximum size of an in-memory temporary table is defined by the tmp_table_size or max_heap_table_size value, whichever is smaller. 内存中临时表的最大大小由tmp_table_sizemax_heap_table_size值定义,以较小者为准。This differs from MEMORY tables explicitly created with CREATE TABLE. 这与使用CREATE TABLE显式创建的MEMORY表不同。For such tables, only the max_heap_table_size variable determines how large a table can grow, and there is no conversion to on-disk format.对于这样的表,只有max_heap_table_size变量确定表可以增长多大,并且没有到磁盘格式的转换。

Storage Engine for On-Disk Internal Temporary Tables磁盘上内部临时表的存储引擎

Starting with MySQL 8.0.16, the server always uses the InnoDB storage engine for managing internal temporary tables on disk.从MySQL 8.0.16开始,服务器总是使用InnoDB存储引擎来管理磁盘上的内部临时表。

In MySQL 8.0.15 and earlier, the internal_tmp_disk_storage_engine variable was used to define the storage engine used for on-disk internal temporary tables. 在MySQL 8.0.15及更早版本中,internal_tmp_disk_storage_engine变量用于定义用于磁盘内部临时表的存储引擎。This variable was removed in MySQL 8.0.16, and the storage engine used for this purpose is no longer user-configurable.MySQL 8.0.16中删除了此变量,用于此目的的存储引擎不再是用户可配置的。

In MySQL 8.0.15 and earlier: For common table expressions (CTEs), the storage engine used for on-disk internal temporary tables cannot be MyISAM. 在MySQL 8.0.15及更早版本中:对于公共表表达式(CTE),用于磁盘内部临时表的存储引擎不能是MyISAMIf internal_tmp_disk_storage_engine=MYISAM, an error occurs for any attempt to materialize a CTE using an on-disk temporary table.如果internal_tmp_disk_storage_engine=MYISAM,则任何使用磁盘上临时表具体化CTE的尝试都会发生错误。

In MySQL 8.0.15 and earlier: When using internal_tmp_disk_storage_engine=INNODB, queries that generate on-disk internal temporary tables that exceed InnoDB row or column limits return Row size too large or Too many columns errors. 在MySQL 8.0.15及更早版本中:当使用internal_tmp_disk_storage_engine=INNODB时,在磁盘上生成超过InnoDB行或列限制的内部临时表的查询会返回行大小过大或列过多的错误。The workaround is to set internal_tmp_disk_storage_engine to MYISAM.变通的解决方法是将internal_tmp_disk_storage_engine设置为MYISAM

Internal Temporary Table Storage Format内部临时表存储格式

When in-memory internal temporary tables are managed by the TempTable storage engine, rows that include VARCHAR columns, VARBINARY columns, and other binary large object type columns (supported as of MySQL 8.0.13) are represented in memory by an array of cells, with each cell containing a NULL flag, the data length, and a data pointer. 当内存中的内部临时表由TempTable存储引擎管理时,包含VARCHAR列、VARBINARY列和其他二进制大对象类型列(MySQL 8.0.13支持)的行在内存中由一个单元格数组表示,每个单元格包含一个空标志、数据长度和一个数据指针。Column values are placed in consecutive order after the array, in a single region of memory, without padding. 列值按连续顺序放置在数组之后,在单个内存区域中,无需填充。Each cell in the array uses 16 bytes of storage. 阵列中的每个单元使用16字节的存储空间。The same storage format applies when the TempTable storage engine exceeds the temptable_max_ram limit and starts allocating space from disk as memory-mapped files or InnoDB on-disk internal temporary tables.TempTable存储引擎超过temptable_max_ram限制,并开始从磁盘分配空间作为内存映射文件或InnoDB on-disk内部临时表时,同样的存储格式适用。

When in-memory internal temporary tables are managed by the MEMORY storage engine, fixed-length row format is used. 当内存中的内部临时表由MEMORY存储引擎管理时,将使用固定长度的行格式。VARCHAR and VARBINARY column values are padded to the maximum column length, in effect storing them as CHAR and BINARY columns.VARCHARVARBINARY列值被填充到最大列长度,实际上将它们存储为CHARBINARY列。

Previous to MySQL 8.0.16, on-disk internal temporary tables were managed by the InnoDB or MyISAM storage engine (depending on the internal_tmp_disk_storage_engine setting). 在MySQL 8.0.16之前,磁盘上的内部临时表由InnoDBMyISAM存储引擎管理(取决于internal_tmp_disk_storage_engine设置)。Both engines store internal temporary tables using dynamic-width row format. Columns take only as much storage as needed, which reduces disk I/O, space requirements, and processing time compared to on-disk tables that use fixed-length rows. 两个引擎都使用动态宽度行格式存储内部临时表。列只占用所需的存储空间,与使用固定长度行的磁盘表相比,这减少了磁盘I/O、空间需求和处理时间。Beginning with MySQL 8.0.16, internal_tmp_disk_storage_engine is not supported, and internal temporary tables on disk are always handled by InnoDB.从MySQL 8.0.16开始,不支持internal_tmp_disk_storage_engine,磁盘上的内部临时表总是由InnoDB处理。

When using the MEMORY storage engine, statements can initially create an in-memory internal temporary table and then convert it to an on-disk table if the table becomes too large. 使用MEMORY存储引擎时,语句最初可以创建内存中的内部临时表,如果表太大,则可以将其转换为磁盘上的表。In such cases, better performance might be achieved by skipping the conversion and creating the internal temporary table on disk to begin with. 在这种情况下,可以通过跳过转换并首先在磁盘上创建内部临时表来实现更好的性能。The big_tables variable can be used to force disk storage of internal temporary tables.big_tables变量可用于强制内部临时表的磁盘存储。

Monitoring Internal Temporary Table Creation监视内部临时表的创建

When an internal temporary table is created in memory or on disk, the server increments the Created_tmp_tables value. 在内存或磁盘上创建内部临时表时,服务器会增加Created_tmp_tables值。When an internal temporary table is created on disk, the server increments the Created_tmp_disk_tables value. 在磁盘上创建内部临时表时,服务器会增加Created_tmp_disk_tables值。If too many internal temporary tables are created on disk, consider increasing the tmp_table_size and max_heap_table_size settings.如果在磁盘上创建了太多的内部临时表,请考虑增加tmp_table_sizemax_heap_table_size设置。

Note注意

Due to a known limitation, Created_tmp_disk_tables does not count on-disk temporary tables created in memory-mapped files. 由于已知的限制,Created_tmp_disk_tables不依赖于在内存映射文件中创建的磁盘临时表。By default, the TempTable storage engine overflow mechanism creates internal temporary tables in memory-mapped files. 默认情况下,诱人的存储引擎溢出机制会在内存映射文件中创建内部临时表。This behavior is controlled by the temptable_use_mmap and temptable_max_mmap variables.此行为由temptable_use_mmaptemptable_max_mmap变量控制。

The memory/temptable/physical_ram and memory/temptable/physical_disk Performance Schema instruments can be used to monitor TempTable space allocation from memory and disk. memory/temptable/physical_rammemory/temptable/physical_disk性能模式工具可用于监控内存和磁盘的可诱惑空间分配。memory/temptable/physical_ram reports the amount of allocated RAM. memory/temptable/physical_ram报告分配的内存量。memory/temptable/physical_disk reports the amount of space allocated from disk when memory-mapped files are used as the TempTable overflow mechanism. memory/temptable/physical_disk报告当内存映射文件用作tentable溢出机制时从磁盘分配的空间量。If the physical_disk instrument reports a value other than 0 and memory-mapped files are used as the TempTable overflow mechanism, the temptable_max_ram threshold was reached at some point. 如果physical_disk器材械报告的值不是0,并且内存映射文件被用作TempTable溢出机制,则在某个时候达到了temptable_max_ram阈值。Data can be queried in Performance Schema memory summary tables such as memory_summary_global_by_event_name. 可以在性能模式内存摘要表中查询数据,例如memory_summary_global_by_event_nameSee Section 27.12.20.10, “Memory Summary Tables”.请参阅第27.12.20.10节,“内存汇总表”