8.2.1.16 ORDER BY Optimization优化

This section describes when MySQL can use an index to satisfy an ORDER BY clause, the filesort operation used when an index cannot be used, and execution plan information available from the optimizer about ORDER BY.本节介绍MySQL何时可以使用索引来满足ORDER BY子句、无法使用索引时使用的filesort操作,以及优化器提供的有关ORDER BY的执行计划信息。

An ORDER BY with and without LIMIT may return rows in different orders, as discussed in Section 8.2.1.19, “LIMIT Query Optimization”.第8.2.1.19节,“LIMIT查询优化”所述,有LIMIT和无LIMITORDER BY可以以不同的顺序返回行。

Use of Indexes to Satisfy 使用索引来满足ORDER BY

In some cases, MySQL may use an index to satisfy an ORDER BY clause and avoid the extra sorting involved in performing a filesort operation.在某些情况下,MySQL可能使用索引来满足ORDER BY子句,并避免执行filesort操作所涉及的额外排序。

The index may also be used even if the ORDER BY does not match the index exactly, as long as all unused portions of the index and all extra ORDER BY columns are constants in the WHERE clause. 即使ORDER BY与索引不完全匹配,也可以使用索引,只要索引的所有未使用部分和所有额外ORDER BY列都是WHERE子句中的常量。If the index does not contain all columns accessed by the query, the index is used only if index access is cheaper than other access methods.如果索引不包含查询访问的所有列,则仅当索引访问比其他访问方法便宜时才使用索引。

Assuming that there is an index on (key_part1, key_part2), the following queries may use the index to resolve the ORDER BY part. 假设(key_part1, key_part2)上有一个索引,下面的查询可以使用该索引解决ORDER BY部分。Whether the optimizer actually does so depends on whether reading the index is more efficient than a table scan if columns not in the index must also be read.优化器是否真的这样做取决于如果还必须读取不在索引中的列,则读取索引是否比表扫描更有效。

  • In this query, the index on (key_part1, key_part2) enables the optimizer to avoid sorting:在这个查询中,(key_part1, key_part2)上的索引使优化器能够避免排序:

    SELECT * FROM t1
      ORDER BY key_part1, key_part2;

    However, the query uses SELECT *, which may select more columns than key_part1 and key_part2. 但是,查询使用SELECT *,它可以选择比key_part1key_part2更多的列。In that case, scanning an entire index and looking up table rows to find columns not in the index may be more expensive than scanning the table and sorting the results. 在这种情况下,扫描整个索引并查找表行以查找不在索引中的列可能比扫描表并对结果进行排序更昂贵。If so, the optimizer probably does not use the index. 如果是这样,优化器可能不使用索引。If SELECT * selects only the index columns, the index is used and sorting avoided.如果SELECT *只选择索引列,则使用了索引,避免了排序。

    If t1 is an InnoDB table, the table primary key is implicitly part of the index, and the index can be used to resolve the ORDER BY for this query:如果t1InnoDB表,则表主键隐式地是索引的一部分,该索引可用于解析此查询的ORDER BY

    SELECT pk, key_part1, key_part2 FROM t1
      ORDER BY key_part1, key_part2;
  • In this query, key_part1 is constant, so all rows accessed through the index are in key_part2 order, and an index on (key_part1, key_part2) avoids sorting if the WHERE clause is selective enough to make an index range scan cheaper than a table scan:在该查询中,key_part1是常量,因此通过索引访问的所有行都是按key_part2排序,如果WHERE子句的选择性足以使索引范围扫描比表扫描便宜,则(key_part1, key_part2)上的索引可以避免排序:

    SELECT * FROM t1
      WHERE key_part1 = constant
      ORDER BY key_part2;
  • In the next two queries, whether the index is used is similar to the same queries without DESC shown previously:在接下来的两个查询中,是否使用索引与前面未显示DESC的相同查询类似:

    SELECT * FROM t1
      ORDER BY key_part1 DESC, key_part2 DESC;
    
    SELECT * FROM t1
      WHERE key_part1 = constant
      ORDER BY key_part2 DESC;
  • Two columns in an ORDER BY can sort in the same direction (both ASC, or both DESC) or in opposite directions (one ASC, one DESC). ORDER BY中的两列可以按相同方向(两个ASC或两个DESC)排序,也可以按相反方向(一个ASC,一个DESC)排序。A condition for index use is that the index must have the same homogeneity, but need not have the same actual direction.使用索引的条件是索引必须具有相同的同质性,但不必具有相同的实际方向。

    If a query mixes ASC and DESC, the optimizer can use an index on the columns if the index also uses corresponding mixed ascending and descending columns:如果查询混合了ASCDESC,那么如果索引还使用相应的升序和降序混合列,则优化器可以在列上使用索引:

    SELECT * FROM t1
      ORDER BY key_part1 DESC, key_part2 ASC;

    The optimizer can use an index on (key_part1, key_part2) if key_part1 is descending and key_part2 is ascending. 如果key_part1是降序,而key_part2<是升序,则优化器可以使用(key_part1, key_part2)上的索引。It can also use an index on those columns (with a backward scan) if key_part1 is ascending and key_part2 is descending. 如果key_part1为升序,key_part2为降序,它还可以在这些列上使用索引(向后扫描)。See Section 8.3.13, “Descending Indexes”.请参阅第8.3.13节,“降序索引”

  • In the next two queries, key_part1 is compared to a constant. 在接下来的两个查询中,将key_part1与一个常量进行比较。The index is used if the WHERE clause is selective enough to make an index range scan cheaper than a table scan:如果WHERE子句的选择性足以使索引范围扫描比表扫描便宜,则使用索引:

    SELECT * FROM t1
      WHERE key_part1 > constant
      ORDER BY key_part1 ASC;
    
    SELECT * FROM t1
      WHERE key_part1 < constant
      ORDER BY key_part1 DESC;
  • In the next query, the ORDER BY does not name key_part1, but all rows selected have a constant key_part1 value, so the index can still be used:在下一个查询中,ORDER BY没有命名key_part1,但所有选中的行都有一个常量key_part1值,因此仍可使用索引:

    SELECT * FROM t1
      WHERE key_part1 = constant1 AND key_part2 > constant2
      ORDER BY key_part2;

In some cases, MySQL cannot use indexes to resolve the ORDER BY, although it may still use indexes to find the rows that match the WHERE clause. 在某些情况下,MySQL不能使用索引来解析ORDER BY,尽管它仍然可以使用索引来查找与WHERE子句匹配的行。Examples:

  • The query uses ORDER BY on different indexes:查询在不同索引上使用ORDER BY

    SELECT * FROM t1 ORDER BY key1, key2;
  • The query uses ORDER BY on nonconsecutive parts of an index:查询对索引的非连续部分使用ORDER BY

    SELECT * FROM t1 WHERE key2=constant ORDER BY key1_part1, key1_part3;
  • The index used to fetch the rows differs from the one used in the ORDER BY:用于获取行的索引与ORDER BY中使用的索引不同,具体如下:

    SELECT * FROM t1 WHERE key2=constant ORDER BY key1;
  • The query uses ORDER BY with an expression that includes terms other than the index column name:查询将ORDER BY与包含索引列名以外的术语的表达式一起使用:

    SELECT * FROM t1 ORDER BY ABS(key);
    SELECT * FROM t1 ORDER BY -key;
  • The query joins many tables, and the columns in the ORDER BY are not all from the first nonconstant table that is used to retrieve rows. 该查询连接了许多表,ORDER BY中的列并不都来自用于检索行的第一个非常量表。(This is the first table in the EXPLAIN output that does not have a const join type.)(这是EXPLAIN输出中第一个没有const联接类型的表。)

  • The query has different ORDER BY and GROUP BY expressions.查询具有不同的ORDER BYGROUP BY表达式。

  • There is an index on only a prefix of a column named in the ORDER BY clause. 只有在ORDER BY子句中命名的列的前缀上有索引。In this case, the index cannot be used to fully resolve the sort order. 在这种情况下,索引不能用于完全解析排序顺序。For example, if only the first 10 bytes of a CHAR(20) column are indexed, the index cannot distinguish values past the 10th byte and a filesort is needed.例如,如果仅对CHAR(20)列的前10个字节进行索引,则索引无法区分超过第10个字节的值,因此需要一个filesort

  • The index does not store rows in order. 索引不按顺序存储行。For example, this is true for a HASH index in a MEMORY table.例如,这对于MEMORY表中的HASH索引是正确的。

Availability of an index for sorting may be affected by the use of column aliases. 用于排序的索引的可用性可能会受到列别名使用的影响。Suppose that the column t1.a is indexed. 假设列t1.a被索引。In this statement, the name of the column in the select list is a. 在此语句中,选择列表中的列的名称为aIt refers to t1.a, as does the reference to a in the ORDER BY, so the index on t1.a can be used:它引用t1.a,正如ORDER BY中引用a一样,因此可以使用t1.a上的索引:

SELECT a FROM t1 ORDER BY a;

In this statement, the name of the column in the select list is also a, but it is the alias name. 在此语句中,选择列表中的列的名称也是a,但它是别名。It refers to ABS(a), as does the reference to a in the ORDER BY, so the index on t1.a cannot be used:它指的是ABS(a),正如在ORDER BY中对a的引用一样,因此不能使用t1.a上的索引:

SELECT ABS(a) AS a FROM t1 ORDER BY a;

In the following statement, the ORDER BY refers to a name that is not the name of a column in the select list. 在下面的语句中,ORDER BY引用的名称不是选择列表中某列的名称。But there is a column in t1 named a, so the ORDER BY refers to t1.a and the index on t1.a can be used. 但是t1中有一列名为a,因此ORDER BY引用t1.a,可以使用t1.a上的索引。(The resulting sort order may be completely different from the order for ABS(a), of course.)(当然,产生的排序顺序可能与ABS(a)的顺序完全不同。)

SELECT ABS(a) AS b FROM t1 ORDER BY a;

Previously (MySQL 5.7 and lower), GROUP BY sorted implicitly under certain conditions. 以前(MySQL 5.7及更低版本),GROUP BY在某些条件下隐式排序。In MySQL 8.0, that no longer occurs, so specifying ORDER BY NULL at the end to suppress implicit sorting (as was done previously) is no longer necessary. 在MySQL 8.0中,这种情况不再发生,因此不再需要在末尾指定ORDER BY NULL来抑制隐式排序(如前所述)。However, query results may differ from previous MySQL versions. 但是,查询结果可能不同于以前的MySQL版本。To produce a given sort order, provide an ORDER BY clause.要生成给定的排序顺序,请提供ORDER BY子句。

Use of filesort to Satisfy 使用filesort来满足ORDER BY

If an index cannot be used to satisfy an ORDER BY clause, MySQL performs a filesort operation that reads table rows and sorts them. 如果索引不能用于满足ORDER BY子句,MySQL将执行filesort操作,读取表行并对其进行排序。A filesort constitutes an extra sorting phase in query execution.filesort构成查询执行中的额外排序阶段。

To obtain memory for filesort operations, as of MySQL 8.0.12, the optimizer allocates memory buffers incrementally as needed, up to the size indicated by the sort_buffer_size system variable, rather than allocating a fixed amount of sort_buffer_size bytes up front, as was done prior to MySQL 8.0.12. 为了获得用于文件排序操作的内存,从MySQL 8.0.12开始,优化器根据需要递增地分配内存缓冲区,直到sort_buffer_size系统变量指示的大小,而不是像MySQL 8.0.12之前那样预先分配固定数量的sort_buffer_size字节。This enables users to set sort_buffer_size to larger values to speed up larger sorts, without concern for excessive memory use for small sorts. 这使得用户能够将sort_buffer_size设置为更大的值,以加快更大的排序,而不必担心小排序会占用过多的内存。(This benefit may not occur for multiple concurrent sorts on Windows, which has a weak multithreaded malloc.)(Windows上的多个并发排序可能不会带来这种好处,因为它的多线程malloc很弱。)

A filesort operation uses temporary disk files as necessary if the result set is too large to fit in memory. 如果结果集太大而无法放入内存,则filesort操作会根据需要使用临时磁盘文件。Some types of queries are particularly suited to completely in-memory filesort operations. 某些类型的查询特别适合于完全在内存中的filesort操作。For example, the optimizer can use filesort to efficiently handle in memory, without temporary files, the ORDER BY operation for queries (and subqueries) of the following form:例如,优化器可以使用filesort在内存中有效地处理以下形式的查询(和子查询)的ORDER BY操作,而不需要临时文件:

SELECT ... FROM single_table ... ORDER BY non_index_column [DESC] LIMIT [M,]N;

Such queries are common in web applications that display only a few rows from a larger result set. 这种查询在只显示较大结果集中的几行的web应用程序中很常见。Examples:示例:

SELECT col1, ... FROM t1 ... ORDER BY name LIMIT 10;
SELECT col1, ... FROM t1 ... ORDER BY RAND() LIMIT 15;
Influencing ORDER BY Optimization影响ORDER BY优化

For slow ORDER BY queries for which filesort is not used, try lowering the max_length_for_sort_data system variable to a value that is appropriate to trigger a filesort. 对于未使用filesort的慢ORDER BY查询,请尝试将系统变量max_length_for_sort_data降低到适合触发filesort的值。(A symptom of setting the value of this variable too high is a combination of high disk activity and low CPU activity.) (将此变量的值设置得过高的一个症状是高磁盘活动和低CPU活动的组合。)This technique applies only before MySQL 8.0.20. 此技术仅适用于MySQL 8.0.20之前的版本。As of 8.0.20, max_length_for_sort_data is deprecated due to optimizer changes that make it obsolete and of no effect.从8.0.20开始,由于优化器的更改使其过时且无效,因此不推荐使用max_length_for_sort_data(排序数据的最大长度)。

To increase ORDER BY speed, check whether you can get MySQL to use indexes rather than an extra sorting phase. 为了提高ORDER BY速度,请检查是否可以让MySQL使用索引,而不是额外的排序阶段。If this is not possible, try the following strategies:如果不可能,请尝试以下策略:

  • Increase the sort_buffer_size variable value. 增加sort_buffer_size变量值。Ideally, the value should be large enough for the entire result set to fit in the sort buffer (to avoid writes to disk and merge passes).理想情况下,该值应足够大,以使整个结果集能够放入排序缓冲区(以避免写入磁盘和合并过程)。

    Take into account that the size of column values stored in the sort buffer is affected by the max_sort_length system variable value. 考虑到存储在排序缓冲区中的列值的大小受max_sort_length系统变量值的影响。For example, if tuples store values of long string columns and you increase the value of max_sort_length, the size of sort buffer tuples increases as well and may require you to increase sort_buffer_size.例如,如果元组存储长字符串列的值,并且您增加了max_sort_length的值,则排序缓冲区元组的大小也会增加,并且可能需要您增加sort_buffer_size

    To monitor the number of merge passes (to merge temporary files), check the Sort_merge_passes status variable.要监视合并过程的数量(要合并临时文件),请选中Sort_merge_passes状态变量。

  • Increase the read_rnd_buffer_size variable value so that more rows are read at a time.增加read_rnd_buffer_size变量值,以便一次读取更多行。

  • Change the tmpdir system variable to point to a dedicated file system with large amounts of free space. tmpdir系统变量更改为指向具有大量可用空间的专用文件系统。The variable value can list several paths that are used in round-robin fashion; you can use this feature to spread the load across several directories. 变量值可以列出以循环方式使用的多个路径;您可以使用此功能将负载分散到多个目录。Separate the paths by colon characters (:) on Unix and semicolon characters (;) on Windows. 在Unix上用冒号字符(:)分隔路径,在Windows上用分号字符(;)分隔路径。The paths should name directories in file systems located on different physical disks, not different partitions on the same disk.路径应该命名位于不同物理磁盘上的文件系统中的目录,而不是同一磁盘上的不同分区。

ORDER BY Execution Plan Information AvailableORDER BY执行计划信息可用

With EXPLAIN (see Section 8.8.1, “Optimizing Queries with EXPLAIN”), you can check whether MySQL can use indexes to resolve an ORDER BY clause:使用EXPLAIN(请参阅第8.8.1节,“使用EXPLAIN优化查询”),您可以检查MySQL是否可以使用索引来解析ORDER BY子句:

  • If the Extra column of EXPLAIN output does not contain Using filesort, the index is used and a filesort is not performed.如果EXPLAIN输出的Extra列不包含Using filesort,则使用索引,而不执行filesort

  • If the Extra column of EXPLAIN output contains Using filesort, the index is not used and a filesort is performed.如果EXPLAIN输出的Extra列包含Using filesort,则不使用索引,而执行filesort

In addition, if a filesort is performed, optimizer trace output includes a filesort_summary block. 此外,如果执行filesort,优化器跟踪输出将包括一个filesort_summary块。For example:例如:

"filesort_summary": {
  "rows": 100,
  "examined_rows": 100,
  "number_of_tmp_files": 0,
  "peak_memory_used": 25192,
  "sort_mode": "<sort_key, packed_additional_fields>"
}

peak_memory_used indicates the maximum memory used at any one time during the sort. peak_memory_used表示排序过程中任何时间使用的最大内存。This is a value up to but not necessarily as large as the value of the sort_buffer_size system variable. 这是一个最大值,但不一定与sort_buffer_size系统变量的值一样大。Prior to MySQL 8.0.12, the output shows sort_buffer_size instead, indicating the value of sort_buffer_size. 在MySQL 8.0.12之前,输出显示sort_buffer_size,表示sort_buffer_size的值。(Prior to MySQL 8.0.12, the optimizer always allocates sort_buffer_size bytes for the sort buffer. (在MySQL 8.0.12之前,优化器总是为排序缓冲区分配sort_buffer_size字节。As of 8.0.12, the optimizer allocates sort-buffer memory incrementally, beginning with a small amount and adding more as necessary, up to sort_buffer_size bytes.)从8.0.12开始,优化器以增量方式分配排序缓冲区内存,从少量开始,根据需要添加更多,直到sort_buffer_size字节。)

The sort_mode value provides information about the contents of tuples in the sort buffer:sort_mode值提供有关排序缓冲区中元组内容的信息:

  • <sort_key, rowid>: This indicates that sort buffer tuples are pairs that contain the sort key value and row ID of the original table row. 这表示排序缓冲区元组是包含原始表行的排序键值和行ID的对。Tuples are sorted by sort key value and the row ID is used to read the row from the table.元组按排序键值排序,行ID用于从表中读取行。

  • <sort_key, additional_fields>: This indicates that sort buffer tuples contain the sort key value and columns referenced by the query. 这表示排序缓冲区元组包含排序键值和查询引用的列。Tuples are sorted by sort key value and column values are read directly from the tuple.元组按排序键值排序,列值直接从元组读取。

  • <sort_key, packed_additional_fields>: Like the previous variant, but the additional columns are packed tightly together instead of using a fixed-length encoding.与前一个变体类似,但是附加列紧密地打包在一起,而不是使用固定长度的编码。

EXPLAIN does not distinguish whether the optimizer does or does not perform a filesort in memory. EXPLAIN不区分优化器是否在内存中执行filesortUse of an in-memory filesort can be seen in optimizer trace output. 在优化器跟踪输出中可以看到内存中filesort的使用。Look for filesort_priority_queue_optimization. 查找filesort_priority_queue_optimizationFor information about the optimizer trace, see MySQL Internals: Tracing the Optimizer.有关优化器跟踪的信息,请参阅MySQL内部:跟踪优化器