mmapMemory-mapped file support内存映射文件支持


Memory-mapped file objects behave like both bytearray and like file objects. 内存映射文件对象的行为既像bytearray,又像文件对象You can use mmap objects in most places where bytearray are expected; for example, you can use the re module to search through a memory-mapped file. 您可以在需要bytearray的大多数地方使用mmap对象;例如,可以使用re模块搜索内存映射文件。You can also change a single byte by doing obj[index] = 97, or change a subsequence by assigning to a slice: obj[i1:i2] = b'...'. 您还可以通过执行obj[index] = 97来更改单个字节,或通过分配给切片来更改子序列:obj[i1:i2] = b'...'You can also read and write data starting at the current file position, and seek() through the file to different positions.您还可以从当前文件位置开始读写数据,并在文件中seek()到不同的位置。

A memory-mapped file is created by the mmap constructor, which is different on Unix and on Windows. 内存映射文件由mmap构造函数创建,这在Unix和Windows上有所不同。In either case you must provide a file descriptor for a file opened for update. 在这两种情况下,您都必须为打开进行更新的文件提供文件描述符。If you wish to map an existing Python file object, use its fileno() method to obtain the correct value for the fileno parameter. 如果希望映射现有的Python文件对象,请使用其fileno()方法获取fileno参数的正确值。Otherwise, you can open the file using the os.open() function, which returns a file descriptor directly (the file still needs to be closed when done).否则,您可以使用os.open()函数打开文件,该函数直接返回文件描述符(完成后仍需要关闭文件)。

Note

If you want to create a memory-mapping for a writable, buffered file, you should flush() the file first. 如果要为可写缓冲文件创建内存映射,应首先flush()该文件。This is necessary to ensure that local modifications to the buffers are actually available to the mapping.这对于确保对缓冲区的本地修改实际上可用于映射是必要的。

For both the Unix and Windows versions of the constructor, access may be specified as an optional keyword parameter. 对于Unix和Windows版本的构造函数,access可以指定为可选的关键字参数。access accepts one of four values: ACCESS_READ, ACCESS_WRITE, or ACCESS_COPY to specify read-only, write-through or copy-on-write memory respectively, or ACCESS_DEFAULT to defer to prot. access接受四个值中的一个:ACCESS_READACCESS_WRITEACCESS_COPY分别指定只读、直写或写内存复制,或ACCESS_DEFAULT延迟到protaccess can be used on both Unix and Windows. access可以在Unix和Windows上使用。If access is not specified, Windows mmap returns a write-through mapping. 如果未指定access权限,Windows mmap将返回一个写入映射。The initial memory values for all three access types are taken from the specified file. 所有三种访问类型的初始内存值都取自指定的文件。Assignment to an ACCESS_READ memory map raises a TypeError exception. 分配给ACCESS_READ内存映射会引发TypeError异常。Assignment to an ACCESS_WRITE memory map affects both memory and the underlying file. 分配给ACCESS_WRITE内存映射会影响内存和基础文件。Assignment to an ACCESS_COPY memory map affects memory but does not update the underlying file.分配给ACCESS_COPY内存映射会影响内存,但不会更新底层文件。

Changed in version 3.7:版本3.7中更改: Added ACCESS_DEFAULT constant.添加了ACCESS_DEFAULT常量。

To map anonymous memory, -1 should be passed as the fileno along with the length.要映射匿名内存,应将-1与长度一起作为fileno传递。

classmmap.mmap(fileno, length, tagname=None, access=ACCESS_DEFAULT[, offset])

(Windows version) Maps length bytes from the file specified by the file handle fileno, and creates a mmap object. 从文件句柄fileno指定的文件映射length字节,并创建一个mmap对象。If length is larger than the current size of the file, the file is extended to contain length bytes. 如果length大于文件的当前大小,则文件将扩展为包含length字节。If length is 0, the maximum length of the map is the current size of the file, except that if the file is empty Windows raises an exception (you cannot create an empty mapping on Windows).如果length为0,则映射的最大长度为文件的当前大小,除非文件为空,否则Windows会引发异常(不能在Windows上创建空映射)。

tagname, if specified and not None, is a string giving a tag name for the mapping. tagname(如果指定而不是None)是一个字符串,为映射提供标记名。Windows allows you to have many different mappings against the same file. Windows允许您对同一文件进行许多不同的映射。If you specify the name of an existing tag, that tag is opened, otherwise a new tag of this name is created. 如果指定现有标记的名称,则将打开该标记,否则将创建具有该名称的新标记。If this parameter is omitted or None, the mapping is created without a name. 如果省略此参数或None,则将创建没有名称的映射。Avoiding the use of the tag parameter will assist in keeping your code portable between Unix and Windows.避免使用标记参数将有助于保持代码在Unix和Windows之间的可移植性。

offset may be specified as a non-negative integer offset. 可以指定为非负整数偏移。mmap references will be relative to the offset from the beginning of the file. offset defaults to 0. mmap引用将相对于文件开头的偏移量。offset默认为0。offset默认为0。offset must be a multiple of the ALLOCATIONGRANULARITY.offset必须是ALLOCATIONGRANULARITY的倍数。

Raises an auditing event mmap.__new__ with arguments fileno, length, access, offset.引发带参数filenolengthaccessoffset审核事件mmap.__new__

classmmap.mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, access=ACCESS_DEFAULT[, offset])

(Unix version) Maps length bytes from the file specified by the file descriptor fileno, and returns a mmap object. 从文件描述符fileno指定的文件映射长度字节,并返回mmap对象。If length is 0, the maximum length of the map will be the current size of the file when mmap is called.如果length0,则当调用mmap时,映射的最大长度将是文件的当前大小。

flags specifies the nature of the mapping. 指定映射的性质。MAP_PRIVATE creates a private copy-on-write mapping, so changes to the contents of the mmap object will be private to this process, and MAP_SHARED creates a mapping that’s shared with all other processes mapping the same areas of the file. MAP_PRIVATE在写入映射时创建一个私有副本,因此对mmap对象内容的更改将对此进程是私有的,MAP_SHARED创建一个映射,该映射与映射文件相同区域的所有其他进程共享。The default value is MAP_SHARED. 默认值为MAP_SHAREDSome systems have additional possible flags with the full list specified in MAP_* constants.某些系统具有MAP_*常量中指定的完整列表的其他可能标志。

prot, if specified, gives the desired memory protection; the two most useful values are PROT_READ and PROT_WRITE, to specify that the pages may be read or written. prot(如果指定)提供所需的内存保护;两个最有用的值是PROT_READPROT_WRITE,以指定可以读取或写入页面。prot defaults to PROT_READ | PROT_WRITE.prot默认为PROT_READ | PROT_WRITE

access may be specified in lieu of flags and prot as an optional keyword parameter. 可以指定access来代替flagsprot作为可选的关键字参数。It is an error to specify both flags, prot and access. 同时指定flagsprotaccess是错误的。See the description of access above for information on how to use this parameter.有关如何使用此参数的信息,请参阅上面的access说明。

offset may be specified as a non-negative integer offset. offset可以被指定为非负整数偏移。mmap references will be relative to the offset from the beginning of the file. mmap引用将相对于文件开头的偏移量。offset defaults to 0. offset默认为0。offset must be a multiple of ALLOCATIONGRANULARITY which is equal to PAGESIZE on Unix systems.offset必须是ALLOCATIONGGRAULARITY的倍数,在Unix系统上等于PAGESIZE

To ensure validity of the created memory mapping the file specified by the descriptor fileno is internally automatically synchronized with physical backing store on macOS and OpenVMS.为了确保创建的内存映射的有效性,描述符fileno指定的文件在内部自动与macOS和OpenVMS上的物理备份存储同步。

This example shows a simple way of using mmap:此示例显示了使用mmap的简单方法:

import mmap
# write a simple example file
with open("hello.txt", "wb") as f:
f.write(b"Hello Python!\n")

with open("hello.txt", "r+b") as f:
# memory-map the file, size 0 means whole file
mm = mmap.mmap(f.fileno(), 0)
# read content via standard file methods
print(mm.readline()) # prints b"Hello Python!\n"
# read content via slice notation
print(mm[:5]) # prints b"Hello"
# update content using slice notation;
# note that new content must have same size
mm[6:] = b" world!\n"
# ... and read again using standard file methods
mm.seek(0)
print(mm.readline()) # prints b"Hello world!\n"
# close the map
mm.close()

mmap can also be used as a context manager in a with statement:mmap还可以用作with语句中的上下文管理器:

import mmap
with mmap.mmap(-1, 13) as mm:
mm.write(b"Hello world!")

New in version 3.2.版本3.2中新增。Context manager support.上下文管理器支持。

The next example demonstrates how to create an anonymous map and exchange data between the parent and child processes:下一个示例演示如何创建匿名映射并在父进程和子进程之间交换数据:

import mmap
import os
mm = mmap.mmap(-1, 13)
mm.write(b"Hello world!")

pid = os.fork()

if pid == 0: # In a child process
mm.seek(0)
print(mm.readline())

mm.close()

Raises an auditing event mmap.__new__ with arguments fileno, length, access, offset.引发带参数filenolengthaccessoffset审核事件mmap.__new__

Memory-mapped file objects support the following methods:内存映射文件对象支持以下方法:

close()

Closes the mmap. 关闭mmap。Subsequent calls to other methods of the object will result in a ValueError exception being raised. 随后调用对象的其他方法将导致引发ValueError异常。This will not close the open file.这不会关闭打开的文件。

closed

True if the file is closed.如果文件已关闭,则为True

New in version 3.2.版本3.2中新增。

find(sub[, start[, end]])

Returns the lowest index in the object where the subsequence sub is found, such that sub is contained in the range [start, end]. 返回找到子序列sub的对象中的最低索引,以便子序列包含在[start, end]范围内。Optional arguments start and end are interpreted as in slice notation. 可选参数startend被解释为切片表示法。Returns -1 on failure.失败时返回-1

Changed in version 3.5:版本3.5中更改: Writable bytes-like object is now accepted.可写的类字节对象现在被接受。

flush([offset[, size]])

Flushes changes made to the in-memory copy of a file back to disk. 将对文件的内存副本所做的更改刷新回磁盘。Without use of this call there is no guarantee that changes are written back before the object is destroyed. 如果不使用此调用,就不能保证在销毁对象之前将更改写回。If offset and size are specified, only changes to the given range of bytes will be flushed to disk; otherwise, the whole extent of the mapping is flushed. 如果指定了offsetsize,则只会将对给定字节范围的更改刷新到磁盘;否则,将刷新映射的整个范围。offset must be a multiple of the PAGESIZE or ALLOCATIONGRANULARITY.offset必须是PAGESIZEALLOCATIONGRANULARITY的倍数。

None is returned to indicate success. 返回以指示成功。An exception is raised when the call failed.调用失败时引发异常。

Changed in version 3.8:版本3.8中更改: Previously, a nonzero value was returned on success; zero was returned on error under Windows. 以前,成功时返回非零值;在Windows下出错时返回零。A zero value was returned on success; an exception was raised on error under Unix.成功时返回零值;在Unix下发生错误时引发异常。

madvise(option[, start[, length]])

Send advice option to the kernel about the memory region beginning at start and extending length bytes. 向内核发送关于start时开始并扩展length字节的内存区域的建optionoption must be one of the MADV_* constants available on the system. option必须是系统上可用的MADV_* constants之一。If start and length are omitted, the entire mapping is spanned. 如果省略了startlength,则整个映射将被跨越。On some systems (including Linux), start must be a multiple of the PAGESIZE.在某些系统(包括Linux)上,start必须是PAGESIZE的倍数。

Availability: Systems with the madvise() system call.可用性:具有madvise()系统调用的系统。

New in version 3.8.版本3.8中新增。

move(dest, src, count)

Copy the count bytes starting at offset src to the destination index dest. 将从偏移量src开始的count字节复制到目标索引dest。If the mmap was created with ACCESS_READ, then calls to move will raise a TypeError exception.如果mmap是用ACCESS_READ创建的,那么调用move将引发TypeError异常。

read([n])

Return a bytes containing up to n bytes starting from the current file position. 返回从当前文件位置开始最多包含n个字节的bytesIf the argument is omitted, None or negative, return all bytes from the current file position to the end of the mapping. 如果省略了参数None或负数,则返回从当前文件位置到映射末尾的所有字节。The file position is updated to point after the bytes that were returned.文件位置将更新到返回的字节之后。

Changed in version 3.3:版本3.3中更改: Argument can be omitted or None.参数可以省略或None

read_byte()

Returns a byte at the current file position as an integer, and advances the file position by 1.以整数形式返回当前文件位置的字节,并将文件位置提前1。

readline()

Returns a single line, starting at the current file position and up to the next newline. 返回单行,从当前文件位置开始,直到下一个新行。The file position is updated to point after the bytes that were returned.文件位置将更新到返回的字节之后。

resize(newsize)

Resizes the map and the underlying file, if any. 调整贴图和基础文件的大小(如果有)。If the mmap was created with ACCESS_READ or ACCESS_COPY, resizing the map will raise a TypeError exception.如果mmap是用ACCESS_READACCESS_COPY创建的,则调整映射大小将引发TypeError异常。

rfind(sub[, start[, end]])

Returns the highest index in the object where the subsequence sub is found, such that sub is contained in the range [start, end]. 返回找到子序列sub的对象中的最高索引,以便子序列包含在[start, end]范围内。Optional arguments start and end are interpreted as in slice notation. 可选参数startend被解释为切片表示法。Returns -1 on failure.失败时返回-1

Changed in version 3.5:版本3.5中更改: Writable bytes-like object is now accepted.可写的类字节对象现在被接受。

seek(pos[, whence])

Set the file’s current position. 设置文件的当前位置。whence argument is optional and defaults to os.SEEK_SET or 0 (absolute file positioning); other values are os.SEEK_CUR or 1 (seek relative to the current position) and os.SEEK_END or 2 (seek relative to the file’s end).where参数是可选的,默认为os.SEEK_SET0(绝对文件定位);其他值为os.SEEK_CUR1(相对于当前位置的搜索)和os.SEEK_END2(相对于文件的结尾的搜索)。

size()

Return the length of the file, which can be larger than the size of the memory-mapped area.返回文件的长度,该长度可以大于内存映射区域的大小。

tell()

Returns the current position of the file pointer.返回文件游标的当前位置。

write(bytes)

Write the bytes in bytes into memory at the current position of the file pointer and return the number of bytes written (never less than len(bytes), since if the write fails, a ValueError will be raised). 将字节(以bytes为单位)写入文件游标当前位置的内存中,并返回写入的字节数(不得小于len(bytes),因为如果写入失败,将引发ValueError)。The file position is updated to point after the bytes that were written. 文件位置将更新到写入的字节之后。If the mmap was created with ACCESS_READ, then writing to it will raise a TypeError exception.如果mmap是用ACCESS_READ创建的,那么写入它将引发TypeError异常。

Changed in version 3.5:版本3.5中更改: Writable bytes-like object is now accepted.可写的类字节对象现在被接受。

Changed in version 3.6:版本3.6中更改: The number of bytes written is now returned.现在返回写入的字节数。

write_byte(byte)

Write the integer byte into memory at the current position of the file pointer; the file position is advanced by 1. 在文件游标的当前位置将整数byte写入内存;文件位置前进1If the mmap was created with ACCESS_READ, then writing to it will raise a TypeError exception.如果mmap是用ACCESS_READ创建的,那么写入它将引发TypeError异常。

MADV_* Constants

mmap.MADV_NORMAL
mmap.MADV_RANDOM
mmap.MADV_SEQUENTIAL
mmap.MADV_WILLNEED
mmap.MADV_DONTNEED
mmap.MADV_REMOVE
mmap.MADV_DONTFORK
mmap.MADV_DOFORK
mmap.MADV_HWPOISON
mmap.MADV_MERGEABLE
mmap.MADV_UNMERGEABLE
mmap.MADV_SOFT_OFFLINE
mmap.MADV_HUGEPAGE
mmap.MADV_NOHUGEPAGE
mmap.MADV_DONTDUMP
mmap.MADV_DODUMP
mmap.MADV_FREE
mmap.MADV_NOSYNC
mmap.MADV_AUTOSYNC
mmap.MADV_NOCORE
mmap.MADV_CORE
mmap.MADV_PROTECT
mmap.MADV_FREE_REUSABLE
mmap.MADV_FREE_REUSE

These options can be passed to mmap.madvise(). 这些选项可以传递给mmap.madvise()Not every option will be present on every system.并非每个系统都有每个选项。

Availability: Systems with the madvise() system call.可用性:具有madvise()系统调用的系统。

New in version 3.8.版本3.8中新增。

MAP_* Constants

mmap.MAP_SHARED
mmap.MAP_PRIVATE
mmap.MAP_DENYWRITE
mmap.MAP_EXECUTABLE
mmap.MAP_ANON
mmap.MAP_ANONYMOUS
mmap.MAP_POPULATE

These are the various flags that can be passed to mmap.mmap(). 这些是可以传递给mmap.mmap()的各种标志。Note that some options might not be present on some systems.请注意,某些系统上可能不存在某些选项。

Changed in version 3.10:版本3.10中更改: Added MAP_POPULATE constant.