Development guide开发指南
Introduction介绍
Code layout代码布局
auto
—Build scripts构建脚本src
core
—Basic types and functions — string, array, log, pool, etc.基本类型和函数-字符串、数组、日志、池等。event
—Event core事件核心modules
—Event notification modules:事件通知模块:epoll
,kqueue
,select
etc.
http
—Core HTTP module and common code核心HTTP模块和通用代码modules
—Other HTTP modules其他HTTP模块v2
— HTTP/2
mail
—Mail modules邮件模块os
—Platform-specific code平台特定代码unix
win32
stream
—Stream modules流模块
Include files包括文件
The following two 以下两个#include
statements must appear at the beginning of every nginx file:#include
语句必须出现在每个nginx文件的开头:
#include <ngx_config.h>
#include <ngx_core.h>
In addition to that, HTTP code should include除此之外,HTTP代码还应该包括
#include <ngx_http.h>
Mail code should include邮件代码应包括
#include <ngx_mail.h>
Stream code should include流代码应该包括
#include <ngx_stream.h>
Integers整数
For general purposes, nginx code uses two integer types, 出于一般目的,nginx代码使用两种整数类型ngx_int_t
and ngx_uint_t
, which are typedefs for intptr_t
and uintptr_t
respectively.ngx_int_t
和ngx_uint_t
,它们分别是intptr_t
和uintptr_t
的typedef。
Common return codes通用返回码
Most functions in nginx return the following codes:nginx中的大多数函数返回以下代码:
NGX_OK
—Operation succeeded.操作成功。NGX_ERROR
—Operation failed.操作失败。NGX_AGAIN
—Operation incomplete; call the function again.操作不全;再次调用该函数。NGX_DECLINED
—Operation rejected, for example, because it is disabled in the configuration.例如,操作被拒绝,因为它在配置中被禁用。This is never an error.这绝不是一个错误。NGX_BUSY
—Resource is not available.资源不可用。NGX_DONE
—Operation complete or continued elsewhere.操作在别处完成或继续。Also used as an alternative success code.也用作替代成功代码。NGX_ABORT
—Function was aborted.函数被中止。Also used as an alternative error code.也用作替代错误代码。
Error handling错误处理
The ngx_errno
macro returns the last system error code. ngx_errno
宏返回最后一个系统错误代码。It's mapped to 它在POSIX平台上映射到errno
on POSIX platforms and to GetLastError()
call in Windows. errno
,在Windows中映射到GetLastError()
调用。The ngx_socket_errno
macro returns the last socket error number. ngx_socket_errno
宏返回最后一个套接字错误号。Like the 与ngx_errno
macro, it's mapped to errno
on POSIX platforms. ngx_errno
宏一样,它被映射到POSIX平台上的errno
。It's mapped to the 在Windows上它被映射到WSAGetLastError()
call on Windows. WSAGetLastError()
调用。Accessing the values of 连续多次访问ngx_errno
or ngx_socket_errno
more than once in a row can cause performance issues. ngx_errno
或ngx_socket_errno
的值可能会导致性能问题。If the error value might be used multiple times, store it in a local variable of type 如果错误值可能被多次使用,请将其存储在ngx_err_t
. ngx_err_t
类型的局部变量中。To set errors, use the 要设置错误,请使用ngx_set_errno(errno)
and ngx_set_socket_errno(errno)
macros.ngx_set_errno(errno)
和ngx_set_socket_errno(errno)
宏。
The values of ngx_errno
and ngx_socket_errno
can be passed to the logging functions ngx_log_error()
and ngx_log_debugX()
, in which case system error text is added to the log message.ngx_errno
和ngx_socket_errno
的值可以传递给日志函数ngx_log_error()
和ngx_log_debugX()
,在这种情况下,系统错误文本会添加到日志消息中。
Example using 使用ngx_errno
:ngx_errno
的示例:
ngx_int_t ngx_my_kill(ngx_pid_t pid, ngx_log_t *log, int signo) { ngx_err_t err; if (kill(pid, signo) == -1) { err = ngx_errno; ngx_log_error(NGX_LOG_ALERT, log, err, "kill(%P, %d) failed", pid, signo); if (err == NGX_ESRCH) { return 2; } return 1; } return 0; }
Strings字符串
Overview概述
For C strings, nginx uses the unsigned character type pointer 对于C字符串,nginx使用无符号字符类型指针u_char *
.u_char *
。
The nginx string type nginx字符串类型ngx_str_t
is defined as follows:ngx_stru_t
的定义如下:
typedef struct { size_t len; u_char *data; } ngx_str_t;
The len
field holds the string length and data
holds the string data. len
字段保存字符串长度,data
保存字符串数据。The string, held in 保存在ngx_str_t
, may or may not be null-terminated after the len
bytes. ngx_stru_t
中的字符串可以在len
字节后以null
结尾,也可以不以null
结尾。In most cases it’s not. 在大多数情况下并非如此。However, in certain parts of the code (for example, when parsing configuration), 但是,在代码的某些部分(例如,在解析配置时),ngx_str_t
objects are known to be null-terminated, which simplifies string comparison and makes it easier to pass the strings to syscalls.ngx_str__t
对象以null
结尾,这简化了字符串比较,并且更容易将字符串传递给系统调用。
The string operations in nginx are declared in nginx中的字符串操作在src/core/ngx_string.h
src/core/ngx_string.h
中声明。Some of them are wrappers around standard C functions:其中一些是围绕标准C函数的包装:
ngx_strcmp()
ngx_strncmp()
ngx_strstr()
ngx_strlen()
ngx_strchr()
ngx_memcmp()
ngx_memset()
ngx_memcpy()
ngx_memmove()
Other string functions are nginx-specific其他字符串函数是特定于nginx的
ngx_memzero()
—Fills memory with zeroes.用零填充内存。ngx_explicit_memzero()
—Does the same as执行与ngx_memzero()
, but this call is never removed by the compiler's dead store elimination optimization.ngx_memzero()
相同的操作,但编译器的死存储消除优化不会删除此调用。This function can be used to clear sensitive data such as passwords and keys.此功能可用于清除密码和密钥等敏感数据。ngx_cpymem()
—Does the same as与ngx_memcpy()
, but returns the final destination address This one is handy for appending multiple strings in a row.ngx_memcpy()
的操作相同,但返回最终的目标地址。该地址便于在一行中追加多个字符串。ngx_movemem()
—Does the same as与ngx_memmove()
, but returns the final destination address.ngx_memmove()
相同,但返回最终目标地址。ngx_strlchr()
—Searches for a character in a string, delimited by two pointers.在字符串中搜索由两个指针分隔的字符。
The following functions perform case conversion and comparison:以下功能执行案例转换和比较:
ngx_tolower()
ngx_toupper()
ngx_strlow()
ngx_strcasecmp()
ngx_strncasecmp()
The following macros simplify string initialization:以下宏简化了字符串初始化:
ngx_string(text)
—static initializer for theC字符串ngx_str_t
type from the C string literaltext
text
中ngx_stru_t
类型的静态初始值设定项ngx_null_string
—static empty string initializer for thengx_str_t
typengx_stru_t
类型的静态空字符串初始值设定项ngx_str_set(str, text)
—initializes string使用C字符串文本初始化str
ofngx_str_t *
type with the C string literaltext
ngx_str_t *
类型的字符串str
ngx_str_null(str)
—initializes string用空字符串初始化str
ofngx_str_t *
type with the empty stringngx_str_t *
类型的字符串str
Formatting格式化
The following formatting functions support nginx-specific types:以下格式化函数支持特定于nginx的类型:
ngx_sprintf(buf, fmt, ...)
ngx_snprintf(buf, max, fmt, ...)
ngx_slprintf(buf, last, fmt, ...)
ngx_vslprintf(buf, last, fmt, args)
ngx_vsnprintf(buf, max, fmt, args)
The full list of formatting options, supported by these functions is in 这些函数支持的格式选项的完整列表位于src/core/ngx_string.c
. src/core/ngx_string.c
中。Some of them are:其中包括:
%O
—off_t
%T
—time_t
%z
—ssize_t
%i
—ngx_int_t
%p
—void *
%V
—ngx_str_t *
%s
—u_char *
(null-terminated)%*s
—size_t + u_char *
You can prepend 您可以在大多数类型上预加u
on most types to make them unsigned. u
以使其不带签名。To convert output to hex, use 要将输出转换为十六进制,请使用X
or x
.X
或x
。
For example:例如:
u_char buf[NGX_INT_T_LEN]; size_t len; ngx_uint_t n; /* set n here */ len = ngx_sprintf(buf, "%ui", n) — buf;
Numeric conversion数字转换
Several functions for numeric conversion are implemented in nginx. nginx中实现了几个数字转换函数。The first four each convert a string of given length to a positive integer of the indicated type. 前四个分别将给定长度的字符串转换为指定类型的正整数。They return 它们在出错时返回NGX_ERROR
on error.NGX_ERROR
。
ngx_atoi(line, n)
—ngx_int_t
ngx_atosz(line, n)
—ssize_t
ngx_atoof(line, n)
—off_t
ngx_atotm(line, n)
—time_t
There are two additional numeric conversion functions. 还有两个附加的数值转换函数。Like the first four, they return 像前四个一样,它们一个接一个地返回NGX_ERROR
on error.NGX_ERROR
。
ngx_atofp(line, n, point)
—Converts a fixed point floating number of given length to a positive integer of type将给定长度的定点浮点数转换为ngx_int_t
.ngx_int_t
类型的正整数。The result is shifted left by结果左移小数点位置。point
decimal positions.The string representation of the number is expected to have no more than数字的字符串表示形式应不超过points
fractional digits.points
小数位数。For example,例如,ngx_atofp("10.5", 4, 2)
returns1050
.ngx_atofp("10.5",4,2)
返回1050
。ngx_hextoi(line, n)
—Converts a hexadecimal representation of a positive integer to将正整数的十六进制表示形式转换为ngx_int_t
.ngx_int_t
。
Regular expressions正则表达式
The regular expressions interface in nginx is a wrapper around the PCRE library. nginx中的正则表达式接口是PCRE库的包装器。The corresponding header file is 相应的头文件是src/core/ngx_regex.h
.src/core/ngx_regex.h
。
To use a regular expression for string matching, it first needs to be compiled, which is usually done at the configuration phase. 要使用正则表达式进行字符串匹配,首先需要对其进行编译,这通常在配置阶段完成。Note that since PCRE support is optional, all code using the interface must be protected by the surrounding 请注意,由于PCRE支持是可选的,因此使用接口的所有代码都必须受到周围NGX_PCRE
macro:NGX_PCRE
宏的保护:
#if (NGX_PCRE) ngx_regex_t *re; ngx_regex_compile_t rc; u_char errstr[NGX_MAX_CONF_ERRSTR]; ngx_str_t value = ngx_string("message (\\d\\d\\d).*Codeword is '(?<cw>\\w+)'"); ngx_memzero(&rc, sizeof(ngx_regex_compile_t)); rc.pattern = value; rc.pool = cf->pool; rc.err.len = NGX_MAX_CONF_ERRSTR; rc.err.data = errstr; /* rc.options are passed as is to pcre_compile() */ if (ngx_regex_compile(&rc) != NGX_OK) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "%V", &rc.err); return NGX_CONF_ERROR; } re = rc.regex; #endif
After successful compilation, the 成功编译后,captures
and named_captures
fields in the ngx_regex_compile_t
structure contain the count of all captures and named captures, respectively, found in the regular expression.ngx_regex_compile_t
结构中的captures
字段和named_captures
字段分别包含正则表达式中找到的所有捕获和命名捕获的计数。
The compiled regular expression can then be used for matching against strings:编译后的正则表达式可用于匹配字符串:
ngx_int_t n; int captures[(1 + rc.captures) * 3]; ngx_str_t input = ngx_string("This is message 123. Codeword is 'foobar'."); n = ngx_regex_exec(re, &input, captures, (1 + rc.captures) * 3); if (n >= 0) { /* string matches expression */ } else if (n == NGX_REGEX_NO_MATCHED) { /* no match was found */ } else { /* some error */ ngx_log_error(NGX_LOG_ALERT, log, 0, ngx_regex_exec_n " failed: %i", n); }
The arguments to ngx_regex_exec()
are the compiled regular expression re
, the string to match input
, an optional array of integers to hold any captures
that are found, and the array's size
. ngx_regex_exec()
的参数包括编译后的正则表达式re
、匹配input
的字符串、保存找到的任何captures
的可选整数数组以及数组大小。The size of the 根据PCRE API的要求,captures
array must be a multiple of three, as required by the PCRE API. captures
数组的大小必须是三的倍数。In the example, the size is calculated from the total number of captures plus one for the matched string itself.在本例中,大小是根据捕获的总数加上匹配字符串本身的一个来计算的。
If there are matches, captures can be accessed as follows:如果存在匹配项,则可以按如下方式访问捕获:
u_char *p; size_t size; ngx_str_t name, value; /* all captures */ for (i = 0; i < n * 2; i += 2) { value.data = input.data + captures[i]; value.len = captures[i + 1] — captures[i]; } /* accessing named captures */ size = rc.name_size; p = rc.names; for (i = 0; i < rc.named_captures; i++, p += size) { /* capture name */ name.data = &p[2]; name.len = ngx_strlen(name.data); n = 2 * ((p[0] << 8) + p[1]); /* captured value */ value.data = &input.data[captures[n]]; value.len = captures[n + 1] — captures[n]; }
The ngx_regex_exec_array()
function accepts the array of ngx_regex_elt_t
elements (which are just compiled regular expressions with associated names), a string to match, and a log. ngx_regex_exec_array()
函数接受ngx_regex_elt_t
元素的数组(这些元素只是带有相关名称的编译正则表达式)、要匹配的字符串和日志。The function applies expressions from the array to the string until either a match is found or no more expressions are left. 函数将数组中的表达式应用于字符串,直到找到匹配项或不再剩下表达式为止。The return value is 如果存在匹配,则返回值为NGX_OK
when there is a match and NGX_DECLINED
otherwise, or NGX_ERROR
in case of error.NGX_OK
,否则返回值为NGX_DECLINED
,如果出现错误,则返回值为NGX_ERROR
。
Time时间
The ngx_time_t
structure represents time with three separate types for seconds, milliseconds, and the GMT offset:ngx_time_t
结构表示时间,有三种不同类型,分别表示秒、毫秒和GMT偏移量:
typedef struct { time_t sec; ngx_uint_t msec; ngx_int_t gmtoff; } ngx_time_t;
The >ngx_tm_t结构是UNIX平台上ngx_tm_t
structure is an alias for struct tm
on UNIX platforms and SYSTEMTIME
on Windows.struct tm
和Windows上SYSTEMTIME
的别名。
To obtain the current time, it is usually sufficient to access one of the available global variables, representing the cached time value in the desired format.要获取当前时间,访问一个可用的全局变量(以所需格式表示缓存的时间值)通常就足够了。
The available string representations are:可用的字符串表示形式有:
ngx_cached_err_log_time
—Used in error log entries:在错误日志条目中使用:"1970/09/28 12:00:00"
ngx_cached_http_log_time
—Used in HTTP access log entries:在HTTP访问日志条目中使用:"28/Sep/1970:12:00:00 +0600"
ngx_cached_syslog_time
—Used in syslog entries:在系统日志条目中使用:"Sep 28 12:00:00"
ngx_cached_http_time
—Used in HTTP headers:在HTTP标头中使用:"Mon, 28 Sep 1970 06:00:00 GMT"
ngx_cached_http_log_iso8601
—The ISO 8601 standard format:ISO 8601标准格式:"1970-09-28T12:00:00+06:00"
The ngx_time()
and ngx_timeofday()
macros return the current time value in seconds and are the preferred way to access the cached time value.ngx_time()
和ngx_timeofday()
宏以秒为单位返回当前时间值,是访问缓存时间值的首选方法。
To obtain the time explicitly, use 要显式获取时间,请使用ngx_gettimeofday()
, which updates its argument (pointer to struct timeval
). ngx_gettimeofday()
,它将更新其参数(指向struct timeval
的指针)。The time is always updated when nginx returns to the event loop from system calls. 当nginx从系统调用返回事件循环时,时间总是更新的。To update the time immediately, call 要立即更新时间,请调用ngx_time_update()
, or ngx_time_sigsafe_update()
if updating the time in the signal handler context.ngx_time_update()
,如果在信号处理程序上下文中更新时间,则调用ngx_time_sigsafe_update()
。
The following functions convert 以下函数将time_t
into the indicated broken-down time representation. time_t
转换为指示的细分时间表示形式。The first function in each pair converts 每对函数中的第一个函数将time_t
to ngx_tm_t
and the second (with the _libc_
infix) to struct tm
:time_t
转换为ngx_tm_t
,第二个函数(使用_libc_
中缀)转换为struct tm
:
ngx_gmtime(), ngx_libc_gmtime()
—Time expressed as UTC以UTC表示的时间ngx_localtime(), ngx_libc_localtime()
—Time expressed relative to the local time zone相对于本地时区表示的时间
The ngx_http_time(buf, time)
function returns a string representation suitable for use in HTTP headers (for example, "Mon, 28 Sep 1970 06:00:00 GMT"
). ngx_http_time(buf, time)
函数返回适合在http头中使用的字符串表示形式(例如,"Mon, 28 Sep 1970 06:00:00 GMT"
)。The ngx_http_cookie_time(buf,time)返回字符串表示法函数返回适合HTTP cookies的字符串表示法(ngx_http_cookie_time(buf, time)
returns a string representation function returns a string representation suitable for HTTP cookies ("Thu, 31-Dec-37 23:55:55 GMT"
)."Thu, 31-Dec-37 23:55:55 GMT"
)。
Containers容器
Array数组
The nginx array type nginx数组类型ngx_array_t
is defined as followsngx_array_t
的定义如下
typedef struct { void *elts; ngx_uint_t nelts; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_array_t;
The elements of the array are available in the 数组的元素在elts
field. elts
字段中可用。The nelts
field holds the number of elements. nelts
字段保存元素的数量。The size
field holds the size of a single element and is set when the array is initialized.size
字段保存单个元素的大小,并在数组初始化时设置。
Use the 使用ngx_array_create(pool, n, size)
call to create an array in a pool, and the ngx_array_init(array, pool, n, size)
call to initialize an array object that has already been allocated.ngx_array_create(pool, n, size)
调用在池中创建数组,使用ngx_array_init(array, pool, n, size)
调用初始化已分配的数组对象。
ngx_array_t *a, b; /* create an array of strings with preallocated memory for 10 elements */ a = ngx_array_create(pool, 10, sizeof(ngx_str_t)); /* initialize string array for 10 elements */ ngx_array_init(&b, pool, 10, sizeof(ngx_str_t));
Use the following functions to add elements to an array:使用以下函数向数组中添加元素:
ngx_array_push(a)
adds one tail element and returns pointer to it添加一个尾部元素并返回指向它的指针ngx_array_push_n(a, n)
adds添加n
tail elements and returns pointer to the first onen
个尾部元素并返回指向第一个元素的指针
If the currently allocated amount of memory is not large enough to accommodate the new elements, a new block of memory is allocated and the existing elements are copied to it. 如果当前分配的内存量不足以容纳新元素,则会分配一个新的内存块,并将现有元素复制到其中。The new memory block is normally twice as large as the existing one.新内存块通常是现有内存块的两倍大。
s = ngx_array_push(a); ss = ngx_array_push_n(&b, 3);
List
In nginx a list is a sequence of arrays, optimized for inserting a potentially large number of items. 在nginx中,列表是一系列数组,针对插入大量潜在项目进行了优化。The ngx_list_t
list type is defined as follows:ngx_list_t
类型定义如下:
typedef struct { ngx_list_part_t *last; ngx_list_part_t part; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_list_t;
The actual items are stored in list parts, which are defined as follows:实际项目存储在列表零件中,其定义如下:
typedef struct ngx_list_part_s ngx_list_part_t; struct ngx_list_part_s { void *elts; ngx_uint_t nelts; ngx_list_part_t *next; };
Before use, a list must be initialized by calling 使用前,必须通过调用ngx_list_init(list, pool, n, size)
or created by calling ngx_list_create(pool, n, size)
. ngx_list_init(list, pool, n, size)
初始化列表,或通过调用ngx_list_create(pool, n, size)
创建列表。Both functions take as arguments the size of a single item and a number of items per list part. 这两个函数都将单个项目的大小和每个列表部分的项目数作为参数。To add an item to a list, use the 要向列表中添加项目,请使用ngx_list_push(list)
function. ngx_list_push(list)
函数。To iterate over the items, directly access the list fields as shown in the example:要迭代项目,请直接访问列表字段,如示例所示:
ngx_str_t *v; ngx_uint_t i; ngx_list_t *list; ngx_list_part_t *part; list = ngx_list_create(pool, 100, sizeof(ngx_str_t)); if (list == NULL) { /* error */ } /* add items to the list */ v = ngx_list_push(list); if (v == NULL) { /* error */ } ngx_str_set(v, "foo"); v = ngx_list_push(list); if (v == NULL) { /* error */ } ngx_str_set(v, "bar"); /* iterate over the list */ part = &list->part; v = part->elts; for (i = 0; /* void */; i++) { if (i >= part->nelts) { if (part->next == NULL) { break; } part = part->next; v = part->elts; i = 0; } ngx_do_smth(&v[i]); }
Lists are primarily used for HTTP input and output headers.列表主要用于HTTP输入和输出头。
Lists do not support item removal. 列表不支持删除项。However, when needed, items can internally be marked as missing without actually being removed from the list. 但是,在需要时,可以在内部将项目标记为缺少,而不从列表中实际删除。For example, to mark HTTP output headers (which are stored as 例如,要将HTTP输出头(存储为ngx_table_elt_t
objects) as missing, set the hash
field in ngx_table_elt_t
to zero. ngx_table_elt_t
对象)标记为缺失,请将ngx_table_elt_t
中的哈希字段设置为零。Items marked in this way are explicitly skipped when the headers are iterated over.当头被迭代时,以这种方式标记的项被显式跳过。
Queue队列
In nginx a queue is an intrusive doubly linked list, with each node defined as follows:在nginx中,队列是一个侵入式双链接列表,每个节点定义如下:
typedef struct ngx_queue_s ngx_queue_t; struct ngx_queue_s { ngx_queue_t *prev; ngx_queue_t *next; };
The head queue node is not linked with any data. 头队列节点未与任何数据链接。Use the 在使用前,使用ngx_queue_init(q)
call to initialize the list head before use. ngx_queue_init(q)
调用初始化列表头。Queues support the following operations:队列支持以下操作:
ngx_queue_insert_head(h, x)
,ngx_queue_insert_tail(h, x)
—Insert a new node插入新节点ngx_queue_remove(x)
—Remove a queue node删除队列节点ngx_queue_split(h, q, n)
—Split a queue at a node, returning the queue tail in a separate queue在节点上拆分队列,在单独的队列中返回队列尾部ngx_queue_add(h, n)
—Add a second queue to the first queue将第二个队列添加到第一个队列ngx_queue_head(h)
,ngx_queue_last(h)
—Get first or last queue node获取第一个或最后一个队列节点ngx_queue_sentinel(h)
-Get a queue sentinel object to end iteration at获取队列sentinel对象以结束迭代ngx_queue_data(q, type, link)
—Get a reference to the beginning of a queue node data structure, considering the queue field offset in it获取对队列节点数据结构开头的引用,并考虑其中的队列字段偏移量
An example:例如:
typedef struct { ngx_str_t value; ngx_queue_t queue; } ngx_foo_t; ngx_foo_t *f; ngx_queue_t values, *q; ngx_queue_init(&values); f = ngx_palloc(pool, sizeof(ngx_foo_t)); if (f == NULL) { /* error */ } ngx_str_set(&f->value, "foo"); ngx_queue_insert_tail(&values, &f->queue); /* insert more nodes here */ for (q = ngx_queue_head(&values); q != ngx_queue_sentinel(&values); q = ngx_queue_next(q)) { f = ngx_queue_data(q, ngx_foo_t, queue); ngx_do_smth(&f->value); }
Red-Black tree红黑树
The src/core/ngx_rbtree.h
header file provides access to the effective implementation of red-black trees.src/core/ngx_rbtree.h
头文件提供了对红黑树的有效实现的访问。
typedef struct { ngx_rbtree_t rbtree; ngx_rbtree_node_t sentinel; /* custom per-tree data here */ } my_tree_t; typedef struct { ngx_rbtree_node_t rbnode; /* custom per-node data */ foo_t val; } my_node_t;
To deal with a tree as a whole, you need two nodes: root and sentinel. 要将树作为一个整体处理,需要两个节点:根节点和哨兵节点。Typically, they are added to a custom structure, allowing you to organize your data into a tree in which the leaves contain a link to or embed your data.通常,它们被添加到自定义结构中,允许您将数据组织到树中,其中叶子包含指向或嵌入数据的链接。
To initialize a tree:要初始化树,请执行以下操作:
my_tree_t root; ngx_rbtree_init(&root.rbtree, &root.sentinel, insert_value_function);
To traverse a tree and insert new values, use the "要遍历树并插入新值,请使用“insert_value
" functions. insert_value
”函数。For example, the 例如,ngx_str_rbtree_insert_value
function deals with the ngx_str_t
type. ngx_str_rbtree_insert_value
函数处理ngx_str_t
类型。Its arguments are pointers to a root node of an insertion, the newly created node to be added, and a tree sentinel.它的参数是指向插入的根节点、要添加的新创建节点和树哨兵的指针。
void ngx_str_rbtree_insert_value(ngx_rbtree_node_t *temp, ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel)
The traversal is pretty straightforward and can be demonstrated with the following lookup function pattern:遍历非常简单,可以通过以下查找函数模式进行演示:
my_node_t * my_rbtree_lookup(ngx_rbtree_t *rbtree, foo_t *val, uint32_t hash) { ngx_int_t rc; my_node_t *n; ngx_rbtree_node_t *node, *sentinel; node = rbtree->root; sentinel = rbtree->sentinel; while (node != sentinel) { n = (my_node_t *) node; if (hash != node->key) { node = (hash < node->key) ? node->left : node->right; continue; } rc = compare(val, node->val); if (rc < 0) { node = node->left; continue; } if (rc > 0) { node = node->right; continue; } return n; } return NULL; }
The compare()
function is a classic comparator function that returns a value less than, equal to, or greater than zero. compare()
函数是一个经典的比较器函数,它返回小于、等于或大于零的值。To speed up lookups and avoid comparing user objects that can be big, an integer hash field is used.为了加快查找速度并避免比较可能很大的用户对象,使用了整数散列字段。
To add a node to a tree, allocate a new node, initialize it and call 要将节点添加到树中,请分配一个新节点,对其进行初始化并调用ngx_rbtree_insert()
:ngx_rbtree_insert()
:
my_node_t *my_node; ngx_rbtree_node_t *node; my_node = ngx_palloc(...); init_custom_data(&my_node->val); node = &my_node->rbnode; node->key = create_key(my_node->val); ngx_rbtree_insert(&root->rbtree, node);
To remove a node, call the 要删除节点,请调用ngx_rbtree_delete()
function:ngx_rbtree_delete()
函数:
ngx_rbtree_delete(&root->rbtree, node);
Hash
Hash table functions are declared in src/core/ngx_hash.h
. src/core/ngx_hash.h
中声明了哈希表函数。Both exact and wildcard matching are supported. 支持精确匹配和通配符匹配。The latter requires extra setup and is described in a separate section below.后者需要额外的设置,并在下面的单独章节中描述。
Before initializing a hash, you need to know the number of elements it will hold so that nginx can build it optimally. 在初始化散列之前,您需要知道它将包含的元素数,以便nginx能够以最佳方式构建它。Two parameters that need to be configured are 需要配置的两个参数是max_size
and bucket_size
, as detailed in a separate document. max_size
和bucket_size
,详见单独的文档。They are usually configurable by the user. 它们通常由用户配置。Hash initialization settings are stored with the 散列初始化设置与ngx_hash_init_t
type, and the hash itself is ngx_hash_t
:ngx_Hash_init_t
类型一起存储,散列本身是ngx_Hash_t
:
ngx_hash_t foo_hash; ngx_hash_init_t hash; hash.hash = &foo_hash; hash.key = ngx_hash_key; hash.max_size = 512; hash.bucket_size = ngx_align(64, ngx_cacheline_size); hash.name = "foo_hash"; hash.pool = cf->pool; hash.temp_pool = cf->temp_pool;
The key
is a pointer to a function that creates the hash integer key from a string. key
是指向从字符串创建哈希整数键的函数的指针。There are two generic key-creation functions:有两个通用密钥创建函数:
ngx_hash_key(data, len)
and ngx_hash_key_lc(data, len)
. The latter converts a string to all lowercase characters, so the passed string must be writable. 后者将字符串转换为所有小写字符,因此传递的字符串必须是可写的。If that is not true, pass the 如果不是这样,则将NGX_HASH_READONLY_KEY
flag to the function, initializing the key array (see below).NGX_HASH_READONLY_KEY
标志传递给函数,初始化密钥数组(见下文)。
The hash keys are stored in 散列键存储在ngx_hash_keys_arrays_t
and are initialized with ngx_hash_keys_array_init(arr, type)
:ngx_hash_keys_arrays_t
中,并用ngx_hash_keys_array_init(arr, type)
初始化:
The second parameter (第二个参数(type
) controls the amount of resources preallocated for the hash and can be either NGX_HASH_SMALL
or NGX_HASH_LARGE
. type
)控制为哈希预分配的资源量,可以是NGX_hash_SMALL
或NGX_HASH_LARGE
。The latter is appropriate if you expect the hash to contain thousands of elements.如果希望散列包含数千个元素,则后者是合适的。
ngx_hash_keys_arrays_t foo_keys; foo_keys.pool = cf->pool; foo_keys.temp_pool = cf->temp_pool; ngx_hash_keys_array_init(&foo_keys, NGX_HASH_SMALL);
To insert keys into a hash keys array, use the 要将键插入哈希键数组,请使用ngx_hash_add_key(keys_array, key, value, flags)
function:ngx_hash_add_key(keys_array, key, value, flags)
函数:
ngx_str_t k1 = ngx_string("key1"); ngx_str_t k2 = ngx_string("key2"); ngx_hash_add_key(&foo_keys, &k1, &my_data_ptr_1, NGX_HASH_READONLY_KEY); ngx_hash_add_key(&foo_keys, &k2, &my_data_ptr_2, NGX_HASH_READONLY_KEY);
To build the hash table, call the 要构建哈希表,请调用ngx_hash_init(hinit, key_names, nelts)
function:ngx_hash_init(hinit, key_names, nelts)
函数:
ngx_hash_init(&hash, foo_keys.keys.elts, foo_keys.keys.nelts);
The function fails if 如果max_size
or bucket_size
parameters are not big enough.max_size
或bucket_size
参数不够大,该函数将失败。
When the hash is built, use the 构建哈希时,使用ngx_hash_find(hash, key, name, len)
function to look up elements:ngx_hash_find(hash, key, name, len)
函数查找元素:
my_data_t *data; ngx_uint_t key; key = ngx_hash_key(k1.data, k1.len); data = ngx_hash_find(&foo_hash, key, k1.data, k1.len); if (data == NULL) { /* key not found */ }
Wildcard matching通配符匹配
To create a hash that works with wildcards, use the 要创建与通配符一起使用的哈希,请使用ngx_hash_combined_t
type. ngx_hash_combined_t
类型。It includes the hash type described above and has two additional keys arrays: 它包括上述散列类型,并具有两个额外的密钥数组:dns_wc_head
and dns_wc_tail
. dns_wc_head
和dns_wc_tail
。The initialization of basic properties is similar to a regular hash:基本属性的初始化类似于常规哈希:
ngx_hash_init_t hash ngx_hash_combined_t foo_hash; hash.hash = &foo_hash.hash; hash.key = ...;
It is possible to add wildcard keys using the 可以使用NGX_HASH_WILDCARD_KEY
flag:NGX_HASH_wildcard_KEY
标志添加通配符键:
/* k1 = ".example.org"; */ /* k2 = "foo.*"; */ ngx_hash_add_key(&foo_keys, &k1, &data1, NGX_HASH_WILDCARD_KEY); ngx_hash_add_key(&foo_keys, &k2, &data2, NGX_HASH_WILDCARD_KEY);
The function recognizes wildcards and adds keys into the corresponding arrays. 该函数识别通配符并将键添加到相应的数组中。Please refer to the map module documentation for the description of the wildcard syntax and the matching algorithm.有关通配符语法和匹配算法的说明,请参阅map模块文档。
Depending on the contents of added keys, you may need to initialize up to three key arrays: one for exact matching (described above), and two more to enable matching starting from the head or tail of a string:根据添加的键的内容,您可能需要初始化最多三个键数组:一个用于精确匹配(如上所述),另外两个用于启用从字符串的开头或结尾开始的匹配:
if (foo_keys.dns_wc_head.nelts) { ngx_qsort(foo_keys.dns_wc_head.elts, (size_t) foo_keys.dns_wc_head.nelts, sizeof(ngx_hash_key_t), cmp_dns_wildcards); hash.hash = NULL; hash.temp_pool = pool; if (ngx_hash_wildcard_init(&hash, foo_keys.dns_wc_head.elts, foo_keys.dns_wc_head.nelts) != NGX_OK) { return NGX_ERROR; } foo_hash.wc_head = (ngx_hash_wildcard_t *) hash.hash; }
The keys array needs to be sorted, and initialization results must be added to the combined hash. 需要对keys数组进行排序,并且必须将初始化结果添加到组合哈希中。The initialization of dns_wc_tail
array is done similarly.dns_wc_tail
数组的初始化也是这样完成的。
The lookup in a combined hash is handled by the 组合哈希中的查找ngx_hash_find_combined(chash, key, name, len)
:ngx_hash_find_combined(chash, key, name, len)
处理:
/* key = "bar.example.org"; — will match ".example.org" */ /* key = "foo.example.com"; — will match "foo.*" */ hkey = ngx_hash_key(key.data, key.len); res = ngx_hash_find_combined(&foo_hash, hkey, key.data, key.len);
Memory management内存管理
Heap堆
To allocate memory from system heap, use the following functions:要从系统堆分配内存,请使用以下函数:
ngx_alloc(size, log)
—Allocate memory from system heap.从系统堆中分配内存。This is a wrapper around这是一个围绕malloc()
with logging support.malloc()
的包装器,支持日志记录。Allocation error and debugging information is logged to分配错误和调试信息记录到log
.log
中。ngx_calloc(size, log)
—Allocate memory from system heap like像ngx_alloc()
, but fill memory with zeros after allocation.ngx_alloc()
一样从系统堆分配内存,但分配后用零填充内存。ngx_memalign(alignment, size, log)
—Allocate aligned memory from system heap.从系统堆中分配对齐的内存。This is a wrapper around这是提供该功能的平台上posix_memalign()
on those platforms that provide that function.posix_memalign()
的包装器。Otherwise implementation falls back to否则,实现将退回到提供最大对齐的ngx_alloc()
which provides maximum alignment.ngx_alloc()
。ngx_free(p)
—Free allocated memory.释放分配的内存。This is a wrapper around这是一个围绕free()
free()
的包装器。
Pool池
Most nginx allocations are done in pools. 大多数nginx分配是在池中完成的。Memory allocated in an nginx pool is freed automatically when the pool is destroyed. nginx池中分配的内存在池被销毁时自动释放。This provides good allocation performance and makes memory control easy.这提供了良好的分配性能,并使内存控制变得容易。
A pool internally allocates objects in continuous blocks of memory. 池在内部分配连续内存块中的对象。Once a block is full, a new one is allocated and added to the pool memory block list. 块已满后,将分配一个新块并将其添加到池内存块列表中。When the requested allocation is too large to fit into a block, the request is forwarded to the system allocator and the returned pointer is stored in the pool for further deallocation.当请求的分配太大而无法装入块时,请求将转发给系统分配器,返回的指针将存储在池中以供进一步释放。
The type for nginx pools is nginx池的类型是ngx_pool_t
. ngx_pool_t
。The following operations are supported:支持以下操作:
ngx_create_pool(size, log)
—Create a pool with specified block size.创建具有指定块大小的池。The pool object returned is allocated in the pool as well.返回的池对象也在池中分配。Thesize
should be at leastNGX_MIN_POOL_SIZE
and a multiple ofNGX_POOL_ALIGNMENT
.size
应至少为NGX_MIN_POOL_SIZE
,并且为NGX_POOL_ALIGNMENT
的倍数。ngx_destroy_pool(pool)
—Free all pool memory, including the pool object itself.释放所有池内存,包括池对象本身。ngx_palloc(pool, size)
—Allocate aligned memory from the specified pool.从指定的池中分配对齐的内存。ngx_pcalloc(pool, size)
—Allocate aligned memory from the specified pool and fill it with zeroes.从指定的池中分配对齐的内存并用零填充。ngx_pnalloc(pool, size)
—Allocate unaligned memory from the specified pool.从指定的池中分配未对齐的内存。Mostly used for allocating strings.主要用于分配字符串。ngx_pfree(pool, p)
—Free memory that was previously allocated in the specified pool.以前在指定池中分配的可用内存。Only allocations that result from requests forwarded to the system allocator can be freed.只能释放转发到系统分配器的请求所产生的分配。
u_char *p; ngx_str_t *s; ngx_pool_t *pool; pool = ngx_create_pool(1024, log); if (pool == NULL) { /* error */ } s = ngx_palloc(pool, sizeof(ngx_str_t)); if (s == NULL) { /* error */ } ngx_str_set(s, "foo"); p = ngx_pnalloc(pool, 3); if (p == NULL) { /* error */ } ngx_memcpy(p, "foo", 3);
Chain links (连缀链接(ngx_chain_t
) are actively used in nginx, so the nginx pool implementation provides a way to reuse them. ngx_Chain_t
)在nginx中被积极使用,因此nginx池实现提供了一种重用它们的方法。The chain
field of ngx_pool_t
keeps a list of previously allocated links ready for reuse. ngx_pool_t
的chain
字段保存一个以前分配的链接列表,以备重用。For efficient allocation of a chain link in a pool, use the 要在池中高效分配链链接,请使用ngx_alloc_chain_link(pool)
function. ngx_alloc_chain_link(pool)
函数。This function looks up a free chain link in the pool list and allocates a new chain link if the pool list is empty. 此函数在池列表中查找空闲链链接,如果池列表为空,则分配新的链链接。To free a link, call the 要释放链接,请调用ngx_free_chain(pool, cl)
function.ngx_free_chain(pool, cl)
函数。
Cleanup handlers can be registered in a pool. 可以在池中注册清理处理程序。A cleanup handler is a callback with an argument which is called when pool is destroyed. 清理处理程序是带有参数的回调,在池被销毁时调用该参数。A pool is usually tied to a specific nginx object (like an HTTP request) and is destroyed when the object reaches the end of its lifetime. 池通常绑定到特定的nginx对象(如HTTP请求),并在对象到达其生命周期结束时销毁。Registering a pool cleanup is a convenient way to release resources, close file descriptors or make final adjustments to the shared data associated with the main object.注册池清理是释放资源、关闭文件描述符或对与主对象关联的共享数据进行最终调整的方便方法。
To register a pool cleanup, call 要注册池清理,请调用ngx_pool_cleanup_add(pool, size)
, which returns a ngx_pool_cleanup_t
pointer to be filled in by the caller. ngx_pool_cleanup_add(pool, size)
,它将返回调用方要填充的ngx_pool_cleanup_t
指针。Use the 使用size
argument to allocate context for the cleanup handler.size
参数为清理处理程序分配上下文。
ngx_pool_cleanup_t *cln; cln = ngx_pool_cleanup_add(pool, 0); if (cln == NULL) { /* error */ } cln->handler = ngx_my_cleanup; cln->data = "foo"; ... static void ngx_my_cleanup(void *data) { u_char *msg = data; ngx_do_smth(msg); }
Shared memory共享内存
Shared memory is used by nginx to share common data between processes. nginx使用共享内存在进程之间共享公共数据。The ngx_shared_memory_add(cf, name, size, tag)
function adds a new shared memory entry ngx_shm_zone_t
to a cycle. ngx_shared_memory_add(cf, name, size, tag)
函数将新的共享内存条目ngx_shm_zone_t
添加到循环中。The function receives the 该函数接收区域的name
and size
of the zone. name
和size
。Each shared zone must have a unique name. 每个共享区域必须具有唯一的名称。If a shared zone entry with the provided 如果已存在具有提供的name
and tag
already exists, the existing zone entry is reused. name
和tag
的共享区域条目,则重用现有区域条目。The function fails with an error if an existing entry with the same name has a different tag. 如果具有相同名称的现有条目具有不同的标记,则函数将失败并出错。Usually, the address of the module structure is passed as 通常,模块结构的地址作为tag
, making it possible to reuse shared zones by name within one nginx module.tag
传递,使得在一个nginx模块内按名称重用共享区域成为可能。
The shared memory entry structure 共享内存条目结构ngx_shm_zone_t
has the following fields:ngx_shm_zone_t
具有以下字段:
init
—Initialization callback, called after the shared zone is mapped to actual memory初始化回调,在共享区域映射到实际内存后调用data
—Data context, used to pass arbitrary data to the数据上下文,用于将任意数据传递给init
callbackinit
回调noreuse
—Flag that disables reuse of a shared zone from the old cycle禁止重用旧循环中的共享区域的标志tag
—Shared zone tag共享区域标签shm
—Platform-specific object of typengx_shm_t
, having at least the following fields:ngx_shm_t
类型的平台特定对象,至少具有以下字段:addr
—Mapped shared memory address, initially NULL映射的共享内存地址,最初为空size
—Shared memory size共享内存大小name
—Shared memory name共享内存名log
—Shared memory log共享内存日志exists
—Flag that indicates shared memory was inherited from the master process (Windows-specific)指示从主进程继承共享内存的标志(特定于Windows)
Shared zone entries are mapped to actual memory in 解析配置后,共享区域条目将映射到ngx_init_cycle()
after the configuration is parsed. ngx_init_cycle()
中的实际内存。On POSIX systems, the 在POSIX系统上,mmap()
syscall is used to create the shared anonymous mapping. mmap()
系统调用用于创建共享匿名映射。On Windows, the 在Windows上,使用CreateFileMapping()
/MapViewOfFileEx()
pair is used.CreateFileMapping()
/MapViewOffilex()
对。
For allocating in shared memory, nginx provides the slab pool 为了在共享内存中进行分配,nginx提供了板池ngx_slab_pool_t
type. ngx_slab_pool_t
类型。A slab pool for allocating memory is automatically created in each nginx shared zone. 在每个nginx共享区域中自动创建用于分配内存的板池。The pool is located in the beginning of the shared zone and can be accessed by the expression 池位于共享区域的开头,可以通过表达式(ngx_slab_pool_t *) shm_zone->shm.addr
. (ngx_slab_pool_t *) shm_zone->shm.addr
来访问。To allocate memory in a shared zone, call either 要在共享区域中分配内存,请调用ngx_slab_alloc(pool, size)
or ngx_slab_calloc(pool, size)
. ngx_slab_alloc(pool, size)
或ngx_slab_calloc(pool, size)
。To free memory, call 要释放内存,请调用ngx_slab_free(pool, p)
.ngx_slab_free(pool, p)
。
Slab pool divides all shared zone into pages. 板池将所有共享区域划分为页面。Each page is used for allocating objects of the same size. 每个页面用于分配相同大小的对象。The specified size must be a power of 2, and greater than the minimum size of 8 bytes. Nonconforming values are rounded up. 指定的大小必须是2的幂,并且大于8字节的最小大小。不符合要求的值被四舍五入。A bitmask for each page tracks which blocks are in use and which are free for allocation. 每个页面的位掩码跟踪哪些块正在使用,哪些块可以自由分配。For sizes greater than a half page (which is usually 2048 bytes), allocation is done an entire page at a time对于大于半页(通常为2048字节)的大小,一次分配一整页
To protect data in shared memory from concurrent access, use the mutex available in the 要保护共享内存中的数据不被并发访问,请使用mutex
field of ngx_slab_pool_t
. ngx_slab_pool_t
的mutex
字段中可用的互斥。A mutex is most commonly used by the slab pool while allocating and freeing memory, but it can be used to protect any other user data structures allocated in the shared zone. 在分配和释放内存时,互斥锁最常用于slab池,但它可用于保护在共享区域中分配的任何其他用户数据结构。To lock or unlock a mutex, call 要锁定或解锁互斥锁,请分别调用ngx_shmtx_lock(&shpool->mutex)
or ngx_shmtx_unlock(&shpool->mutex)
respectively.ngx_shmtx_lock(&shpool->mutex)
或ngx_shmtx_unlock(&shpool->mutex)
。
ngx_str_t name; ngx_foo_ctx_t *ctx; ngx_shm_zone_t *shm_zone; ngx_str_set(&name, "foo"); /* allocate shared zone context */ ctx = ngx_pcalloc(cf->pool, sizeof(ngx_foo_ctx_t)); if (ctx == NULL) { /* error */ } /* add an entry for 64k shared zone */ shm_zone = ngx_shared_memory_add(cf, &name, 65536, &ngx_foo_module); if (shm_zone == NULL) { /* error */ } /* register init callback and context */ shm_zone->init = ngx_foo_init_zone; shm_zone->data = ctx; ... static ngx_int_t ngx_foo_init_zone(ngx_shm_zone_t *shm_zone, void *data) { ngx_foo_ctx_t *octx = data; size_t len; ngx_foo_ctx_t *ctx; ngx_slab_pool_t *shpool; value = shm_zone->data; if (octx) { /* reusing a shared zone from old cycle */ ctx->value = octx->value; return NGX_OK; } shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; if (shm_zone->shm.exists) { /* initialize shared zone context in Windows nginx worker */ ctx->value = shpool->data; return NGX_OK; } /* initialize shared zone */ ctx->value = ngx_slab_alloc(shpool, sizeof(ngx_uint_t)); if (ctx->value == NULL) { return NGX_ERROR; } shpool->data = ctx->value; return NGX_OK; }
Logging登录
For logging nginx uses 对于日志记录,nginx使用ngx_log_t
objects. ngx_log_t
对象。The nginx logger supports several types of output:nginx记录器支持几种类型的输出:
- stderr —
Logging to standard error (stderr)记录到标准错误(stderr) - file —
Logging to a file记录到文件 - syslog —
Logging to syslog登录到系统日志 - memory —
Logging to internal memory storage for development purposes; the memory can be accessed later with a debugger为开发目的记录到内存存储器;以后可以使用调试器访问内存
A logger instance can be a chain of loggers, linked to each other with the 记录器实例可以是记录器链,通过next
field. next
字段相互链接。In this case, each message is written to all loggers in the chain.在这种情况下,每条消息都会写入链中的所有记录器。
For each logger, a severity level controls which messages are written to the log (only events assigned that level or higher are logged). 对于每个记录器,严重性级别控制哪些消息写入日志(仅记录指定该级别或更高级别的事件)。The following severity levels are supported:支持以下严重性级别:
NGX_LOG_EMERG
NGX_LOG_ALERT
NGX_LOG_CRIT
NGX_LOG_ERR
NGX_LOG_WARN
NGX_LOG_NOTICE
NGX_LOG_INFO
NGX_LOG_DEBUG
For debug logging, the debug mask is checked as well. 对于调试日志记录,还将检查调试掩码。The debug masks are:调试掩码包括:
NGX_LOG_DEBUG_CORE
NGX_LOG_DEBUG_ALLOC
NGX_LOG_DEBUG_MUTEX
NGX_LOG_DEBUG_EVENT
NGX_LOG_DEBUG_HTTP
NGX_LOG_DEBUG_MAIL
NGX_LOG_DEBUG_STREAM
Normally, loggers are created by existing nginx code from 通常,记录器是由现有的nginx代码根据error_log
directives and are available at nearly every stage of processing in cycle, configuration, client connection and other objects.error_log
指令创建的,在处理周期、配置、客户端连接和其他对象的几乎每个阶段都可用。
Nginx provides the following logging macros:Nginx提供以下日志宏:
ngx_log_error(level, log, err, fmt, ...)
—Error logging错误记录ngx_log_debug0(level, log, err, fmt)
,ngx_log_debug1(level, log, err, fmt, arg1)
etc —Debug logging with up to eight supported formatting arguments使用最多八个受支持的格式参数调试日志记录
A log message is formatted in a buffer of size 日志消息在堆栈上大小为NGX_MAX_ERROR_STR
(currently, 2048 bytes) on stack. NGX_MAX_ERROR_STR
(当前为2048字节)的缓冲区中格式化。The message is prepended with the severity level, process ID (PID), connection ID (stored in 消息前面有严重性级别、进程ID(PID)、连接ID(存储在log->connection
), and the system error text. log->connection
),以及系统错误文本。For non-debug messages 对于非调试消息还将调用log->handler
is called as well to prepend more specific information to the log message. log->handler
,以便在日志消息中预先添加更多特定信息。HTTP module sets HTTP模块将ngx_http_log_error()
function as log handler to log client and server addresses, current action (stored in log->action
), client request line, server name etc.ngx_HTTP_log_error()
函数设置为日志处理程序,以记录客户端和服务器地址、当前操作(存储在log->action
中)、客户端请求行、服务器名称等。
/* specify what is currently done */ log->action = "sending mp4 to client"; /* error and debug log */ ngx_log_error(NGX_LOG_INFO, c->log, 0, "client prematurely closed connection"); ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "mp4 start:%ui, length:%ui", mp4->start, mp4->length);
The example above results in log entries like these:上面的示例生成如下日志条目:
2016/09/16 22:08:52 [info] 17445#0: *1 client prematurely closed connection while sending mp4 to client, client: 127.0.0.1, server: , request: "GET /file.mp4 HTTP/1.1" 2016/09/16 23:28:33 [debug] 22140#0: *1 mp4 start:0, length:10000
Cycle循环
A cycle object stores the nginx runtime context created from a specific configuration. cycle对象存储从特定配置创建的nginx运行时上下文。Its type is 其类型为ngx_cycle_t
. ngx_cycle_t
。The current cycle is referenced by the 当前周期由ngx_cycle
global variable and inherited by nginx workers as they start. ngx_cycle
全局变量引用,并由nginx工作线程在开始时继承。Each time the nginx configuration is reloaded, a new cycle is created from the new nginx configuration; the old cycle is usually deleted after the new one is successfully created.每次重新加载nginx配置时,从新nginx配置创建一个新循环;旧周期通常在成功创建新周期后删除。
A cycle is created by the 循环由ngx_init_cycle()
function, which takes the previous cycle as its argument. ngx_init_cycle()
函数创建,该函数将上一个循环作为其参数。The function locates the previous cycle's configuration file and inherits as many resources as possible from the previous cycle. 该函数定位上一个周期的配置文件,并从上一个周期继承尽可能多的资源。A placeholder cycle called "init cycle" is created as nginx start, then is replaced by an actual cycle built from configuration.名为“init cycle”的占位符循环被创建为nginx start,然后被根据配置构建的实际循环所替代。
Members of the cycle include:该周期的成员包括:
pool
—Cycle pool.循环池。Created for each new cycle.为每个新周期创建。log
—Cycle log.循环日志。Initially inherited from the old cycle, it is set to point to最初从旧循环继承,在读取配置后将其设置为指向new_log
after the configuration is read.new_log
。new_log
—Cycle log, created by the configuration.循环日志,由配置创建。It's affected by the root-scope它受根作用域error_log
directive.error_log
指令的影响。connections
,connection_n
—Array of connections of typengx_connection_t
, created by the event module while initializing each nginx worker.ngx_connection_t
类型的连接数组,由事件模块在初始化每个nginx工作进程时创建。Thenginx配置中的worker_connections
directive in the nginx configuration sets the number of connectionsconnection_n
.worker_connections
指令设置连接数connection_n
。free_connections
,free_connection_n
—List and number of currently available connections.当前可用连接的列表和数量。If no connections are available, an nginx worker refuses to accept new clients or connect to upstream servers.如果没有可用的连接,nginx工作线程将拒绝接受新客户端或连接到上游服务器。files
,files_n
—Array for mapping file descriptors to nginx connections.用于将文件描述符映射到nginx连接的数组。This mapping is used by the event modules, having the此映射由事件模块使用,具有NGX_USE_FD_EVENT
flag (currently, it'spoll
anddevpoll
).NGX_USE_FD_EVENT
标志(当前为poll
和devpoll
)。conf_ctx
—Array of core module configurations.核心模块配置阵列。The configurations are created and filled during reading of nginx configuration files.在读取nginx配置文件期间创建并填充配置。modules
,modules_n
—Array of modules of type由当前配置加载的ngx_module_t
, both static and dynamic, loaded by the current configuration.ngx_module_t
类型的静态和动态模块阵列。listening
—Array of listening objects of typengx_listening_t
.ngx_listening_t
类型的侦听对象数组。Listening objects are normally added by the侦听对象通常由调用listen
directive of different modules which call thengx_create_listening()
function.ngx_create_Listening()
函数的不同模块的listen
指令添加。Listen sockets are created based on the listening objects.基于侦听对象创建侦听套接字。paths
—Array of paths of typengx_path_t
.ngx_path_t
类型的路径数组。Paths are added by calling the function通过调用函数ngx_add_path()
from modules which are going to operate on certain directories.ngx_add_path()
来添加路径,该函数来自将在某些目录上操作的模块。These directories are created by nginx after reading configuration, if missing.如果缺少,这些目录将由nginx在读取配置后创建。Moreover, two handlers can be added for each path:此外,可以为每个路径添加两个处理程序:path loader — Executes only once in 60 seconds after starting or reloading nginx.路径加载器 — 在启动或重新加载nginx后,仅在60秒内执行一次。Normally, the loader reads the directory and stores data in nginx shared memory.通常,加载程序读取目录并将数据存储在nginx共享内存中。The handler is called from the dedicated nginx process “nginx cache loader”.从专用nginx进程“nginx缓存加载程序”调用该处理程序。path manager — Executes periodically.路径管理器 — 定期执行。Normally, the manager removes old files from the directory and updates nginx memory to reflect the changes.通常,管理器会从目录中删除旧文件,并更新nginx内存以反映更改。The handler is called from the dedicated “nginx cache manager” process.从专用的“nginx缓存管理器”进程调用该处理程序。
open_files
—List of open file objects of type通过调用函数ngx_open_file_t
, which are created by calling the functionngx_conf_open_file()
.ngx_conf_open_file()
创建的类型为ngx_open_file_t
的打开文件对象列表。Currently, nginx uses this kind of open files for logging.目前,nginx使用这种打开的文件进行日志记录。After reading the configuration, nginx opens all files in the读取配置后,nginx打开open_files
list and stores each file descriptor in the object'sfd
field.open_files
列表中的所有文件,并将每个文件描述符存储在对象的fd
字段中。The files are opened in append mode and are created if missing.文件以附加模式打开,如果丢失,则创建。The files in the list are reopened by nginx workers upon receiving the reopen signal (most oftennginx工作线程在收到重新打开信号(通常为USR1
).USR1
)后,会重新打开列表中的文件。In this case the descriptor in the在这种情况下,fd
field is changed to a new value.fd
字段中的描述符将更改为新值。shared_memory
—List of shared memory zones, each added by calling the共享内存区域列表,每个区域通过调用ngx_shared_memory_add()
function.ngx_shared_memory_add()
函数添加。Shared zones are mapped to the same address range in all nginx processes and are used to share common data, for example the HTTP cache in-memory tree.共享区域在所有nginx进程中映射到相同的地址范围,并用于共享公共数据,例如内存树中的HTTP缓存。
Buffer缓冲器
For input/output operations, nginx provides the buffer type 对于输入/输出操作,nginx提供缓冲区类型ngx_buf_t
. ngx_buf_t
。Normally, it's used to hold data to be written to a destination or read from a source. 通常,它用于保存要写入目标或从源读取的数据。A buffer can reference data in memory or in a file and it's technically possible for a buffer to reference both at the same time. 缓冲区可以引用内存或文件中的数据,从技术上讲,缓冲区可以同时引用这两个数据。Memory for the buffer is allocated separately and is not related to the buffer structure 缓冲区的内存是单独分配的,与缓冲区结构ngx_buf_t
.ngx_buf_t
无关。
The ngx_buf_t
structure has the following fields:ngx_buf_t
结构具有以下字段:
start
,end
—The boundaries of the memory block allocated for the buffer.为缓冲区分配的内存块的边界。pos
,last
—The boundaries of the memory buffer; normally a subrange of内存缓冲区的边界;通常是start
..end
.start
..end
的子范围。file_pos
,file_last
—The boundaries of a file buffer, expressed as offsets from the beginning of the file.文件缓冲区的边界,表示为距文件开头的偏移量。tag
—Unique value used to distinguish buffers; created by different nginx modules, usually for the purpose of buffer reuse.用于区分缓冲区的唯一值;由不同的nginx模块创建,通常用于缓冲区重用。file
—File object.文件对象。temporary
—Flag indicating that the buffer references writable memory.指示缓冲区引用可写内存的标志。memory
—Flag indicating that the buffer references read-only memory.指示缓冲区引用只读内存的标志。in_file
—Flag indicating that the buffer references data in a file.指示缓冲区引用文件中数据的标志。flush
—Flag indicating that all data prior to the buffer need to be flushed.指示需要刷新缓冲区之前的所有数据的标志。recycled
—Flag indicating that the buffer can be reused and needs to be consumed as soon as possible.指示缓冲区可重用且需要尽快使用的标志。sync
—Flag indicating that the buffer carries no data or special signal like指示缓冲器不携带数据或特殊信号(如flush
orlast_buf
.flush
或last_buf
)的标志。By default nginx considers such buffers an error condition, but this flag tells nginx to skip the error check.默认情况下,nginx将此类缓冲区视为错误条件,但此标志告诉nginx跳过错误检查。last_buf
—Flag indicating that the buffer is the last in output.指示缓冲区是输出中最后一个的标志。last_in_chain
—Flag indicating that there are no more data buffers in a request or subrequest.指示请求或子请求中没有更多数据缓冲区的标志。shadow
—Reference to another ("shadow") buffer related to the current buffer, usually in the sense that the buffer uses data from the shadow.引用与当前缓冲区相关的另一个(“阴影”)缓冲区,通常是指缓冲区使用来自阴影的数据。When the buffer is consumed, the shadow buffer is normally also marked as consumed.当缓冲区被使用时,阴影缓冲区通常也被标记为已使用。last_shadow
—Flag indicating that the buffer is the last one that references a particular shadow buffer.指示缓冲区是引用特定阴影缓冲区的最后一个缓冲区的标志。temp_file
—Flag indicating that the buffer is in a temporary file.指示缓冲区位于临时文件中的标志。
For input and output operations buffers are linked in chains. 对于输入和输出操作,缓冲区以链的形式链接。A chain is a sequence of chain links of type 链是ngx_chain_t
, defined as follows:ngx_chain_t
类型的链节序列,定义如下:
typedef struct ngx_chain_s ngx_chain_t; struct ngx_chain_s { ngx_buf_t *buf; ngx_chain_t *next; };
Each chain link keeps a reference to its buffer and a reference to the next chain link.每个链链接保留对其缓冲区的引用和对下一个链链接的引用。
An example of using buffers and chains:使用缓冲区和链的示例:
ngx_chain_t * ngx_get_my_chain(ngx_pool_t *pool) { ngx_buf_t *b; ngx_chain_t *out, *cl, **ll; /* first buf */ cl = ngx_alloc_chain_link(pool); if (cl == NULL) { /* error */ } b = ngx_calloc_buf(pool); if (b == NULL) { /* error */ } b->start = (u_char *) "foo"; b->pos = b->start; b->end = b->start + 3; b->last = b->end; b->memory = 1; /* read-only memory */ cl->buf = b; out = cl; ll = &cl->next; /* second buf */ cl = ngx_alloc_chain_link(pool); if (cl == NULL) { /* error */ } b = ngx_create_temp_buf(pool, 3); if (b == NULL) { /* error */ } b->last = ngx_cpymem(b->last, "foo", 3); cl->buf = b; cl->next = NULL; *ll = cl; return out; }
Networking网络
Connection联系
The connection type 连接类型ngx_connection_t
is a wrapper around a socket descriptor. ngx_connection_t
是套接字描述符的包装器。It includes the following fields:它包括以下字段:
fd
—Socket descriptor套接字描述符data
—Arbitrary connection context.任意连接上下文。Normally, it is a pointer to a higher-level object built on top of the connection, such as an HTTP request or a Stream session.通常,它是指向建立在连接之上的更高级别对象的指针,例如HTTP请求或流会话。read
,write
—Read and write events for the connection.读取和写入连接的事件。recv
,send
,recv_chain
,send_chain
—I/O operations for the connection.连接的I/O操作。pool
—Connection pool.连接池。log
—Connection log.连接日志。sockaddr
,socklen
,addr_text
—Remote socket address in binary and text forms.二进制和文本形式的远程套接字地址。local_sockaddr
,local_socklen
—Local socket address in binary form. Initially, these fields are empty.二进制形式的本地套接字地址。最初,这些字段是空的。Use the使用ngx_connection_local_sockaddr()
function to get the local socket address.ngx_connection_local_sockaddr()
函数获取本地套接字地址。proxy_protocol_addr
,proxy_protocol_port
-PROXY protocol client address and port, if the PROXY protocol is enabled for the connection.代理协议客户端地址和端口(如果为连接启用了代理协议)。ssl
—SSL context for the connection.连接的SSL上下文。reusable
—Flag indicating the connection is in a state that makes it eligible for reuse.指示连接处于可重用状态的标志。close
—Flag indicating that the connection is being reused and needs to be closed.指示连接正在被重用且需要关闭的标志。
An nginx connection can transparently encapsulate the SSL layer. nginx连接可以透明地封装SSL层。In this case the connection's 在这种情况下,连接的ssl
field holds a pointer to an ngx_ssl_connection_t
structure, keeping all SSL-related data for the connection, including SSL_CTX
and SSL
. ssl
字段保存一个指向ngx_ssl_connection_t
结构的指针,保留连接的所有ssl相关数据,包括SSL_CTX
和SSL
。The 处理程序recv
, send
, recv_chain
, and send_chain
handlers are set to SSL-enabled functions as well.recv
、send
、recv_chain
和send_chain
也设置为启用SSL的函数。
The nginx配置中的worker_connections
directive in the nginx configuration limits the number of connections per nginx worker. worker_connections
指令限制每个nginx工作线程的连接数。All connection structures are precreated when a worker starts and stored in the 当工作进程启动时,所有连接结构都将预创建,并存储在cycle对象的connections
field of the cycle object. connections
字段中。To retrieve a connection structure, use the 要检索连接结构,请使用ngx_get_connection(s, log)
function. ngx_get_connection(s, log)
函数。It takes as its 它将套接字描述符作为其s
argument a socket descriptor, which needs to be wrapped in a connection structure.s
参数,该描述符需要包装在连接结构中。
Because the number of connections per worker is limited, nginx provides a way to grab connections that are currently in use. 由于每个工作线程的连接数量有限,nginx提供了一种获取当前正在使用的连接的方法。To enable or disable reuse of a connection, call the 要启用或禁用连接的重用,请调用ngx_reusable_connection(c, reusable)
function. ngx_reusable_connection(c, reusable)
函数。Calling 调用ngx_reusable_connection(c, 1)
sets the reuse
flag in the connection structure and inserts the connection into the reusable_connections_queue
of the cycle. ngx_reusable_connection(c, 1)
在连接结构中设置reuse
标志,并将连接插入循环的reusable_connections_queue
。Whenever 每当ngx_get_connection()
finds out there are no available connections in the cycle's free_connections
list, it calls ngx_drain_connections()
to release a specific number of reusable connections. ngx_get_connection()
发现循环的空闲连接列表中没有可用连接时,它就会调用ngx_drain_connections()
来释放特定数量的可重用连接。For each such connection, the 对于每个这样的连接,都会设置close
flag is set and its read handler is called which is supposed to free the connection by calling ngx_close_connection(c)
and make it available for reuse. close
标志并调用其读取处理程序,该处理程序通过调用ngx_close_connection(c)
释放连接,并使其可重用。To exit the state when a connection can be reused 要退出连接可重用时的状态,将调用ngx_reusable_connection(c, 0)
is called. ngx_reusable_connection(c, 0)
。HTTP client connections are an example of reusable connections in nginx; they are marked as reusable until the first request byte is received from the client.HTTP客户端连接是nginx中可重用连接的一个示例;它们被标记为可重用,直到从客户端接收到第一个请求字节。
Events事件
Event object nginx中的事件对象ngx_event_t
in nginx provides a mechanism for notification that a specific event has occurred.ngx_Event_t
提供了一种通知特定事件已发生的机制。
Fields in ngx_event_t
include:ngx_event_t
中的字段包括:
data
—Arbitrary event context used in event handlers, usually as pointer to a connection related to the event.事件处理程序中使用的任意事件上下文,通常作为指向与事件相关的连接的指针。handler
—Callback function to be invoked when the event happens.事件发生时要调用的回调函数。write
—Flag indicating a write event. Absence of the flag indicates a read event.指示写入事件的标志。缺少该标志表示读取事件。active
—Flag indicating that the event is registered for receiving I/O notifications, normally from notification mechanisms like指示已注册事件以接收I/O通知的标志,通常来自epoll
,kqueue
,poll
.epoll
、kqueue
、poll
等通知机制。ready
—Flag indicating that the event has received an I/O notification.指示事件已收到I/O通知的标志。delayed
—Flag indicating that I/O is delayed due to rate limiting.指示I/O由于速率限制而延迟的标志。timer
—Red-black tree node for inserting the event into the timer tree.用于将事件插入计时器树的红黑树节点。timer_set
—Flag indicating that the event timer is set and not yet expired.指示事件计时器已设置且尚未过期的标志。timedout
—Flag indicating that the event timer has expired.指示事件计时器已过期的标志。eof
—Flag indicating that EOF occurred while reading data.指示读取数据时发生EOF的标志。pending_eof
—Flag indicating that EOF is pending on the socket, even though there may be some data available before it.指示EOF在套接字上挂起的标志,即使在此之前可能有一些数据可用。The flag is delivered via the该标志通过EPOLLRDHUP
epoll
event orEV_EOF
kqueue
flag.EPOLLRDHUP
epoll
事件或EV_EOF
kqueue
标志传递。error
—Flag indicating that an error occurred during reading (for a read event) or writing (for a write event).指示在读取(对于读取事件)或写入(对于写入事件)期间发生错误的标志。cancelable
—Timer event flag indicating that the event should be ignored while shutting down the worker.计时器事件标志,指示关闭工作进程时应忽略该事件。Graceful worker shutdown is delayed until there are no non-cancelable timer events scheduled.正常工作进程关闭延迟,直到没有计划的不可取消计时器事件。posted
—Flag indicating that the event is posted to a queue.指示事件已发布到队列的标志。queue
—Queue node for posting the event to a queue.用于将事件发布到队列的队列节点。
I/O eventsI/O事件
Each connection obtained by calling the 通过调用ngx_get_connection()
function has two attached events, c->read
and c->write
, which are used for receiving notification that the socket is ready for reading or writing. ngx_get_connection()
函数获得的每个连接都有两个附加的事件,c->read
和c->write
,用于接收套接字已准备好读取或写入的通知。All such events operate in Edge-Triggered mode, meaning that they only trigger notifications when the state of the socket changes. 所有这些事件都在边缘触发模式下运行,这意味着它们仅在套接字状态更改时触发通知。For example, doing a partial read on a socket does not make nginx deliver a repeated read notification until more data arrives on the socket. 例如,在套接字上执行部分读取不会使nginx在更多数据到达套接字之前发送重复读取通知。Even when the underlying I/O notification mechanism is essentially Level-Triggered (即使底层I/O通知机制本质上是级别触发的(poll
, select
etc), nginx converts the notifications to Edge-Triggered. poll
、select
等),nginx也会将通知转换为边缘触发。To make nginx event notifications consistent across all notifications systems on different platforms, the functions 为了使nginx事件通知在不同平台上的所有通知系统中保持一致,必须在处理I/O套接字通知或调用该套接字上的任何I/O函数后调用函数ngx_handle_read_event(rev, flags)
and ngx_handle_write_event(wev, lowat)
must be called after handling an I/O socket notification or calling any I/O functions on that socket. ngx_handle_read_event(rev, flags)
和ngx_handle_write_event(wev, lowat)
。Normally, the functions are called once at the end of each read or write event handler.通常,在每个读或写事件处理程序的末尾调用函数一次。
Timer events计时器事件
An event can be set to send a notification when a timeout expires. 可以将事件设置为在超时过期时发送通知。The timer used by events counts milliseconds since some unspecified point in the past truncated to 自过去的某个未指定点被截断为ngx_msec_t
type. ngx_msec_t
类型以来,事件使用的计时器以毫秒计。Its current value can be obtained from the 其当前值可从ngx_current_msec
variable.ngx_current_msec
变量中获得。
The function 函数ngx_add_timer(ev, timer)
sets a timeout for an event, ngx_del_timer(ev)
deletes a previously set timeout. ngx_add_timer(ev, timer)
为事件设置超时,ngx_del_timer(ev)
删除先前设置的超时。The global timeout red-black tree 全局超时红黑树ngx_event_timer_rbtree
stores all timeouts currently set. ngx_event_timer_rbtree
存储当前设置的所有超时。The key in the tree is of type 树中的键为ngx_msec_t
and is the time when the event occurs. ngx_msec_t
类型,是事件发生的时间。The tree structure enables fast insertion and deletion operations, as well as access to the nearest timeouts, which nginx uses to find out how long to wait for I/O events and for expiring timeout events.树状结构支持快速插入和删除操作,以及访问最近的超时,nginx使用这些超时来确定等待I/O事件和过期超时事件的时间。
Posted events发布的事件
An event can be posted which means that its handler will be called at some point later within the current event loop iteration. 可以发布一个事件,这意味着它的处理程序将在当前事件循环迭代中稍后的某个时间被调用。Posting events is a good practice for simplifying code and escaping stack overflows. 发布事件对于简化代码和避免堆栈溢出是一个很好的实践。Posted events are held in a post queue. 发布的事件保存在发布队列中。The ngx_post_event(ev, q)
macro posts the event ev
to the post queue q
. ngx_delete_posted_event(ev)
宏将事件ev
发布到post队列q
。The ngx_delete_posted_event(ev)
macro deletes the event ev
from the queue it's currently posted in. ngx_delete_posted_event(ev)
宏从当前发布的队列中删除事件ev
。Normally, events are posted to the 通常情况下,事件被发布到ngx_posted_events
queue, which is processed late in the event loop — after all I/O and timer events are already handled. ngx_posted_events
队列,该队列在事件循环的后期处理-在所有I/O和计时器事件都已处理之后。The function 调用函数ngx_event_process_posted()
is called to process an event queue. ngx_event_process_posted()
来处理事件队列。It calls event handlers until the queue is not empty. 它调用事件处理程序,直到队列不为空。This means that a posted event handler can post more events to be processed within the current event loop iteration.这意味着已发布的事件处理程序可以发布更多要在当前事件循环迭代中处理的事件。
An example:例如:
void ngx_my_connection_read(ngx_connection_t *c) { ngx_event_t *rev; rev = c->read; ngx_add_timer(rev, 1000); rev->handler = ngx_my_read_handler; ngx_my_read(rev); } void ngx_my_read_handler(ngx_event_t *rev) { ssize_t n; ngx_connection_t *c; u_char buf[256]; if (rev->timedout) { /* timeout expired */ } c = rev->data; while (rev->ready) { n = c->recv(c, buf, sizeof(buf)); if (n == NGX_AGAIN) { break; } if (n == NGX_ERROR) { /* error */ } /* process buf */ } if (ngx_handle_read_event(rev, 0) != NGX_OK) { /* error */ } }
Event loop事件循环
Except for the nginx master process, all nginx processes do I/O and so have an event loop. 除了nginx主进程之外,所有nginx进程都进行I/O,因此都有一个事件循环。(The nginx master process instead spends most of its time in the (nginx主进程将其大部分时间花费在sigsuspend()
call waiting for signals to arrive.)sigssuspend()
调用中,等待信号到达。)
The nginx event loop is implemented in the nginx事件循环是在ngx_process_events_and_timers()
function, which is called repeatedly until the process exits.ngx_process_events_and_timers()
函数中实现的,该函数会重复调用,直到进程退出。
The event loop has the following stages:事件循环有以下几个阶段:
Find the timeout that is closest to expiring, by calling通过调用ngx_event_find_timer()
.ngx_event_Find_timer()
查找最接近过期的超时。This function finds the leftmost node in the timer tree and returns the number of milliseconds until the node expires.此函数用于查找计时器树中最左侧的节点,并返回节点过期前的毫秒数。Process I/O events by calling a handler, specific to the event notification mechanism, chosen by nginx configuration.通过调用nginx配置选择的特定于事件通知机制的处理程序来处理I/O事件。This handler waits for at least one I/O event to happen, but only until the next timeout expires.此处理程序至少等待一个I/O事件发生,但只等待下一个超时过期。When a read or write event occurs, the发生读或写事件时,设置ready
flag is set and the event's handler is called.ready
标志并调用事件的处理程序。For Linux, the对于Linux,通常使用ngx_epoll_process_events()
handler is normally used, which callsepoll_wait()
to wait for I/O events.ngx_epoll_process_events()
处理程序,它调用epoll_wait()
来等待I/O事件。Expire timers by calling通过调用ngx_event_expire_timers()
.ngx_event_Expire_timers()
使计时器过期。The timer tree is iterated from the leftmost element to the right until an unexpired timeout is found.计时器树从最左边的元素向右迭代,直到找到未过期的超时。For each expired node the对于每个过期节点,设置timedout
event flag is set, thetimer_set
flag is reset, and the event handler is calledtimedout
事件标志,重置timer_set
标志,并调用事件处理程序Process posted events by calling通过调用ngx_event_process_posted()
.ngx_event_Process_posted()
来处理已发布的事件。The function repeatedly removes the first element from the posted events queue and calls the element's handler, until the queue is empty.该函数反复从posted事件队列中删除第一个元素,并调用该元素的处理程序,直到队列为空。
All nginx processes handle signals as well. 所有nginx进程也处理信号。Signal handlers only set global variables which are checked after the 信号处理程序仅设置在ngx_process_events_and_timers()
call.ngx_process_events_and_timers()
调用后检查的全局变量。
Processes过程
There are several types of processes in nginx. nginx中有几种类型的进程。The type of a process is kept in the 流程类型保存在ngx_process
global variable, and is one of the following:ngx_process
全局变量中,是以下类型之一:
-
NGX_PROCESS_MASTER
—The master process, which reads the NGINX configuration, creates cycles, and starts and controls child processes.主进程读取NGINX配置,创建循环,启动和控制子进程。It does not perform any I/O and responds only to signals.它不执行任何I/O,只响应信号。Its cycle function is其循环功能是ngx_master_process_cycle()
.ngx_master_process_cycle()
。 -
NGX_PROCESS_WORKER
—The worker process, which handles client connections.工作进程,用于处理客户端连接。It is started by the master process and responds to its signals and channel commands as well.它由主进程启动,并响应其信号和通道命令。Its cycle function is其循环函数为ngx_worker_process_cycle()
.ngx_worker_process_cycle()
。There can be multiple worker processes, as configured by the根据worker_processes
directive.worker_processs
指令的配置,可以有多个工作进程。 -
NGX_PROCESS_SINGLE
—The single process, which exists only in单个进程,仅存在于master_process off
mode, and is the only process running in that mode.master_process off
(主进程关闭)模式下,并且是在该模式下运行的唯一进程。It creates cycles (like the master process does) and handles client connections (like the worker process does).它创建循环(像主进程那样)并处理客户端连接(像辅助进程那样)。Its cycle function is其循环函数为ngx_single_process_cycle()
.ngx_single_process_cycle()
。 -
NGX_PROCESS_HELPER
—The helper process, of which currently there are two types: cache manager and cache loader.助手进程,目前有两种类型:缓存管理器和缓存加载程序。The cycle function for both is两者的循环函数都是ngx_cache_manager_process_cycle()
.ngx_cache_manager_process_cycle()
。
The nginx processes handle the following signals:nginx进程处理以下信号:
-
NGX_SHUTDOWN_SIGNAL
((大多数系统上为SIGQUIT
on most systems)SIGQUIT
) —Gracefully shutdown. Upon receiving this signal, the master process sends a shutdown signal to all child processes. When no child processes are left, the master destroys the cycle pool and exits.优雅地关机。收到此信号后,主进程向所有子进程发送一个关闭信号。当没有剩下子进程时,主进程将销毁循环池并退出。When a worker process receives this signal, it closes all listening sockets and waits until there are no non-cancelable events scheduled, then destroys the cycle pool and exits.当工作进程收到此信号时,它将关闭所有侦听套接字并等待,直到没有计划不可取消的事件,然后销毁循环池并退出。When the cache manager or the cache loader process receives this signal, it exits immediately.当缓存管理器或缓存加载程序进程收到此信号时,它将立即退出。The当进程收到此信号时,ngx_quit
variable is set to1
when a process receives this signal, and is immediately reset after being processed.ngx_quit
变量设置为1
,并在处理后立即重置。The当工作进程处于关闭状态时,ngx_exiting
variable is set to1
while a worker process is in the shutdown state.ngx_exiting
变量设置为1
。 -
NGX_TERMINATE_SIGNAL
((大多数系统上为SIGTERM
on most systems)SIGTERM
) —Terminate.终止Upon receiving this signal, the master process sends a terminate signal to all child processes.在接收到该信号后,主进程向所有子进程发送终止信号。If a child process does not exit within 1 second, the master process sends the如果子进程在1秒内没有退出,主进程将发送SIGKILL
signal to kill it.SIGKILL
信号以终止它。When no child processes are left, the master process destroys the cycle pool and exits.当没有剩下子进程时,主进程将销毁循环池并退出。When a worker process, the cache manager process or the cache loader process receives this signal, it destroys the cycle pool and exits.当工作进程、缓存管理器进程或缓存加载程序进程收到此信号时,它将销毁循环池并退出。The variable收到此信号时,变量ngx_terminate
is set to1
when this signal is received.ngx_terminate
设置为1
。 -
NGX_NOACCEPT_SIGNAL
((大多数系统上为SIGWINCH
on most systems)SIGWINCH
) -Shut down all worker and helper processes.关闭所有辅助进程和辅助进程。Upon receiving this signal, the master process shuts down its child processes.接收到此信号后,主进程关闭其子进程。If a previously started new nginx binary exits, the child processes of the old master are started again.如果先前启动的新nginx二进制文件退出,则旧主机的子进程将再次启动。When a worker process receives this signal, it shuts down in debug mode set by the当工作进程收到此信号时,它将在debug_points
directive.debug_points
指令设置的调试模式下关闭。 -
NGX_RECONFIGURE_SIGNAL
((大多数系统上为SIGHUP
on most systems)SIGHUP
) -Reconfigure. Upon receiving this signal, the master process re-reads the configuration and creates a new cycle based on it.重新配置。在接收到该信号后,主进程重新读取配置并基于它创建一个新周期。If the new cycle is created successfully, the old cycle is deleted and new child processes are started.如果成功创建新周期,则删除旧周期并启动新的子进程。Meanwhile, the old child processes receive the同时,旧的子进程接收NGX_SHUTDOWN_SIGNAL
signal.NGX_SHUTDOWN_SIGNAL
信号。In single-process mode, nginx creates a new cycle, but keeps the old one until there are no longer clients with active connections tied to it.在单进程模式下,nginx会创建一个新的周期,但会保留旧的周期,直到不再有与之关联的活动连接的客户端。The worker and helper processes ignore this signal.辅助进程和辅助进程忽略此信号。 -
NGX_REOPEN_SIGNAL
((大多数系统上为SIGUSR1
on most systems)SIGUSR1
) —Reopen files.重新打开文件。The master process sends this signal to workers, which reopen all主进程将此信号发送给工作线程,工作线程重新打开与循环相关的所有open_files
related to the cycle.open_files
。 -
NGX_CHANGEBIN_SIGNAL
((大多数系统上为SIGUSR2
on most systems)SIGUSR2
) —Change the nginx binary. The master process starts a new nginx binary and passes in a list of all listen sockets.更改nginx二进制文件。主进程启动一个新的nginx二进制文件,并传入所有侦听套接字的列表。The text-format list, passed in the在“NGINX”
environment variable, consists of descriptor numbers separated with semicolons.“NGINX”
环境变量中传递的文本格式列表由用分号分隔的描述符数字组成。The new nginx binary reads the新的nginx二进制文件读取“NGINX”
variable and adds the sockets to its init cycle.“nginx”
变量,并将套接字添加到其init循环中。Other processes ignore this signal.其他进程忽略此信号。
While all nginx worker processes are able to receive and properly handle POSIX signals, the master process does not use the standard 虽然所有nginx工作进程都能够接收并正确处理POSIX信号,但主进程不使用标准kill()
syscall to pass signals to workers and helpers. kill()
系统调用将信号传递给工作进程和助手。Instead, nginx uses inter-process socket pairs which allow sending messages between all nginx processes. 相反,nginx使用进程间套接字对,允许在所有nginx进程之间发送消息。Currently, however, messages are only sent from the master to its children. 但是,目前,消息仅从主服务器发送到其子服务器。The messages carry the standard signals.这些信息带有标准信号。
Threads线程
It is possible to offload into a separate thread tasks that would otherwise block the nginx worker process. 可以将任务卸载到单独的线程中,否则会阻塞nginx工作进程。For example, nginx can be configured to use threads to perform file I/O. 例如,nginx可以配置为使用线程执行文件I/O。Another use case is a library that doesn't have asynchronous interface and thus cannot be normally used with nginx. 另一个用例是没有异步接口的库,因此不能与nginx一起正常使用。Keep in mind that the threads interface is a helper for the existing asynchronous approach to processing client connections, and by no means intended as a replacement.请记住,threads接口是处理客户机连接的现有异步方法的帮助器,决不是用来替代的。
To deal with synchronization, the following wrappers over 要处理同步,可以使用pthreads
primitives are available:pthreads
原语上的以下包装器:
typedef pthread_mutex_t ngx_thread_mutex_t;
ngx_int_t ngx_thread_mutex_create(ngx_thread_mutex_t *mtx, ngx_log_t *log);
ngx_int_t ngx_thread_mutex_destroy(ngx_thread_mutex_t *mtx, ngx_log_t *log);
ngx_int_t ngx_thread_mutex_lock(ngx_thread_mutex_t *mtx, ngx_log_t *log);
ngx_int_t ngx_thread_mutex_unlock(ngx_thread_mutex_t *mtx, ngx_log_t *log);
typedef pthread_cond_t ngx_thread_cond_t;
ngx_int_t ngx_thread_cond_create(ngx_thread_cond_t *cond, ngx_log_t *log);
ngx_int_t ngx_thread_cond_destroy(ngx_thread_cond_t *cond, ngx_log_t *log);
ngx_int_t ngx_thread_cond_signal(ngx_thread_cond_t *cond, ngx_log_t *log);
ngx_int_t ngx_thread_cond_wait(ngx_thread_cond_t *cond, ngx_thread_mutex_t *mtx, ngx_log_t *log);
Instead of creating a new thread for each task, nginx implements a thread_pool strategy. nginx实现了thread_pool策略,而不是为每个任务创建一个新线程。Multiple thread pools may be configured for different purposes (for example, performing I/O on different sets of disks). 可以为不同的目的配置多个线程池(例如,在不同的磁盘集上执行I/O)。Each thread pool is created at startup and contains a limited number of threads that process a queue of tasks. 每个线程池都是在启动时创建的,并且包含处理任务队列的有限数量的线程。When a task is completed, a predefined completion handler is called.任务完成后,将调用预定义的完成处理程序。
The src/core/ngx_thread_pool.h
header file contains relevant definitions:src/core/ngx_thread_pool.h
头文件包含相关定义:
struct ngx_thread_task_s { ngx_thread_task_t *next; ngx_uint_t id; void *ctx; void (*handler)(void *data, ngx_log_t *log); ngx_event_t event; }; typedef struct ngx_thread_pool_s ngx_thread_pool_t; ngx_thread_pool_t *ngx_thread_pool_add(ngx_conf_t *cf, ngx_str_t *name); ngx_thread_pool_t *ngx_thread_pool_get(ngx_cycle_t *cycle, ngx_str_t *name); ngx_thread_task_t *ngx_thread_task_alloc(ngx_pool_t *pool, size_t size); ngx_int_t ngx_thread_task_post(ngx_thread_pool_t *tp, ngx_thread_task_t *task);
At configuration time, a module willing to use threads has to obtain a reference to a thread pool by calling 在配置时,愿意使用线程的模块必须通过调用ngx_thread_pool_add(cf, name)
, which either creates a new thread pool with the given name
or returns a reference to the pool with that name if it already exists.ngx_thread_pool_add(cf, name)
来获取对线程池的引用,ngx_thread_pool_add(cf, name)
创建一个具有给定名称的新线程池,或者返回一个具有该名称的池的引用(如果该池已经存在)。
To add a 要在运行时将task
into a queue of a specified thread pool tp
at runtime, use the ngx_thread_task_post(tp, task)
function.task
添加到指定线程池tp
的队列中,请使用ngx_thread_task_post(tp, task)
函数。
To execute a function in a thread, pass parameters and setup a completion handler using the 要在线程中执行函数,请使用ngx_thread_task_t
structure:ngx_thread_task_t
结构传递参数并设置完成处理程序:
typedef struct { int foo; } my_thread_ctx_t; static void my_thread_func(void *data, ngx_log_t *log) { my_thread_ctx_t *ctx = data; /* this function is executed in a separate thread */ } static void my_thread_completion(ngx_event_t *ev) { my_thread_ctx_t *ctx = ev->data; /* executed in nginx event loop */ } ngx_int_t my_task_offload(my_conf_t *conf) { my_thread_ctx_t *ctx; ngx_thread_task_t *task; task = ngx_thread_task_alloc(conf->pool, sizeof(my_thread_ctx_t)); if (task == NULL) { return NGX_ERROR; } ctx = task->ctx; ctx->foo = 42; task->handler = my_thread_func; task->event.handler = my_thread_completion; task->event.data = ctx; if (ngx_thread_task_post(conf->thread_pool, task) != NGX_OK) { return NGX_ERROR; } return NGX_OK; }
Modules模块
Adding new modules添加新模块
Each standalone nginx module resides in a separate directory that contains at least two files:每个独立的nginx模块位于一个单独的目录中,其中至少包含两个文件:
config
and a file with the module source code. config
和一个包含模块源代码的文件。The config
file contains all information needed for nginx to integrate the module, for example:config
文件包含nginx集成模块所需的所有信息,例如:
ngx_module_type=CORE ngx_module_name=ngx_foo_module ngx_module_srcs="$ngx_addon_dir/ngx_foo_module.c" . auto/module ngx_addon_name=$ngx_module_name
The config
file is a POSIX shell script that can set and access the following variables:config
文件是一个POSIX shell脚本,可以设置和访问以下变量:
ngx_module_type
—Type of module to build.要生成的模块的类型。Possible values are可能的值是CORE
,HTTP
,HTTP_FILTER
,HTTP_INIT_FILTER
,HTTP_AUX_FILTER
,MAIL
,STREAM
, orMISC
.CORE
、HTTP
、HTTP_FILTER
、HTTP_INIT_FILTER
、HTTP_AUX_FILTER
、MAIL
、STREAM
或MISC
。ngx_module_name
—Module names. To build multiple modules from a set of source files, specify a whitespace-separated list of names.模块名称。要从一组源文件构建多个模块,请指定以空格分隔的名称列表。The first name indicates the name of the output binary for the dynamic module.第一个名称表示动态模块的输出二进制文件的名称。The names in the list must match the names used in the source code.列表中的名称必须与源代码中使用的名称匹配。ngx_addon_name
—Name of the module as it appears in output on the console from the configure script.从配置脚本在控制台上的输出中显示的模块名称。ngx_module_srcs
—Whitespace-separated list of source files used to compile the module.用于编译模块的源文件的空白分隔列表。The$ngx_addon_dir
variable can be used to represent the path to the module directory.$ngx_addon_dir
变量可用于表示模块目录的路径。ngx_module_incs
—Include paths required to build the module包括构建模块所需的路径ngx_module_deps
—Whitespace-separated list of the module's dependencies.以空格分隔的模块依赖项列表。Usually, it is the list of header files.通常,它是头文件的列表。ngx_module_libs
—Whitespace-separated list of libraries to link with the module.要与模块链接的库的空白分隔列表。For example, usengx_module_libs=-lpthread
to linklibpthread
library. The following macros can be used to link against the same libraries as nginx:LIBXSLT
,LIBGD
,GEOIP
,PCRE
,OPENSSL
,MD5
,SHA1
,ZLIB
, andPERL
.ngx_module_link
— Variable set by the build system toDYNAMIC
for a dynamic module orADDON
for a static module and used to determine different actions to perform depending on linking type.ngx_module_order
— Load order for the module; useful for theHTTP_FILTER
andHTTP_AUX_FILTER
module types. The format for this option is a whitespace-separated list of modules. All modules in the list following the current module's name end up after it in the global list of modules, which sets up the order for modules initialization. For filter modules later initialization means earlier execution.The following modules are typically used as references. The
ngx_http_copy_filter_module
reads the data for other filter modules and is placed near the bottom of the list so that it is one of the first to be executed. Thengx_http_write_filter_module
writes the data to the client socket and is placed near the top of the list, and is the last to be executed.By default, filter modules are placed before the
ngx_http_copy_filter
in the module list so that the filter handler is executed after the copy filter handler. For other module types the default is the empty string.
To compile a module into nginx statically, use the 要静态地将模块编译成nginx,请在配置脚本中使用--add-module=/path/to/module
argument to the configure script. --add-module=/path/to/module
参数。To compile a module for later dynamic loading into nginx, use the 要编译模块以便以后动态加载到nginx中,请使用--add-dynamic-module=/path/to/module
argument.--add-dynamic-module=/path/to/module
参数。
Core Modules核心模块
Modules are the building blocks of nginx, and most of its functionality is implemented as modules. 模块是nginx的构建块,它的大部分功能都是作为模块实现的。The module source file must contain a global variable of type 模块源文件必须包含ngx_module_t
, which is defined as follows:ngx_module_t
类型的全局变量,其定义如下:
struct ngx_module_s { /* private part is omitted */ void *ctx; ngx_command_t *commands; ngx_uint_t type; ngx_int_t (*init_master)(ngx_log_t *log); ngx_int_t (*init_module)(ngx_cycle_t *cycle); ngx_int_t (*init_process)(ngx_cycle_t *cycle); ngx_int_t (*init_thread)(ngx_cycle_t *cycle); void (*exit_thread)(ngx_cycle_t *cycle); void (*exit_process)(ngx_cycle_t *cycle); void (*exit_master)(ngx_cycle_t *cycle); /* stubs for future extensions are omitted */ };
The omitted private part includes the module version and a signature and is filled using the predefined macro 省略的私有部分包括模块版本和签名,并使用预定义的宏NGX_MODULE_V1
.NGX_MODULE_V1
填充。
Each module keeps its private data in the 每个模块将其私有数据保存在ctx
field, recognizes the configuration directives, specified in the commands
array, and can be invoked at certain stages of nginx lifecycle. ctx
字段中,识别在commands
数组中指定的配置指令,并可在nginx生命周期的某些阶段调用。The module lifecycle consists of the following events:模块生命周期由以下事件组成:
Configuration directive handlers are called as they appear in configuration files in the context of the master process.配置指令处理程序在主进程上下文中的配置文件中出现时被调用。After the configuration is parsed successfully,成功解析配置后,将在主进程的上下文中调用init_module
handler is called in the context of the master process.init_module
处理程序。The每次加载配置时,都会在主进程中调用init_module
handler is called in the master process each time a configuration is loaded.init_module
处理程序。The master process creates one or more worker processes and the主进程创建一个或多个工作进程,并在每个进程中调用init_process
handler is called in each of them.init_process
处理程序。When a worker process receives the shutdown or terminate command from the master, it invokes the当工作进程从主进程接收到shutdown或terminate命令时,它将调用exit_process
handler.exit_process
处理程序。The master process calls the主进程在退出之前调用exit_master
handler before exiting.exit_master
处理程序。
Because threads are used in nginx only as a supplementary I/O facility with its own API, 由于线程在nginx中仅作为具有自己API的补充I/O工具使用,因此当前不调用init_thread
and exit_thread
handlers are not currently called. init_thread
和exit_thread
处理程序。There is also no 也没有init_master
handler, because it would be unnecessary overhead.init_master
处理程序,因为这将是不必要的开销。
The module 模块type
defines exactly what is stored in the ctx
field. type
准确定义了ctx
字段中存储的内容。Its value is one of the following types:其值为以下类型之一:
NGX_CORE_MODULE
NGX_EVENT_MODULE
NGX_HTTP_MODULE
NGX_MAIL_MODULE
NGX_STREAM_MODULE
The NGX_CORE_MODULE
is the most basic and thus the most generic and most low-level type of module. NGX_CORE_MODULE
是最基本的模块,因此也是最通用、最低级的模块类型。The other module types are implemented on top of it and provide a more convenient way to deal with corresponding domains, like handling events or HTTP requests.其他模块类型在其上实现,并提供更方便的方式来处理相应的域,如处理事件或HTTP请求。
The set of core modules includes ngx_core_module
, ngx_errlog_module
, ngx_regex_module
, ngx_thread_pool_module
and ngx_openssl_module
modules. The HTTP module, the stream module, the mail module and event modules are core modules too. HTTP模块、流模块、邮件模块和事件模块也是核心模块。The context of a core module is defined as:核心模块的上下文定义为:
typedef struct { ngx_str_t name; void *(*create_conf)(ngx_cycle_t *cycle); char *(*init_conf)(ngx_cycle_t *cycle, void *conf); } ngx_core_module_t;
where the name
is a module name string, create_conf
and init_conf
are pointers to functions that create and initialize module configuration respectively. For core modules, nginx calls create_conf
before parsing a new configuration and init_conf
after all configuration is parsed successfully. The typical create_conf
function allocates memory for the configuration and sets default values.
For example, a simplistic module called ngx_foo_module
might look like this:
/* * Copyright (C) Author. */ #include <ngx_config.h> #include <ngx_core.h> typedef struct { ngx_flag_t enable; } ngx_foo_conf_t; static void *ngx_foo_create_conf(ngx_cycle_t *cycle); static char *ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf); static char *ngx_foo_enable(ngx_conf_t *cf, void *post, void *data); static ngx_conf_post_t ngx_foo_enable_post = { ngx_foo_enable }; static ngx_command_t ngx_foo_commands[] = { { ngx_string("foo_enabled"), NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, 0, offsetof(ngx_foo_conf_t, enable), &ngx_foo_enable_post }, ngx_null_command }; static ngx_core_module_t ngx_foo_module_ctx = { ngx_string("foo"), ngx_foo_create_conf, ngx_foo_init_conf }; ngx_module_t ngx_foo_module = { NGX_MODULE_V1, &ngx_foo_module_ctx, /* module context */ ngx_foo_commands, /* module directives */ NGX_CORE_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static void * ngx_foo_create_conf(ngx_cycle_t *cycle) { ngx_foo_conf_t *fcf; fcf = ngx_pcalloc(cycle->pool, sizeof(ngx_foo_conf_t)); if (fcf == NULL) { return NULL; } fcf->enable = NGX_CONF_UNSET; return fcf; } static char * ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf) { ngx_foo_conf_t *fcf = conf; ngx_conf_init_value(fcf->enable, 0); return NGX_CONF_OK; } static char * ngx_foo_enable(ngx_conf_t *cf, void *post, void *data) { ngx_flag_t *fp = data; if (*fp == 0) { return NGX_CONF_OK; } ngx_log_error(NGX_LOG_NOTICE, cf->log, 0, "Foo Module is enabled"); return NGX_CONF_OK; }
Configuration Directives配置指令
The ngx_command_t
type defines a single configuration directive. ngx_command_t
类型定义了单个配置指令。Each module that supports configuration provides an array of such structures that describe how to process arguments and what handlers to call:每个支持配置的模块都提供了一系列这样的结构,这些结构描述了如何处理参数以及要调用的处理程序:
typedef struct ngx_command_s ngx_command_t; struct ngx_command_s { ngx_str_t name; ngx_uint_t type; char *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_uint_t conf; ngx_uint_t offset; void *post; };
Terminate the array with the special value 使用特殊值ngx_null_command
. ngx_null_command
终止数组。The name
is the name of a directive as it appears in the configuration file, for example "worker_processes" or "listen". name
是在配置文件中显示的指令的名称,例如“worker_processes”或“listen”。The type
is a bit-field of flags that specify the number of arguments the directive takes, its type, and the context in which it appears. type
是一个由标志组成的位字段,用于指定指令所采用的参数数量、类型及其出现的上下文。The flags are:旗为:
NGX_CONF_NOARGS
—Directive takes no arguments.指令不接受任何参数。NGX_CONF_1MORE
—Directive takes one or more arguments.指令接受一个或多个参数。NGX_CONF_2MORE
—Directive takes two or more arguments.指令接受两个或多个参数。NGX_CONF_TAKE1
..NGX_CONF_TAKE7
—Directive takes exactly the indicated number of arguments.指令完全接受指定数量的参数。NGX_CONF_TAKE12
,NGX_CONF_TAKE13
,NGX_CONF_TAKE23
,NGX_CONF_TAKE123
,NGX_CONF_TAKE1234
—Directive may take different number of arguments.指令可以采用不同数量的参数。Options are limited to the given numbers.选项仅限于给定的数字。For example,例如,NGX_CONF_TAKE12
means it takes one or two arguments.NGX_CONF_TAKE12
意味着它需要一个或两个参数。
The flags for directive types are:指令类型的标志为:
NGX_CONF_BLOCK
—Directive is a block, that is, it can contain other directives within its opening and closing braces, or even implement its own parser to handle contents inside.指令是一个块,也就是说,它可以在其开始和结束大括号中包含其他指令,甚至可以实现自己的解析器来处理其中的内容。NGX_CONF_FLAG
—Directive takes a boolean value, either指令获取一个布尔值,on
oroff
.on
或off
。
A directive's context defines where it may appear in the configuration:指令的上下文定义了它可能出现在配置中的位置:
NGX_MAIN_CONF
—In the top level context.在顶级上下文中。NGX_HTTP_MAIN_CONF
— In thehttp
block.NGX_HTTP_SRV_CONF
— In aserver
block within thehttp
block.NGX_HTTP_LOC_CONF
— In alocation
block within thehttp
block.NGX_HTTP_UPS_CONF
— In anupstream
block within thehttp
block.NGX_HTTP_SIF_CONF
— In anif
block within aserver
block in thehttp
block.NGX_HTTP_LIF_CONF
— In anif
block within alocation
block in thehttp
block.NGX_HTTP_LMT_CONF
— In alimit_except
block within thehttp
block.NGX_STREAM_MAIN_CONF
— In thestream
block.NGX_STREAM_SRV_CONF
— In aserver
block within thestream
block.NGX_STREAM_UPS_CONF
— In anupstream
block within thestream
block.NGX_MAIL_MAIN_CONF
— In themail
block.NGX_MAIL_SRV_CONF
— In aserver
block within themail
block.NGX_EVENT_CONF
— In theevent
block.NGX_DIRECT_CONF
—Used by modules that don't create a hierarchy of contexts and only have one global configuration.由不创建上下文层次结构且只有一个全局配置的模块使用。This configuration is passed to the handler as the此配置作为conf
argument.conf
参数传递给处理程序。
The configuration parser uses these flags to throw an error in case of a misplaced directive and calls directive handlers supplied with a proper configuration pointer, so that the same directives in different locations can store their values in distinct places.配置解析器使用这些标志在指令放错位置时抛出错误,并调用随正确配置指针提供的指令处理程序,以便不同位置的相同指令可以将其值存储在不同的位置。
The set
field defines a handler that processes a directive and stores parsed values into the corresponding configuration. set
字段定义处理指令并将解析值存储到相应配置中的处理程序。There's a number of functions that perform common conversions:有许多函数可以执行常见的转换:
ngx_conf_set_flag_slot
— Converts the literal stringson
andoff
into anngx_flag_t
value with values 1 or 0, respectively.ngx_conf_set_str_slot
— Stores a string as a value of thengx_str_t
type.ngx_conf_set_str_array_slot
— Appends a value to an arrayngx_array_t
of stringsngx_str_t
. The array is created if does not already exist.ngx_conf_set_keyval_slot
— Appends a key-value pair to an arrayngx_array_t
of key-value pairsngx_keyval_t
. The first string becomes the key and the second the value. The array is created if it does not already exist.ngx_conf_set_num_slot
— Converts a directive's argument to anngx_int_t
value.ngx_conf_set_size_slot
— Converts a size to asize_t
value expressed in bytes.ngx_conf_set_off_slot
— Converts an offset to anoff_t
value expressed in bytes.ngx_conf_set_msec_slot
— Converts a time to anngx_msec_t
value expressed in milliseconds.ngx_conf_set_sec_slot
— Converts a time to atime_t
value expressed in in seconds.ngx_conf_set_bufs_slot
— Converts the two supplied arguments into anngx_bufs_t
object that holds the number and size of buffers.ngx_conf_set_enum_slot
— Converts the supplied argument into anngx_uint_t
value. The null-terminated array ofngx_conf_enum_t
passed in thepost
field defines the acceptable strings and corresponding integer values.ngx_conf_set_bitmask_slot
— Converts the supplied arguments into anngx_uint_t
value. The mask values for each argument are ORed producing the result. The null-terminated array ofngx_conf_bitmask_t
passed in thepost
field defines the acceptable strings and corresponding mask values.set_path_slot
— Converts the supplied arguments to anngx_path_t
value and performs all required initializations. For details, see the documentation for the proxy_temp_path directive.set_access_slot
— Converts the supplied arguments to a file permissions mask. For details, see the documentation for the proxy_store_access directive.
The conf
field defines which configuration structure is passed to the directory handler. Core modules only have the global configuration and set NGX_DIRECT_CONF
flag to access it. Modules like HTTP, Stream or Mail create hierarchies of configurations. For example, a module's configuration is created for server
, location
and if
scopes.
NGX_HTTP_MAIN_CONF_OFFSET
— Configuration for thehttp
block.NGX_HTTP_SRV_CONF_OFFSET
— Configuration for aserver
block within thehttp
block.NGX_HTTP_LOC_CONF_OFFSET
— Configuration for alocation
block within thehttp
.NGX_STREAM_MAIN_CONF_OFFSET
— Configuration for thestream
block.NGX_STREAM_SRV_CONF_OFFSET
— Configuration for aserver
block within thestream
block.NGX_MAIL_MAIN_CONF_OFFSET
— Configuration for themail
block.NGX_MAIL_SRV_CONF_OFFSET
— Configuration for aserver
block within themail
block.
The offset
defines the offset of a field in a module configuration structure that holds values for this particular directive. The typical use is to employ the offsetof()
macro.
The post
field has two purposes: it may be used to define a handler to be called after the main handler has completed, or to pass additional data to the main handler. In the first case, the ngx_conf_post_t
structure needs to be initialized with a pointer to the handler, for example:
static char *ngx_do_foo(ngx_conf_t *cf, void *post, void *data); static ngx_conf_post_t ngx_foo_post = { ngx_do_foo };
The post
argument is the ngx_conf_post_t
object itself, and the data
is a pointer to the value, converted from arguments by the main handler with the appropriate type.
HTTP
Connection
Each HTTP client connection runs through the following stages:
ngx_event_accept()
accepts a client TCP connection. This handler is called in response to a read notification on a listen socket. A newngx_connection_t
object is created at this stage to wrap the newly accepted client socket. Each nginx listener provides a handler to pass the new connection object to. For HTTP connections it'sngx_http_init_connection(c)
.ngx_http_init_connection()
performs early initialization of the HTTP connection. At this stage anngx_http_connection_t
object is created for the connection and its reference is stored in the connection'sdata
field. Later it will be replaced by an HTTP request object. A PROXY protocol parser and the SSL handshake are started at this stage as well.ngx_http_wait_request_handler()
read event handler is called when data is available on the client socket. At this stage an HTTP request objectngx_http_request_t
is created and set to the connection'sdata
field.ngx_http_process_request_line()
read event handler reads client request line. The handler is set byngx_http_wait_request_handler()
. The data is read into connection'sbuffer
. The size of the buffer is initially set by the directive client_header_buffer_size. The entire client header is supposed to fit in the buffer. If the initial size is not sufficient, a bigger buffer is allocated, with the capacity set by thelarge_client_header_buffers
directive.ngx_http_process_request_headers()
read event handler, is set afterngx_http_process_request_line()
to read the client request header.ngx_http_core_run_phases()
is called when the request header is completely read and parsed. This function runs request phases fromNGX_HTTP_POST_READ_PHASE
toNGX_HTTP_CONTENT_PHASE
. The last phase is intended to generate a response and pass it along the filter chain. The response is not necessarily sent to the client at this phase. It might remain buffered and be sent at the finalization stage.ngx_http_finalize_request()
is usually called when the request has generated all the output or produced an error. In the latter case an appropriate error page is looked up and used as the response. If the response is not completely sent to the client by this point, an HTTP writerngx_http_writer()
is activated to finish sending outstanding data.ngx_http_finalize_connection()
is called when the complete response has been sent to the client and the request can be destroyed. If the client connection keepalive feature is enabled,ngx_http_set_keepalive()
is called, which destroys the current request and waits for the next request on the connection. Otherwise,ngx_http_close_request()
destroys both the request and the connection.
Request
For each client HTTP request the ngx_http_request_t
object is created. Some of the fields of this object are:
-
connection
— Pointer to angx_connection_t
client connection object. Several requests can reference the same connection object at the same time - one main request and its subrequests. After a request is deleted, a new request can be created on the same connection.Note that for HTTP connections
ngx_connection_t
'sdata
field points back to the request. Such requests are called active, as opposed to the other requests tied to the connection. An active request is used to handle client connection events and is allowed to output its response to the client. Normally, each request becomes active at some point so that it can send its output. -
ctx
— Array of HTTP module contexts. Each module of typeNGX_HTTP_MODULE
can store any value (normally, a pointer to a structure) in the request. The value is stored in thectx
array at the module'sctx_index
position. The following macros provide a convenient way to get and set request contexts:ngx_http_get_module_ctx(r, module)
— Returns themodule
's contextngx_http_set_ctx(r, c, module)
— Setsc
as themodule
's context
main_conf
,srv_conf
,loc_conf
— Arrays of current request configurations. Configurations are stored at the module'sctx_index
positions.read_event_handler
,write_event_handler
- Read and write event handlers for the request. Normally, both the read and write event handlers for an HTTP connection are set tongx_http_request_handler()
. This function calls theread_event_handler
andwrite_event_handler
handlers for the currently active request.cache
— Request cache object for caching the upstream response.upstream
— Request upstream object for proxying.pool
— Request pool. The request object itself is allocated in this pool, which is destroyed when the request is deleted. For allocations that need to be available throughout the client connection's lifetime, usengx_connection_t
's pool instead.header_in
— Buffer into which the client HTTP request header is read.headers_in
,headers_out
— Input and output HTTP headers objects. Both objects contain theheaders
field of typengx_list_t
for keeping the raw list of headers. In addition to that, specific headers are available for getting and setting as separate fields, for examplecontent_length_n
,status
etc.request_body
— Client request body object.start_sec
,start_msec
— Time point when the request was created, used for tracking request duration.method
,method_name
— Numeric and text representation of the client HTTP request method. Numeric values for methods are defined insrc/http/ngx_http_request.h
with the macrosNGX_HTTP_GET
,NGX_HTTP_HEAD
,NGX_HTTP_POST
, etc.http_protocol
— Client HTTP protocol version in its original text form (“HTTP/1.0”, “HTTP/1.1” etc).http_version
— Client HTTP protocol version in numeric form (NGX_HTTP_VERSION_10
,NGX_HTTP_VERSION_11
, etc.).http_major
,http_minor
— Client HTTP protocol version in numeric form split into major and minor parts.request_line
,unparsed_uri
— Request line and URI in the original client request.uri
,args
,exten
— URI, arguments and file extension for the current request. The URI value here might differ from the original URI sent by the client due to normalization. Throughout request processing, these values can change as internal redirects are performed.main
— Pointer to a main request object. This object is created to process a client HTTP request, as opposed to subrequests, which are created to perform a specific subtask within the main request.parent
— Pointer to the parent request of a subrequest.postponed
— List of output buffers and subrequests, in the order in which they are sent and created. The list is used by the postpone filter to provide consistent request output when parts of it are created by subrequests.post_subrequest
— Pointer to a handler with the context to be called when a subrequest gets finalized. Unused for main requests.-
posted_requests
— List of requests to be started or resumed, which is done by calling the request'swrite_event_handler
. Normally, this handler holds the request main function, which at first runs request phases and then produces the output.A request is usually posted by the
ngx_http_post_request(r, NULL)
call. It is always posted to the main requestposted_requests
list. The functionngx_http_run_posted_requests(c)
runs all requests that are posted in the main request of the passed connection's active request. All event handlers callngx_http_run_posted_requests
, which can lead to new posted requests. Normally, it is called after invoking a request's read or write handler. phase_handler
— Index of current request phase.ncaptures
,captures
,captures_data
— Regex captures produced by the last regex match of the request. A regex match can occur at a number of places during request processing: map lookup, server lookup by SNI or HTTP Host, rewrite, proxy_redirect, etc. Captures produced by a lookup are stored in the above mentioned fields. The fieldncaptures
holds the number of captures,captures
holds captures boundaries andcaptures_data
holds the string against which the regex was matched and which is used to extract captures. After each new regex match, request captures are reset to hold new values.count
— Request reference counter. The field only makes sense for the main request. Increasing the counter is done by simpler->main->count++
. To decrease the counter, callngx_http_finalize_request(r, rc)
. Creating of a subrequest and running the request body read process both increment the counter.subrequests
— Current subrequest nesting level. Each subrequest inherits its parent's nesting level, decreased by one. An error is generated if the value reaches zero. The value for the main request is defined by theNGX_HTTP_MAX_SUBREQUESTS
constant.uri_changes
— Number of URI changes remaining for the request. The total number of times a request can change its URI is limited by theNGX_HTTP_MAX_URI_CHANGES
constant. With each change the value is decremented until it reaches zero, at which time an error is generated. Rewrites and internal redirects to normal or named locations are considered URI changes.blocked
— Counter of blocks held on the request. While this value is non-zero, the request cannot be terminated. Currently, this value is increased by pending AIO operations (POSIX AIO and thread operations) and active cache lock.buffered
— Bitmask showing which modules have buffered the output produced by the request. A number of filters can buffer output; for example, sub_filter can buffer data because of a partial string match, copy filter can buffer data because of the lack of free output buffers etc. As long as this value is non-zero, the request is not finalized pending the flush.header_only
— Flag indicating that the output does not require a body. For example, this flag is used by HTTP HEAD requests.-
keepalive
— Flag indicating whether client connection keepalive is supported. The value is inferred from the HTTP version and the value of the “Connection” header. header_sent
— Flag indicating that the output header has already been sent by the request.internal
— Flag indicating that the current request is internal. To enter the internal state, a request must pass through an internal redirect or be a subrequest. Internal requests are allowed to enter internal locations.allow_ranges
— Flag indicating that a partial response can be sent to the client, as requested by the HTTP Range header.subrequest_ranges
— Flag indicating that a partial response can be sent while a subrequest is being processed.single_range
— Flag indicating that only a single continuous range of output data can be sent to the client. This flag is usually set when sending a stream of data, for example from a proxied server, and the entire response is not available in one buffer.main_filter_need_in_memory
,filter_need_in_memory
— Flags requesting that the output produced in memory buffers rather than files. This is a signal to the copy filter to read data from file buffers even if sendfile is enabled. The difference between the two flags is the location of the filter modules that set them. Filters called before the postpone filter in the filter chain setfilter_need_in_memory
, requesting that only the current request output come in memory buffers. Filters called later in the filter chain setmain_filter_need_in_memory
, requesting that both the main request and all subrequests read files in memory while sending output.filter_need_temporary
— Flag requesting that the request output be produced in temporary buffers, but not in readonly memory buffers or file buffers. This is used by filters which may change output directly in the buffers where it's sent.
Configuration
Each HTTP module can have three types of configuration:
- Main configuration — Applies to the entire
http
block. Functions as global settings for a module. - Server configuration — Applies to a single
server
block. Functions as server-specific settings for a module. - Location configuration — Applies to a single
location
,if
orlimit_except
block. Functions as location-specific settings for a module.
Configuration structures are created at the nginx configuration stage by calling functions, which allocate the structures, initialize them and merge them. The following example shows how to create a simple location configuration for a module. The configuration has one setting, foo
, of type unsigned integer.
typedef struct { ngx_uint_t foo; } ngx_http_foo_loc_conf_t; static ngx_http_module_t ngx_http_foo_module_ctx = { NULL, /* preconfiguration */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ ngx_http_foo_create_loc_conf, /* create location configuration */ ngx_http_foo_merge_loc_conf /* merge location configuration */ }; static void * ngx_http_foo_create_loc_conf(ngx_conf_t *cf) { ngx_http_foo_loc_conf_t *conf; conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_foo_loc_conf_t)); if (conf == NULL) { return NULL; } conf->foo = NGX_CONF_UNSET_UINT; return conf; } static char * ngx_http_foo_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) { ngx_http_foo_loc_conf_t *prev = parent; ngx_http_foo_loc_conf_t *conf = child; ngx_conf_merge_uint_value(conf->foo, prev->foo, 1); }
As seen in the example, the ngx_http_foo_create_loc_conf()
function creates a new configuration structure, and ngx_http_foo_merge_loc_conf()
merges a configuration with configuration from a higher level. In fact, server and location configuration do not exist only at the server and location levels, but are also created for all levels above them. Specifically, a server configuration is also created at the main level and location configurations are created at the main, server, and location levels. These configurations make it possible to specify server- and location-specific settings at any level of an nginx configuration file. Eventually configurations are merged down. A number of macros like NGX_CONF_UNSET
and NGX_CONF_UNSET_UINT
are provided for indicating a missing setting and ignoring it while merging. Standard nginx merge macros like ngx_conf_merge_value()
and ngx_conf_merge_uint_value()
provide a convenient way to merge a setting and set the default value if none of the configurations provided an explicit value. For complete list of macros for different types, see src/core/ngx_conf_file.h
.
The following macros are available. for accessing configuration for HTTP modules at configuration time. They all take ngx_conf_t
reference as the first argument.
ngx_http_conf_get_module_main_conf(cf, module)
ngx_http_conf_get_module_srv_conf(cf, module)
ngx_http_conf_get_module_loc_conf(cf, module)
The following example gets a pointer to a location configuration of standard nginx core module ngx_http_core_module and replaces the location content handler kept in the handler
field of the structure.
static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r); static ngx_command_t ngx_http_foo_commands[] = { { ngx_string("foo"), NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS, ngx_http_foo, 0, 0, NULL }, ngx_null_command }; static char * ngx_http_foo(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_bar_handler; return NGX_CONF_OK; }
The following macros are available for accessing configuration for HTTP modules at runtime.
ngx_http_get_module_main_conf(r, module)
ngx_http_get_module_srv_conf(r, module)
ngx_http_get_module_loc_conf(r, module)
These macros receive a reference to an HTTP request ngx_http_request_t
. The main configuration of a request never changes. Server configuration can change from the default after the virtual server for the request is chosen. Location configuration selected for processing a request can change multiple times as a result of a rewrite operation or internal redirect. The following example shows how to access a module's HTTP configuration at runtime.
static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r) { ngx_http_foo_loc_conf_t *flcf; flcf = ngx_http_get_module_loc_conf(r, ngx_http_foo_module); ... }
Phases
Each HTTP request passes through a sequence of phases. In each phase a distinct type of processing is performed on the request. Module-specific handlers can be registered in most phases, and many standard nginx modules register their phase handlers as a way to get called at a specific stage of request processing. Phases are processed successively and the phase handlers are called once the request reaches the phase. Following is the list of nginx HTTP phases.
NGX_HTTP_POST_READ_PHASE
— First phase. The ngx_http_realip_module registers its handler at this phase to enable substitution of client addresses before any other module is invoked.NGX_HTTP_SERVER_REWRITE_PHASE
— Phase where rewrite directives defined in aserver
block (but outside alocation
block) are processed. The ngx_http_rewrite_module installs its handler at this phase.NGX_HTTP_FIND_CONFIG_PHASE
— Special phase where a location is chosen based on the request URI. Before this phase, the default location for the relevant virtual server is assigned to the request, and any module requesting a location configuration receives the configuration for the default server location. This phase assigns a new location to the request. No additional handlers can be registered at this phase.NGX_HTTP_REWRITE_PHASE
— Same asNGX_HTTP_SERVER_REWRITE_PHASE
, but for rewrite rules defined in the location, chosen in the previous phase.NGX_HTTP_POST_REWRITE_PHASE
— Special phase where the request is redirected to a new location if its URI changed during a rewrite. This is implemented by the request going through theNGX_HTTP_FIND_CONFIG_PHASE
again. No additional handlers can be registered at this phase.NGX_HTTP_PREACCESS_PHASE
— A common phase for different types of handlers, not associated with access control. The standard nginx modules ngx_http_limit_conn_module and ngx_http_limit_req_module register their handlers at this phase.NGX_HTTP_ACCESS_PHASE
— Phase where it is verified that the client is authorized to make the request. Standard nginx modules such as ngx_http_access_module and ngx_http_auth_basic_module register their handlers at this phase. By default the client must pass the authorization check of all handlers registered at this phase for the request to continue to the next phase. The satisfy directive, can be used to permit processing to continue if any of the phase handlers authorizes the client.NGX_HTTP_POST_ACCESS_PHASE
— Special phase where the satisfy any directive is processed. If some access phase handlers denied access and none explicitly allowed it, the request is finalized. No additional handlers can be registered at this phase.NGX_HTTP_PRECONTENT_PHASE
— Phase for handlers to be called prior to generating content. Standard modules such as ngx_http_try_files_module and ngx_http_mirror_module register their handlers at this phase.NGX_HTTP_CONTENT_PHASE
— Phase where the response is normally generated. Multiple nginx standard modules register their handlers at this phase, including ngx_http_index_module orngx_http_static_module
. They are called sequentially until one of them produces the output. It's also possible to set content handlers on a per-location basis. If the ngx_http_core_module's location configuration hashandler
set, it is called as the content handler and the handlers installed at this phase are ignored.NGX_HTTP_LOG_PHASE
— Phase where request logging is performed. Currently, only the ngx_http_log_module registers its handler at this stage for access logging. Log phase handlers are called at the very end of request processing, right before freeing the request.
Following is the example of a preaccess phase handler.
static ngx_http_module_t ngx_http_foo_module_ctx = { NULL, /* preconfiguration */ ngx_http_foo_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r) { ngx_str_t *ua; ua = r->headers_in->user_agent; if (ua == NULL) { return NGX_DECLINED; } /* reject requests with "User-Agent: foo" */ if (ua->value.len == 3 && ngx_strncmp(ua->value.data, "foo", 3) == 0) { return NGX_HTTP_FORBIDDEN; } return NGX_DECLINED; } static ngx_int_t ngx_http_foo_init(ngx_conf_t *cf) { ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cmcf; cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cmcf->phases[NGX_HTTP_PREACCESS_PHASE].handlers); if (h == NULL) { return NGX_ERROR; } *h = ngx_http_foo_handler; return NGX_OK; }
Phase handlers are expected to return specific codes:
NGX_OK
— Proceed to the next phase.NGX_DECLINED
— Proceed to the next handler of the current phase. If the current handler is the last in the current phase, move to the next phase.NGX_AGAIN
,NGX_DONE
— Suspend phase handling until some future event which can be an asynchronous I/O operation or just a delay, for example. It is assumed, that phase handling will be resumed later by callingngx_http_core_run_phases()
.- Any other value returned by the phase handler is treated as a request finalization code, in particular, an HTTP response code. The request is finalized with the code provided.
For some phases, return codes are treated in a slightly different way. At the content phase, any return code other that NGX_DECLINED
is considered a finalization code. Any return code from the location content handlers is considered a finalization code. At the access phase, in satisfy any mode, any return code other than NGX_OK
, NGX_DECLINED
, NGX_AGAIN
, NGX_DONE
is considered a denial. If no subsequent access handlers allow or deny access with a different code, the denial code will become the finalization code.
Variables
Accessing existing variables
Variables can be referenced by index (this is the most common method)
or name (see below). The index is created at configuration stage, when a variable is added to the configuration. To obtain the variable index, use ngx_http_get_variable_index()
:
ngx_str_t name; /* ngx_string("foo") */ ngx_int_t index; index = ngx_http_get_variable_index(cf, &name);
Here, cf
is a pointer to nginx configuration and name
points to a string containing the variable name. The function returns NGX_ERROR
on error or a valid index otherwise, which is typically stored somewhere in the module's configuration for future use.
All HTTP variables are evaluated in the context of a given HTTP request, and results are specific to and cached in that HTTP request. All functions that evaluate variables return the ngx_http_variable_value_t
type, representing the variable value:
typedef ngx_variable_value_t ngx_http_variable_value_t; typedef struct { unsigned len:28; unsigned valid:1; unsigned no_cacheable:1; unsigned not_found:1; unsigned escape:1; u_char *data; } ngx_variable_value_t;
where:
len
— The length of the valuedata
— The value itselfvalid
— The value is validnot_found
— The variable was not found and thus thedata
andlen
fields are irrelevant; this can happen, for example, with variables like$arg_foo
when a corresponding argument was not passed in a requestno_cacheable
— Do not cache resultescape
— Used internally by the logging module to mark values that require escaping on output.
The ngx_http_get_flushed_variable()
and ngx_http_get_indexed_variable()
functions are used to obtain the value of a variable. They have the same interface - accepting an HTTP request r
as a context for evaluating the variable and an index
that identifies it. An example of typical usage:
ngx_http_variable_value_t *v; v = ngx_http_get_flushed_variable(r, index); if (v == NULL || v->not_found) { /* we failed to get value or there is no such variable, handle it */ return NGX_ERROR; } /* some meaningful value is found */
The difference between functions is that the ngx_http_get_indexed_variable()
returns a cached value and ngx_http_get_flushed_variable()
flushes the cache for non-cacheable variables.
Some modules, such as SSI and Perl, need to deal with variables for which the name is not known at configuration time. An index therefore cannot be used to access them, but the ngx_http_get_variable(r, name, key)
function is available. It searches for a variable with a given name
and its hash key
derived from the name.
Creating variables
To create a variable, use the ngx_http_add_variable()
function. It takes as arguments a configuration (where the variable is registered), the variable name and flags that control the function's behaviour:
NGX_HTTP_VAR_CHANGEABLE
— Enables redefinition of the variable: there is no conflict if another module defines a variable with the same name. This allows the set directive to override variables.NGX_HTTP_VAR_NOCACHEABLE
— Disables caching, which is useful for variables such as$time_local
.NGX_HTTP_VAR_NOHASH
— Indicates that this variable is only accessible by index, not by name. This is a small optimization for use when it is known that the variable is not needed in modules like SSI or Perl.NGX_HTTP_VAR_PREFIX
— The name of the variable is a prefix. In this case, a handler must implement additional logic to obtain the value of a specific variable. For example, all “arg_
” variables are processed by the same handler, which performs lookup in request arguments and returns the value of a specific argument.
The function returns NULL in case of error or a pointer to ngx_http_variable_t
otherwise:
struct ngx_http_variable_s { ngx_str_t name; ngx_http_set_variable_pt set_handler; ngx_http_get_variable_pt get_handler; uintptr_t data; ngx_uint_t flags; ngx_uint_t index; };
The get
and set
handlers are called to obtain or set the variable value, data
is passed to variable handlers, and index
holds assigned variable index used to reference the variable.
Usually, a null-terminated static array of ngx_http_variable_t
structures is created by a module and processed at the preconfiguration stage to add variables into the configuration, for example:
static ngx_http_variable_t ngx_http_foo_vars[] = { { ngx_string("foo_v1"), NULL, ngx_http_foo_v1_variable, 0, 0, 0 }, ngx_http_null_variable }; static ngx_int_t ngx_http_foo_add_variables(ngx_conf_t *cf) { ngx_http_variable_t *var, *v; for (v = ngx_http_foo_vars; v->name.len; v++) { var = ngx_http_add_variable(cf, &v->name, v->flags); if (var == NULL) { return NGX_ERROR; } var->get_handler = v->get_handler; var->data = v->data; } return NGX_OK; }
This function in the example is used to initialize the preconfiguration
field of the HTTP module context and is called before the parsing of HTTP configuration, so that the parser can refer to these variables.
The get
handler is responsible for evaluating a variable in the context of a specific request, for example:
static ngx_int_t ngx_http_variable_connection(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { u_char *p; p = ngx_pnalloc(r->pool, NGX_ATOMIC_T_LEN); if (p == NULL) { return NGX_ERROR; } v->len = ngx_sprintf(p, "%uA", r->connection->number) - p; v->valid = 1; v->no_cacheable = 0; v->not_found = 0; v->data = p; return NGX_OK; }
It returns NGX_ERROR
in case of internal error (for example, failed memory allocation) or NGX_OK
otherwise. To learn the status of variable evaluation, inspect the flags in ngx_http_variable_value_t
(see the description above).
The set
handler allows setting the property referenced by the variable. For example, the set handler of the $limit_rate
variable modifies the request's limit_rate
field:
... { ngx_string("limit_rate"), ngx_http_variable_request_set_size, ngx_http_variable_request_get_size, offsetof(ngx_http_request_t, limit_rate), NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOCACHEABLE, 0 }, ... static void ngx_http_variable_request_set_size(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { ssize_t s, *sp; ngx_str_t val; val.len = v->len; val.data = v->data; s = ngx_parse_size(&val); if (s == NGX_ERROR) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "invalid size \"%V\"", &val); return; } sp = (ssize_t *) ((char *) r + data); *sp = s; return; }
Complex values
A complex value, despite its name, provides an easy way to evaluate expressions which can contain text, variables, and their combination.
The complex value description in ngx_http_compile_complex_value
is compiled at the configuration stage into ngx_http_complex_value_t
which is used at runtime to obtain results of expression evaluation.
ngx_str_t *value; ngx_http_complex_value_t cv; ngx_http_compile_complex_value_t ccv; value = cf->args->elts; /* directive arguments */ ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); ccv.cf = cf; ccv.value = &value[1]; ccv.complex_value = &cv; ccv.zero = 1; ccv.conf_prefix = 1; if (ngx_http_compile_complex_value(&ccv) != NGX_OK) { return NGX_CONF_ERROR; }
Here, ccv
holds all parameters that are required to initialize the complex value cv
:
cf
— Configuration pointervalue
— String to be parsed (input)complex_value
— Compiled value (output)zero
— Flag that enables zero-terminating valueconf_prefix
— Prefixes the result with the configuration prefix (the directory where nginx is currently looking for configuration)root_prefix
— Prefixes the result with the root prefix (the normal nginx installation prefix)
The zero
flag is useful when results are to be passed to libraries that require zero-terminated strings, and prefixes are handy when dealing with filenames.
Upon successful compilation, cv.lengths
contains information about the presence of variables in the expression. The NULL value means that the expression contained static text only, and so can be stored in a simple string rather than as a complex value.
The ngx_http_set_complex_value_slot()
is a convenient function used to initialize a complex value completely in the directive declaration itself.
At runtime, a complex value can be calculated using the ngx_http_complex_value()
function:
ngx_str_t res; if (ngx_http_complex_value(r, &cv, &res) != NGX_OK) { return NGX_ERROR; }
Given the request r
and previously compiled value cv
, the function evaluates the expression and writes the result to res
.
Request redirection
An HTTP request is always connected to a location via the loc_conf
field of the ngx_http_request_t
structure. This means that at any point the location configuration of any module can be retrieved from the request by calling ngx_http_get_module_loc_conf(r, module)
. Request location can change several times during the request's lifetime. Initially, a default server location of the default server is assigned to a request. If the request switches to a different server (chosen by the HTTP “Host” header or SSL SNI extension), the request switches to the default location of that server as well. The next change of the location takes place at the NGX_HTTP_FIND_CONFIG_PHASE
request phase. At this phase a location is chosen by request URI among all non-named locations configured for the server. The ngx_http_rewrite_module can change the request URI at the NGX_HTTP_REWRITE_PHASE
request phase as a result of the rewrite directive and send the request back to the NGX_HTTP_FIND_CONFIG_PHASE
phase for selection of a new location based on the new URI.
It is also possible to redirect a request to a new location at any point by calling one of ngx_http_internal_redirect(r, uri, args)
or ngx_http_named_location(r, name)
.
The ngx_http_internal_redirect(r, uri, args)
function changes the request URI and returns the request to the NGX_HTTP_SERVER_REWRITE_PHASE
phase. The request proceeds with a server default location. Later at NGX_HTTP_FIND_CONFIG_PHASE
a new location is chosen based on the new request URI.
The following example performs an internal redirect with the new request arguments.
ngx_int_t ngx_http_foo_redirect(ngx_http_request_t *r) { ngx_str_t uri, args; ngx_str_set(&uri, "/foo"); ngx_str_set(&args, "bar=1"); return ngx_http_internal_redirect(r, &uri, &args); }
The function ngx_http_named_location(r, name)
redirects a request to a named location. The name of the location is passed as the argument. The location is looked up among all named locations of the current server, after which the requests switches to the NGX_HTTP_REWRITE_PHASE
phase.
The following example performs a redirect to a named location @foo.
ngx_int_t ngx_http_foo_named_redirect(ngx_http_request_t *r) { ngx_str_t name; ngx_str_set(&name, "foo"); return ngx_http_named_location(r, &name); }
Both functions - ngx_http_internal_redirect(r, uri, args)
and ngx_http_named_location(r, name)
can be called when nginx modules have already stored some contexts in a request's ctx
field. It's possible for these contexts to become inconsistent with the new location configuration. To prevent inconsistency, all request contexts are erased by both redirect functions.
Calling ngx_http_internal_redirect(r, uri, args)
or ngx_http_named_location(r, name)
increases the request count
. For consistent request reference counting, call ngx_http_finalize_request(r, NGX_DONE)
after redirecting the request. This will finalize current request code path and decrease the counter.
Redirected and rewritten requests become internal and can access the internal locations. Internal requests have the internal
flag set.
Subrequests
Subrequests are primarily used to insert output of one request into another, possibly mixed with other data. A subrequest looks like a normal request, but shares some data with its parent. In particular, all fields related to client input are shared because a subrequest does not receive any other input from the client. The request field parent
for a subrequest contains a link to its parent request and is NULL for the main request. The field main
contains a link to the main request in a group of requests.
A subrequest starts in the NGX_HTTP_SERVER_REWRITE_PHASE
phase. It passes through the same subsequent phases as a normal request and is assigned a location based on its own URI.
The output header in a subrequest is always ignored. The ngx_http_postpone_filter
places the subrequest's output body in the right position relative to other data produced by the parent request.
Subrequests are related to the concept of active requests. A request r
is considered active if c->data == r
, where c
is the client connection object. At any given point, only the active request in a request group is allowed to output its buffers to the client. An inactive request can still send its output to the filter chain, but it does not pass beyond the ngx_http_postpone_filter
and remains buffered by that filter until the request becomes active. Here are some rules of request activation:
- Initially, the main request is active.
- The first subrequest of an active request becomes active right after creation.
- The
ngx_http_postpone_filter
activates the next request in the active request's subrequest list, once all data prior to that request are sent. - When a request is finalized, its parent is activated.
Create a subrequest by calling the function ngx_http_subrequest(r, uri, args, psr, ps, flags)
, where r
is the parent request, uri
and args
are the URI and arguments of the subrequest, psr
is the output parameter, which receives the newly created subrequest reference, ps
is a callback object for notifying the parent request that the subrequest is being finalized, and flags
is bitmask of flags. The following flags are available:
NGX_HTTP_SUBREQUEST_IN_MEMORY
- Output is not sent to the client, but rather stored in memory. The flag only affects subrequests which are processed by one of the proxying modules. After a subrequest is finalized its output is available inr->out
of typengx_buf_t
.NGX_HTTP_SUBREQUEST_WAITED
- The subrequest'sdone
flag is set even if the subrequest is not active when it is finalized. This subrequest flag is used by the SSI filter.NGX_HTTP_SUBREQUEST_CLONE
- The subrequest is created as a clone of its parent. It is started at the same location and proceeds from the same phase as the parent request.
The following example creates a subrequest with the URI of /foo
.
ngx_int_t rc; ngx_str_t uri; ngx_http_request_t *sr; ... ngx_str_set(&uri, "/foo"); rc = ngx_http_subrequest(r, &uri, NULL, &sr, NULL, 0); if (rc == NGX_ERROR) { /* error */ }
This example clones the current request and sets a finalization callback for the subrequest.
ngx_int_t ngx_http_foo_clone(ngx_http_request_t *r) { ngx_http_request_t *sr; ngx_http_post_subrequest_t *ps; ps = ngx_palloc(r->pool, sizeof(ngx_http_post_subrequest_t)); if (ps == NULL) { return NGX_ERROR; } ps->handler = ngx_http_foo_subrequest_done; ps->data = "foo"; return ngx_http_subrequest(r, &r->uri, &r->args, &sr, ps, NGX_HTTP_SUBREQUEST_CLONE); } ngx_int_t ngx_http_foo_subrequest_done(ngx_http_request_t *r, void *data, ngx_int_t rc) { char *msg = (char *) data; ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, "done subrequest r:%p msg:%s rc:%i", r, msg, rc); return rc; }
Subrequests are normally created in a body filter, in which case their output can be treated like the output from any explicit request. This means that eventually the output of a subrequest is sent to the client, after all explicit buffers that are passed before subrequest creation and before any buffers that are passed after creation. This ordering is preserved even for large hierarchies of subrequests. The following example inserts output from a subrequest after all request data buffers, but before the final buffer with the last_buf
flag.
ngx_int_t ngx_http_foo_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_buf_t *b; ngx_uint_t last; ngx_chain_t *cl, out; ngx_http_request_t *sr; ngx_http_foo_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_foo_filter_module); if (ctx == NULL) { return ngx_http_next_body_filter(r, in); } last = 0; for (cl = in; cl; cl = cl->next) { if (cl->buf->last_buf) { cl->buf->last_buf = 0; cl->buf->last_in_chain = 1; cl->buf->sync = 1; last = 1; } } /* Output explicit output buffers */ rc = ngx_http_next_body_filter(r, in); if (rc == NGX_ERROR || !last) { return rc; } /* * Create the subrequest. The output of the subrequest * will automatically be sent after all preceding buffers, * but before the last_buf buffer passed later in this function. */ if (ngx_http_subrequest(r, ctx->uri, NULL, &sr, NULL, 0) != NGX_OK) { return NGX_ERROR; } ngx_http_set_ctx(r, NULL, ngx_http_foo_filter_module); /* Output the final buffer with the last_buf flag */ b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_ERROR; } b->last_buf = 1; out.buf = b; out.next = NULL; return ngx_http_output_filter(r, &out); }
A subrequest can also be created for other purposes than data output. For example, the ngx_http_auth_request_module module creates a subrequest at the NGX_HTTP_ACCESS_PHASE
phase. To disable output at this point, the header_only
flag is set on the subrequest. This prevents the subrequest body from being sent to the client. Note that the subrequest's header is never sent to the client. The result of the subrequest can be analyzed in the callback handler.
Request finalization
An HTTP request is finalized by calling the function ngx_http_finalize_request(r, rc)
. It is usually finalized by the content handler after all output buffers are sent to the filter chain. At this point all of the output might not be sent to the client, with some of it remaining buffered somewhere along the filter chain. If it is, the ngx_http_finalize_request(r, rc)
function automatically installs a special handler ngx_http_writer(r)
to finish sending the output. A request is also finalized in case of an error or if a standard HTTP response code needs to be returned to the client.
The function ngx_http_finalize_request(r, rc)
expects the following rc
values:
NGX_DONE
- Fast finalization. Decrement the requestcount
and destroy the request if it reaches zero. The client connection can be used for more requests after the current request is destroyed.NGX_ERROR
,NGX_HTTP_REQUEST_TIME_OUT
(408
),NGX_HTTP_CLIENT_CLOSED_REQUEST
(499
) - Error finalization. Terminate the request as soon as possible and close the client connection.NGX_HTTP_CREATED
(201
),NGX_HTTP_NO_CONTENT
(204
), codes greater than or equal toNGX_HTTP_SPECIAL_RESPONSE
(300
) - Special response finalization. For these values nginx either sends to the client a default response page for the code or performs the internal redirect to an error_page location if that is configured for the code.- Other codes are considered successful finalization codes and might activate the request writer to finish sending the response body. Once the body is completely sent, the request
count
is decremented. If it reaches zero, the request is destroyed, but the client connection can still be used for other requests. Ifcount
is positive, there are unfinished activities within the request, which will be finalized at a later point.
Request body
For dealing with the body of a client request, nginx provides the ngx_http_read_client_request_body(r, post_handler)
and ngx_http_discard_request_body(r)
functions. The first function reads the request body and makes it available via the request_body
request field. The second function instructs nginx to discard (read and ignore) the request body. One of these functions must be called for every request. Normally, the content handler makes the call.
Reading or discarding the client request body from a subrequest is not allowed. It must always be done in the main request. When a subrequest is created, it inherits the parent's request_body
object which can be used by the subrequest if the main request has previously read the request body.
The function ngx_http_read_client_request_body(r, post_handler)
starts the process of reading the request body. When the body is completely read, the post_handler
callback is called to continue processing the request. If the request body is missing or has already been read, the callback is called immediately. The function ngx_http_read_client_request_body(r, post_handler)
allocates the request_body
request field of type ngx_http_request_body_t
. The field bufs
of this object keeps the result as a buffer chain. The body can be saved in memory buffers or file buffers, if the capacity specified by the client_body_buffer_size directive is not enough to fit the entire body in memory.
The following example reads a client request body and returns its size.
ngx_int_t ngx_http_foo_content_handler(ngx_http_request_t *r) { ngx_int_t rc; rc = ngx_http_read_client_request_body(r, ngx_http_foo_init); if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { /* error */ return rc; } return NGX_DONE; } void ngx_http_foo_init(ngx_http_request_t *r) { off_t len; ngx_buf_t *b; ngx_int_t rc; ngx_chain_t *in, out; if (r->request_body == NULL) { ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } len = 0; for (in = r->request_body->bufs; in; in = in->next) { len += ngx_buf_size(in->buf); } b = ngx_create_temp_buf(r->pool, NGX_OFF_T_LEN); if (b == NULL) { ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } b->last = ngx_sprintf(b->pos, "%O", len); b->last_buf = (r == r->main) ? 1: 0; b->last_in_chain = 1; r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = b->last - b->pos; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { ngx_http_finalize_request(r, rc); return; } out.buf = b; out.next = NULL; rc = ngx_http_output_filter(r, &out); ngx_http_finalize_request(r, rc); }
The following fields of the request determine how the request body is read:
request_body_in_single_buf
- Read the body to a single memory buffer.request_body_in_file_only
- Always read the body to a file, even if fits in the memory buffer.request_body_in_persistent_file
- Do not unlink the file immediately after creation. A file with this flag can be moved to another directory.request_body_in_clean_file
- Unlink the file when the request is finalized. This can be useful when a file was supposed to be moved to another directory but was not moved for some reason.request_body_file_group_access
- Enable group access to the file by replacing the default 0600 access mask with 0660.request_body_file_log_level
- Severity level at which to log file errors.request_body_no_buffering
- Read the request body without buffering.
The request_body_no_buffering
flag enables the unbuffered mode of reading a request body. In this mode, after calling ngx_http_read_client_request_body()
, the bufs
chain might keep only a part of the body. To read the next part, call the ngx_http_read_unbuffered_request_body(r)
function. The return value NGX_AGAIN
and the request flag reading_body
indicate that more data is available. If bufs
is NULL after calling this function, there is nothing to read at the moment. The request callback read_event_handler
will be called when the next part of request body is available.
Response
In nginx an HTTP response is produced by sending the response header followed by the optional response body. Both header and body are passed through a chain of filters and eventually get written to the client socket. An nginx module can install its handler into the header or body filter chain and process the output coming from the previous handler.
Response header
The ngx_http_send_header(r)
function sends the output header. Do not call this function until r->headers_out
contains all of the data required to produce the HTTP response header. The status
field in r->headers_out
must always be set. If the response status indicates that a response body follows the header, content_length_n
can be set as well. The default value for this field is -1
, which means that the body size is unknown. In this case, chunked transfer encoding is used. To output an arbitrary header, append the headers
list.
static ngx_int_t ngx_http_foo_content_handler(ngx_http_request_t *r) { ngx_int_t rc; ngx_table_elt_t *h; /* send header */ r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = 3; /* X-Foo: foo */ h = ngx_list_push(&r->headers_out.headers); if (h == NULL) { return NGX_ERROR; } h->hash = 1; ngx_str_set(&h->key, "X-Foo"); ngx_str_set(&h->value, "foo"); rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; } /* send body */ ... }
Header filters
The ngx_http_send_header(r)
function invokes the header filter chain by calling the first header filter handler stored in the ngx_http_top_header_filter
variable. It's assumed that every header handler calls the next handler in the chain until the final handler ngx_http_header_filter(r)
is called. The final header handler constructs the HTTP response based on r->headers_out
and passes it to the ngx_http_writer_filter
for output.
To add a handler to the header filter chain, store its address in the global variable ngx_http_top_header_filter
at configuration time. The previous handler address is normally stored in a static variable in a module and is called by the newly added handler before exiting.
The following example of a header filter module adds the HTTP header "X-Foo: foo
" to every response with status 200
.
#include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> static ngx_int_t ngx_http_foo_header_filter(ngx_http_request_t *r); static ngx_int_t ngx_http_foo_header_filter_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_foo_header_filter_module_ctx = { NULL, /* preconfiguration */ ngx_http_foo_header_filter_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_foo_header_filter_module = { NGX_MODULE_V1, &ngx_http_foo_header_filter_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_output_header_filter_pt ngx_http_next_header_filter; static ngx_int_t ngx_http_foo_header_filter(ngx_http_request_t *r) { ngx_table_elt_t *h; /* * The filter handler adds "X-Foo: foo" header * to every HTTP 200 response */ if (r->headers_out.status != NGX_HTTP_OK) { return ngx_http_next_header_filter(r); } h = ngx_list_push(&r->headers_out.headers); if (h == NULL) { return NGX_ERROR; } h->hash = 1; ngx_str_set(&h->key, "X-Foo"); ngx_str_set(&h->value, "foo"); return ngx_http_next_header_filter(r); } static ngx_int_t ngx_http_foo_header_filter_init(ngx_conf_t *cf) { ngx_http_next_header_filter = ngx_http_top_header_filter; ngx_http_top_header_filter = ngx_http_foo_header_filter; return NGX_OK; }
Response body
To send the response body, call the ngx_http_output_filter(r, cl)
function. The function can be called multiple times. Each time, it sends a part of the response body in the form of a buffer chain. Set the last_buf
flag in the last body buffer.
The following example produces a complete HTTP response with "foo" as its body. For the example to work as subrequest as well as a main request, the last_in_chain
flag is set in the last buffer of the output. The last_buf
flag is set only for the main request because the last buffer for a subrequest does not end the entire output.
static ngx_int_t ngx_http_bar_content_handler(ngx_http_request_t *r) { ngx_int_t rc; ngx_buf_t *b; ngx_chain_t out; /* send header */ r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = 3; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; } /* send body */ b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_ERROR; } b->last_buf = (r == r->main) ? 1: 0; b->last_in_chain = 1; b->memory = 1; b->pos = (u_char *) "foo"; b->last = b->pos + 3; out.buf = b; out.next = NULL; return ngx_http_output_filter(r, &out); }
Body filters
The function ngx_http_output_filter(r, cl)
invokes the body filter chain by calling the first body filter handler stored in the ngx_http_top_body_filter
variable. It's assumed that every body handler calls the next handler in the chain until the final handler ngx_http_write_filter(r, cl)
is called.
A body filter handler receives a chain of buffers. The handler is supposed to process the buffers and pass a possibly new chain to the next handler. It's worth noting that the chain links ngx_chain_t
of the incoming chain belong to the caller, and must not be reused or changed. Right after the handler completes, the caller can use its output chain links to keep track of the buffers it has sent. To save the buffer chain or to substitute some buffers before passing to the next filter, a handler needs to allocate its own chain links.
Following is an example of a simple body filter that counts the number of bytes in the body. The result is available as the $counter
variable which can be used in the access log.
#include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> typedef struct { off_t count; } ngx_http_counter_filter_ctx_t; static ngx_int_t ngx_http_counter_body_filter(ngx_http_request_t *r, ngx_chain_t *in); static ngx_int_t ngx_http_counter_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_counter_add_variables(ngx_conf_t *cf); static ngx_int_t ngx_http_counter_filter_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_counter_filter_module_ctx = { ngx_http_counter_add_variables, /* preconfiguration */ ngx_http_counter_filter_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_counter_filter_module = { NGX_MODULE_V1, &ngx_http_counter_filter_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_output_body_filter_pt ngx_http_next_body_filter; static ngx_str_t ngx_http_counter_name = ngx_string("counter"); static ngx_int_t ngx_http_counter_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_chain_t *cl; ngx_http_counter_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_counter_filter_ctx_t)); if (ctx == NULL) { return NGX_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_counter_filter_module); } for (cl = in; cl; cl = cl->next) { ctx->count += ngx_buf_size(cl->buf); } return ngx_http_next_body_filter(r, in); } static ngx_int_t ngx_http_counter_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { u_char *p; ngx_http_counter_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module); if (ctx == NULL) { v->not_found = 1; return NGX_OK; } p = ngx_pnalloc(r->pool, NGX_OFF_T_LEN); if (p == NULL) { return NGX_ERROR; } v->data = p; v->len = ngx_sprintf(p, "%O", ctx->count) - p; v->valid = 1; v->no_cacheable = 0; v->not_found = 0; return NGX_OK; } static ngx_int_t ngx_http_counter_add_variables(ngx_conf_t *cf) { ngx_http_variable_t *var; var = ngx_http_add_variable(cf, &ngx_http_counter_name, 0); if (var == NULL) { return NGX_ERROR; } var->get_handler = ngx_http_counter_variable; return NGX_OK; } static ngx_int_t ngx_http_counter_filter_init(ngx_conf_t *cf) { ngx_http_next_body_filter = ngx_http_top_body_filter; ngx_http_top_body_filter = ngx_http_counter_body_filter; return NGX_OK; }
Building filter modules
When writing a body or header filter, pay special attention to the filter's position in the filter order. There's a number of header and body filters registered by nginx standard modules. The nginx standard modules register a number of head and body filters and it's important to register a new filter module in the right place with respect to them. Normally, modules register filters in their postconfiguration handlers. The order in which filters are called during processing is obviously the reverse of the order in which they are registered.
For third-party filter modules nginx provides a special slot HTTP_AUX_FILTER_MODULES
. To register a filter module in this slot, set the ngx_module_type
variable to HTTP_AUX_FILTER
in the module's configuration.
The following example shows a filter module config file assuming for a module with just one source file, ngx_http_foo_filter_module.c
.
ngx_module_type=HTTP_AUX_FILTER ngx_module_name=ngx_http_foo_filter_module ngx_module_srcs="$ngx_addon_dir/ngx_http_foo_filter_module.c" . auto/module
Buffer reuse
When issuing or altering a stream of buffers, it's often desirable to reuse the allocated buffers. A standard and widely adopted approach in nginx code is to keep two buffer chains for this purpose:
free
and busy
. The free
chain keeps all free buffers, which can be reused. The busy
chain keeps all buffers sent by the current module that are still in use by some other filter handler. A buffer is considered in use if its size is greater than zero. Normally, when a buffer is consumed by a filter, its pos
(or file_pos
for a file buffer) is moved towards last
(file_last
for a file buffer). Once a buffer is completely consumed, it's ready to be reused. To add newly freed buffers to the free
chain it's enough to iterate over the busy
chain and move the zero size buffers at the head of it to free
. This operation is so common that there is a special function for it, ngx_chain_update_chains(free, busy, out, tag)
. The function appends the output chain out
to busy
and moves free buffers from the top of busy
to free
. Only the buffers with the specified tag
are reused. This lets a module reuse only the buffers that it allocated itself.
The following example is a body filter that inserts the string “foo” before each incoming buffer. The new buffers allocated by the module are reused if possible. Note that for this example to work properly, setting up a header filter and resetting content_length_n
to -1
is also required, but the relevant code is not provided here.
typedef struct { ngx_chain_t *free; ngx_chain_t *busy; } ngx_http_foo_filter_ctx_t; ngx_int_t ngx_http_foo_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_buf_t *b; ngx_chain_t *cl, *tl, *out, **ll; ngx_http_foo_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_foo_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_foo_filter_ctx_t)); if (ctx == NULL) { return NGX_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_foo_filter_module); } /* create a new chain "out" from "in" with all the changes */ ll = &out; for (cl = in; cl; cl = cl->next) { /* append "foo" in a reused buffer if possible */ tl = ngx_chain_get_free_buf(r->pool, &ctx->free); if (tl == NULL) { return NGX_ERROR; } b = tl->buf; b->tag = (ngx_buf_tag_t) &ngx_http_foo_filter_module; b->memory = 1; b->pos = (u_char *) "foo"; b->last = b->pos + 3; *ll = tl; ll = &tl->next; /* append the next incoming buffer */ tl = ngx_alloc_chain_link(r->pool); if (tl == NULL) { return NGX_ERROR; } tl->buf = cl->buf; *ll = tl; ll = &tl->next; } *ll = NULL; /* send the new chain */ rc = ngx_http_next_body_filter(r, out); /* update "busy" and "free" chains for reuse */ ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &out, (ngx_buf_tag_t) &ngx_http_foo_filter_module); return rc; }
Load balancing
The ngx_http_upstream_module provides the basic functionality needed to pass requests to remote servers. Modules that implement specific protocols, such as HTTP or FastCGI, use this functionality. The module also provides an interface for creating custom load-balancing modules and implements a default round-robin method.
The least_conn and hash modules implement alternative load-balancing methods, but are actually implemented as extensions of the upstream round-robin module and share a lot of code with it, such as the representation of a server group. The keepalive module is an independent module that extends upstream functionality.
The ngx_http_upstream_module can be configured explicitly by placing the corresponding upstream block into the configuration file, or implicitly by using directives such as proxy_pass that accept a URL that gets evaluated at some point into a list of servers. The alternative load-balancing methods are available only with an explicit upstream configuration. The upstream module configuration has its own directive context NGX_HTTP_UPS_CONF
. The structure is defined as follows:
struct ngx_http_upstream_srv_conf_s { ngx_http_upstream_peer_t peer; void **srv_conf; ngx_array_t *servers; /* ngx_http_upstream_server_t */ ngx_uint_t flags; ngx_str_t host; u_char *file_name; ngx_uint_t line; in_port_t port; ngx_uint_t no_port; /* unsigned no_port:1 */ #if (NGX_HTTP_UPSTREAM_ZONE) ngx_shm_zone_t *shm_zone; #endif };
srv_conf
— Configuration context of upstream modules.servers
— Array ofngx_http_upstream_server_t
, the result of parsing a set of server directives in theupstream
block.flags
— Flags that mostly mark which features are supported by the load-balancing method. The features are configured as parameters of the server directive:NGX_HTTP_UPSTREAM_CREATE
— Distinguishes explicitly defined upstreams from those that are automatically created by the proxy_pass directive and “friends” (FastCGI, SCGI, etc.)NGX_HTTP_UPSTREAM_WEIGHT
— The “weight
” parameter is supportedNGX_HTTP_UPSTREAM_MAX_FAILS
— The “max_fails
” parameter is supportedNGX_HTTP_UPSTREAM_FAIL_TIMEOUT
— The “fail_timeout
” parameter is supportedNGX_HTTP_UPSTREAM_DOWN
— The “down
” parameter is supportedNGX_HTTP_UPSTREAM_BACKUP
— The “backup
” parameter is supportedNGX_HTTP_UPSTREAM_MAX_CONNS
— The “max_conns
” parameter is supported
host
— Name of the upstream.file_name, line
— Name of the configuration file and the line where theupstream
block is located.port
andno_port
— Not used for explicitly defined upstream groups.shm_zone
— Shared memory zone used by this upstream group, if any.peer
— object that holds generic methods for initializing upstream configuration:
A module that implements a load-balancing algorithm must set these methods and initialize privatetypedef struct { ngx_http_upstream_init_pt init_upstream; ngx_http_upstream_init_peer_pt init; void *data; } ngx_http_upstream_peer_t;
data
. Ifinit_upstream
was not initialized during configuration parsing,ngx_http_upstream_module
sets it to the defaultngx_http_upstream_init_round_robin
algorithm.init_upstream(cf, us)
— Configuration-time method responsible for initializing a group of servers and initializing theinit()
method in case of success. A typical load-balancing module uses a list of servers in theupstream
block to create an efficient data structure that it uses and saves its own configuration to thedata
field.init(r, us)
— Initializes a per-requestngx_http_upstream_peer_t.peer
structure that is used for load balancing (not to be confused with thengx_http_upstream_srv_conf_t.peer
described above which is per-upstream). It is passed as thedata
argument to all callbacks that deal with server selection.
When nginx has to pass a request to another host for processing, it uses the configured load-balancing method to obtain an address to connect to. The method is obtained from the ngx_http_upstream_t.peer
object of type ngx_peer_connection_t
:
struct ngx_peer_connection_s { ... struct sockaddr *sockaddr; socklen_t socklen; ngx_str_t *name; ngx_uint_t tries; ngx_event_get_peer_pt get; ngx_event_free_peer_pt free; ngx_event_notify_peer_pt notify; void *data; #if (NGX_SSL || NGX_COMPAT) ngx_event_set_peer_session_pt set_session; ngx_event_save_peer_session_pt save_session; #endif ... };
The structure has the following fields:
sockaddr
,socklen
,name
— Address of the upstream server to connect to; this is the output parameter of a load-balancing method.data
— The per-request data of a load-balancing method; keeps the state of the selection algorithm and usually includes the link to the upstream configuration. It is passed as an argument to all methods that deal with server selection (see below).tries
— Allowed number of attempts to connect to an upstream server.get
,free
,notify
,set_session
, andsave_session
- Methods of the load-balancing module, described below.
All methods accept at least two arguments: a peer connection object pc
and the data
created by ngx_http_upstream_srv_conf_t.peer.init()
. Note that it might differ from pc.data
due to “chaining” of load-balancing modules.
get(pc, data)
— The method called when the upstream module is ready to pass a request to an upstream server and needs to know its address. The method has to fill thesockaddr
,socklen
, andname
fields ofngx_peer_connection_t
structure. The return is one of:NGX_OK
— Server was selected.NGX_ERROR
— Internal error occurred.NGX_BUSY
— no servers are currently available. This can happen due to many reasons, including: the dynamic server group is empty, all servers in the group are in the failed state, or all servers in the group are already handling the maximum number of connections.NGX_DONE
— the underlying connection was reused and there is no need to create a new connection to the upstream server. This value is set by thekeepalive
module.
free(pc, data, state)
— The method called when an upstream module has finished work with a particular server. Thestate
argument is the completion status of the upstream connection, a bitmask with the following possible values:NGX_PEER_FAILED
— Attempt was unsuccessfulNGX_PEER_NEXT
— A special case when upstream server returns codes403
or404
, which are not considered a failure.NGX_PEER_KEEPALIVE
— Currently unused
tries
counter.notify(pc, data, type)
— Currently unused in the OSS version.set_session(pc, data)
andsave_session(pc, data)
— SSL-specific methods that enable caching sessions to upstream servers. The implementation is provided by the round-robin balancing method.
Examples
The nginx-dev-examples repository provides nginx module examples.
Code style
General rules
- maximum text width is 80 characters
- indentation is 4 spaces
- no tabs, no trailing spaces
- list elements on the same line are separated with spaces
- hexadecimal literals are lowercase
- file names, function and type names, and global variables have the
ngx_
or more specific prefix such asngx_http_
andngx_mail_
size_t ngx_utf8_length(u_char *p, size_t n) { u_char c, *last; size_t len; last = p + n; for (len = 0; p < last; len++) { c = *p; if (c < 0x80) { p++; continue; } if (ngx_utf8_decode(&p, last - p) > 0x10ffff) { /* invalid UTF-8 */ return n; } } return len; }
Files
A typical source file may contain the following sections separated by two empty lines:
- copyright statements
- includes
- preprocessor definitions
- type definitions
- function prototypes
- variable definitions
- function definitions
Copyright statements look like this:
/* * Copyright (C) Author Name * Copyright (C) Organization, Inc. */
If the file is modified significantly, the list of authors should be updated, the new author is added to the top.
The ngx_config.h
and ngx_core.h
files are always included first, followed by one of ngx_http.h
, ngx_stream.h
, or ngx_mail.h
. Then follow optional external header files:
#include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> #include <libxml/parser.h> #include <libxml/tree.h> #include <libxslt/xslt.h> #if (NGX_HAVE_EXSLT) #include <libexslt/exslt.h> #endif
Header files should include the so called "header protection":
#ifndef _NGX_PROCESS_CYCLE_H_INCLUDED_ #define _NGX_PROCESS_CYCLE_H_INCLUDED_ ... #endif /* _NGX_PROCESS_CYCLE_H_INCLUDED_ */
Comments
- “
//
” comments are not used - text is written in English, American spelling is preferred
- multi-line comments are formatted like this:
/* * The red-black tree code is based on the algorithm described in * the "Introduction to Algorithms" by Cormen, Leiserson and Rivest. */
/* find the server configuration for the address:port */
Preprocessor
Macro names start from ngx_
or NGX_
(or more specific) prefix. Macro names for constants are uppercase. Parameterized macros and macros for initializers are lowercase. The macro name and value are separated by at least two spaces:
#define NGX_CONF_BUFFER 4096 #define ngx_buf_in_memory(b) (b->temporary || b->memory || b->mmap) #define ngx_buf_size(b) \ (ngx_buf_in_memory(b) ? (off_t) (b->last - b->pos): \ (b->file_last - b->file_pos)) #define ngx_null_string { 0, NULL }
Conditions are inside parentheses, negation is outside:
#if (NGX_HAVE_KQUEUE) ... #elif ((NGX_HAVE_DEVPOLL && !(NGX_TEST_BUILD_DEVPOLL)) \ || (NGX_HAVE_EVENTPORT && !(NGX_TEST_BUILD_EVENTPORT))) ... #elif (NGX_HAVE_EPOLL && !(NGX_TEST_BUILD_EPOLL)) ... #elif (NGX_HAVE_POLL) ... #else /* select */ ... #endif /* NGX_HAVE_KQUEUE */
Types
Type names end with the “_t
” suffix. A defined type name is separated by at least two spaces:
typedef ngx_uint_t ngx_rbtree_key_t;
Structure types are defined using typedef
. Inside structures, member types and names are aligned:
typedef struct { size_t len; u_char *data; } ngx_str_t;
Keep alignment identical among different structures in the file. A structure that points to itself has the name, ending with “_s
”. Adjacent structure definitions are separated with two empty lines:
typedef struct ngx_list_part_s ngx_list_part_t; struct ngx_list_part_s { void *elts; ngx_uint_t nelts; ngx_list_part_t *next; }; typedef struct { ngx_list_part_t *last; ngx_list_part_t part; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_list_t;
Each structure member is declared on its own line:
typedef struct { ngx_uint_t hash; ngx_str_t key; ngx_str_t value; u_char *lowcase_key; } ngx_table_elt_t;
Function pointers inside structures have defined types ending with “_pt
”:
typedef ssize_t (*ngx_recv_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ssize_t (*ngx_recv_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); typedef ssize_t (*ngx_send_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ngx_chain_t *(*ngx_send_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); typedef struct { ngx_recv_pt recv; ngx_recv_chain_pt recv_chain; ngx_recv_pt udp_recv; ngx_send_pt send; ngx_send_pt udp_send; ngx_send_chain_pt udp_send_chain; ngx_send_chain_pt send_chain; ngx_uint_t flags; } ngx_os_io_t;
Enumerations have types ending with “_e
”:
typedef enum { ngx_http_fastcgi_st_version = 0, ngx_http_fastcgi_st_type, ... ngx_http_fastcgi_st_padding } ngx_http_fastcgi_state_e;
Variables
Variables are declared sorted by length of a base type, then alphabetically. Type names and variable names are aligned. The type and name “columns” are separated with two spaces. Large arrays are put at the end of a declaration block:
u_char | | *rv, *p; ngx_conf_t | | *cf; ngx_uint_t | | i, j, k; unsigned int | | len; struct sockaddr | | *sa; const unsigned char | | *data; ngx_peer_connection_t | | *pc; ngx_http_core_srv_conf_t | |**cscfp; ngx_http_upstream_srv_conf_t| | *us, *uscf; u_char | | text[NGX_SOCKADDR_STRLEN];
Static and global variables may be initialized on declaration:
static ngx_str_t ngx_http_memcached_key = ngx_string("memcached_key");
static ngx_uint_t mday[] = { 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 };
static uint32_t ngx_crc32_table16[] = { 0x00000000, 0x1db71064, 0x3b6e20c8, 0x26d930ac, ... 0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c };
There is a bunch of commonly used type/name combinations:
u_char *rv; ngx_int_t rc; ngx_conf_t *cf; ngx_connection_t *c; ngx_http_request_t *r; ngx_peer_connection_t *pc; ngx_http_upstream_srv_conf_t *us, *uscf;
Functions
All functions (even static ones) should have prototypes. Prototypes include argument names. Long prototypes are wrapped with a single indentation on continuation lines:
static char *ngx_http_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static ngx_int_t ngx_http_init_phases(ngx_conf_t *cf, ngx_http_core_main_conf_t *cmcf); static char *ngx_http_merge_servers(ngx_conf_t *cf, ngx_http_core_main_conf_t *cmcf, ngx_http_module_t *module, ngx_uint_t ctx_index);
The function name in a definition starts with a new line. The function body opening and closing braces are on separate lines. The body of a function is indented. There are two empty lines between functions:
static ngx_int_t ngx_http_find_virtual_server(ngx_http_request_t *r, u_char *host, size_t len) { ... } static ngx_int_t ngx_http_add_addresses(ngx_conf_t *cf, ngx_http_core_srv_conf_t *cscf, ngx_http_conf_port_t *port, ngx_http_listen_opt_t *lsopt) { ... }
There is no space after the function name and opening parenthesis. Long function calls are wrapped such that continuation lines start from the position of the first function argument. If this is impossible, format the first continuation line such that it ends at position 79:
ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http header: \"%V: %V\"", &h->key, &h->value); hc->busy = ngx_palloc(r->connection->pool, cscf->large_client_header_buffers.num * sizeof(ngx_buf_t *));
The ngx_inline
macro should be used instead of inline
:
static ngx_inline void ngx_cpuid(uint32_t i, uint32_t *buf);
Expressions
Binary operators except “.
” and “−>
”
should be separated from their operands by one space. Unary operators and subscripts are not separated from their operands by spaces:
width = width * 10 + (*fmt++ - '0');
ch = (u_char) ((decoded << 4) + (ch - '0'));
r->exten.data = &r->uri.data[i + 1];
Type casts are separated by one space from casted expressions. An asterisk inside type cast is separated with space from type name:
len = ngx_sock_ntop((struct sockaddr *) sin6, p, len, 1);
If an expression does not fit into single line, it is wrapped. The preferred point to break a line is a binary operator. The continuation line is lined up with the start of expression:
if (status == NGX_HTTP_MOVED_PERMANENTLY || status == NGX_HTTP_MOVED_TEMPORARILY || status == NGX_HTTP_SEE_OTHER || status == NGX_HTTP_TEMPORARY_REDIRECT || status == NGX_HTTP_PERMANENT_REDIRECT) { ... }
p->temp_file->warn = "an upstream response is buffered " "to a temporary file";
As a last resort, it is possible to wrap an expression so that the continuation line ends at position 79:
hinit->hash = ngx_pcalloc(hinit->pool, sizeof(ngx_hash_wildcard_t) + size * sizeof(ngx_hash_elt_t *));
The above rules also apply to sub-expressions, where each sub-expression has its own indentation level:
if (((u->conf->cache_use_stale & NGX_HTTP_UPSTREAM_FT_UPDATING) || c->stale_updating) && !r->background && u->conf->cache_background_update) { ... }
Sometimes, it is convenient to wrap an expression after a cast. In this case, the continuation line is indented:
node = (ngx_rbtree_node_t *) ((u_char *) lr - offsetof(ngx_rbtree_node_t, color));
Pointers are explicitly compared to NULL
(not 0
):
if (ptr != NULL) { ... }
Conditionals and Loops
The “if
” keyword is separated from the condition by one space. Opening brace is located on the same line, or on a dedicated line if the condition takes several lines. Closing brace is located on a dedicated line, optionally followed by “else if
/ else
”. Usually, there is an empty line before the “else if
/ else
” part:
if (node->left == sentinel) { temp = node->right; subst = node; } else if (node->right == sentinel) { temp = node->left; subst = node; } else { subst = ngx_rbtree_min(node->right, sentinel); if (subst->left != sentinel) { temp = subst->left; } else { temp = subst->right; } }
Similar formatting rules are applied to “do
”
and “while
” loops:
while (p < last && *p == ' ') { p++; }
do { ctx->node = rn; ctx = ctx->next; } while (ctx);
The “switch
” keyword is separated from the condition by one space. Opening brace is located on the same line. Closing brace is located on a dedicated line. The “case
” keywords are lined up with “switch
”:
switch (ch) { case '!': looked = 2; state = ssi_comment0_state; break; case '<': copy_end = p; break; default: copy_end = p; looked = 0; state = ssi_start_state; break; }
Most “for
” loops are formatted like this:
for (i = 0; i < ccf->env.nelts; i++) { ... }
for (q = ngx_queue_head(locations); q != ngx_queue_sentinel(locations); q = ngx_queue_next(q)) { ... }
If some part of the “for
” statement is omitted, this is indicated by the “/* void */
” comment:
for (i = 0; /* void */ ; i++) { ... }
A loop with an empty body is also indicated by the “/* void */
” comment which may be put on the same line:
for (cl = *busy; cl->next; cl = cl->next) { /* void */ }
An endless loop looks like this:
for ( ;; ) { ... }
Labels
Labels are surrounded with empty lines and are indented at the previous level:
if (i == 0) { u->err = "host not found"; goto failed; } u->addrs = ngx_pcalloc(pool, i * sizeof(ngx_addr_t)); if (u->addrs == NULL) { goto failed; } u->naddrs = i; ... return NGX_OK; failed: freeaddrinfo(res); return NGX_ERROR;
Debugging memory issues
To debug memory issues such as buffer overruns or use-after-free errors, you can use the AddressSanitizer (ASan) supported by some modern compilers. To enable ASan with gcc
and clang
, use the -fsanitize=address
compiler and linker option. When building nginx, this can be done by adding the option to --with-cc-opt
and --with-ld-opt
parameters of the configure
script.
Since most allocations in nginx are made from nginx internal pool, enabling ASan may not always be enough to debug memory issues. The internal pool allocates a big chunk of memory from the system and cuts smaller allocations from it. However, this mechanism can be disabled by setting the NGX_DEBUG_PALLOC
macro to 1
. In this case, allocations are passed directly to the system allocator giving it full control over the buffers boundaries.
The following configuration line summarizes the information provided above. It is recommended while developing third-party modules and testing nginx on different platforms.
auto/configure --with-cc-opt='-fsanitize=address -DNGX_DEBUG_PALLOC=1' --with-ld-opt=-fsanitize=address
Common Pitfalls
Writing a C module
The most common pitfall is an attempt to write a full-fledged C module when it can be avoided. In most cases your task can be accomplished by creating a proper configuration. If writing a module is inevitable, try to make it as small and simple as possible. For example, a module can only export some variables.
Before starting a module, consider the following questions:
- Is it possible to implement a desired feature using already available modules?
- Is it possible to solve an issue using built-in scripting languages, such as Perl or njs?
C Strings
The most used string type in nginx, ngx_str_t is not a C-Style zero-terminated string. You cannot pass the data to standard C library functions such as strlen()
or strstr()
. Instead, nginx counterparts that accept either ngx_str_t
should be used or pointer to data and a length. However, there is a case when ngx_str_t
holds a pointer to a zero-terminated string: strings that come as a result of configuration file parsing are zero-terminated.
Global Variables
Avoid using global variables in your modules. Most likely this is an error to have a global variable. Any global data should be tied to a configuration cycle and be allocated from the corresponding memory pool. This allows nginx to perform graceful configuration reloads. An attempt to use global variables will likely break this feature, because it will be impossible to have two configurations at the same time and get rid of them. Sometimes global variables are required. In this case, special attention is needed to manage reconfiguration properly. Also, check if libraries used by your code have implicit global state that may be broken on reload.
Manual Memory Management
Instead of dealing with malloc/free approach which is error prone, learn how to use nginx pools. A pool is created and tied to an object - configuration, cycle, connection, or HTTP request. When the object is destroyed, the associated pool is destroyed too. So when working with an object, it is possible to allocate the amount needed from the corresponding pool and don't care about freeing memory even in case of errors.
Threads
It is recommended to avoid using threads in nginx because it will definitely break things: most nginx functions are not thread-safe. It is expected that a thread will be executing only system calls and thread-safe library functions. If you need to run some code that is not related to client request processing, the proper way is to schedule a timer in the init_process
module handler and perform required actions in timer handler. Internally nginx makes use of threads to boost IO-related operations, but this is a special case with a lot of limitations.
Blocking Libraries
A common mistake is to use libraries that are blocking internally. Most libraries out there are synchronous and blocking by nature. In other words, they perform one operation at a time and waste time waiting for response from other peer. As a result, when a request is processed with such library, whole nginx worker is blocked, thus destroying performance. Use only libraries that provide asynchronous interface and don't block whole process.
HTTP Requests to External Services
Often modules need to perform an HTTP call to some external service. A common mistake is to use some external library, such as libcurl, to perform the HTTP request. It is absolutely unnecessary to bring a huge amount of external (probably blocking!) code for the task which can be accomplished by nginx itself.
There are two basic usage scenarios when an external request is needed:
- in the context of processing a client request (for example, in content handler)
- in the context of a worker process (for example, timer handler)
In the first case, the best is to use subrequests API. Instead of directly accessing external service, you declare a location in nginx configuration and direct your subrequest to this location. This location is not limited to proxying requests, but may contain other nginx directives. An example of such approach is the auth_request directive implemented in ngx_http_auth_request module.
For the second case, it is possible to use basic HTTP client functionality available in nginx. For example, OCSP module implements simple HTTP client.