This memory leak was found by running the Valgrind testing mode against
lua-resty-core's `ssl-session-fetch.t` test suite:
TEST 5: yield during doing handshake with client which uses low version OpenSSL
==16956== 64 (32 direct, 32 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 15
==16956== at 0x4C2B002: malloc (vg_replace_malloc.c:298)
==16956== by 0x5FFC868: CRYPTO_malloc (mem.c:222)
==16956== by 0x5FFC96F: CRYPTO_zalloc (mem.c:230)
==16956== by 0x603C54A: OPENSSL_sk_new_reserve (stack.c:209)
==16956== by 0x603C597: OPENSSL_sk_new_null (stack.c:118)
==16956== by 0x5C94A86: sk_SSL_CIPHER_new_null (ssl.h:960)
==16956== by 0x5C94A86: bytes_to_cipher_list (ssl_lib.c:5361)
==16956== by 0x5CB52E9: tls_early_post_process_client_hello (statem_srvr.c:1713)
==16956== by 0x5CB52E9: tls_post_process_client_hello (statem_srvr.c:2231)
==16956== by 0x5CB6F39: ossl_statem_server_post_process_message (statem_srvr.c:1218)
==16956== by 0x5CA4C11: read_state_machine (statem.c:664)
==16956== by 0x5CA4C11: state_machine (statem.c:434)
==16956== by 0x5CA538A: ossl_statem_accept (statem.c:255)
==16956== by 0x5C91759: SSL_do_handshake (ssl_lib.c:3609)
==16956== by 0x45456B: ngx_ssl_handshake (ngx_event_openssl.c:1606)
==16956== by 0x4698D3: ngx_http_ssl_handshake (ngx_http_request.c:751)
==16956== by 0x44ECA8: ngx_epoll_process_events (ngx_epoll_module.c:901)
==16956== by 0x443E94: ngx_process_events_and_timers (ngx_event.c:257)
==16956== by 0x44DC25: ngx_single_process_cycle (ngx_process_cycle.c:333)
==16956== by 0x4236AB: main (nginx.c:382)
==16956==
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
fun:CRYPTO_malloc
fun:CRYPTO_zalloc
fun:OPENSSL_sk_new_reserve
fun:OPENSSL_sk_new_null
fun:sk_SSL_CIPHER_new_null
fun:bytes_to_cipher_list
fun:tls_early_post_process_client_hello
fun:tls_post_process_client_hello
fun:ossl_statem_server_post_process_message
fun:read_state_machine
fun:state_machine
fun:ossl_statem_accept
fun:SSL_do_handshake
fun:ngx_ssl_handshake
fun:ngx_http_ssl_handshake
fun:ngx_epoll_process_events
fun:ngx_process_events_and_timers
fun:ngx_single_process_cycle
fun:main
}
==16956== 368 (32 direct, 336 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 15
==16956== at 0x4C2B002: malloc (vg_replace_malloc.c:298)
==16956== by 0x5FFC868: CRYPTO_malloc (mem.c:222)
==16956== by 0x5FFC96F: CRYPTO_zalloc (mem.c:230)
==16956== by 0x603C54A: OPENSSL_sk_new_reserve (stack.c:209)
==16956== by 0x603C597: OPENSSL_sk_new_null (stack.c:118)
==16956== by 0x5C94A79: sk_SSL_CIPHER_new_null (ssl.h:960)
==16956== by 0x5C94A79: bytes_to_cipher_list (ssl_lib.c:5360)
==16956== by 0x5CB52E9: tls_early_post_process_client_hello (statem_srvr.c:1713)
==16956== by 0x5CB52E9: tls_post_process_client_hello (statem_srvr.c:2231)
==16956== by 0x5CB6F39: ossl_statem_server_post_process_message (statem_srvr.c:1218)
==16956== by 0x5CA4C11: read_state_machine (statem.c:664)
==16956== by 0x5CA4C11: state_machine (statem.c:434)
==16956== by 0x5CA538A: ossl_statem_accept (statem.c:255)
==16956== by 0x5C91759: SSL_do_handshake (ssl_lib.c:3609)
==16956== by 0x45456B: ngx_ssl_handshake (ngx_event_openssl.c:1606)
==16956== by 0x4698D3: ngx_http_ssl_handshake (ngx_http_request.c:751)
==16956== by 0x44ECA8: ngx_epoll_process_events (ngx_epoll_module.c:901)
==16956== by 0x443E94: ngx_process_events_and_timers (ngx_event.c:257)
==16956== by 0x44DC25: ngx_single_process_cycle (ngx_process_cycle.c:333)
==16956== by 0x4236AB: main (nginx.c:382)
==16956==
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
fun:CRYPTO_malloc
fun:CRYPTO_zalloc
fun:OPENSSL_sk_new_reserve
fun:OPENSSL_sk_new_null
fun:sk_SSL_CIPHER_new_null
fun:bytes_to_cipher_list
fun:tls_early_post_process_client_hello
fun:tls_post_process_client_hello
fun:ossl_statem_server_post_process_message
fun:read_state_machine
fun:state_machine
fun:ossl_statem_accept
fun:SSL_do_handshake
fun:ngx_ssl_handshake
fun:ngx_http_ssl_handshake
fun:ngx_epoll_process_events
fun:ngx_process_events_and_timers
fun:ngx_single_process_cycle
fun:main
}
Previously, we used the OpenSSL 1.1.1 ClientHello callback to do ssl
session fetching non-blockingly. However, this way cannot handle an edge
case: the ssl session resumption via session ticket might fail, and the
client fallbacks to session ID resumption. The ClientHello callback is
run too early to know if the client will fallback to use session ID
resumption.
Therefore, we have to take back the OpenSSL sess_set_get_cb_yield patch
and upgrade it to adapt OpenSSL 1.1.1.
Thanks Yongjian Xu and crasyangel for their help.
See 08e9e50.
Signed-off-by: Thibault Charbonnier <thibaultcha@me.com>
Thibault Charbonnier
spacewander