Red Hat JBossEAP: Allocation of Resources Without Limits or Throttling (CVE-2023-34462)

Description

Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients. The `SniHandler` can allocate up to 16MB of heap for each channel during the TLS handshake. When the handler or the channel does not have an idle timeout, it can be used to make a TCP server using the `SniHandler` to allocate 16MB of heap. The `SniHandler` class is a handler that waits for the TLS handshake to configure a `SslHandler` according to the indicated server name by the `ClientHello` record. For this matter it allocates a `ByteBuf` using the value defined in the `ClientHello` record. Normally the value of the packet should be smaller than the handshake packet but there are not checks done here and the way the code is written, it is possible to craft a packet that makes the `SslClientHelloHandler`. This vulnerability has been fixed in version 4.1.94.Final.. A flaw was found in Netty's SniHandler while navigating TLS handshake which may permit a large heap allocation if the handler did not have a timeout configured. This issue may allow an attacker to send a client hello packet which would cause the server to buffer large amounts of data per connection, potentially causing an out of memory error, resulting in Denial of Service.