issue: 3586273 Use XLIO_DEFERRED_CLOSE by default #136

Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

At most a single element of this vector is always used. Once rfs constructor is complete there must be exactly one attach_flow_data element in case of ring_simple. For ring_tap this element remains null. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Set ETIMEDOUT errno and return -1 from recv in case a socket was timed out, instead of 0 return value and 0 errno. For instance, in case of TCP keep alive timeout. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

The idea is to scan all rpm/deb packages for personal emails we should not be releasing packages with such emails the scan is done on both the metadat info and the changelog of a specific package Issue: HPCINFRA-919 Signed-off-by: Daniel Pressler <danielpr@nvidia.com>

- Enable LTO by default - Add support for PGO - Switch orig_os_api with direct syscalls Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Signed-off-by: Alex Briskin <abriskin@nvidia.com>

PRM allows minimum of 512 RX strides per WQE. Adjust minimum value for NGINX master process from 32 to 512. Using also minimum stride size allowed - 64 instead of 512. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

If all unacked queue is acked - update last_unacked to null. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

In case of pcb->is_last_seg_dropped - we might end up with wrong pcb->last_unsent. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

Update last_unacked to null after RTO. In this case we move all unacked queue to unsent. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

In tcp_rexmit we take the first segment from unacked and put it in unsent. In case its the only segment in unacked, we need to update last_unacked. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

Remove iterating over unsent / unacked to find last in lists. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

Checking for last_unsent and unsent for NULL is the same. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

This is leftover from the Multi Packet RQ removal. In current code, this option doesn't have effect and only wastes 96 bytes per socket. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Remove multiple fields from the ring_allocation_logic class: - m_active is redundant. - m_type / m_owner are needed only for debug logs. 16 bytes per object is a high cost taking into account that ring migration is disabled by default. Also compact the structure. This patch saves 64 bytes per socket (sockinfo and dst_entry). Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Ring migration is disabled in most scenarios, so improve is_logic_support_migration() - check whether migration is disabled first. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Add to_str() which returns pointer to "this". Also, print the type in constructor, so each further debug message can be identified by the pointer. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Fields m_n_pbufs_rcvd and m_n_pbufs_freed are unused. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Jira task: HPCINFRA-1430 Signed-off-by: Viacheslav Login <vlogin@nvidia.com>

Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Add tcp_tx_express to tcp_sockinfo_ops_tls::tx. I was unable to perform any refactoring, since: - tx_arg is the method argument - tls_arg is initialized at the beginning of the method and used in the nested loops. - tls_record class needs constant reassurance whether it is zerocopy or not. - tls_record class needs constant reassurance whether it is TLS1.2 or TLS1.3 version. - The blocking, non-blocking, zerocopy, non-zerocopy, TLS1.2., and TLS1.2 parts are tightly-coupled and no clean separation is possible without rewriting the code. - Need to allow blocking on small sndbuf in TLS Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Signed-off-by: Alex Briskin <abriskin@nvidia.com>

- Refactor cannot_do_requested_partial_write - Extract static function is_invalid_iovec - Extract method is_connected_and_ready_to_send - Extract method tcp_tx_slow_path - Extract method tcp_tx_handle_errno_and_unlock - Extract method tcp_tx_handle_done_and_unlock - Extract method tcp_tx_handle_partial_send_and_unlock - Check ready-to-send socket in express path Signed-off-by: Alex Briskin <abriskin@nvidia.com>

- Remove oversize from pbuf_prealloc_express - Extract tcp_tx_handle_sndbuf_unavailable method - Promote pbuf.len checks to u32_t - Combine express_mkey and mkey into mkey - Remove pbuf_clen Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Signed-off-by: Ben Walker <ben@nvidia.com> Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Signed-off-by: Alex Briskin <abriskin@nvidia.com>

The deleted define returned always true because it the address of orig_os_api._func instead of checking it's value. Signed-off-by: Alex Briskin <abriskin@nvidia.com>

Signed-off-by: Gal Noam <gnoam@nvidia.com>

Enum with underlying type is C++ or C23 feature. Move the enum definition to an internal header, because currently, it's unused in public API. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Signed-off-by: Gal Noam <gnoam@nvidia.com>

The false positive warning is generated during linking phase. Cast array size to 32bit type to suppress the warning. The warning appears in --enable-lto configuration likely due to inlined new[] operator and generated to the underlying malloc() call. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Accessing completion event channel for dummy RQ, that was created automatically by ibv when we create QP. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

XLIO receives any incoming TCP payload in pbuf chains less than 64k. There is no point to do the splits on each receive. Sockinfo_tcp has no 64k limitation and this avoid extra operations. To clean the code and achieve possible minor performance improvement, remove pbuf_split_64k() and accept entire pbuf chain as is. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

refused_data was used to keep TCP payload which couldn't be received by sockinfo_tcp object and retry receive operation later. Only a single receive operation could be postponed in such a way and all the subsequent failures led to packet drops. sockinfo_tcp::rx_lwip_cb() doesn't produce an error by itself. All the possible errors are propagated by L3_tcp_input(). And all the errors lead to connection reset. So, there is no point to keep refused_data in current implementation. Remove refused_data to clean the code and save extra cache line read in fast TCP timer. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Split poll_sn to poll_sn_rx and poll_sn_tx. Required after splitting cq to cq_rx and cq_tx. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

Panic level of logs terminate the process and don't return. Call std::terminate() to terminate the process unconditionally. Also, this fixes cppcheck warning that an exception is thrown without an object. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

IF_VERBS_FAILURE if supposed to be used with ibverbs API as a workaround for inconsistent API. However, using it for an internal method make the code dirty. Replace 'throw' with 'throw_xlio_exception' to suppress a cppcheck warning. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

cppcheck generates a warning about uninitialized fields even if they're not used in fact. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

This field is used as a local variable in 3 places. Allocate a variable on stack in the respective places instead of keeping it per socket. This fixes cppcheck warning about lifecycles of local and non-local variables. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

cppcheck complains about 'int' type overflow in 0x3 << 30. Fix this with explicit unsigned type. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Debug code is disabled by default in ip_frag and has become broken. It assumes that mem_buf_desc_t::n_ref_count has a non atomic type. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Use <> for the system headers. This suppresses "information" level warning of cppcheck. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Signed-off-by: Alex Briskin <abriskin@nvidia.com>

It's not thread_local specific anymore. This event handler will also be used within poll_group to provide TCP timers for the respective sockets. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

After the 1st call to xlio_get_socket_rings_fds(), XLIO remembers the resulting array within the socket object. However, subsequent calls return the array immediately without setting number of elements in the array. As result, the function returns 0 rings by mistake. Always set the number of elements to resolve the issue. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

xlio_get_socket_rings_fds() and xlio_get_socket_rings_num() return the TX ring now. This can be useful in case of TX ring != RX ring when extra API is used. Also don't remember resulting array and construct it on the fly. Usually, this API is called only once for a socket, so remembering the array is a waste of memory. Remove the check_rings() and hardcoded number of rings for SockeXtreme flow. Return actual rings to make more precise result. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

XLIO_TCP_ABORT_ON_CLOSE option forces XLIO to send RST instead of FIN handshake. In this case, XLIO reset each connection twice. Fix this and check only once whether connection needs to send RST in prepare_to_close(). Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Once TCP_NODELAY option is set, XLIO disables Nagle's algorithm and sets TCP send buffer to a hardcoded value. This applies to setsockopt() and XLIO_TCP_NODELAY configuration. Avoid changing the send buffer and continues using either SO_SNDBUF or XLIO_TCP_SEND_BUFFER_SIZE value. Also, remove duplicated send buffer size from sockinfo_tcp object which was used only by getsockopt(). Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

The tests fail to receive notifications via the error queue and disabled in CI as result. However, running the tests manually is inconvenient, because of the known failing test cases. Disable the failing tests in the code instead of CI script. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

XLIO_ZC_TX_SIZE limited zerocopy segment size to the specified 16bit value. By default, this was 32KB. However, pbuf len is not limited to 16bit anymore and TCP segment has TSO limitation instead. Remove the option and use TSO capability to limit zerocopy TCP segments. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

tcp_pcb::max_send_sge is a copy of tcp_pcb::tso.max_send_sge, but reduced by 1. It was used in zerocopy POC implementation, because zerocopy segments were TSO segments with the TCP header in the 1st sge. Current implementation uses tcp_seg::l2_l3_tcphdr_zc[] to hold the header and doesn't require the reduced limit. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

tcp_write_express() is a zerocopy version of tcp_write() and not limited by the sndbuf or queue length. Therefore, it's unlikely to fail. However, it still can fail to allocate pbuf or TCP segment in theory. Move the loop over iovec from higher level methods to tcp_write_express(), so we keep consistent TCP state if the memory allocation error happens in the middle of a complex send operation. Additionally, improve tcp_write_express() robustness and don't allow to append data to last segment if it's a retransmit. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Condition is_connected_and_ready_to_send() does RX polling in case of TCP_CONN_CONNECTING state. The idea is to speedup the connected state, however, this has few drawbacks: - This is unlikely path, because there are mechanisms to async connect() to finish; - RX polling requires to be unlocked, so there is a race window between TX thread and concurrent RX which may change the state; - The RX polling doesn't guarantee that there is the last handshake packet in the RQ. Remove the polling, so checking for connected state is more lightweight and allowed under the socket lock. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Replace u16_t with u32_t to avoid type overflow. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

pcb::snd_buf can become negative in some flows. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Higher level logic checks for sndbuf, so no need to check it again in tcp_write(). Also account SYN/FIN segments in snd_buf. Previously, this code was disabled, because of the corner case when snd_buf is 0. But now it can become negative. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

This field was used to hold mapping_t pointer, but the mapping is a mem_desc object now. Remove the leftover. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

This is performance oriented and event based API for TCP sockets. It introduces a polling group (xlio_poll_group_t) which is a collection of sockets (xlio_socket_t) and a serialization context. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Remove stub initialization functions to clean the code. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Structure pbuf_custom doesn't serve its original purpose anymore and contains a single field pbuf. Remove the structure and use pbuf directly. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Signed-off-by: Gal Noam <gnoam@nvidia.com>

Signed-off-by: Viacheslav Login <vlogin@nvidia.com>

There is no path to create pipeinfo objects. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

To simplify inheritance and make socket objects cache friendly, we avoid inheriting cleanable_obj which inserts only one boolean but trashes the sapce after vptr. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

The class pkt_sndr_source is unused empty interface. This adds undesired vptr to inheriting classes and trashes the cache. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

The only consumer of pkt_rcvr_source is sockinfo hierarchy objects. Removing this abstraction allows cache friendly layout of sockinfo. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Removing inheritance of timer_handler from sockinfo_tcp allows o order fields in cache friendly manner. timer_hander add vptr ad related fields in an undesired location in the object layout. sockinfo_tcp is always handled by tcp_timer_collection which itslef is timer_handler. Thus, there is no real need in registering the socket itself into the internal thread. This removes complex code with handling groups inside timers registration which are used only for tcp_collection. For XLIO_TCP_CTL_THREAD=with_wakeup/no_wakeup we put the listen socket events to be handled as part of tcp_timer_collections instead of being registered directly on internal thread. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Having sockinfo inheriting from wakeup_pipe inserts wakeup_pipe members in an undesired location in term of cache utilization for data path. The same functionallity can be achieved by having the wakeup_pipe as a member. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Before merging socket_fd_api with sockinfo, replace all access through socket_fd_api with sockinfo. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Removing redundant base from sockinfo allows to put data path members near the vptr for better cache utilization. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Having inline methods with long implementation outside the class makes the class definition more readable. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Reordering sockinfo methods for more readable definition. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Moving socket stats to be a global resource instead of inside of each socket. This reduces socket size by +500B and allows to disable stats for sockets or make few sockets to use stats. Socket stats are generally unneeded and limited in monitoring to 1024 anyway. In cases of 1K+ sockets these stats are overhead. This also allows to disable stats for sockets and improve cache misses. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Reordering the sockinfo fields from random locations to group data path frequently accessed fields together. Placing frequently fields at the beggining of the object allows these members to be fetched with the cache line that fetches the vptr. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

The check for m_flow_tag_enabled is redundant as we already got flow-tag packet and know the socket. In worst case the flow_tag_id can be checked. Avoid unneeded memory access for critical data path. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

This support was added long ago. However, there are no more users for this option. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Checking if timestamps are needed per each packet consumes more cpu cycles than checking once and then running another loop is better for most cases when timestamps are not required. wq Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Precaching constant sysvar inside socket enlarges the socket and requires more cache lines. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Socketxtrme does not need to wakeup sockets by using m_sock_wakeup_pipe. This mechanism is used when a seperate thread sleeps on a blocking socket. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

This mechanism is used for poll/select only. Socketxtrme has its own polling mechanism. Accesing and checking this member/method is cycles/cache overhead. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Cache optimization for data path. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

It is enough to hold a thread_local dummy lock instead of allocating a seperate object for each socket. Dummy lock contains only empty methods. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

TCP 5T flow-tag data path always contains src/dst addresses in m_connected/m_bound memebers. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

There is no bug, this fix is aimed at suppressing the optimizer errors. Signed-off-by: Alex Briskin <abriskin@nvidia.com>

A TCP time-wait used socket can call register_timer although it is already registered. This made the socket to be added more than once to the timers list. Adding a check to verify that the socket is not already in the list. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

A TCP reused socket calls socket_stats_init. However, the check inside socket_stats_init was for m_p_socket_stats which may point to sock_stats::t_dummy_stats and thus pass the condition and init the sockets as one that has stats. Further a pointer to dummy is returned to stats vector as regular stats object. Checking for m_has_stats instead of for m_p_socket_stats solves the issue. Since, m_has_stats initialized to true only if a real stats object is returned by sock_stats. This stats object will be held by the socket untill its destruction. A reused timewait socket that has stats will just reset its stats as part of socket_stats_init. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

XLIO Socket API introduced memory callback which provides information about internal memory allocation for RX/TX buffers. The callback also notifies about hugepage size and guarantees hugepage boundary alignment if hugepages are used. This makes XLIO hugepage configuration is more critical to user if their logic depends on hugepage properties. XLIO_HUGEPAGE_LOG2 is not user friendly, so replace it with XLIO_HUGEPAGE_SIZE configuration which accepts user readable format of the hugepage size. For example, XLIO_HUGEPAGE_SIZE=2MB. XLIO checks whether the requested hugepage size is supported by the system. But this check is moved to hugepage_mgr constructor, because g_hugepage_mgr is not initialized during reading XLIO configuration. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

XLIO socket API doesn't support crypto offload on this stage. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

xlio.h and xlio_extra.h provides public API for different scenarios: explicit linkage and LD_PRELOAD respectively. Therefore, usually, it makes sense to use a single header. However, the headers have common types definitions. Move them to a separate xlio_types.h instead of including xlio_extra.h from xlio.h. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Allow user to take responsibility for XLIO buffers allocation. This can be useful if RX buffers needs to be placed to a special memory for further handling. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

This allows to use XLIO Socket API with LD_PRELOAD approach via pointers. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

While lock_spin requires 16 bytes, the lock_spin_simple requires 4 bytes. This allows squashing data path members into less cache lines. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Adding a class to act as a pool of different objects. Other objects that use this pool are introduced next. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Introduce ring pool of ring_ec objects which are used for Socketxtreme completion. Using such a pool decreases significantly the number of ring_ec obejcts needed. This, in turn, improves cache utilization. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Compiler defines __BYTE_ORDER__ and others, but XLIO uses wrong names in the macros. Preprocessor replaces undefined names with 0, so XLIO always builds little endian branch. Fix the byte order names and fail a build if __BYTE_ORDER__ is not defined. Otherwise, the build would be broken on a big endian system. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Signed-off-by: Gal Noam <gnoam@nvidia.com>

Keep a global collection to destroy remaining groups at exit. Usually, polling groups are pre-initialized in a small number (up to the number of CPU cores) and destroyed at exit. Therefore, the collection doesn't have to be efficient. Moreover, a group destruction is a heavy operation itself, therefore, optimizing the collection doesn't make sense. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Keep a sockets list per polling group to close the remaining sockets on a group destruction. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

To avoid a race and extra destruction operations, take additional reference to each ring within a group. The race can happen if a socket destruction leads to its ring destruction, but user returns RX buffers after that. User may return buffers to a group regardless of socket lifecycle. In a scenario when user closes all the sockets in runtime, the respective ring would be destroyed and created with new socket again. This is expensive operation and the reference avoids the destruction. Current implementation keeps all the rings until net_device_val destructor. In the future, poll_group should release the reference and destroy the ring. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Rings are created in context of a net_device_val. Therefore, the net_device_val object must be present for an existing ring. However, API allows to return NULL by the netdev lookup and coverity generates a warning because of that. Throw an exception to suppress the coverity warning. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

poll_group takes additional reference to each its ring. But it doesn't release it once the group is destroyed. This leads to two issues: 1. Extra resources are utilized if user destroys a polling group before the application terminates. 2. Polling is not possible for a destroyed group. Therefore, if there are not completed WQEs in the SQ, respective sockets won't report TX completions and cannot be fully terminated. The ring needs to be destroyed to flush all the completions. Release all the native rings explicitly in the poll_group destructor to resolve the above issues. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

When an RX packet event happens, XLIO passes the ownership to user. Further, user releases the buffer explicitly. However, XLIO frees the buffer unconditionally just after emitting the event. Fix this and free buffers only if user doesn't provides the RX event callback. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

reclaim_recv_single_buffer() accumulates buffers in a list. In the performance oriented API we want to reuse hot buffers immediately, so reclaim_recv_buffers() implementation is more suitable. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

The memory callback provides hugepage size of the underlying pages. Replace hardcoded 0 with real hugepage size. Keep the page size in xlio_allocator object. This field a relevant only the hugepage allocation method and 0 in all other cases. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

XLIO Socket API must guarantee that the XLIO_SOCKET_EVENT_TERMINATED is not followed by any other events. Therefore, all the TX completion events must be completed by that moment. Do a polling iteration before calling socket destructor to increase the chance that all the relevant WQEs are completed. This mechanism needs to be improved in the future. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

xlio_init_ex() changes some default parameters. However, a global object can trigger safe_mce_sys() constructor at the start. Therefore, we need to re-read the environment variables again to guarantee that the changed parameters take place. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Avoid using connect() with sock fd interface, because fd_collection doesn't keep xlio_socket_t objects. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

xlio_socket_t objects aren't connected to the fd_collection anymore. Therefore, all the methods must be called from the sockinfo_tcp objects directly. Also, xlio_socket_fd() is not relevant anymore and can be removed. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Iterate over std::list of TCP sockets while erasing socket during iteration. Overcomed by increasing iterator before erase. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

rdma-core limits number of UARs per context to 16 by default. After creating 16 QPs, XLIO receives duplicates of blueflame registers for each subsequent QP. As results, blueflame doorbell method can write WQEs concurrently without serialization and this leads to a data corruption. BlueFlame can make impact on throughput, since copy to the blueflame register is expensive. It can improve latency in some low latency scenarios, however, XLIO targets high traffic/PPS rates. Removing blueflame method also slightly improves performance in some scenarios. BlueFlame can be returned back in the future to improve low-latency scenarios, however, it will need some rework to avoid the data corruption. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

The inline WQE branch is not likely in most throughput scenarios. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

Avoid calling register_socket_timer_event when a socket is already registered (TIME-WAIT). Although there is no functionality issue with that, it produces too high rate of posting events for internal-thread. This leads to lock contantion inside internal-thread and degraded performance of HTTP CPS. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Signed-off-by: Gal Noam <gnoam@nvidia.com>

UTLS uses tcp_tx_express() for non blocking sockets. However, this TX method doesn't support XLIO_RX_POLL_ON_TX_TCP. Additional RX polling improves scenarios such as WEB servers. Insert RX polling into UTLS TX path to resolve performance degradation. Signed-off-by: Dmytro Podgornyi <dmytrop@nvidia.com>

In heavy CPS scenarios a socket may go to TIME-WAIT state and be reused before first TCP timer registration is performed by internal-thread. 1. Setting timer_registered=true while posting the event prevents the second attemp to try and post the event again. 2. Adding sanity check in add_new_timer that verifies that the socket is not already in the timer map. Signed-off-by: Alexander Grissik <agrissik@nvidia.com>

Added new env parameter - XLIO_MAX_TSO_SIZE. It allows the user to control maximum size of TSO, instead of taking the maximum cap by HW. The default size is 256KB (maximum by current HW). Values higher than HW capabilities won't be taken into account. Signed-off-by: Iftah Levi <iftahl@nvidia.com>

Signed-off-by: Gal Noam <gnoam@nvidia.com>

For incoming sockets - no change. For outgoing sockets - since outgoing sockets occupy a local port, we should release it on the socket destructor to prevent race from another socket to use the same port. This race might cause XLIO to hold 2 sockets with the same RFS object at the same time, and this is fatal (particularly for TCP). Signed-off-by: Iftah Levi <iftahl@nvidia.com>

For tests related to bind() Signed-off-by: Iftah Levi <iftahl@nvidia.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

issue: 3586273 Use XLIO_DEFERRED_CLOSE by default #136

issue: 3586273 Use XLIO_DEFERRED_CLOSE by default #136

Commits on Jan 14, 2024

Commits on Jan 17, 2024

Commits on Jan 18, 2024

Commits on Jan 22, 2024

Commits on Jan 25, 2024

Commits on Jan 30, 2024

Commits on Feb 1, 2024

Commits on Feb 6, 2024

Commits on Feb 7, 2024

Commits on Feb 12, 2024

Commits on Feb 22, 2024

Commits on Feb 25, 2024

Commits on Feb 26, 2024

Commits on Mar 10, 2024

Commits on Mar 11, 2024

Commits on Mar 12, 2024

Commits on Mar 18, 2024

Commits on Mar 19, 2024

Commits on Mar 20, 2024

Commits on Mar 31, 2024

Commits on Apr 1, 2024

Commits on Apr 3, 2024

Commits on Apr 4, 2024

Commits on Apr 8, 2024

Commits on Apr 9, 2024

Commits on Apr 14, 2024

Commits on Apr 17, 2024

issue: 3586273 Use XLIO_DEFERRED_CLOSE by default #136

Are you sure you want to change the base?

issue: 3586273 Use XLIO_DEFERRED_CLOSE by default #136

Commits on Jan 14, 2024

Commits on Jan 17, 2024

Commits on Jan 18, 2024

Commits on Jan 22, 2024

Commits on Jan 25, 2024

Commits on Jan 30, 2024

Commits on Feb 1, 2024

Commits on Feb 6, 2024

Commits on Feb 7, 2024

Commits on Feb 12, 2024

Commits on Feb 22, 2024

Commits on Feb 25, 2024

Commits on Feb 26, 2024

Commits on Mar 10, 2024

Commits on Mar 11, 2024

Commits on Mar 12, 2024

Commits on Mar 18, 2024

Commits on Mar 19, 2024

Commits on Mar 20, 2024

Commits on Mar 31, 2024

Commits on Apr 1, 2024

Commits on Apr 3, 2024

Commits on Apr 4, 2024

Commits on Apr 8, 2024

Commits on Apr 9, 2024

Commits on Apr 14, 2024

Commits on Apr 17, 2024