Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Hmm, there will still be a buffer in the kernel, that the NIC DMAs into.

You will also need to buffer incoming data from a stream protocol like TCP to combine consecutive reads into complete application layer packets.

A better approach can be seen with Registered I/O on Windows, which lets user-mode programs register buffers with the kernel so that they can be locked for the NIC to DMA into: https://technet.microsoft.com/en-us/library/hh997032(v=ws.11...



This is something i've wondered about for years. If you have a million sockets, and you've set SO_SNDBUF to 4 kilobytes for all of them, does that mean the kernel has 4 gigabytes of memory sitting around waiting for TCP segments?

In the classic C10k scenario, most sockets don't have a segment in flight at any point in time, so their buffers will be empty. It would be a huge waste if they were actually occupying memory.


Those DMA buffers are shared. You don't have a different DMA buffer for each of a million sockets.


You're right. But there will realistically still be some protocol-level buffering in user-mode for every socket. Although that might be a good use-case for a hash table of buffers, as most of the sockets will not have outstanding partial packets most of the time in typical (frequent small message) applications.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: