This is something i've wondered about for years. If you have a million sockets, and you've set SO_SNDBUF to 4 kilobytes for all of them, does that mean the kernel has 4 gigabytes of memory sitting around waiting for TCP segments?
In the classic C10k scenario, most sockets don't have a segment in flight at any point in time, so their buffers will be empty. It would be a huge waste if they were actually occupying memory.
You're right. But there will realistically still be some protocol-level buffering in user-mode for every socket.
Although that might be a good use-case for a hash table of buffers, as most of the sockets will not have outstanding partial packets most of the time in typical (frequent small message) applications.
You will also need to buffer incoming data from a stream protocol like TCP to combine consecutive reads into complete application layer packets.
A better approach can be seen with Registered I/O on Windows, which lets user-mode programs register buffers with the kernel so that they can be locked for the NIC to DMA into: https://technet.microsoft.com/en-us/library/hh997032(v=ws.11...