Right. Obviously. What I'm pointing out is that if the kernel decided to include dynamically linking to certain drivers within its scope (which it does not wish to do at present as you rightly point out) then it would need to specify a scheme for doing so. At which point glibc is all but a foregone conclusion.
So just pretend they did that and include a second dlopen routine and get on with life because at that point you're compatible with ~all major linux distros.
Far more pressing is a blessed filesystem path for where to find the things.