Distributed teams work when the company, its culture and processes are stable and clear. Most software related companies don't get anywhere near that in the first five to ten years. In the mean time, fast growing start-ups look for the engineers they hire to be their future technical leaders, and to be deeply involved in the entire company, not just writing code.

In short: drinking coffee/beer/koolaid with the rest of the team is of vital importance, even if the actual work can be done remotely.

Also, the people that don't get this aren't good hires for remote work anyway, because one primary skill needed for a remote worker is knowing how to compensate for not being physically present.