There should be some sort of open-source testing environment for common breakages. For example, maybe a bunch of VirtualBox VM's running Linux. Maybe a unit test for the problem mentioned in the article sets up machines A, B, FW, and C connected A <-> B <-> FW <-> C, has B fragment packets, and has A perform path MTU discovery to C. The firewall configuration under test goes on FW and running the test will catch this problem.
With a good enough framework of this type, all the testing could be done "out of the box" so all you have to do is set up a disk image or IP address of a firewall box to test, and the testing is fully automatic. The firewall under test can run any OS or firewall that will run as a VirtualBox client -- or even be its own box connected to the testing machine's Ethernet port. Heck, if you had machines on both sides of some third-party you don't control, like your ISP, you could even use it to probe their network configuration for issues without any special cooperation from them.
If the test suite gets good enough, maybe eventually pressure will build on vendors to make their products pass and we'll see firewall brokenness start to disappear.
As well as cloud services like AWS, such tests could be used by Linux distros, operating system vendors, and network equipment manufacturers.
I'd build it myself, but I'm not a networking expert and I'm not particularly enthusiastic about becoming one.
With a good enough framework of this type, all the testing could be done "out of the box" so all you have to do is set up a disk image or IP address of a firewall box to test, and the testing is fully automatic. The firewall under test can run any OS or firewall that will run as a VirtualBox client -- or even be its own box connected to the testing machine's Ethernet port. Heck, if you had machines on both sides of some third-party you don't control, like your ISP, you could even use it to probe their network configuration for issues without any special cooperation from them.
If the test suite gets good enough, maybe eventually pressure will build on vendors to make their products pass and we'll see firewall brokenness start to disappear.
As well as cloud services like AWS, such tests could be used by Linux distros, operating system vendors, and network equipment manufacturers.
I'd build it myself, but I'm not a networking expert and I'm not particularly enthusiastic about becoming one.