Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

They wanted a measure of quality. We are an embedded system where we sometimes have to pay a tech to go our and update customer devices. Last time we had to do this the recall cost us something like 10 million dollars - that is just the price we paid the techs to drive to the customer and do the update, and doesn't count the engineering costs to create the fix. As such not having a recall event is important. (We do have customer installable updates, but for "reasons" some code cannot be updated by customers)

The negative is some people don't believe in through tests. They write a few unit tests for things they know are tricky. Then when their code coverage is low they get their coverage up by writing "tests" that take all the branches, but never actually assert anything. They know all the tricks to sneaking these bad tests in and the result is the metrics look good while hiding the fact that code is not covered.



Interesting. Sounds like tests were really needed in this case, but some people didn't want to write them. And the code coverage tools were just gamed. ie, an additional level of oversight would be needed there if you really wanted to ensure good test coverage.

I don't see how code coverage is making the situation worse though, it seems like it just wasn't enough in this case. I guess in that it resulted in useless tests by some people, it's a minor setback, but surely it encouraged others to write more proper tests.


Code coverage wasn't making it better. As one of the developers on the project, I already knew who was writing good code or not (but I couldn't do anything about it).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: