>So the best you can do is estimate the costs of doing the necessary work in each scenario (do it now but don't need it later, do it now and do need it later, do it later when known to need it), the likelihood that it will in fact be required at some point, and therefore the expected benefit of doing it now vs. later.
So you do these to estimations and sometimes you are wrong. When you are wrong it costs you. It costs you opportunity cost, cost of carry, cost of delay, cost of building, cost of repair.
With YAGNI, we build feature A then possibly feature B. Let assume that YAGNI and TDD offer no benefit, and the cost of doing A then B is twice the cost of doing A and B together, but sometimes we only do A, or we do A, and then later do B and in between A makes two months of revenue.
The question is, on average, that is to say, over a large number of features, which method costs less?
If you estimates and predictions are accurate, then clearly your solution is the lowest cost over the whole project.
If however, your estimates and predictions are poor then in fact you cause waste and increase costs. There is a point at which YAGNI offers the lower cost over the whole project.
It is my experience, and, it appears, that of Martin Fowler, that in fact we are very bad at making estimations, and very poor at predicting the future.
So this is obviously counter intuitive and not something you want to believe: on average, your "best choice given the information available at the time" will cost more money on average than just implementing feature A now and worrying about B later.
You have to do a risk assessment and cost/benefit analysis of your risk assessment and cost/benefit analysis. It turns out that your risk assessment is highly risky and your cost/benefit analysis is too costly for the supposed benefits. Its cheaper to YAGNI.
So you do these to estimations and sometimes you are wrong. When you are wrong it costs you. It costs you opportunity cost, cost of carry, cost of delay, cost of building, cost of repair.
But unless you are very lucky, always disregarding any expectations about the future that turn out to be true also has a cost. YAGNI seems to assume that such costs are negligible, but in reality both false positives and false negatives can cost you.
As I noted in another comment, this is the danger of generalising from a single person's experience or a small set of anecdotes in such a large and diverse field. I have also been around the industry for a while, but personally I'm still waiting to run into these projects that incur crippling costs because they are so bad at predicting future needs that they write loads of code that is never used, just as I'm still waiting to see a project where refactoring fails catastrophically if you don't have 100% test coverage, or whatever other absolute metric someone wants to argue for this week.
Sure, sometimes you make the wrong call, because as everyone keeps saying, for most projects you can't predict the future with 100% accuracy in this business. But my personal experience has been that much of the time, either you have a reasonable idea of generally where things are going or you know you don't have enough confidence in future directions yet to be worth building specifically. And when you do have a reasonable idea, there are plenty of projects where failing to take advantage of that knowledge really will hurt you because you can't just conveniently refactor it out later -- many fields with specific resource or performance constraints will fall into this category for example.
So you do these to estimations and sometimes you are wrong. When you are wrong it costs you. It costs you opportunity cost, cost of carry, cost of delay, cost of building, cost of repair.
With YAGNI, we build feature A then possibly feature B. Let assume that YAGNI and TDD offer no benefit, and the cost of doing A then B is twice the cost of doing A and B together, but sometimes we only do A, or we do A, and then later do B and in between A makes two months of revenue.
The question is, on average, that is to say, over a large number of features, which method costs less?
If you estimates and predictions are accurate, then clearly your solution is the lowest cost over the whole project.
If however, your estimates and predictions are poor then in fact you cause waste and increase costs. There is a point at which YAGNI offers the lower cost over the whole project.
It is my experience, and, it appears, that of Martin Fowler, that in fact we are very bad at making estimations, and very poor at predicting the future.
So this is obviously counter intuitive and not something you want to believe: on average, your "best choice given the information available at the time" will cost more money on average than just implementing feature A now and worrying about B later.
You have to do a risk assessment and cost/benefit analysis of your risk assessment and cost/benefit analysis. It turns out that your risk assessment is highly risky and your cost/benefit analysis is too costly for the supposed benefits. Its cheaper to YAGNI.