Every programmer writes either an OS or a programming language. Every programmer-blogger writes about "Root of all evil". It is time for me to write how I see it.
I agree that trying to optimize something too early leads to some most broken designs. But I have a very clear idea of when it is not too early anymore, and is about to be late already.
The biggest "too early" criteria is - if you have the data yet?
Do you know the performance of your DB's write operation? How it compares to a read operation? Is there a cache?
Do you know the size of a message? Medium size, maximum size, etc.
Do you know all the SLAs of the system?
If you do not any of those, then optimization is premature. That's how I've seen systems end up with layers and layers of caching, all on the same level. Just different developers thought of "it must be an expensive operation, I'd better cache the result", but they all did it. And it worked just fine and may even passed the QA, until one day somebody asked a simple question "what is our cache policy for blah?" and got a horrifying answer.
In order to begin optimizations you need to have a Proof-of-Concept code somewhat working already. You should be able to observe the kind of errors the systems you use emit when they enter various error states. You should know how exactly your "happy path" looks like.
That Proof-of-Concept should be simple. It should not have any optimizations baked in it already because you have a gut feel your end product will need it. Your feeling is most likely correct, but having a test system without optimization is your way to validate it.
In my humble opinion, the best way to go about implementing any system is:
- Prototype it.
- Evaluate the performance of various parts and dependencies.
- Design your end product. You may be adding some optimizations to your prototype at this point. Just to validate the way they work.
- Throw away the prototype. Really important thing to do. You started writing it without the full picture in your head, it is guaranteed to have some wrong assumptions baked in and you are likely to not remember them all.
- Write the code up to your design. Don't forget to write automatic tests for it. Hopefully you were not writing unit tests for your Proof-of-Concept, because nobody needs them there.
This approach saved me a lot of time in my career.