Half of the tech articles I read from the FB engineers are leaving me in a very puzzled state. They read to me like this:
"We decided to shoot ourselves in the foot. Then it started to ache and we came up with this novel drug that heals feet in under 5 seconds! Now we can shoot ourselves in the foot every morning". After reading them, all I want to say is "This drug is no doubt will do a great good for the humanity, but let me take that gun away first..."
One recent example is the post about spam filter. The authors wrote a spam filter in Haskell, then stepped on all sorts of performance problems. From the need to write ingestion pipeline in C to fixing GHC garbage collector. While reading this post I was thiking about spam filter I've seen in one of my previous companies. I am not its author. But it was awesome.
It was a small compiler that builds the set of spam rules into a single FSM. It also had a tiny executor for that FSM written in plain C. It was plain and simple - all the burden of processing complex rules was handled during the compile time. The result of it - a relatively small FSM definition file was all that was needed to put to the production environment to update the spam filter. The executor was small and stateless (beside the tiny state of the compiled FSM). It processed the rules as fast as the server could inject the stream. It was running in constant memory. It was probably able to handle 1M requests per second on a modern commodity box. And it scaled incredibly well.
But nobody ever wrote a post about it. Because that post would have been a very short one. Something along the lines of: we applied core CS principles described half a century ago and they worked.
Facebook engineering keeps creating good things for the humanity. But it puzzles me that more often than not those are created as a response to a very odd and crooked problem.
They have thousands of smartest engineers. But maybe, just maybe, if they spent a little more time designing the system, they would not even need them.
Also, 90% of content on FB is spam and garbage. I can say, because there is other social network, that exposes feed filtering controls to the end user - and I can compare the two. My VK feed has 3-5 posts a day - and all of those are things I really want to read. FB has hundreds of posts a day, and most of them are just an exercise for my scrolling thumb.