There is one skill that is exceptionally difficult to teach or implement in any software engineering situation…
To Get To That I Have To Tell A Funny Story
Many years ago Google rolled out this incredible new feature. It was an automatic autocomplete dropdown that populated as you type. It was seriously impressive.
Technically, they had to solve a few major issues to even make the feature work.
Second, they needed a sensible list of autocomplete suggestions based on what the user was typing. Given that the user can type literally anything into Google, there are A LOT of variations.
Third, this was potentially going to generate an enormous amount of load on the system. As I recall, the blog post announcing the feature estimated they had to increase their infrastructure by 6x to support the added HTTP requests, database calls, etc…
Doing all of that at Google scale to hundreds of millions of users and having it “just work” is pretty incredible.
Also, for those of you thinking you could implement this in a weekend with React and Elastic Search, neither of those tools really existed at the time.
Anyhow, like I said this was a very cool feature and at the time I was working on a very large nonprofit fundraising website that brought in millions of dollars in donations.
Someone saw what Google was doing and thought we needed to implement that feature on our site. It was useful for looking up events and people a bit faster, so it seemed like a reasonable idea at the time.
So, after maybe a month of development, we launched the feature. It didn’t seem like a big deal until a few hours later…
The Site Went Down!
It looked like we had been DDOS’ed and it took about an hour to roll back everything and get back on-line.
It turns out that the JS was sending an AJAX request on every keystroke. So, if a user quickly typed “John Doe” into the search, it fired off eight HTTP requests. Even worse, the first 7 requests were instantly useless.
It became obvious why Google had to increase their infrastructure to match the feature. The team lead hadn’t considered that at all.
The solution was simple enough, set a timeout to wait at least a half second or second after the last keystroke before doing the autosuggest. Also, we ended up later using ElasticSearch for the search backend once it was available.
The Point Of This Story Is Two-Fold…
One, the website being down for an hour that day probably cost anywhere from $10,000 to $100,000 in donations. That is nearly one FTE salary for a whole year (depending on the position and the actual loss amount).
Two, the entire feature didn’t need to exist. It was a “nice to have” that wasn’t guaranteed to make the app better. Yet, it came with huge downside risk that hit us very hard.
All of this could have been avoided if the team had approached software engineering from a minimalist approach.
The Fastest Code Is The Code That Doesn’t Run At All
Running fewer SQL queries is better than running more queries.
Fewer features means less complexity.
Fewer libraries means less complexity.
Fewer tools means less complexity.
Complexity costs time.
Complexity costs money.
Doing nothing is better than doing something useless.
A mature software engineer will do everything in their power to avoid complexity and code where possible.
Minimalism In Software Engineering Is Nearly Impossible To Teach.
It’s difficult to build a team around. Most people would rather pile on libraries, gems, new technology, and so on in the pursuit of laziness.
Technical debt is the new normal and everyone is happy to never pay it off.
So, in practice minimalism in software engineering, that is to say the approach of building the least amount of software with the fewest tools, is not a skill that common or easy to teach.
Programmers love to pile on layers of “stuff”. It’s practically impossible to get around that in an organization larger than 1 person.