Escaping complexity in the world of Clean Code and OOP
or Getting Job Done- oriented programming
(This article is also available as Youtube video)
I want to start off with short story of my programming career.
I started working full time as a programmer in 2010.
Before that time I was doing some programming, but mostly as a hobby.
And since 2010 it was full-time.
At the time I was gaining experience and skills and I was pretty much happy with how the software was built.
Then at around mark 2015 I started feeling that I was not as productive anymore as before. The programming was not as joyful experience anymore. Instead of building, I was slamming together all these frameworks, and layers upon layers of stuff. It also required me to relearn every 12 months how to do the same job with a new tool and a new framework.
So, I didn’t feel I was solving real world problems with this activity.
Building software felt too complicated and I felt that we as the industry were heading in the wrong direction with all this complexity. It was not as joyful anymore and it felt too complex.
This feeling of frustration grew inside of me to a point where at mark 2020 I abandoned programming altogether. I started my own company with my co-founder. And I was not doing any programming whatsoever. I was hiring people, doing sales, marketing, management, and all this fun stuff that founders have to do.
The company is still growing fine till this day. But that’s not the point. The main point here is, that at mark 2022 we decided that as a part of this company we gonna create our own software. It was gonna be a SaaS web application that we would offer to our customers.
And since the company was doing quite well, I knew that I could dedicate some of my time to programming on this project.
I knew that I didn’t want to end up in the same situation that I had before, where I was required to follow all these “best practices” in order to keep my job.
Right now since I don’t have any employer anymore, or rather I’m my own employer, I can build software the way I want. And I don’t have to impress anyone, or conform to any formalities from employer, or prove anything to anyone. I can just do my job in the best way I can and see what happens.
And thanks to some inspiration on the Internet, especially Jonathan Blow and Casey Muratori, I saw that if you think for yourself, and make your own decisions based on reality, and not what the industry is telling you, then there is a possibility to discover better ways of building software. So maybe if I try that, I could find such a way that could be successful and make me productive again.
And if turns out that I’m wrong, so let it be, and I can always come back to the “best practices” that are considered a “safe path”.
So I said to myself, that I’m gonna rediscover programming from scratch.
I would just write some stupid code that makes the computer do what I need it to do, similarly to what I was doing in school when I was programming as a hobby. And I would trust myself that as the problems arise in the code or in architecture, I will try to solve them with whatever makes sense, without trying to use any principles of OOP or whatever else was written in popular books on programming.
So I focused my attention to solving actual problems and doing things that computer needs to do.
As a result it turned out to be a success and now I’m the most productive I ever been, and the most happy I ever been with programming.
The software doesn’t feel as complicated anymore.
Of course it’s not perfect, and I’m still on my journey of discovering the most sane way of programming, but it’s already way easier and simpler than it was in the past.
And today I will try to summarise some of the concepts that I see, looking at how our small team is building software nowadays. The ideas and mindsets that help us make sane decisions in this world of insane software practices.
These will be high level ideas, because I find that most of the problems with software is not actually in the coding practice itself, but it’s in everything around that.
Hopefully some of the ideas listed below can be helpful.
Side note:
Consider these ideas as a seed to growing your own judgement and not the final rules that you have to conform.
There are no hard rules in programming (or life in general).
My background since 2010:
C++, desktop games.
Java, internet banks.
Scala, online games, banking systems.
Kotlin, scraping the web, bots.
Elixir, web applications.
Typescript, web applications.
2020: founded my own company.
I listed here the projects that I worked on. It’s not an exhaustive list, but just to give you an idea where I’m coming from.
I started my career working in C++ and building desktop games. I was fired from that place actually. It's probably the most embarrassing part of my career. Game programming is actually quite hard and I was very junior at the time, so I lacked the skills required to do the job.
But over time I shifted to web development. Learned some Java and for the next 3 years was doing some web applications and internet banking systems, started on the frontend and then moved to the backend.
Then I was doing Scala for the next 5 years.
I built some internal web applications for stock exchanges, and integrations between different banking protocols.
Made backend for some online games, where people could play on different platforms online.
Then I did scraping with Kotlin, also built some bots for the web.
Then I build some more web applications with Elixir.
And since 2020 I’m building my own company as a founder, and for programming our software I use Typescript.
As you see now, it’s mostly web development, so that’s where I’m coming from.
And the languages themselves do not matter as much.
That's actually how long it took me to realise that languages do not matter to the extent that we usually think they do.
The first idea that I want to talk about is:
Cost
Pay attention to the cost of everything in software.
Every benefit has a cost.
Prefer tools with only the benefits that you need. Avoid using benefits that you don’t need. (Also see Tim Berners-Lee has a great article on the Principle of Least Power).
If you don’t see the cost, it’s still there. You just don’t see it now. You will discover it down the road.
Example:
You can either write code just as a block of code, or you can write it in a function and then call this function:
The function has a benefit that you can call it multiple times from different places. But it also has a cost - it has to be abstract now, and work for all the cases.
Whereas a block of code written in place can do only what you need it to do, and exactly how you need it to do, and doesn’t have to work for all the cases in a general way.
There are other concerns of course.
And it’s not that one is always better than another.
But this is a trivial example that illustrates the point. We as programmers make choices every day all day, and these choices all have different cost/benefit ratio.
Costs and benefits are like building your player profile in a game.
Imagine every time you increase strength by +1, your endurance automatically gets decreased by -1.
Is it that strength is always better than endurance? No. You pick what you want in your specific circumstance.
The problem, is that picking the right thing requires you to do this cost/benefit analysis.
The next concept is:
Goal achievement
The more goals/priorities you have, the less is the chance of achieving all of them.
Example:
If you want to become stronger and smarter then you are less likely to succeed at both of these priorities at the same time. If you were to choose a single priority, then your chances of success are higher.
Every additional priority that you have in a company - makes other priorities weaker.
Don’t try to optimise 100% of situations.
I optimise for solving important problems, and I can afford suffering some inconvenience with problems that do not matter as much.
For example, for iteration I will only use a “for” loop (in Typescript example), and I will avoid using ‘map’, ‘fold’, and other ‘functional’ styles of iteration. Why is that?
Because one of my job’s as a programmer is to be able to quickly see all the loops in the code just by glancing at it.
And if I know that any loop starts with ‘for’ keyword, then my brain gets used to that and automatically finds all the loops in code just by glancing at it. So ability to very easily see every loop in the code, and see possible performance bottlenecks is more important job, than the convenience of writing a concise ‘map’ or ‘fold’.
I can even use my code editor to quickly find all the loops in the code.
Also, if I decide that I want to break out of for loop, I can do it.
Whereas if I were to use .map() and then decide I want to break out of it early, I have to change .map() to something else or throw an exception.
And If I need to find all the iterations in the code, I am not able to do it if I use different ways of iterations all over the place.
Also, “for” loops do not create closures that I don't need as well.
All these subtle benefits add-up, and make for loops more versatile, in my opinion.
Now, as you can see in the example above, I will still use .map() sometimes in very few restricted cases that don't have large impact on the entire flow and are very local. But that's a tradeoff that I'm willing to make. And I don’t make this choice consciously, it’s already happening on the subconscious level. That’s what is known as professional intuition.
The next thing is:
Solve hard problems first (solving easy problems is easy)
Choose tools that solve hard problems first, and trust that you will be able to solve easy problems with any tool.
Example: rendering some HTML on a page depending on some state is not really a hard problem. I don’t need React for that. Yes, React makes it nicer to do. But does React help me solve hard problems in software?
Let’s take a look at some of the hard problems that I care about in a typical web application:
Hard problem #1. Making software run fast. Does React help me with optimising my code for performance? No. It’s very hard to optimise React applications, especially once they get big.
Hard problem #2. Once the code is fast, keep it fast. Does React help me keeping my code fast and not introduce performance regressions by accident? No. It’s very easy to cause a chain of recalculations in React, that will cause performance regression.
Hard problem #3. Making software work well for 10 years and maintain it without regressions in quality. React is changing every 12 months, so we have to constantly re-learn new ways of doing the same stuff but with different APIs. React is not helping me maintain software for 10 years at all. I have to keep upgrading to newer versions and fix the regressions every time the code is migrated to a newer version of the framework.
So if a framework doesn’t let me solve hard problems, but only solves easy problems - I try to stay away from that. I try to optimise for solving hard problems as my first priority, and I trust that I can solve easy problems with any tool.
“But React makes you super productive!” Someone could say.
Well, there are gazillion ways to be productive that don't involve compromising on performance or predictability of your program.
And then, if you take into account the maintenance cost of React application and the time to debug issues and optimise performance, and the time to migrate to a newer version of React, and the fact that you are almost required to have React engineers as a separate role in the team because React requires too much knowledge. So basically if you take into account full cycle cost of software development with React and the impact on the entire company structure, then it's not looking as good of a deal now.
Side note: for some companies, React actually may be a solution to really hard problems(such as how can you make 100k programmers work in the same codebase and stay productive?) Now, that’s a really hard problem to solve, and maybe React is really a good solution to that. I don't know. But you have to think if your specific company has this problem or not.
The next thing is:
Point of leverage
Any given problem can be solved at different levels.
Examples of such levels are: coding practices, determining the right problem, hiring process, etc.
You can leverage any level to solve the problem.
For any given problem there is a level where your effort will produce more results.
Also, the team must have skills on all the levels.
For example, if you struggle identifying what problem needs to be solved, then no matter how well you write the code, you will have hard time being effective.
Some solutions require making trade offs on other levels. Such as React example we talked about before
If you were to decide not to use React in your program, then your hiring process has to change dramatically. Instead of hiring for React skills, you have to hire for problem-solving skills, which are actually harder to find and even harder to evaluate. And if your company is not able to solve this hiring problem, then you are almost destined to stick to the mainstream “React way” of doing things, because that’s where most of the programmers skills are. So it’s a problem, that you have to solve on the hiring level, not in the coding level.
If you wanna avoid React, but don’t solve hiring, then your decision of building without React (or a similar framework) will not work.
Another example is Ruby on Rails and their famous ‘Convention over Configuration’, which totally makes sense. The problem that its solving is configuration for simple systems, and Rails solved it by simple convention and agreeing on defaults. So the problem is solved not with the code at all, but just with conventions.
The more levels you can utilise in your solutions, the more effective you are as a programmer. That’s what I call 10x programmer.
People usually call it “think outside the box”. But that’s just too abstract to me.
Here is the levels that I use in our team to solve problems:
I interview a lot of programmers, and most issues I see in candidates is not in how they write code, but rather in all the rest of the levels: how they determine what problem they are solving, how they design a solution, how they debug problems, etc.
A lot of the times I see programmers who can write very sophisticated code, but are very weak at determining the problem they are solving and designing a solution.
If you cannot determine the problem precisely, you cannot solve it precisely and effectively.
Caveat,
Not every level is available to you of course, but you still have top management in the company, and it’s their job to solve all the levels, including hiring and team organisation. That’s another problem, that it’s not only programmers job, but the company’s job as well to execute their part of responsibilities.
And me being a CEO I’m trying to make trade offs at every level in order to optimise the entire company, so that our team could build the best products they can with the resources they have. I start at the business model and the company strategy, then I optimise the hiring process, and all the way down to coding.
One example how we optimise hiring process is: we do not allow junior developers on a team. Just by making this one decision in hiring, we already eliminate a lot of issues in coding.
Another Example: Twitter is solving spam problems in DMs by requiring premium subscription.
Now, if you want to send a DM to someone who does not follow you, Twitter requires you to have a premium subscription.
Now, whether you like this solution or not, that's not the point.
The point is, it's a great example of solving problem at the level of business model and it’s very effective and requires less effort from the Twitter engineering team compared to previous solutions of trying to analyse all the DMs in the entire system.
Sidenote 1:
The more levels you can have in one head, the better you can optimise your decisions, making trade offs at any level you want.
Of course the bigger the company is, the harder it is to have all the levels in one head.
If you are not making tradeoffs, then you don’t really have priorities.
Jobs To Be Done
This is a framework that lets companies look at products they are making from a different angle. It’s almost like a mindset.
Instead of thinking what features we have to build, we are thinking what job our product is solving. So we are trying to optimise the product to solve this job better and better.
Example: McDonald’s Milkshake is solving a job of ‘satisfying hunger on commute’. When McDonald’s determined precisely what job their Milkshake is solving, they can optimise solving this specific job.
So every product has ‘jobs’ that it’s solving.
Now, when we have all of these concepts in mind,
That’s how I see software development process.
I try to solve as few jobs as possible, but pick the most important ones, and use all the company levels to optimise for solving these specific jobs.
The ideal situation is having only two priorities:
Priority #1. Get users jobs done
The software has to:
Solve problems / satisfy needs
Run fast enough
Work as expected
Not require too much time to learn how to use it
Not change too frequently (not force users to learn new UI if it doesn’t bring new value)
Priority #2. Get programmers jobs done
The programmer has to:
Write code
Deploy it
Run it
Identify issues and fix them quickly
Change code according to new requirements as we discover them
Make the code understandable for other team members
Do all of this and make sure that the value created for users is greater than the value invested into building the product.
value created must be > value invested
So that we can capture some of that value and have a sustainable business that let’s us keep building the product in the future.
Now, let’s look at that from a different angle.
There is this concept:
Optimising from point A to point B
Let’s say we have point A (we don’t have any product yet), and our point B is (making a first version of the product that users can use).
OR it can be point A (we have version 1 of the product) and we want to go to point B (which is version 2 of the product)
In ideal situation we want to go from A to B in the most straightforward line we can.
But real life is usually not like that.
The reality is that we have some essential complexity that exists in the domain of the problem, that we cannot escape.
So essential complexity looks like this squiggly line from A to B:
Example of such essential complexity is: our product that we are building is using Amazon retail and advertising APIs and builds some UI on top of that, to provide users with analytics and give them some tools to take action on this data and improve their business as a seller on Amazon.
Now, since Amazon API has some inconsistencies and different parts of API have different versions, that evolve and we have to migrate between them. This is essential complexity to us and we cannot escape it.
Then additionally to our essential complexity, we have to solve the jobs that need to be solved: for users to use the product and get value out of it and for programmers to be able to maintain the product and keep developing it in the future.
So if we add these two priorities to already existing essential complexity, now our line is even more squiggly. That’s our most efficient way in going from A to B. And we try to follow that.
Now, the problem is that the mainstream way of building software is something little more complicated like this:
Where we have everything we mentioned above + some made-up additional complexity.
Examples of made-up complexity
SOLID principles. Now, I don’t have anything against every specific principle in itself, but the whole point of trying to apply these principles as much as possible doesn’t make sense. I would argue that they are not usually solving the hardest problems in software.
The next thing is Layers. Examples of that are 3-layered architecture, onion architecture, or whatever architecture. That’s also not the best way of solving problems, because it doesn’t take into account any specific knowledge about your specific domain and the specific program you are trying to build.
Layers can be fine, if you end up with them. But you should not try to create layers upfront without solving hard problems with these layers. In my experience, most programs need way less layers than they actually have.
The next thing is Libraries and Frameworks. I don’t have problems with libraries as a thing. The problem is not with the libraries themselves. The problem happens when programmer decides to use a library where it doesn’t strike a good cost/benefit ratio.
Every additional dependency adds a weight to the project. Now you as a programmer have additional job to do: you need to upgrade to newer version of library, read migration guide and fix regressions when they happen.
If you consider your work seriously and want your project to work 10 years or more, then on this timeline any open source library or framework can die. And when it happens, you got additional job to do - either fork the project and become a maintainer of codebase that you didn’t write and may not necessarily agree with the decisions that being made in this codebase, or replace this dependency with another alternative (if it’s possible), and fix regressions as they inevitably come up.
Libraries and frameworks don’t have any knowledge of your specific application, domain, and requirements. Thus, you cannot leverage any trade offs that are allowed in your case, but couldn’t be anticipated by the creator of the library.
The next sting is OOP. Object-oriented programming. I don’t have any problems with objects themselves. But orienting the entire program around objects - that’s the issue. You don’t wanna orient your program around anything, except solving the jobs that need to be solved.
The next thing is the need to feel emotional comfort. There is a need in programmers to feel that everything is nice and the code is beautiful and not ‘dirty’. These concepts do not exist in real life, they only exist in programmers mind and just make programmer more comfortable. So trying to solve emotional comfort is just introducing additional friction in your path from A to B. This can be very subtle. For example a solution that you apply seems to be solving the problem you have, but the solution is not as directly and precisely focused on the problem as it could be, but it’s a little sideway to make the programmer feel emotionally comfortable. Whereas a direct and precise solution will be the most effective, but it will not provide such a comfort to programmer. So you have to learn to get rid of this need of feeling comfortable.
The next thing is trying to look smart. In the past I was required to use some complex tools that are used by Googles and Facebooks of this world. I thought that these companies were considered ‘smart’, and I wanted to prove that I was also smart. So I’m gonna do the same stuff that they do. But these companies have different problems and priorities, and you may not necessarily have the same problems in your company.
So all of the above made-up complexity can be summarised as solving problems that you don’t have.
The more precisely you can determine the exact problem you are trying to solve, the more effective your solution will be.
And if on top of that, you are not trying to solve problems that do not exist in your company - you gonna be the most effective.
(Also see Mike Acton on problem solving).
Summary
Understand your priorities. Have as few of them as possible.
Solve exactly and precisely the problems that you have. Do not try to solve some other problems, that you don’t have.
Optimise at all levels in the company.
Pay attention to the Cost. Everything has a cost. There is no benefit without the cost.
Your job is to build products that get users jobs done and in the meantime while doing that, help programmers to get their jobs done.
All of this is about your judgement and how you make trade offs.
It has very little to do with actual coding practice.
You are problem solver first. And it’s just happens so that you solve problems by means of making software.
Do not delegate your judgement to someone else. Make up your own mind from your own experience.
Thank you.
Subscribe to this blog and my Twitter.