Python is Easy. Go is Simple. Simple != Easy.

There is a common misconception that simple and easy refer to the same thing. After all, if something is easy to use, its inner workings must be simple to understand, right? Or vice versa? Actually, it is quite the opposite. While the two concepts spiritually point to the same outcome, making something seem easy on the outside requires enormous complexity under the hood.

Take Python, a language known for its low barrier to entry and, therefore, a favorite choice for entry programming language. Schools, universities, research centers, and a large number of businesses across the globe have chosen Python precisely because of its accessibility to anyone, regardless of their level of education or academic background (or total lack thereof). One rarely needs much type theory or understanding of how and where things get stored in memory, which threads some piece of code is running on, etc. Moreover, Python is the entry gateway to some of the most profound scientific and system-level libraries. Being able to control this amount of power with a single line of code speaks a lot in favor of it becoming one of the most popular programming languages on the planet.

And here comes the catch - the easiness of expressing things in Python code comes at a cost. Under the hood, the Python interpreter is massive, and many operations must take place for even a single line of code to be executed. When you hear someone referring to Python as a “slow” language, much of the perceived “slowness” comes from the number of decisions the interpreter makes at runtime. But that’s not even the biggest issue, in my view. The complexity of the Python runtime ecosystem, together with some liberal design decisions around its package management, makes for a very fragile environment, and updates often lead to incompatibilities and runtime crashes. It is not uncommon to leave a Python application to go back to it after a few months, only to realize that the host environment has changed enough that it is no longer possible to even to start the application anymore.

Of course, this is a gross over-simplification, and even kids nowadays know that containers exist to solve problems like this. Indeed, thanks to Docker and its likes, it is possible to “freeze” a Python codebase’s dependencies in time so that it can practically run forever. However, this comes at the cost of shifting the responsibility and complexity to the OS infrastructure. It is not the end of the world, but it is also not something to underestimate and overlook.

From Easiness to Simplicity #

If we were to address the issues with Python, we would end up with something like Rust - extremely performant but with a notoriously high barrier to entry. Rust is in my view, not easy to use, and what is more, not simple. While it is in total hype these days, despite 20 years of programming and having had my first steps in C and C++, I cannot look at a piece of Rust code and say with certainty that I understand what is going on there.

I discovered Go about five years ago while working on a Python-based system. While it took me a few tries to get to like the syntax, I immediately fell for the simplicity idea. Go is meant to be simple to understand by anyone in an organization - from the junior developer fresh out of school to the senior-level engineering manager who only occasionally looks at code. What’s more, being a simple language, Go gets syntax updates very rarely - the last significant one has been the addition of generics in v1.18, which is only after a decade of serious discussion. For the most part, whether you look at Go code written five days ago or five years ago, it is mostly the same and should just work.

Simplicity requires discipline, though. It can feel limiting and even somewhat backward at first. Especially when compared to a succinct expression, such as a list or a dictionary comprehension in Python:

temperatures = [
    {"city": "City1", "temp": 19},
    {"city": "City2", "temp": 22},
    {"city": "City3", "temp": 21},
]

filtered_temps = {
    entry["city"]: entry["temp"] for entry in temperatures if entry["temp"] > 20
}

The same code in Go requires a few more keystrokes but should be ideally one idea closer to what the Python interpreter is doing under the hood:

type CityTemperature struct {
    City      string
    Temp float64
}

// ...

temperatures := []CityTemperature{
    {"City1", 19},
    {"City2", 22},
    {"City3", 21},
}

filteredTemps := make(map[string]float64)
for _, ct := range temperatures {
    if ct.Temp > 20 {
        filteredTemps[ct.City] = ct.Temp
    }
}

While you can write equivalent code in Python, an unwritten rule in programming says that if the language provides an easier (as in, more concise, more elegant) option, programmers will gravitate towards it. But easy is subjective, and simple should be equally applicable to everyone. The availability of alternatives to perform the same action leads to different programming styles, and one can often find multiple styles within the same codebase.

With Go being verbose and “boring,” it naturally ticks another box - the Go compiler has much less work to do when compiling an executable. Compiling and running a Go application is often as fast, or even quicker, than getting the Python interpreter or Java’s virtual machine to load before even running the actual application. Not surprisingly, being a native executable is as fast as one executable can be. It’s not as fast as its C/C++ or Rust counterparts but at a fraction of the code complexity. I am willing to neglect this minor “drawback” of Go. Last but not least, Go binaries are statically-bound, meaning you can build one anywhere and run it on the target host - without any runtimes or library dependencies whatsoever. For the sake of convenience, we still wrap our Go applications in Docker containers. Still, those are significantly smaller and have a fraction of the memory and CPU consumption of their Python or Java counterparts.

How we use both Python and Go to our advantage #

The most pragmatic solution we have found in our work is combining the powers of Python’s easiness and Go’s simplicity. For us, Python is a great prototyping playground. It’s where ideas are born and where scientific hypotheses get accepted and rejected. Python is a natural fit for data science and machine learning, and since we deal with lots of that stuff, it makes little sense to try and reinvent the wheel with something else. Python is also at the core of Django, which speaks to its motto of allowing rapid application development like few other tools (of course, Ruby on Rails and Elixir’s Phoenix deserve a noteworthy mention here).

Suppose a project needs the slightest bit of user management and internal data administration (like most of our projects do). In that case, we’d start with a Django skeleton because of its built-in Admin, which is fantastic. Once the rough Django proof-of-concept starts resembling a product, we identify how much of it can be rewritten in Go. Since the Django application has already defined the structure of the database and how data models look, writing the Go code that steps up on top of it is quite easy. After a few iterations, we reach a symbiosis, where the two sides peacefully co-exist on top of the same database and use bare-bones messaging to communicate with one another. Eventually, the Django “shell” becomes an orchestrator - it serves our administration purposes and triggers tasks that are then handled by its Go counterpart. The Go part serves everything else, from the front-facing APIs and endpoints to the business logic and backend job processing.

It’s a symbiosis that has worked well so far, and I hope it stays this way in the future. In a future post, I will outline some more details on the architecture itself.

Thanks for reading!