Lessons from Go: Keep Calm and Use the Byte Array

Along with error handling and the lack of generics, the prevalent use of byte slices evoked resistance at first, but I quickly got used to it.


3 min read
Lessons from Go: Keep Calm and Use the Byte Array

Before coming to Go, I had spent years cushioned by the high level of abstraction of Web frameworks such as Spring or ASP.NET. Thus, I was set aback when I first encountered Go's ubiquitous byte slice ([]byte).

To say that byte arrays are unique to Go will of course be a big lie. Computers communicate using bytes. Thus, any form of data exchange between a software application and its external environment is an array of bytes. This includes IO, database, network, data conversion, and many other kinds of operations.

While other technologies try to abstract away from working with byte arrays directly, in Go they are everywhere. Go's term for byte arrays of unspecified length is called a "byte slice", but the concept is pretty much the same. You will use byte slices when reading a file, for example:

f, err := os.Open("./input.txt")
defer f.Close()
if err != nil {
    // handle the error
}

buf := bytes.NewBuffer(make([]byte, 0))
buf.ReadFrom(f)
fmt.Printf(string(buf.Bytes()))

Converting to/from XML, JSON, or other transport formats:

jsonString := `{"Foo": "Bar"}`

// Parse the incoming JSON string
type S struct{ Foo string }
var s S
json.Unmarshal([]byte(jsonString), &s)

// Turn into XML
xmlResult, _ := xml.MarshalIndent(&s, "", "")

// Write to a file
ioutil.WriteFile("out.txt", xmlResult, 0644)

Or the mother-of-all-examples, handling an HTTP request:

func postTodo(w http.ResponseWriter, r *http.Request) {
    var todo Todo
    err := json.NewDecoder(r.Body).Decode(&todo)
    defer r.Body.Close()
    if err != nil {
        // handle the error
    }

    err = store.addTodo(todo)
    if err != nil {
        // handle the error
    }

    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(todo)
}

Reader and Writer

Go's standard library provides two extremely convenient interfaces to wrap around the use of byte array's: Reader and Writer. Those two are the poster children of the concept of interfaces in Go. They are super simple (each having only a single method to implement). This is the reason why they are used all over the place in the standard library and in many Go projects. Make your custom type implement one or the other (or both), and it gets superpowers. You can serialize it, store it on disk, or send in network requests. All that using the standard library alone, without having to write any additional code.

The Cost of Abstractions

As mentioned in the beginning, I frowned upon seeing byte slices the first time around. I had worked with Java's InputStream and OutputStream before, as well as with their .NET Stream equivalent. While one can find them used in foundational libraries and frameworks, their use in applications is rather sparse.

The reasons could be many. One that comes to mind are the complex class hierarchies built around each, which make the right choice difficult, and the usage cumbersome. Another, might be the use of reflection everywhere. While an HTTP request in Spring or ASP.NET can be handled similarly to its Go equivalent, working with bytes directly is an exceptional case. The accepted solution is to use reflection, in order to cast incoming byte arrays to typed objects:

[HttpPost]
public async Task<ActionResult<TodoItem>> PostTodoItem(TodoItem todoItem)
{
    _store.TodoItems.Add(todoItem);
    await _store.SaveChangesAsync();
    return todoItem;
}

This is a great example of hiding complexity behind abstractions, and is how the majority of modern software gets developed. For decades, the Computer Science curriculum preached to students the values of not repeating oneself (DRY). Only in the last few years, did people start to question DRY's practicality.

Abstractions are great when things work fine, but terrible when they break. The probability of something crumbling down is directly proportional to its level of complexity. The deeper in the abstraction chain a problem occurs, the more difficult and costly it is to fix. This is where Computer Science theory meets hard reality.

Sticking to Go's Minimalism

Go's creators wanted a pragmatic language for tooling, backend, and systems programming. One that doesn't hide complexity from the developer. Go intentionally makes certain things verbose, and favors copying over premature dependency. It may not be the most elegant programming language, but it plays its part really well.

Along with error handling and the lack of generics, the prevalent use of byte slices evoked resistance at first, but I quickly got used to it.


Bits, Bytes, and Byte Slices in Go
If you come from a background other than Computer Science and learned programming through a higher level language such as Ruby (like I did), you may have never really “needed” to be concerned about…

Related Articles

Elixir Tip: Case vs. With
3 min read
Elixir Is Not Ruby. Elixir Is Erlang
2 min read

Write Stupid Code

This is very much a re-interpretation of a post by Thorsten Ball, written back in 2015. Like him, I too

1 min read

GO TOP