How to Structure a Golang Project

Photo by Ricardo Gomez Angel on Unsplash

A while back (2020), I published an article on How to Structure a Go Command-Line Project. Within that article, I stated that while there are many recommendations on structuring Go Projects, there is no set standard.

The lack of a standard causes many new Gophers to question the best way to structure a project, A question I still see to this today.

This article will share how I structure Go Projects for backend services, similar but slightly different from how I structure Command-Line Projects.

While this article reiterates some positions, I first recommend reading the original Command-Line Projects article to understand my structure’s reasoning and decisions.

Where does main.go go?

For command-line applications, my original article suggested placing the main.go file in the top-level directory of the repository. This recommendation made installing the project easier with go install(formerly go get). But that recommendation is less useful for a backend service style project.

It’s not very common for users of a backend service to install anything locally, so for this use case, I tend to follow the common practice of placing the main.go inside of a cmd directory.

Specifically, for an application named myapp, I would create the main.go as cmd/myapp/main.go.

Suppose this project will produce both a server-side application and a command-line application. Then I tend to create two directories within cmd/, the first being myapp-server, which contains the startup logic for the server-side application. And the second is myapp, which includes the startup logic for the client-side command-line application.

Keep main.go small and simple.

My original article explained the benefits of keeping main.go trim, with only enough code to start the service.

For backend services, this is still very true. If we look at an example I’ve created below, I only start a very basic logger and load my application config.

As we can see, there is very little code, but some would argue there is still too much. This example strikes a good balance; the logger is only for logging shutdown errors, and passing Config to the app.Run() function makes my application more testable (more on this later).

Creating an app Package.

In my previous article, I explained that I typically create a runtime package named app, which houses the main execution logic of my service.

Now, some may look at this recommendation and assume that all of the service code exists within app/ and there are no other packages. While that is a tempting structure for many developers, that is not the recommendation.

When creating a service, I have two package types: the app package and non-app packages. Both of these package types have rules around them.

Non-app package rules.

For non-app packages, the rules below apply universally.

• Code that can be made modular or reusable must be within a non-app package.
• Non-app packages cannot call the logger directly. Any execution errors must return an error type.
• Non-app packages can call dependencies such as Databases, other services, etc. But, errors must be returned via error types and not masked from the caller.
• Configuration should be passed to the non-app package in the form of a Config or Opts struct defined within the package. Users should not give the service’s Config directly, making package code harder to test and reuse.
• Except for a Database package or wrapper package, non-app packages should be passed a Database type and not initiated locally. Essentially ensuring connectivity is managed centrally within the app package.

We help ensure that we can test non-app packages individually without loading the complete application by applying these rules.

Rules for the app package.

While there are fewer, there are also rules for the app package.

• Any Run() or Start() function must have a subsequent Stop() function (use context).
• While package globals are ok, they must not be exportable. Exceptions exist for structs that need to render into JSON or other formats.
• Anything that must run initiates via the Run() function, with errors returned to the caller of app.Run().
• The app package must coordinate dependency connectivity.

The goal of the app package is to manage dependencies, coordinate execution, and handle errors. Handling errors alongside the execution logic of the application is a crucial goal.

Often error handling is subjective to the execution of the application. What triggered the error? Is it ok to ignore it, or do we need to retry? Keeping the error handling and execution coordination within the same package ensures that we correctly handle errors.

While including an example app package is a bit too much code for a blog post, you can find an example of how I structure the app package via the Tarmac Project.

Where to put non-app packages

Previously I recommended against one of the most commonly recommended (but not always followed) practices for Go Projects in my original article.

Initially, I recommended against using the pkg and internal/pkg directories for non-app packages. As I felt that putting packages in these directories tends to discourage reuse. I’ve never agreed with having internal-only packages, which makes the internal directory difficult for me to recommend.

However, I have changed my mind about the pkg directory. While having watched others take and try to use my package structure for their projects, I’ve seen many struggles with non-app package structures.

I recently found that placing all non-app packages within pkg helps newcomers organize their code better. It’s clear and straightforward instructions on where to put your package and doesn’t bloat your import paths too much.

Making app even easier to test

In the previous article, I said that having a Stop() or Shutdown() function as part of your app package makes testing easier. The idea is with the app package being your application, when you run Run() or Start(), you are starting the whole application. To shut down that same application once testing is complete allows you to write multiple tests against the application.

Along that same vein, I’ve recently started passing my Config data in the Run() or Start() function.

I can create more unique testing scenarios by handing Config as a parameter to Run().

Configuration drives application behavior. Some applications work differently based on what is enabled or disabled. By giving myself an easy way to create a test config and pass it while executing my tests, I am making my application more testable.

The example below shows a test that creates a test config and launches my application with that.

Summary

For the most part, how I structure a Go Project for command-line applications is very similar to how I structure services. What is different is where the main.go file lives. The rest of my recommendations in this article are updates to my advice from the previous post.

As with anything, opinions and best practices evolve. Suppose you see a valuable snippet from my post but don’t agree 100%. That’s ok, adopt what you want.

To stay up to date with my “current” approach to project structure, I’d suggest following one of my Open Source projects, specifically Tarmac, which has an up-to-date design and is a reasonably complex application.