Benchmarking

In principal, benchmarking is still a part of testing. It is used to measure the performance of a function or a piece of code. In Go, you can use the testing package to create benchmarks. A benchmark function should start with the word Benchmark and take a pointer to testing.B as its only argument. Here's an example of a simple benchmark:

// file: main.go
package main

import (
  "fmt"
)

func primeNumbers(n int) []int {
  primes := []int{}

  for i := 2; i <= n; i++ {
    isPrime := true
    for j := 2; j*j <= i; j++ {
      if i%j == 0 {
        isPrime = false
        break
      }
    }
    if isPrime {
      primes = append(primes, i)
    }
  }

  return primes
}

func main(){
  fmt.Println(primeNumbers(100))
}

  // file: main_test.go
  package main

  import (
    "testing"
  )

  func BenchmarkPrimeNumbers(b *testing.B) {
    for i := 0; i < b.N; i++ {
      primeNumbers(100)
    }
  }

To run the benchmark, you can use the go test command with the -bench flag. This will run all benchmarks in your package:

  $ go test -bench=.
  goos: darwin
  goarch: arm64
  pkg: example.com/prime
  BenchmarkPrimeNumbers-8   	1000000	      0.00123 ns/op
  PASS
  ok  	example.com/prime	1.234s

The output shows the number of iterations and the time taken for each iteration. This information can help you identify performance bottlenecks in your code.

What the heck is b.N?

The b.N variable is a special variable provided by the testing package that represents the number of iterations to run in the benchmark. The Go testing framework automatically determines the value of b.N based on the performance of the code being benchmarked. It will run the benchmark multiple times, adjusting b.N to get a more accurate measurement of the performance.

The testing framework automatically sets b.N by running your benchmark with increasing values until achieving statistical significance. It starts with 1 and increases until the benchmark runs for about 1 second, which is why your code must execute the tested function exactly b.N times:

    // sample code borrowed from
    // https://betterstack.com/community/guides/scaling-go/golang-benchmarking/
    func BenchmarkSomething(b *testing.B) {
        // Optional setup code

        // Reset the timer if setup took significant time
        b.ResetTimer() 

        for i := 0; i < b.N; i++ {
            // Code you want to measure
        }
    }

If you testing a function that's having cleanup code, you must call b.ResetTimer() to reset the timer before running the benchmark loop. This is important because any setup code that runs before the loop can skew the results. By calling b.ResetTimer(), you ensure that only the code inside the loop is measured.

    // sample code borrowed from
    // https://betterstack.com/community/guides/scaling-go/golang-benchmarking/
    func BenchmarkComplexOperation(b *testing.B) {
        // Setup
        data := createLargeDataset()

        // Reset the timer to exclude setup time
        b.ResetTimer()

        for i := 0; i < b.N; i++ {
            processData(data)
        }

        // Optionally pause timer during cleanup
        b.StopTimer()
        cleanupResources()
    }

Load Testing with k6

Load testing is a type of performance testing that simulates a large number of users accessing your application simultaneously. This helps you identify how your application behaves under heavy load and whether it can handle the expected traffic.

This is really useful when you are building a web application or an API. One of the frameworks that you can use for load testing is k6. It is a modern load testing tool that allows you to write tests in JavaScript and run them in a distributed manner. You can use k6 to simulate thousands of virtual users and measure the performance of your application under load.

Assume you have simple API written with Gin and deployed to public using example.com domain:

  package main

  import "github.com/gin-gonic/gin"

  func main() {
    router := gin.Default()
    router.GET("/ping", func(c *gin.Context) {
      c.JSON(200, gin.H{
        "message": "pong",
      })
    })
    router.Run() // listen and serve on 0.0.0.0:8080
  }

You then write a k6 script to test this API endpoint:

  // file:: script.js
  import http from "k6/http";
  import { check, sleep } from "k6";

  // Test configuration
  export const options = {
    thresholds: {
      // Assert that 99% of requests finish within 3000ms.
      http_req_duration: ["p(99) < 3000"],
    },
    // Ramp the number of virtual users up and down
    stages: [
      { duration: "30s", target: 15 },
      { duration: "1m", target: 15 },
      { duration: "20s", target: 0 },
    ],
  };

  // Simulated user behavior
  export default function () {
    let res = http.get("https://example.com/ping");
    // Validate response status
    check(res, { "status was 200": (r) => r.status == 200 });
    sleep(1);
  }

You can run this script using the k6 command line tool:

  $ k6 run script.js

This will simulate 15 virtual users hitting the /ping endpoint of your API. Or you can run it in distributed mode using the k6 cloud command:

  $ k6 cloud script.js

More information about k6 can be found in the k6 documentation.