Currying, or partial capture in Go?

Truth be told, you’ve probably already used it many many times when writing http handlers.

For this article though, I’m going to use a real life, actual piece of code I’m working with in our product. It’s open source, the PR is publicly visible, so I’m fairly certain I’m not violating anything in my contract, but will double check with Legal™. If you’re reading this, they said it’s fine, and I have the email to prove it.

For clarity, I’m using the code as a vehicle to demonstrate a concept, rather than talking about the software I’m working on itself.

Anyways, on to the code.

At some point in NGINX Ingress Controller, we need to take config map values, parse them, and convert them to our internal Go structure. The config map values live in a yaml file, and the format is string keys to string values, per the kubernetes config map spec. Occasionally the string values that come in from the yaml get turned into a different format: bool, integer, float, or some list of allowed values (Go does not have enums).

I’m working on the specific part that takes in a bunch of config map values related to OIDC functionality. Here’s what the current code looks like:

// parseConfigMapOIDC parses OIDC timeout configuration from ConfigMap.
func parseConfigMapOIDC(l *slog.Logger, cfgm *v1.ConfigMap, cfgParams *ConfigParams, eventLog record.EventRecorder) error {
	if oidcPKCETimeout, exists := cfgm.Data["oidc-pkce-timeout"]; exists {
		pkceTimeout, err := ParseTime(oidcPKCETimeout)
		if err != nil {
			errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for 'oidc-pkce-timeout': %q, must be a valid nginx time (e.g. '90s', '5m', '1h')", cfgm.Namespace, cfgm.Name, oidcPKCETimeout)
			nl.Warn(l, errorText)
			eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
			return err
		}
		cfgParams.OIDC.PKCETimeout = pkceTimeout
	}
	if oidcIDTokensTimeout, exists := cfgm.Data["oidc-id-tokens-timeout"]; exists {
		idTokensTimeout, err := ParseTime(oidcIDTokensTimeout)
		if err != nil {
			errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for 'oidc-id-tokens-timeout': %q, must be a valid nginx time (e.g. '1h', '30m', '2h')", cfgm.Namespace, cfgm.Name, oidcIDTokensTimeout)
			nl.Warn(l, errorText)
			eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
			return err
		}
		cfgParams.OIDC.IDTokenTimeout = idTokensTimeout
	}
	if oidcAccessTokensTimeout, exists := cfgm.Data["oidc-access-tokens-timeout"]; exists {
		accessTokensTimeout, err := ParseTime(oidcAccessTokensTimeout)
		if err != nil {
			errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for 'oidc-access-tokens-timeout': %q, must be a valid nginx time (e.g. '1h', '30m', '2h')", cfgm.Namespace, cfgm.Name, oidcAccessTokensTimeout)
			nl.Warn(l, errorText)
			eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
			return err
		}
		cfgParams.OIDC.AccessTimeout = accessTokensTimeout
	}
	if oidcRefreshTokensTimeout, exists := cfgm.Data["oidc-refresh-tokens-timeout"]; exists {
		refreshTokensTimeout, err := ParseTime(oidcRefreshTokensTimeout)
		if err != nil {
			errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for 'oidc-refresh-tokens-timeout': %q, must be a valid nginx time (e.g. '8h', '12h', '24h')", cfgm.Namespace, cfgm.Name, oidcRefreshTokensTimeout)
			nl.Warn(l, errorText)
			eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
			return err
		}
		cfgParams.OIDC.RefreshTimeout = refreshTokensTimeout
	}
	if oidcSIDSTimeout, exists := cfgm.Data["oidc-sids-timeout"]; exists {
		sidsTimeout, err := ParseTime(oidcSIDSTimeout)
		if err != nil {
			errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for 'oidc-sids-timeout': %q, must be a valid nginx time (e.g. '8h', '12h', '24h')", cfgm.Namespace, cfgm.Name, oidcSIDSTimeout)
			nl.Warn(l, errorText)
			eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
			return err
		}
		cfgParams.OIDC.SIDSTimeout = sidsTimeout
	}
	return nil
}

This is a lot of repetition between the individual blocks. For fun, I extracted two of them into different files and ran a diff between them: https://www.diffchecker.com/vlL0olI8/.

Hover over this box to see all the different things that are getting replaced:

if oidcPKCETimeoutoidcIDTokensTimeout, exists := cfgm.Data["oidc-pkce-timeoutoidc-id-tokens-timeout"]; exists {
    pkceTimeoutidTokensTimeout, err := ParseTime(oidcPKCETimeoutoidcIDTokensTimeout)
    if err != nil {
        errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for 'oidc-pkce-timeoutoidc-id-tokens-timeout': %q, must be a valid nginx time (e.g. '90s', '5m', '1h')", cfgm.Namespace, cfgm.Name, oidcPKCETimeoutoidcIDTokensTimeout)
        nl.Warn(l, errorText)
        eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
        return err
    }
    cfgParams.OIDC.PKCETimeoutIDTokenTimeout = pkceTimeoutidTokensTimeout
}

This means I can practically split this out into its own function. With some modification, the generic naming would look like this:

if oidcPKCETimeoutvalue, exists := cfgm.Data["oidc-pkce-timeoutkey"]; exists {
    pkceTimeoutparsedValue, err := ParseTime(oidcPKCETimeoutvalue)
    if err != nil {
        errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for 'oidc-pkce-timeoutkey': %q, must be a valid nginx time (e.g. '90s', '5m', '1h')", cfgm.Namespace, cfgm.Name, oidcPKCETimeoutvalue)
        nl.Warn(l, errorText)
        eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
        return err
    }
    cfgParams.OIDC.PKCETimeoutPropertyName = pkceTimeoutparsedValue
}

After some slight refactoring, I can arrive at the following function. Hover still works:

func parseStringField(cfgm *v1.ConfigMap, key string, parseFunc func(string) (string, error), assignFunc func(string), suggestion string, l *slog.Logger, eventLog record.EventRecorder) error {
	if value, exists := cfgm.Data[key]; exists {
		parsedValue, err := parseFunc(value)
		if err != nil {
			errorText := fmt.Sprintf("ConfigMap %s/%s: invalid value for '%s': %q, %v, %s", cfgm.GetNamespace(), cfgm.GetName(), key, value, err, suggestion)
			nl.Error(l, errorText)
			eventLog.Event(cfgm, v1.EventTypeWarning, nl.EventReasonInvalidValue, errorText)
			return err
		}
		assignFunc(parsedValue)
	}
	return nil
}

This would work like this:

// parseConfigMapOIDC parses OIDC timeout configuration from ConfigMap.
func parseConfigMapOIDC(l *slog.Logger, cfgm *v1.ConfigMap, cfgParams *ConfigParams, eventLog record.EventRecorder) error {
    timeSuggestion := "must be a valid nginx time (e.g. '90s', '5m', '1h')"
    
    err := parseStringField(cfgm, "oidc-pkce-timeout", ParseTime, func(value string) {
        cfgParams.OIDC.PKCETimeout = value
    }, timeSuggestion, l, eventLog)
    if err != nil {
        return fmt.Errorf("parsing 'oidc-pkce-timeout': %v", err)
    }

    err := parseStringField(cfgm, "oidc-id-tokens-timeout", ParseTime, func(value string) {
        cfgParams.OIDC.IDTokenTimeout = value
    }, timeSuggestion, l, eventLog)
    if err != nil {
        return fmt.Errorf("parsing 'oidc-id-tokens-timeout': %v", err)
    }

    // and the others
    return nil
}

This is significantly cleaner already. That said there are still a few things that I keep passing in which do not need to be passed in all the time. I mean they do, but I don’t need to do it every time individually.

Introducing: partial application! Or currying, whichever you prefer!

The definition of currying is when you turn a function that has many arguments into many functions that have each one argument. Or in functional notation:

f(a, b, c) turns into f(a)(b)(c).

func addAll(a, b, c int) int {
    return a + b + c
}

// extract them into each layer
func addEach(a int) func(int) func(int) int {
	return func(b int) func(int) int {
		return func(c int) int {
			return a + b + c
		}
	}
}

// to use
func main() {
    d := addAll(1, 2, 3) // 6

    e := addEach(1)(2)(3) // also 6
}

Okay, looks nice, but what does that even mean!?

func main() {
    intermediary := addEach(1)

    // the above is the same as this
    intermediary := func(b int) func(int) int {
	    return func(c int) int {
	        return 1 + b + c // this 1 comes from addEach(1). We stored
                             // the 1 from that function call into here
	    }
	}

    // and then
    second := intermediary(2)

    // is the same as
    second := func(c int) int {
        return 1 + 2 + c // 1 is from addEach(1), and
                         // 2 is from intermediary(2)
    }

    value := second(3) // 6

    // to recap
    intermediary := addEach(1)
    second := intermediary(2)
    value := second(3)

    // or to collapse calling the function "second"
    intermediary := addEach(1)
    value := intermediary(2)(3)

    // and also collapsing "intermediary"
    value := addEach(1)(2)(3)
}

Now the fun thing is that you don’t need to do all this work all the time. Technically nothing prescribes you to create functions that only have a single argument each time, you can separate the arguments between things you care to change every time, and things that should stay the same all the time.

To go back to our parseStringField function example, for the time version I can create a partially curried, or partially applied function:

parseTimeField := func(key string, assignFunc func(value string)) error {
    return parseStringField(cfgm, key, ParseTime, assignFunc, timeSuggestion, l, eventLog)
}

If you compare that to the signature of the parseStringField function, you’ll notice I separated the arguments:

func parseStringField(cfgm *v1.ConfigMap, key string, parseFunc func(string) (string, error), assignFunc func(string), suggestion string, l *slog.Logger, eventLog record.EventRecorder) error { ... }

Anything in orange (cfgm, parseFunc, suggestion, l, eventLog) gets fixed, or partially applied, and anything in purple (key, assignFunc) is still an argument in the returned function. That way when you call the parseTimeField function, the orange functions are always going to be passed in by default without you having to explicitly deal with it, so you can focus on only the bits that you want.

Using it will look like this:

// parseConfigMapOIDC parses OIDC timeout configuration from ConfigMap.
func parseConfigMapOIDC(l *slog.Logger, cfgm *v1.ConfigMap, cfgParams *ConfigParams, eventLog record.EventRecorder) error {
    timeSuggestion := "must be a valid nginx time (e.g. '90s', '5m', '1h')"
    
    parseTimeField := func(fieldName string, assignFunc func(value string)) error {
		return parseStringField(cfgm, fieldName, ParseTime, assignFunc, timeSuggestion, l, eventLog)
	}
    
    err := parseTimeField("oidc-pkce-timeout", func(value string) {
        cfgParams.OIDC.PKCETimeout = value
    })
    if err != nil {
        return fmt.Errorf("parsing 'oidc-pkce-timeout': %v", err)
    }

    err := parseTimeField("oidc-id-tokens-timeout", func(value string) {
        cfgParams.OIDC.IDTokenTimeout = value
    })
    if err != nil {
        return fmt.Errorf("parsing 'oidc-id-tokens-timeout': %v", err)
    }

    // and the others
    return nil
}

Extending this to also work with other field types, like sizes, becomes fairly easy with minimal code additions.

Slight segue regarding function definition scope

Oh, you might notice that the function is weird, because essentially I’m doing the following:

variable := func(arg1 string, arg2 string) error {
    return otherFunction(outsideArg1 *someType, arg1 string, outsideArg2 *someOtherType, arg2 string)
}

This works, as long as outsideArg1 and outsideArg2 are in scope of where I’m declaring variable. A lot of the examples you see in the Go library and very popular packages, including echo (the http framework), have a somewhat different signature that would look like this for me:

func makeSimplerFunction(outsideArg1 *someType, outsideArg2 *someOtherType) func(string, string) error {
    return func(arg1 string, arg2 string) error {
        return otherFunction(outsideArg1 *someType, arg1 string, outsideArg2 *someOtherType, arg2 string)
    }
}

And then the way I’d use it above is:

variable := makeSimplerFunction(outsideArg1, outsideArg2)

err := variable(arg1, arg2)
if err != nil {
    // the usual, sir?
}

Anyways, slight segue! Let’s get back to extending things!

To add zone sizes, I need to parse some more config maps, and make sure that they look like sizes that NGINX will like. Values can be things like 1024, 16k, 4m, and so on. There’s a function already for this, called ParseSize, so I get to use it! I’d also need to change what the error message says, which means a new sizeSuggestion too, but otherwise it’s mostly the same. Thankfully I don’t need to do anything other than creating a different partially applied function. Behold:

// parseConfigMapOIDC parses OIDC timeout configuration from ConfigMap.
func parseConfigMapOIDC(l *slog.Logger, cfgm *v1.ConfigMap, cfgParams *ConfigParams, eventLog record.EventRecorder) error {
    ...
    sizeSuggestion := "must be a valid nginx size (e.g. '16k', '1m')"

    parseSizeField := func(fieldName string, assignFunc func(value string)) error {
        return parseStringField(cfgm, fieldName, ParseSize, assignFunc, sizeSuggestion, l, eventLog)
	}
    
    err := parseSizeField("oidc-pkce-zone-size", func(value string) {
    	cfgParams.OIDC.PKCEZoneSize = value
    })
    if err != nil {
    	return fmt.Errorf("parsing 'oidc-pkce-zone-size': %v", err)
    }

    err := parseSizeField("oidc-id-tokens-zone-size", func(value string) {
    	cfgParams.OIDC.IDTokenZoneSize = value
    })
    if err != nil {
    	return fmt.Errorf("parsing 'oidc-id-tokens-zone-size': %v", err)
    }

    // and the others
    return nil
}

Does it take some time to wrap your head around it? Absolutely!

Is it significantly better than repeating the same block of code each time? Without a doubt.

Is it idiomatic Go? Yes. You’re already using it when dealing with http handlers, see the wiki for writing web applications:

Writing Web Applications - The Go Programming Language

Google logo

The makeHandler function there returns a different function, an http.HandlerFunc, which does some work, and then calls the function you passed into it. The above does the same job.

Now go forth, and deduplicate some code!

Photo by Yubraj Timsina on Unsplash