2017/11/25

Minimal reverse proxy in go

During the last Copenhagen Gophers meetup, we had some talks about:

  1. Writing applications on google appengine
  2. A personal top 5 of projects in golang.org/x repository where “acme/autocert” was top 1.

The talk about google appengine was good and the speaker made me re-remember that when doing side projects, we do not want to spend all our time setting up complex systems for getting our side project out there. Here i’m thinking, configuring webservers, setting up lets-encrypt, configuring databases, containers, orchestration, etc. etc. Unless that is what your side project is about of course, or that is what scratches our itch. Sometimes all we want is a solution with “batteries included” to get it out there quick.

For the kind of side projects i like to work on, i would love to have someting in between. I’d like to host it myself on the same server but i don’t want to bother with configuring nginx every time i want to put a new side project online.

So how hard could it be to replace nginx with my own home written reverse proxy that fetches lets-encrypt certificates on the fly for the domains that i add. I wanted a solution with minimal configuration.

I came up with a a solution less than 100 lines of code and that does what i need it to do.

package main

import (
	"fmt"
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
	"sync"

	"golang.org/x/crypto/acme/autocert"
)

var (
	proxies map[string]*httputil.ReverseProxy
	statics map[string]http.Handler
)

type Config struct {
	Host   string
	Server string
}

func main() {

	cfgs := []Config{
		Config{
			Host:   "blog.sketchground.dk",
			Server: "http://127.0.0.1:9900",
		},
		Config{
			Host:   "journal.sketchground.dk",
			Server: "http://127.0.0.1:9900",
		},
		Config{
			Host:   "www.ikurven.dk",
			Server: "static:///var/www/ikurvendk",
		},
	}
	hosts := []string{}

	// Load services...
	proxies = map[string]*httputil.ReverseProxy{}
	statics = map[string]http.Handler{}
	for _, cfg := range cfgs {
		u, _ := url.Parse(cfg.Server)
		if u.Scheme == "static" {
			log.Printf("Initializing static server for %v -> %v\n", cfg.Host, cfg.Server)
			statics[cfg.Host] = http.FileServer(http.Dir(u.Path))
		} else {
			log.Printf("Initializing proxy connection for %v -> %v\n", cfg.Host, cfg.Server)
			proxies[cfg.Host] = httputil.NewSingleHostReverseProxy(u)
		}
		hosts = append(hosts, cfg.Host)
	}

	wg := sync.WaitGroup{}
	wg.Add(1)
	go func() {
		log.Println("Starting reverse proxy for ssl connections")
		log.Fatal(http.Serve(autocert.NewListener(hosts...), &P{secure: true}))
		wg.Done()
	}()
	wg.Add(1)
	go func() {
		log.Println("Starting reverse proxy for http connections")
		log.Fatal(http.ListenAndServe(":80", &P{})) // port 80
		wg.Done()
	}()
	wg.Wait()
}

type P struct {
	secure bool
}

func (p *P) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
	if !p.secure { // Redirect always if not secure.
		u := fmt.Sprintf("https://%v%v", req.Host, req.URL.Path)
		http.Redirect(rw, req, u, http.StatusFound)
		return
	}

	if h, ok := proxies[req.Host]; ok { // Check if we have proxies
		h.ServeHTTP(rw, req)
		return
	}
	if h, ok := statics[req.Host]; ok { // Check if we have statics
		h.ServeHTTP(rw, req)
		return
	}

	fmt.Fprintf(rw, "Nothing here. Go elsewhere.") // Return if no hosts match
	return
}

This code is also available at github

When it’s time for some more complex, it’s always possible to replace it with nginx/apache/etc. again.

If you enjoyed reading this, you are more than welcome to support my writing/work/whatever on patreon.


2017/11/18

Sorting the gophers

Sorting in go has always been a bit different than other languages. Not that the algorithms for sorting is different, but merely by the fact that go does not have generics.

This is not going to be about the pros/cons of generics. There are plenty of that on the web.

On the other hand i will go through a bunch of examples how to sort your data.

Go contains built in sorting functions for strings and ints and floats, however if you have a slice of structs you are bound to implement some sorting code.

We will sort the gophers using this structure:

type Gopher struct {
	Name string
	Age  int
}

var gophers = []Gopher{
	{"Bob", 31},
	{"John", 42},
	{"Michael", 17},
	{"Jenny", 26},
	{"Alice", 17},
	{"John", 17},
}

Before go 1.8, sorting had to be done by implementing the sort interface (https://golang.org/pkg/sort/#Interface).

// ByAge helper type to sort the gophers by age
type ByAge []Gopher

func (a ByAge) Len() int           { return len(a) }
func (a ByAge) Swap(i, j int)      { a[i], a[j] = a[j], a[i] }
func (a ByAge) Less(i, j int) bool { return a[i].Age < a[j].Age }

While this required a bunch of boiler plate code to be written, it was reusable across your codebase.

In go 1.8 we got an extra function in the sort package “sort.Slice” that essentially took a function identically to the Less function in the old sort interface. Yay! We can now get rid of some boiler plate code.

sort.Slice(a, func(i, j int) bool {
	return a[i].Age < a[j].Age
})

However we still have to write such a func for each struct we want sorted. Now what if you want to reuse your new sort function? With the old sort interface reusability was easy as it was declared as a type. Well, you still have the possibility to wrap it in another func to help you reusing it.

byAge := func(a []Gopher) func(int, int) bool {
	return func(i, j int) bool {
		return a[i].Age < a[j].Age
	}
}
sort.Slice(a, byAge(a))

If you have multiple reusable sorting functions you can still wrap them in a helper type as what we did with the original Interface.

// NewSorter helper type for multiple sort funcs
type NewSorter []Gopher

// ByAge sorts by age
func (a NewSorter) ByAge(i, j int) bool {
	return a[i].Age < a[j].Age
}

// ByName sorts by name
func (a NewSorter) ByName(i, j int) bool {
	return a[i].Name < a[j].Name
}
sorter := NewSorter(gophers)
sort.Slice(gophers, sorter.ByAge)

This opens up for the possibility to choose order of sorting, sort by multiple keys in the struct etc.

// Ordersorter is a helper type to support order
type Ordersorter struct {
	a     []Gopher
	order bool // asc = true // desc = false
}

// Asc sets asc flag
func (s Ordersorter) Asc() Ordersorter {
	s.order = true
	return s
}

// Desc sets desc flag
func (s Ordersorter) Desc() Ordersorter {
	s.order = false
	return s
}

// ByAge sorts by age
func (s Ordersorter) ByAge(i, j int) bool {
	if s.order {
		return s.a[i].Age < s.a[j].Age
	}
	return s.a[i].Age > s.a[j].Age
}

// NewOrderSorter returns a new instance of ordersorter
func NewOrderSorter(a []Gopher) Ordersorter {
	return Ordersorter{a, true}
}

sorter := NewOrderSorter(gophers)
sort.Slice(gophers, sorter.Desc().ByAge)
sort.Slice(gophers, sorter.Asc().ByAge)

Now you migh think that it is a whole lot of code to write for sorting your struct. And i agree. It is. Which leads me the conclusion.

1) Keep your implementation as simple as possible. 2) Expand your implementation if the need arises. 3) Don’t go all in if all you need is to sort a slice of structs by an int field. Simply use the new sort.Slice func.

How about writing a generator to generate go code to sort your struct in all your possible ways? I haven’t gotten around to that yet but it’s certainly possible to take advantage of go generate for this task. I’ll leave the task for somebody else to implement.

Don’t go generate a bunch of boiler plate code if it’s not needed. It only leads to more code to reason about, more code to maintain, etc.

If you enjoyed reading this, you are more than welcome to support my writing/work/whatever on patreon. For disussions of alternative implementations etc. (not generics related) go here.