Go to content

De Wet Blomerus - Building a globally distributed router

Leave comments at: https://elixirforum.com/t/elixirconf-2023-de-wet-blomerus-building-a-globally-distributed-router/59595 To meet and exceed GDPR, we moved our EU customers to a new instance of our legacy Rails login service in the EU. But before we could do this, we needed a way to route traffic. We built a new Elixir service to sit in front of our legacy Rails login service to route traffic to the correct region. This router has an instance in the EU and the US so that EU traffic never touches the US. Elixir is uniquely suited for this purpose for several reasons. 1. All login traffic passes through this router, so it should add minimal latency ✅. 2. IT is a proxy, so it needs to hold on to each connection for a request to a large cluster of Rails servers for as long as the Rails server takes to respond, which ends up being a lot of requests if the Rails server slows down for any reason ✅ 3. The legacy service has over 200 routes. Different requests have different properties that can tell us which region the request should go to. Having a single `Plug.Conn` where we could place information as we figure it out and then pass responsibility to the next module in the pipeline helped us break a very complex problem into something solvable ✅ Mistakes. 1. I spent a lot of time thinking about optimizing routing speed. Trying not to pattern match all possible routes in a single request. Pattern matching 10s or even 100s of patterns is so fast that finding a quicker way was a waste of time. 2. We tried to flex our Elixir muscles by showing how few resources we could run in, which caused an outage about five months after launch. 3. We underestimated the project, it turns out a 6-year-old Rails app might be doing a lot of things. If you want to proxy it and be a router for it, you need to plan for many edge cases.

September 5, 2023