Discord's Use of Elixir to Handle Concurrency: A Template for Achieving High Performance

Stanislav Vishnevskiy (Discord CTO) recently shared in a blog post the details of Discord's elegant use of Elixir to handle its intense messaging needs. His article was of particular interest to me and to the Nanobox team for a couple reasons. Our distributed team uses Discord as a critical part of our team communication infrastructure. Also, our founder, Tyler Flint, and members our development team have been using and promoting the use of Erlang since before Elixir came along in 2011.

Using Discord To Manage A Remote Team

The Nanobox team uses Discord to keep our team of remote employees, most of whom are developers, connected and productive.

When we made the decision to move most of our team to a remote work environment (essentially disbanding our main office in Rexburg, Idaho as we moved south to Lehi, Utah), we spent hours looking for a solution that would allow us to effectively organize our developers, support people, marketing, and other teams. We wanted to simulate online how our physical office was setup, including having the ability to create virtual rooms where team members could meet. Naturally, like other communication apps designed to support gamers, Discord handles that well.

Just as important as the capability of mapping our physical office into a online communication app was the notion of not having meetings constantly interrupted by statements like, "Sorry, can you repeat that? You're breaking up". Many of the other voice communication apps available are dependent upon users having access to high speed internet, a resource which, even in our highly-connected world, is taken for granted more than it should be.

What has really impressed us with Discord is how robust its performance is. We've even found ourselves on occasion asking, "How does Discord do it?" When I first started with the Nanobox team, I was living in Costa Rica, working with an internet connection that tended to be slow and inconsistent. My remote introduction to the Nanobox team was an unintentional test of how well Discord handles shipping voice packets upstream and downstream when used on a not-all-that-high-speed internet backbone. Discord passed the test. With very few exceptions, mostly when my internet was completely down, I was able to stay connected with my team via Discord.

What Elixir Does for App Performance

Like others who have followed Elixir's progress and adoption over the past several years, we love that Elixir has put the concurrency capabilities of Erlang within reach of the common developer, making it easy to crank out solid functionality that handles intense usage environments and traffic spikes.

I've talked before about situations where engineers went out on a bit of a limb to choose Elixir over other more conventional language options (like Ruby), such as with the well-known example of Bleacher Report. In another recent example, Moz Director of Engineering Jeremy Modjeska in 2016 presented a thorough explanation of how his company used Elixir to support an aggressive data indexing model that upgraded the performance of their existing tools and paved the way for them to add highly-demanded new functionality.

It's no doubt Elixir is making accessible to app developers opportunities they otherwise might not even consider, or that they'd have to go to great efforts to realize.

One of the reasons we've liked Elixir at Nanobox is because out what it does to empower developers, which is exactly what our development platform is designed to do. We've created quickstart guides for Elixir programmers, and we've contributed to further developing the Elixir app infrastructure knowledge base, including explaining how to automate Elixir deployment. We'd love to see more marquee examples of scalable, highly concurrent app examples written in Elixir and running in the cloud.

Elixir and Discord

Discord's early choice of Elixir for coding their app foresaw the extent of their user base today: "five million concurrent users and millions of events per second flowing through the system". Vishnevskiy's description of how Discord put Elixir/Erlang to work describes meticulous attention to shaving off microseconds from each of the publish-subscribe messaging transactions that comprise the application. As Discord servers were setup to handle large groups, including as many as 30,000 people in the Overwatch reddit community, adjustments had to be made to fine-tune performance. The Discord engineers created an Elixir project they named Manifold to handle passing messages between nodes more efficiently.

Ultimately, Discord's work to stay ahead of increasing demand for their communication app validated their choice of Elixir. Their creation of new Elixir projects they published along the way (a Semaphore library was also needed for locking to prevent overloading processes) also helps to move the entire Elixir community forward. Their public documentation of how they handle concurrency and their code contributions add another layer to the language's foundation and provide a template for development teams who may be considering whether Elixir is right for their concurrency-intensive application.

Posted in Elixir