Perfect Elixir: Foundations of a Web App

By /

Today we’ll make a simple web app, and try to decipher what nuances and tradeoffs exist between the available solutions. So let’s dive in and see if one stands out as more applicable to our needs than the others. Ideally we’ll end the day with examples of apps that call into our database and returns some HTML, but let’s see how that ambition fares.

Table of Contents

Nothing At All

What if we… don’t choose a solution? It is technically possible to serve HTML directly with no dependencies at all by using the Erlang TCP/IP socket module, what would that look like? Not that this will be viable solution, but it might be illuminating to see what it takes to work without abstractions.

We’ll basically need to listen on a TCP socket and wait for traffic, and then throw data down the wire when a request arrives. Here’s a script that does that:

defmodule SimpleServer do def start(port) do # Listen on a TCP socket on the specified port # :binary - Treat data as raw binary, instead of # being automatically converted into # Elixir strings (which are UTF-8 encoded). # It'd be unnecessary to convert, as the # HTTP protocol uses raw bytes. # packet: :line - Frame messages using newline delimiters, # which is the expected shape of HTTP-data # active: false - Require manual fetching of messages. In # Erlang, active mode controls the # automatic sending of messages to the # socket's controlling process. We disable # this behavior, so our server can control # when and how it reads data {:ok, socket} = :gen_tcp.listen(port, [ :binary, packet: :line, active: false ]) IO.puts("Listening on port #{port}") loop_handle_client_connection(socket) end defp loop_handle_client_connection(socket) do # Wait for a new client connection. This is a blocking call # that waits until a new connection arrives. # A connection returns a `client_socket` which is connected # to the client, so we can send a reply back. {:ok, client_socket} = :gen_tcp.accept(socket) send_hello_world_response(client_socket) :gen_tcp.close(client_socket) # Recursively wait for the next client connection loop_handle_client_connection(socket) end defp send_hello_world_response(client_socket) do # Simple HTML content for the response. content = "<h1>Hello, World!</h1>" # Generate the entire raw HTTP response, which includes  # calculating content-length header. response = """ HTTP/1.1 200 OK content-length: #{byte_size(content)} content-type: text/html #{content} """ :gen_tcp.send(client_socket, response) end
end SimpleServer.start(8080)
Enter fullscreen mode
Exit fullscreen mode

It’s not exactly easy code to read because it deals with such low-level concepts (hence all the comments to clarify what the various commands do 😅). But it does let us start the script in one terminal and probe it with curl in another, and actually see HTML getting returned:

$ elixir simple_server.exs | $ curl http://localhost:8080
Listening on port 8080 | <h1>Hello, World!</h1>%
Enter fullscreen mode
Exit fullscreen mode

A very basic Hello world

So that works, if by “works” we mean this is technically returning HTML.

Is it actually useful? Not in any practical sense no, it’d be quite an adventure to properly implement all the features missing from this! But it hints at the hard work that gets solved by the libraries we’ll dive into next; it’s no exaggeration to say we stand on the shoulders of giants.

ℹ️ BTW the full code to this section is here.

Cowboy

Cowboy is a classic choice in Elixir: A small, minimalist HTTP server that’s proven itself for over a decade. It’s actually an Erlang library, the main language of the BEAM VM, but it’s worth including here because it’s used in so many Elixir projects.

To try out Cowboy we first need to scaffold an Elixir project, and there’s a straightforward to do that with mix, the Elixir build tool:

$ mix new cowboy_example
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/cowboy_example.ex
* creating test
* creating test/test_helper.exs
* creating test/cowboy_example_test.exs Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more: cd cowboy_example mix test Run "mix help" for more commands.
Enter fullscreen mode
Exit fullscreen mode

And then we add Cowboy as a dependency and install dependencies:

$ cd cowboy_example
$ git-nice-diff -U1
cowboy_example/mix.exs
L#23: [
+ {:cowboy, "~> 2.11"} # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get
Resolving Hex dependencies...
Resolution completed in 0.043s
New: cowboy 2.12.0 cowlib 2.13.0 ranch 1.8.0
* Getting cowboy (Hex package)
* Getting cowlib (Hex package)
* Getting ranch (Hex package)
You have added/upgraded packages you could sponsor, run `mix hex.sponsor` to learn more
Enter fullscreen mode
Exit fullscreen mode

ℹ️ BTW git-nice-diff is just a small script that works like git diff but simplifies the output to make it easier to show diffs in this article. You can find it here if you’re curious.

And then we need to write some code that actually configures and starts the Cowboy server:

defmodule CowboyExample do def start_server do # Set up the routing table for the Cowboy server, so root  #requests ("/") direct to our handler. dispatch = :cowboy_router.compile([{:_, [ {"/", CowboyExample.HelloWorldHandler, []} ]}]) # Start the Cowboy server in "clear mode" aka plain HTTP # options - Configuration options for the server itself # (this also supports which IP to bind to, # SSL details, etc.) # `env` - Configuration map for how the server # handles HTTP requests # (this also allows configuring timeouts, # compression settings, etc.) {:ok, _} = :cowboy.start_clear( :my_name, [{:port, 8080}], %{env: %{dispatch: dispatch}} ) IO.puts("Cowboy server started on port 8080") end
end defmodule CowboyExample.HelloWorldHandler do # `init/2` is the entry point for handling a new HTTP request # in Cowboy def init(req, _opts) do req = :cowboy_req.reply(200, %{ "content-type" => "text/html" }, "<h1>Hello World!</h1>", req) # Return `{:ok, req, state}` where `state` is # handler-specific state data; here, it's `:nostate` # as we do not maintain any state between requests. {:ok, req, :nostate} end
end
Enter fullscreen mode
Exit fullscreen mode

Previously in the “Nothing At All” example we ran our code as a script via elixir <script.exs>, but this time we’ll take a small step towards a more common approach for applications where we start an IEx (Interactive Elixir) shell. From that we manually (for now) start our server:

$ iex -S mix | $ curl http://localhost:8080
Generated cowboy_example app. | <h1>Hello World!</h1>% Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12] |
[ds:12:12:10] [async-threads: |
1] [dtrace] | |
Interactive Elixir (1.16.1) - | press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> CowboyExample.start_s |
erver |
Cowboy server started on port | 8080 |
:ok |
iex(2)> |
Enter fullscreen mode
Exit fullscreen mode

🎉 Hooray, we (once again) have an incredibly basic webserver. We’ve definitely moved up an abstraction level: No more dealing directly with sockets, the Cowboy server handles all that and “just” invokes our request/response handler function.

I still don’t find this easy to read, maybe mostly because the Erlang syntax adds an extra layer of mental translation. Not only in the code itself but also because Cowboy’s documentation is all Erlang-centric and so we have to understand how to map its examples into Elixir to make it all work. But at least the server works, and that’s a pretty good step!

ℹ️ BTW the full code to this section is here.

Plugged Cowboy

The previous example used plain Cowboy, but actually Cowboy seems to be more commonly used via a dependency called Plug, which is an Elixir library that that makes it easy to write html-responding functions. To use Plug with Cowboy the Plug documentation suggests using a library called plug_cowboy which combines both (it’s an adapter library that lets Plug control the Cowboy server). Let’s see how that looks!

First, we need a new project. But this time let’s do it so our server starts automatically when opening IEx. The standard solution for this is to generate an “OTP application“, which includes a main supervisor that will take care of our server process. We could create those files ourselves, but mix has a --sup flag that does it for us:

$ mix new --sup plugged_cowboy
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/plugged_cowboy.ex
* creating lib/plugged_cowboy/application.ex
* creating test
* creating test/test_helper.exs
* creating test/plugged_cowboy_test.exs Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more: cd plugged_cowboy mix test Run "mix help" for more commands.
Enter fullscreen mode
Exit fullscreen mode

And then we add our dependency:

$ cd plugged_cowboy
$ git-nice-diff -U1
/plugged_cowboy/mix.exs
L#24: [
+ {:plug_cowboy, "~> 2.0"} # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get
Resolving Hex dependencies...
Resolution completed in 0.104s
New: cowboy 2.10.0 cowboy_telemetry 0.4.0 cowlib 2.12.1 mime 2.0.5 plug 1.15.3 plug_cowboy 2.7.0 plug_crypto 2.0.0 ranch 1.8.0 telemetry 1.2.1
* Getting plug_cowboy (Hex package)
* Getting cowboy (Hex package)
* Getting cowboy_telemetry (Hex package)
* Getting plug (Hex package)
* Getting mime (Hex package)
* Getting plug_crypto (Hex package)
* Getting telemetry (Hex package)
* Getting cowlib (Hex package)
* Getting ranch (Hex package)
You have added/upgraded packages you could sponsor, run `mix hex.sponsor` to learn more
Enter fullscreen mode
Exit fullscreen mode

Then it’s time to write our request-handler plug, and we get to see why the Plug library is so widely used: It has nice and simple self-describing functions that makes it straightforward to generate a proper HTML response:

defmodule PluggedCowboy.MyPlug do import Plug.Conn # Plugs must define `init/1`, but we have nothing to configure so it's just a no-op implementation def init(options), do: options # `call/2` is the main function of a Plug, and is expected to process the request and generate a response def call(conn, _options) do conn # Both functions below are part of `Plug.Conn`s functions, they're available because we imported it |> put_resp_content_type("text/plain") |> send_resp(200, "Hello, World!") end
end
Enter fullscreen mode
Exit fullscreen mode

And finally we configure and register Plug.Cowboy to use our plug:

$ git-nice-diff -U1
/plugged_cowboy/lib/plugged_cowboy/application.ex
L#10: children = [
+ # Connect Plug.Cowboy plug handler
+ {Plug.Cowboy, plug: PluggedCowboy.MyPlug, scheme: :http, options: [port: 8080]} # Starts a worker by calling: PluggedCowboy.Worker.start_link(arg)
Enter fullscreen mode
Exit fullscreen mode

And… that all just works:

iex -S mix | $ curl http://localhost:8080
Erlang/OTP 26 [erts-14.2.2] [ | Hello, World!%
source] [64-bit] [smp:12:12] | [ds:12:12:10] [async-threads: |
1] [dtrace] |
Interactive Elixir (1.16.1) - | press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> |
Enter fullscreen mode
Exit fullscreen mode

This plug-variant of Cowboy has quite nicely simplified what it takes to start the server, which is exactly why Plug exists. And we even got our code more aligned to normal idiomatic Elixir 👌

ℹ️ BTW the full code to this section is here.

Bandit

This project has a funny name, because it’s an alternative to Cowboy: Bandit is a webserver written in pure Elixir, and it has an internal design that more naturally integrates with the underlying Erlang concurrent programming model which results in greater performance.

ℹ️ BTW the naming-fun goes even deeper: Cowboy pulls in Ranch as a dependency, and so Bandit pulls in… Thousand Island 😂 (non-native English speakers might have to look up the dictionary definitions to follow these puns).

Like before: New project, then add and get our dependencies. And we’ll need to pull in Plug directly this time, because Bandit is already designed to be used with it:

$ mix new --sup bandit_example > /dev/null
$ cd bandit_example
$ git-nice-diff -U1
/bandit_example/mix.exs
L#24: [
+ {:bandit, "~> 1.0"},
+ {:plug, "~> 1.0"} # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get
Enter fullscreen mode
Exit fullscreen mode

ℹ️ BTW the > /dev/null just silences the mix new command, so we don’t have to cover as much output.

And then we write… pretty much exactly the same code as we did for Plugged Cowboy:

git-nice-diff -U1
/bandit_example/lib/bandit_example/application.ex
L#10: children = [
+ {Bandit, plug: BanditExample.MyPlug, port: 8080} # Starts a worker by calling: BanditExample.Worker.start_link(arg)
/bandit_example/lib/bandit_example/my_plug.ex
L#1:
+defmodule BanditExample.MyPlug do
+ import Plug.Conn
+
+ def init(options), do: options
+
+ def call(conn, _options) do
+ conn
+ |> put_resp_content_type("text/plain")
+ |> send_resp(200, "Hello, World!")
+ end
+end
Enter fullscreen mode
Exit fullscreen mode

And that just works:

$ iex -S mix | $ curl http://localhost:8080 | Hello, World!%
Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12] |
[ds:12:12:10] [async-threads: |
1] [dtrace] | |
Interactive Elixir (1.16.1) - | press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> |
Enter fullscreen mode
Exit fullscreen mode

Bandit ends up being almost a drop-in replacement for Cowboy, and that’s by design because Plug is really the primary developer-facing part of this sever-equation. So that’s a pretty cool separation of concerns there, Elixir community. Well done 👍

But this time, could we go one step further and connect to our database?

Connecting To Database

First we need a way for Elixir to connect to our database, and Elixir’s Postgres database adapter is called Postgrex. So let’s download that:

$ git-nice-diff -U1
bandit_example/mix.exs
L#25: {:bandit, "~> 1.0"},
- {:plug, "~> 1.0"}
+ {:plug, "~> 1.0"},
+ {:postgrex, ">= 0.0.0"} # {:dep_from_hexpm, "~> 0.3.0"},
$ mix deps.get > /dev/null
Enter fullscreen mode
Exit fullscreen mode

And the second step to connecting to a database is to start that database 😅. We need to store its files somewhere, and it’s an Elixir convention (actually Erlang, but, same point) to put them in priv/db because that’s a folder meant for private, static files that are “part of the application but aren’t source code”:

$ mkdir -p priv/db $ initdb -D priv/db
$ pg_ctl -D priv/db -l logfile start
Enter fullscreen mode
Exit fullscreen mode

And we also need a user and a database so there’s something to connect to:

$ createuser -d bandit $ createdb -O bandit bandit
Enter fullscreen mode
Exit fullscreen mode

Then we register the Postgrex process as part of our main supervisor, so it starts alongside our webserver:

$ git-nice-diff -U1
bandit_example/lib/application.ex
L#10: children = [
- {Bandit, plug: BanditExample.MyPlug, port: 8080}
+ {Bandit, plug: BanditExample.MyPlug, port: 8080},
+ {Postgrex,
+ [
+ name: :bandit_db,
+ hostname: "localhost",
+ username: "bandit",
+ password: "bandit",
+ database: "bandit"
+ ]
+ } # Starts a worker by calling: BanditExample.Worker.start_link(arg)
Enter fullscreen mode
Exit fullscreen mode

And now we can jam a database query into our plug to prove it all works:

git-nice-diff -U1
bandit_example/lib/my_plug.ex
L#6: def call(conn, _options) do
+ %Postgrex.Result{rows: [[current_time]]} =
+ Postgrex.query(:bandit_db, "SELECT NOW() as current_time", [])
+ conn |> put_resp_content_type("text/plain")
- |> send_resp(200, "Hello, World!")
+ |> send_resp(200, "Hello, World! It's #{current_time}") end
Enter fullscreen mode
Exit fullscreen mode

And indeed it works:

20:04:22.598 [info] Running B | $ curl http://localhost:8080
anditExample.MyPlug with Band | Hello, World! It's 2024-03-17
it 1.3.0 at 0.0.0.0:8080 (htt | 19:04:24.547992Z%
p) | Erlang/OTP 26 [erts-14.2.2] [ |
source] [64-bit] [smp:12:12] |
[ds:12:12:10] [async-threads: |
1] [dtrace] | |
Interactive Elixir (1.16.1) - | press Ctrl+C to exit (type h |
() ENTER for help) |
iex(1)> |
Enter fullscreen mode
Exit fullscreen mode

🎉 We now have a basic web app connected to its database.

Granted, “basic” is doing a lot of heavy lifting in that sentence, because we would still have much work to do to have a truly useful and flexible app (For starters we could probably do better than jamming a raw query directly into a plug, to decouple HTML concerns from data-persistence concerns). But instead of getting stuck in that, let’s try out the next solution and see what it has to offer.

ℹ️ BTW the full code to this section is here.

Phoenix Framework

Of course we’re not skipping Phoenix in this article, it’s the elephant in the room for any topic that intersects Elixir and web. And it’s not unearned: Phoenix is a powerful and flexible framework, with tons of great developer-ergonomic features, and it supports creating all kinds of web applications small and huge.

ℹ️ BTW I’m usually weary of big web frameworks, having often gotten tangled up in their abstractions in other languages and come away regretting their hard-to-debug abstractions. Instead I prefer composing individual dependencies together. But… Phoenix is different, as we’ll see hints of in the below exploration.

Phoenix has a very easy way to get started, courtesy of a well-documented Getting Started guide that lay out the commands to generate and bootstrap an entire Phoenix sample app:

$ mix local.hex --force && mix archive.install hex phx_new --force
$ mix phx.new my_app # answer yes when prompted
$ cd my_app
$ mkdir -p priv/db && initdb -D priv/db && pg_ctl -D priv/db -l logfile start && createuser -d postgres
$ mix deps.get && mix ecto.create
Enter fullscreen mode
Exit fullscreen mode

ℹ️ BTW if you get errors from the database command you might already be running a database from earlier sections. You can reset Postgres using these commands as necessary, to get back on track:

$ lsof -ti :5432 | xargs -I {} kill {}; rm -rf priv/db # Kill all processes on Postgres' default port
$ rm -rf priv/db # Delete the local DB data folder

And done!, this gets us a web-app we can start with iex -S mix phx.server and visit by browsing to http://localhost:4000:

Screenshot of Phoenix Framework's default page. It shows the Phoenix logo of an abstract bird, with the text "Phoenix Framework v1.7.11 in small text, and a big title "Peace of mind from prototype to production."

What Phoenix Gives Us

The Phoenix generator made us a project that already connects to our Postgres database, which is quite convenient. It does this by integrating with Ecto, which is a highly popular Elixir database wrapper and query generator that itself uses Postgrex under the hood to connect to the database.

If we check out config/dev.exs we can actually spot the same pattern of hardcoded database credentials as we used in the Bandit section:

$ cat config/dev.exs
…
# Configure your database
config :my_app, MyApp.Repo, username: "postgres", password: "postgres", hostname: "localhost", database: "my_app_dev",
…
Enter fullscreen mode
Exit fullscreen mode

Well, almost like we did in Bandit section, because this configures something called MyApp.Repo. That’s a module we can see is implementing an Ecto.Repo behavior and it gets started as a child of the main supervisor very much like we did with Postgrex:

$ cat lib/my_app/repo.ex
defmodule MyApp.Repo do use Ecto.Repo, otp_app: :my_app, adapter: Ecto.Adapters.Postgres
end
$ cat lib/my_app/application.ex
… def start(_type, _args) do children = [ … MyApp.Repo,
…
Enter fullscreen mode
Exit fullscreen mode

And that adapter: Ecto.Adapters.Postgres configuration field is how Ecto gets configured to resolve queries via Postgrex. So, when Ecto generates an SQL query we can imagine how it passes it off to Postgrex somewhere within its logic. Ecto even requires us to depend on Postgrex as a dependency to use that adapter, and indeed we can see Postgrex in our list of dependencies:

$ cat mix.exs | grep postgrex
 {:postgrex, ">= 0.0.0"},
Enter fullscreen mode
Exit fullscreen mode

To actually use the database we can create a module that does a query, and then call that function from our page controller:

$ git-nice-diff -U1
my_app/lib/my_app/query.ex
L#1:
+defmodule MyApp.Query do
+ import Ecto.Query
+
+ alias MyApp.Repo
+
+ def get_db_time do
+ # The SELECT 1 is a dummy table to perform a query without a table
+ query = from u in fragment("SELECT 1"), select: fragment("NOW()")
+ query |> Repo.all() |> List.first()
+ end
+end
my_app/lib/my_app_web/controllers/page_controller.ex
L#6:
 # so skip the default app layout.
- render(conn, :home, layout: false)
+ db_time = MyApp.Query.get_db_time()
+ render(conn, :home, layout: false, db_time: db_time) end
my_app/lib/my_app_web/controllers/page_html/home.html.heex
L#42:
 <div class="mx-auto max-w-xl lg:mx-0">
+ <h1 class="text-lg">Database Time: <%= @db_time %></h1>
 <svg viewBox="0 0 71 48" class="h-12" aria-hidden="true">
Enter fullscreen mode
Exit fullscreen mode

It’s not a visually stunning result, but we’re certainly making a database query 🎉:

Screenshot of Phoenix Framework's default page with extra text added to the top that reads "Database Time" followed by a timestamp. The rest of the page is the Phoenix default page, showing the Phoenix logo of an abstract bird, with the text "Phoenix Framework v1.7.11 in small text, and a big title "Peace of mind from prototype to production."

ℹ️ BTW the full code to this section is here.

It’s the ability to follow these abstractions that makes me particularly happy with Phoenix: It’s not a framework with a hundred levels of abstracted and complex code; if we wanted to add proper database support to our own Bandit project we’d end up doing it pretty much exactly like Phoenix is doing it by pulling in Ecto and configuring the whole thing from a configuration file. It’s… simple. Which is a tremendous quality.

Conclusion

It’s been a lot of fun to try various solutions, but it’s hard to see a realistic alternative to Phoenix for our needs. I’m sure there are contexts where it makes sense to not choose Phoenix, but for our case it matches like a glove: Simple and easy to get started with, supporting incredible scale, with several really impressive features that we didn’t even touch on today, with clear and easy to approach documentation, and tons of examples in various articles to lean on. Phoenix is a juggernaut in the Elixir community and for good reason.

So I’ll continue this article-series based on Phoenix.

Discover more from Coursity

Subscribe to get the latest posts sent to your email.

Leave a Comment

Your email address will not be published. Required fields are marked *

Discover more from Coursity

Subscribe now to keep reading and get access to the full archive.

Continue reading

Scroll to Top