"Code reuse is the Holy Grail of Software Engineering." ~ Douglas Crockford
Once you understand basic Elixir
syntax
you may be wondering how to reuse
both your own code across projects
and other people's code in your projects ...
The more (high quality) code you are able to reuse, the more creative and interesting work you can do because you aren't wasting time reimplementing basic functionality or writing boring "boilerplate".
Let's do it!
- Reusing
Elixir
: How to Use + Publish Code on Hex.pm 📦 - Why?
- What?
- How?
"Good programmers know what to write. Great ones know what to rewrite (and reuse)."
~ Eric S. Raymond (The Cathedral and the Bazaar)
The biggest advantages of code reuse are:
- Independently tested small pieces of code that do only one thing. (Curly's Law) 🥇
- Work can be subdivided among people/teams
with clear responsibilities. ✅
Or if you are solo developer, having small chunks of code helps you bitesize your work so it's more manageable. 🙌 - Leverage other people's code to reduce your own efforts and ship faster. 🚀
"If I have seen further than others, it is by standing upon the shoulders of giants." ~ Isaac Newton
We can adapt this quote to a software engineering context as:
"If I have shipped faster and more interesting apps it is by building on the work of giants." ~ Experienced Engineer
In this example we are going to build a simple Elixir module
that returns a random inspiring quote.
The functionality of the module is intentionally
simple to illustrate code reuse in the most basic form.
Along the way we will demonstrate how to:
- Write, document and test a basic package.
- Reuse code without publishing to Hex.pm.
- Publish a package to Hex.pm
- Use the code in a different project.
A quotation, often abbreviated to quote,
is the repetition of someone else's statement or thought.
Quotes are usually an expression of wisdom in a concise form.
They often condense a lifetime of learning into a single sentence
and as such are worthy of our attention.
In our example we will be focussing on
a subset of quotes; the thought-provoking kind
(often called inspirational or motivational).
e.g:
"If you think you are too small to make a difference, try sleeping with a mosquito." ~ Dalai Lama
"Your time is limited, so don’t waste it living someone else’s life." ~ Steve Jobs
"If you get tired, learn to rest, not to quit." ~ Banksy
There are many uses for quotes. If you're having trouble thinking of how/why this is useful. Imagine a browser home page that displays a different inspiring/motivating/uplifting quote each time you view it to remind you to stay focussed/motivated on your goal for the day.1
“First, solve the problem. Then, write the code.” ~ John Johnson
The problem we are solving in this example is: we want to display quotes on our app/website home screen.1
First we will source some quotes. Then we will create an Elixir module, that when invoked returns a random quote to display.
When Quotes.random()
is invoked
a map
will be returned with the following form:
%{
"author" => "Peter Drucker",
"source" => "https://www.goodreads.com/quotes/784267",
"tags" => "time, management",
"text" => "Until we can manage time, we can manage nothing else."
}
This is a step-by-step example of creating a reusable Elixir package from scratch.
"Before software can be reusable, it first has to be usable." ~ Ralph Johnson
Our first step is always to write useable code. Let's begin by creating a new repository: https://github.com/new
Once you've created the repository, create an issue with the first task. e.g: quotes/issues/1
This makes it clear to yourself (and others) what the next step is.
In a terminal window on your localhost
,
run the following command:
mix new quotes
That will create all the files needed for our quotes
package.
The code created by mix new
is:
commit/14e7a08
80 additions.
Using the tree
command (tree -a
lists all files the directory tree
and -I '.git'
just means "ignore .git directory"):
tree -a -I '.git'
We see that our directory/file structure for the project is:
├── .formatter.exs
├── .gitignore
├── LICENSE
├── README.md
├── lib
│ └── quotes.ex
├── mix.exs
└── test
├── quotes_test.exs
└── test_helper.exs
The interesting/relevant files are these four:
├── lib
│ └── quotes.ex
├── mix.exs
└── test
├── quotes_test.exs
└── test_helper.exs
lib/quotes.ex
defmodule Quotes do
@moduledoc """
Documentation for Quotes.
"""
@doc """
Hello world.
## Examples
iex> Quotes.hello()
:world
"""
def hello do
:world
end
end
On creation, the Quotes
module
has a hello
function that
returns the :world
atom.
This is standard in newly created Elixir projects.
It will eventually contain our random
function.
mix.exs
defmodule Quotes.MixProject do
use Mix.Project
def project do
[
app: :quotes,
version: "0.1.0",
elixir: "~> 1.9",
start_permanent: Mix.env() == :prod,
deps: deps()
]
end
# Run "mix help compile.app" to learn about applications.
def application do
[
extra_applications: [:logger]
]
end
# Run "mix help deps" to learn about dependencies.
defp deps do
[
# {:dep_from_hexpm, "~> 0.3.0"},
# {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}
]
end
end
The mix.exs
file is the configuration file for the project/package.
test/quotes_test.exs
defmodule QuotesTest do
use ExUnit.Case
doctest Quotes
test "greets the world" do
assert Quotes.hello() == :world
end
end
test/test_helper.exs
ExUnit.start()
Before writing any code, run the tests to ensure that everything you expect to be working is in fact working:
mix test
You should see something like this:
==> jason
Compiling 8 files (.ex)
Generated jason app
==> quotes
Compiling 2 files (.ex)
Generated quotes app
..
Finished in 0.02 seconds
1 doctest, 1 test, 0 failures
Randomized with seed 771068
That informs us that jason
(the dependency we downloaded previously)
compiled successfully as did the quotes
app.
It also tells us: 1 doctest, 1 test, 0 failures.
The doctest (see below)
which is the "living documentation"
for the hello
function
executed the example successfully.
Recall that the Example in the @doc
block is:
@doc """
Hello world.
## Examples
iex> Quotes.hello()
:world
"""
def hello do
:world
end
If you open iex
in your terminal
by running iex -S mix
and then input the module and function and run it,
you will see the :world
atom as the result:
iex> Quotes.hello()
:world
Doctests are an awesome way of documenting functions because if the function changes the doctest must change with it to avoid breaking.
If we update the hello
function
to return the atom :kitty
instead of :world
the doctest will fail.
Try it!
Open the lib/quotes.ex
file
and change the hello
function
from:
def hello do
:world
end
To:
def hello do
:kitty
end
(don't update the @doc/Example yet)
Rerun the tests:
mix test
1) test greets the world (QuotesTest)
test/quotes_test.exs:5
Assertion with == failed
code: assert Quotes.hello() == :world
left: :kitty
right: :world
stacktrace:
test/quotes_test.exs:6: (test)
2) doctest Quotes.hello/0 (1) (QuotesTest)
test/quotes_test.exs:3
Doctest failed
doctest:
iex> Quotes.hello()
:world
code: Quotes.hello() === :world
left: :kitty
right: :world
stacktrace:
lib/quotes.ex:11: Quotes (module)
Finished in 0.03 seconds
1 doctest, 1 test, 2 failures
The doctest failed because the function was updated.
It might seem redundant to have two (similar) tests for the same function. In this simplistic example both the doctest and ExUnit test are testing for the same thing
assert Quotes.hello() == :world
but the difference is that the doctest example will be included in the module/function's documentation. Always keep in mind that people using your code (including yourself) might not read the tests, but they will rely on the docs so writing doctests are an excellent step.
Change the hello
function back
to what it was before
(returning :world
)
and let's move on.
Before we can return quotes, we need source a bank of quotes!
Now that we have the basics of an Elixir project, our next task is to create (or find) a list of quotes.
We could manually compile our list of quotes by combing through a few popular quotes websites. e.g:
- Wikiquote: https://en.wikiquote.org/wiki/Motivation
- Brainyquote: https://www.brainyquote.com/topics/motivational-quotes
- Goodreads: https://www.goodreads.com/quotes
Or we can feed our favourite search engine with specific targeted keywords. e.g: "inspirational quotes database json free"
Again there are many results so we need to do some sifting ...
- https://github.com/public-apis/public-apis
- https://opendata.stackexchange.com/questions/3488/large-list-of-quotes
- https://type.fit/api/quotes
- https://github.com/JamesFT/Database-Quotes-JSON
- https://github.com/lukePeavey/quotable
- https://github.com/skolakoda/programming-quotes-api
- https://github.com/jamesseanwright/ron-swanson-quotes
Abracadabra hey presto!
[
{
"text": "If I know how you spend your time, then I know what might become of you.",
"author": "Goethe",
"source": "https://www.goodreads.com/quotes/6774650",
"tags": "time, effectiveness"
},
{
"text": "Until we can manage time, we can manage nothing else.",
"author": "Peter Drucker",
"source": "https://www.goodreads.com/quotes/784267",
"tags": "time, management"
},
{
"text": "There is no greater harm than that of time wasted.",
"author": "Michelangelo",
"source": "https://www.brainyquote.com/quotes/michelangelo_183580",
"tags": "time, waste"
},
{
"text": "Those who make the worse use of their time are the first to complain of its shortness",
"author": "Jean de la Bruyere",
"source": "https://www.brainyquote.com/quotes/jean_de_la_bruyere_104446",
"tags": "time, complain"
},
{
"text": "The price of anything is the amount of life you exchange for it.",
"author": "Henry David Thoreau",
"source": "https://www.brainyquote.com/quotes/henry_david_thoreau_106427",
"tags": "price, priorities, life"
},
{
"text": "Life isn't about finding yourself. Life is about creating yourself.",
"author": "Bernard Shaw",
"source": "https://www.goodreads.com/quotes/8727",
"tags": "meaning, creativity"
},
{
"text": "Knowing is not enough, we must apply. Willing is not enough, we must do.",
"author": "Bruce Lee",
"source": "https://www.goodreads.com/quotes/302319",
"tags": "knowledge, action"
}
]
Full file containing a curated list of quotes:
quotes.json
In order to parse JSON data in Elixir, we need to import a module.
This might seem tedious if you have used other programming languages such as Python or JavaScript which have built-in JSON parsers, but it means we can use a faster parser. And since it all gets compiled down to BEAM bytecode without any effort from the developer, this extra step is automatic.
There are several options to choose from
for parsing JSON data
on hex.pm
(Elixir's package manager)
just search for the keyword "json":
https://hex.pm/packages?search=json
In our case we are going to use
jason
because we have read the code and
benchmarks
and know that it's good.
Phoenix
is moving to jason
from poison
in the next major release.
For a .json
file containing only a few thousand quotes
it probably does not matter which parser you use.
Elixir (or the Erlang VM "BEAM")
will cache the decoded JSON map in memory
so any of the options will work.
Pick one and move on.
Open the mix.exs
file in your editor
and locate the line that starts with
defp deps do
In a new Elixir project the list of deps (dependencies) is empty. Add the following line to the list:
{:jason, "~> 1.1"}
For a snapshot of what the mix.exs
file should look like at this point,
see:
quotes/mix.exs
With jason
added to the list of deps
,
you need to run the following command in your terminal:
mix deps.get
This will download the dependency from Hex.pm.
As always, our first step is to create the user story issue that describes what we are aiming to achieve: quotes/issues/4
The functions we need to create are:
-
parse_json
- open thequotes.json
file and parse the contents. -
random
- get a random quote for any author or topicQuotes.random()
-
random_by_tag
- get a quote by a specific tag e.g:Quotes.tag("time")
-
random_by_author
- get a random quote by a specific author e.g:Quotes.author("Einstein")
Let's start with the first function,
opening the quotes.json
file
and parsing its content.
The functionality for parse_json
is quite simple:
- open the
quotes.json
file - parse the data contained in the file
- return the parsed data (a List of Maps)
Open the lib/quotes.ex
file in your editor
and locate the hello
function:
def hello do
:world
end
We are keeping the
hello
function as reference for writing our own functions for the time being because it's a known state (the tests pass). We will remove it - and the corresponding tests - once therandom
tests are passing.
Below the hello
function,
add a new @doc """
block with the following info:
@doc """
parse_json returns a list of maps with quotes in the following form:
[
%{
"author" => "Albert Einstein",
"text" => "Once we accept our limits, we go beyond them."
},
%{
"author" => "Peter Drucker",
"source" => "https://www.goodreads.com/quotes/784267",
"tags" => "time, management",
"text" => "Until we can manage time, we can manage nothing else."
}
%{...},
...
]
All quotes MUST have an `author` and `text` field.
Some quotes have `tags` and `source`, please help to expand/verify others.
"""
The most often overlooked feature in software development is documentation. People naively think that writing the code is all that needs to be done, but that could not be further from the truth. Documentation is at least 30% of the project. Even if you are the only person who will "consume" the reusable code, it still pays to write comprehensive documentation. The relatively small investment pays handsomely when you return to the code in a week/month/year you don't have to waste hours trying to understand it.
"Documentation is a love letter that you write to your future self." ~ Damian Conway
"Incorrect documentation is often worse than no documentation." ~ Bertrand Meyer
Elixir has a superb Doctest feature that helps ensure documentation is kept current. As we saw above, if a function changes and the docs are not updated, the doctests will fail and thus prevent releasing the update.
Given that parse_json
returns a large list of maps,
it's impractical to add a Doctest example to the @doc
block;
the doctest would be thousands of lines
and would need to be manually updated
each time someone adds a quote.
Open the test/quotes_test.exs
file and add the following code:
test "parse_json returns a list of maps containing quotes" do
list = Quotes.parse_json()
assert Enum.count(list) == Utils.count()
# sample quote we know is in the list
sample = %{
"author" => "Albert Einstein",
"text" => "A person who never made a mistake never tried anything new."
}
# find the sample quote in the List of Maps:
[found] = Enum.map(list, fn q ->
if q["author"] == sample["author"] && q["text"] == sample["text"] do
q
end
end)
|> Enum.filter(& !is_nil(&1)) # filter out any nil values
assert sample == found # sample quote was found in the list
end
Run the tests in your terminal:
mix test
You should expect to see it fail:
1) test parse_json returns a list of maps containing quotes (QuotesTest)
test/quotes_test.exs:9
** (UndefinedFunctionError) function Quotes.parse_json/0 is undefined or private
code: list = Quotes.parse_json()
stacktrace:
(quotes) Quotes.parse_json()
test/quotes_test.exs:10: (test)
.
Finished in 0.04 seconds
1 doctest, 2 tests, 1 failure
Add the following code to the lib/quotes.ex
file below the @doc
definition
relevant to the parse_json
function
def parse_json do
File.read!("quotes.json") |> Jason.decode!()
end
Note: For the test to pass, You will also need to create a file called
lib/utils.ex
and add acount
function. See:lib/utils.ex
Re-run the tests:
mix test
You should expect to see the test pass:
...
Finished in 0.06 seconds
1 doctest, 2 tests, 0 failures
Randomized with seed 30116
For good measure,
let's write a test that ensures all quotes in quotes.json
have an "author"
and "text"
fields:
test "all quotes have author and text property" do
Quotes.parse_json()
|> Enum.each(fn(q) ->
assert Map.has_key?(q, "author")
assert Map.has_key?(q, "text")
assert String.length(q["author"]) > 2 # see: https://git.io/Je8CO
assert String.length(q["text"]) > 10
end)
end
This test might seem redundant,
but it ensures that people contributing new
quotes are not tempted to introduce incomplete data.
And having a test that runs on CI,
means that the build will fail if quotes are incomplete,
which makes the project more reliable.
Now that we have the parse_json
helper function,
we can move on to the main course!
Open the lib/quotes.ex
file (if you don't already have it open),
scroll to the bottom and
add the following @doc
comment:
@doc """
random returns a random quote.
e.g:
[
%{
"author" => "Peter Drucker",
"source" => "https://www.goodreads.com/quotes/784267",
"tags" => "time, management",
"text" => "Until we can manage time, we can manage nothing else."
}
]
"""
Given that our principal function is random
nondeterministic
it can be tempting to think that there is "no way to test" it.
In reality it's quite easy to test for randomness,
and we can even have a little fun doing it!
We currently have 1565 quotes in quotes.json
.
By running the Quotes.random()
function
there is a 1 / 1565 x 100 = 0.063% chance
of any given quote being returned.
That's great because it means
people using the quotes
will be highly unlikely
to see the same quote twice
in any given invocation.
But if the person were to keep track of the random quotes they see, the chance of seeing the same quote twice increases with each invocation. This is fairly intuitive, with a finite set of quotes, repetition is inevitable. What is less obvious is how soon the repetition will occur.
Because of a neat feature of compound probability commonly referred to as the "Birthday Paradox", we can calculate exactly when the "random" quotes will be repeated.
We aren't going to dive too deep into probability theory or math, if you are curious about The Birthday Paradox, read Kalid Azad's article (and/or watch his video): https://betterexplained.com/articles/understanding-the-birthday-paradox
We can apply the birthday paradox formula to determine how soon we will see the same "random" quote twice: (replace the word people for quote and days for quotes)
people (number of items we have already seen) = 200
days (the "population" of available data) = 1,565
pairs = (people * (people -1)) / 2 = 20,100
chance per pair = pairs / days = 12.84345047923
chance different = E^(-chance per pair) * 100 = 0.00026433844
chance of match = (100 - chance different) = 99.99973566156
There is a 99.9997% probability that at a quote selected at random will match a quote we have already seen after the 200 random events.
In other words
if we execute Quotes.random()
multiple times
and store the result in an List,
we are almost certain
to see a repeated quote
before
we reach 200 invocations.
We can translate this into code
that tests the Quotes.random
function.
Open test/quotes_test.exs
file and add the following code:
# This recursive function calls Quotes.random until a quote is repeated
def get_random_quote_until_collision(random_quotes_list) do
random_quote = Quotes.random()
if Enum.member?(random_quotes_list, random_quote) do
random_quotes_list
else
get_random_quote_until_collision([random_quote | random_quotes_list])
end
end
test "Quotes.random returns a random quote" do
# execute Quotes.random and accumulate until a collision occurs
random_quotes_list = get_random_quote_until_collision([])
# this is the birthday paradox at work! ;-)
# IO.inspect Enum.count(random_quotes_list)
assert Enum.count(random_quotes_list) < 200
end
If you save the file and run the tests:
mix test
You should expect to see it fail:
1) test Quotes.random returns a random quote (QuotesTest)
test/quotes_test.exs:49
** (UndefinedFunctionError) function Quotes.random/0 is undefined or private
code: random_quotes_list = get_random_quote_until_collision([])
stacktrace:
(quotes) Quotes.random()
test/quotes_test.exs:41: QuotesTest.get_random_quote_until_collision/1
test/quotes_test.exs:51: (test)
...
Finished in 0.1 seconds
1 doctest, 4 tests, 1 failure
The test fails
because we haven't yet implemented
the Quotes.random
function.
Let's make the test pass by implementing the function!
In the lib/quotes.ex
,
add the following function definition
below the @doc
block:
def random do
parse_json() |> Enum.random()
end
Yep, it's that simple.
Elixir
is awesome! 🎉
Re-run the tests:
mix test
They should now pass:
.....
Finished in 0.3 seconds
1 doctest, 4 tests, 0 failures
Now that our Quotes.random
function is working as expected,
we can move on to using the functionality to display quotes.
Before continuing,
take a moment to tidy up the lib/quotes.ex
and test/quotes_test.exs
files.
- Delete the
@doc
comment and function definition for thehello
function. (we no longer need it) - Delete the corresponding test
for in the
hello
function intest/quotes_test.exs
Your files should now look likelib/quotes.ex
andtest/quotes_test.exs
Ensure that the remaining tests still pass (as expected):
mix test
There are fewer tests (because we removed one test and a doctest) but the remaining tests still pass:
Generated quotes app
...
Finished in 0.4 seconds
3 tests, 0 failures
One of the biggest benefits of writing @doc
comments up-front,
is that our functions are already documented
and we don't have to think about going back and doing it.
Elixir can automatically generate the documentation for us!
Add the following line to your mix.exs
file in the deps
section:
{:ex_doc, "~> 0.21", only: :dev},
Then run the following command to download the ex_doc
dependency.
mix deps.get
Now you can run ex_docs
with the command:
mix docs
You will see output similar to this:
Compiling 1 file (.ex)
Docs successfully generated.
View them at "doc/index.html".
In your terminal type the following command:
open doc/index.html
That will open the doc/index.html
file
in your default web browser.
e.g:
Hex.pm is the package manager for the Elixir
(and Erlang
) ecosystem.
It allows you to publish packages free of charge
and share them with your other projects
and the community.
Authenticate with hex.pm in the terminal of your localhost
:
mix hex.user auth
Publish the package:
mix hex.publish
Building quotes 1.0.0
Dependencies:
jason ~> 1.1 (app: jason)
App: quotes
Name: quotes
Files:
lib
lib/index.js
lib/quotes.ex
lib/utils.ex
.formatter.exs
mix.exs
README.md
LICENSE
Version: 1.0.0
Build tools: mix
Description: a collection of inspiring quotes and methods to return them.
Licenses: GNU GPL v2.0
Links:
GitHub: https://github.com/dwyl/quotes
Elixir: ~> 1.9
Before publishing, please read the Code of Conduct: https://hex.pm/policies/codeofconduct
Publishing package to public repository hexpm.
Proceed? [Yn]
Type y
in your terminal and hit the [Enter]
key.
You will see the following output to confirm the docs have been built:
Building docs...
Compiling 2 files (.ex)
Generated quotes app
Docs successfully generated.
View them at "doc/index.html".
Local password:
Enter the password you defined
as part of runnig mix hex.user auth
above.
If the package name is available (which we knew it was), then it will be successfully published:
Publishing package...
[#########################] 100%
Package published to https://hex.pm/packages/quotes/1.0.0 (7147b94fa97ee739d8b8a324ed334f7f50566c9ed8632bf07036c31a50bf9c64)
Publishing docs...
[#########################] 100%
Docs published to https://hexdocs.pm/quotes/1.0.0
See: https://hex.pm/packages/quotes
Docs: https://hexdocs.pm/quotes/Quotes.html
We re-used the quotes
package in:
https://github.com/dwyl/phoenix-content-negotiation-tutorial
Visit: https://phoenix-content-negotiation.herokuapp.com
You should see a random inspiring quote:
In this brief tutorial we learned how to write reusable Elixir code. We reused the our Quotes
This example could easily be modified or extended for any purpose.
## Transfer a Package to Another User/Org
https://hex.pm/docs/faq#can-i-transfer-ownership-of-a-package
- Good background on code reuse: https://en.wikipedia.org/wiki/Code_reuse
- Landscape photos: https://unsplash.com/s/photos/landscape
- Generating Random Numbers in Erlang and Elixir: https://hashrocket.com/blog/posts/the-adventures-of-generating-random-numbers-in-erlang-and-elixir
- TIL: Margaret Hamilton, director of the Software Engineering Division of the MIT Instrumentation Laboratory, which developed on-board flight software for NASA's Apollo space program. (one of the people who coined the term "software engineering") https://en.wikipedia.org/wiki/Margaret_Hamilton_(software_engineer)
Momentum is an example of where inspiring quotes are used. https://momentumdash.com
Though as far as inspiring quotes go, "Yesterday you said tomorrow" is about as inspiring a direct link to the Netflix homepage. 🙄
(we can do so much better than this!)
Even if you feel that having a person homepage/dashboard - that reminds you to stay focussed - is not for you, you can at least acknowledge that there is a huge "market" for it:
If you are sceptical of motivational quotes, or "self-help" in general, remember that words have motivated many masses.
““Of course motivation is not permanent. But then, neither is bathing;
but it is something you should do on a regular basis.” ~ Zig Ziglar
“I am not young enough to know everything.” ~ Oscar Wilde
You might not think that motivational quotes work on you in the same way that most people feel they aren't influenced to advertising.
Examples of popular quotes (as upvoted or "liked" by the users of goodreads.com): goodreads.com/quotes