Web Dev Solutions

Catalin Mititiuc

Web Log

Elixir, JavaScript, SVG, Containers, Git, Linux

Questions, comments, feedback? Contact the author.

Temporary Directories For Testing Mix Tasks That Modify Files

Intro

Last time, we added a Mix task to our project that writes an HTML file to a directory /public in the container’s filesystem. Today, we will write a test for that task.

Here is the code we want to write a test for.

@markup """
  <!DOCTYPE html>
  <html>
    <head></head>
    <body>hello world</body>
  </html>
"""

@output_directory "/public"

@impl Mix.Task
def run(_args) do
  Mix.shell().info("running build task")

  File.mkdir_p!(@output_directory)
  @output_directory |> Path.join("index.html") |> File.write!(@markup)
end

First pass

The task writes some HTML markup to a file, /public/index.html, so our test should ensure that after the task is ran, that file exists.

test/mix/tasks/build_test.exs

defmodule Mix.Tasks.BuildTest do
  use ExUnit.Case

  test "creates output files" do
    Mix.Tasks.Build.run([])

    assert File.exists?("/public/index.html")
  end
end

Let’s try running the test.

$ docker run --rm -w /opt -v $PWD:/opt hw_dev mix test
running build task
...
Finished in 0.02 seconds (0.02s async, 0.00s sync)
1 doctest, 2 tests, 0 failures

While it passes, this is not a valid test. /public/index.html already exists before our test runs the build task, because our Dockerfile tells Docker to run the task when it builds our image. To make the test valid, we need it to use a clean directory.

If we make the output directory configurable during runtime, we can create a temporary directory to use as a mock output directory during test runs. We need to move where the output directory is stored, from a hard-coded module attribute into something configureable at runtime.

Separating test and dev environment data

Right now the value for the output directory is stored in a module attribute in the task module. If we had it stored in a variable, we could change it when we run our tests.

Configuring the output directory

1. Store the value in an application environment variable

To store it in an application environment variable, add this in mix.exs

  def application do
    [
      env: [output_directory: "/public"],
      ...
    ]
  end

Then, in run() in lib/mix/tasks/build.ex, we fetch the output directory from the application environment instead of hard-coding it in a module attribute

  def run(_args) do
    Mix.shell().info("running build task")

    output_directory = Application.fetch_env!(:static_site, :output_directory)

    File.mkdir_p!(output_directory)
    output_directory |> Path.join("index.html") |> File.write!(@markup)
  end

Now that the output directory is stored in an application env var, let’s see about changing it for our test run.

2. Change the value when running tests

Now we can dynamically set the output directory in our test. Really we just have to set it a relative path instead of an absolute. Once we make it a relative path, that mix task will create the output directory relative to the current directory.

  setup do
    output_dir = "public"
    Application.put_env(:static_site, :output_directory, output_dir)

    [output_dir: output_dir]
  end


  test "creates output file", %{output_path: output_dir} do
    ...
    assert File.exists?(Path.join(output_dir, "index.html"))
  end

3. Restore the original value

But this changes the output directory application-wide, and it’s good practice to put anything the tests change back to the way they were before. We can use on_exit() to set the original value when the test run finishes. We have to save it before changing the application variable and pass it to the call in on_exit()

setup do
  output_dir = "public"
  original = Application.fetch_env!(:static_site, :output_directory)
  ...
  on_exit(fn ->
    Application.put_env(:static_site, :output_directory, original)
  end)
  ...
end

Now that we can change the output directory in our tests, we can point to a temporary directory that is separate from our dev files. But where should we put it?

Adding a temporary tmp directory for test artifacts

1. Locate the temporary test directory

Before we spend too much time thinking about it, let’s take a look how it’s done in the Elixir source code. In lib/mix/test/test_helper.exs there’s

  def tmp_path do
    Path.expand("../tmp", __DIR__)
  end

  def tmp_path(extension) do
    Path.join(tmp_path(), remove_colons(extension))
  end

__DIR__ is whatever directory the current file is in, so for our task test in test/mix/tasks the temp directory would be test/mix/tmp.

That seems like a good place to us. Let’s copy tmp_path() into our test file. We can leave off the call to remove_colons(), since we don’t have any colons to deal with.

defp tmp_path, do: Path.expand("../tmp", __DIR__)
defp tmp_path(extension), do: Path.join(tmp_path(), extension)

Now let’s see about changing the location of the output directory.

2. Change the current directory during the test

We can use tmp_path/1 to create a temporary directory specifically for this test.

test "creates output file", %{output_dir: output_dir} do
  File.mkdir_p!(tmp_path("build"))
  File.cd!(tmp_path("build"))
  ...

And at the end of our test, we need to change back into the test file’s directory with

  ...
  File.cd!(__DIR__)
end

3. Add fixture for missing file

And since we’re no longer in the project root directory, our task will not find the root index.html file. We need to add a fixture that our task can read from.

File.write("index.html", """
<!-- Auto-generated fixture -->
<!DOCTYPE html>
<html>
  <body>hello world</body>
</html>
""")

After running the test,

$ docker run --rm -w /opt -v $PWD:/opt hw_dev mix test
running build task
...
Finished in 0.02 seconds (0.02s async, 0.00s sync)
1 doctest, 2 tests, 0 failures

we have our test directory, test/mix/tmp,

$ find test/mix/tmp
test/mix/tmp
test/mix/tmp/build
test/mix/tmp/build/public
test/mix/tmp/build/public/index.html
test/mix/tmp/build/index.html

and our fixture file.

$ cat test/mix/tmp/build/index.html
<!-- Auto-generated fixture -->
<!DOCTYPE html>
<html>
  <body>hello world</body>
</html>

Cleaning up temporary directories

Our tests are pretty solid now, but it is a good idea to clean the tmp directory up when the tests are done. We can do that in the on_exit() call in our test setup block

on_exit(fn ->
  ...
  File.rm_rf!(tmp_path())
end)

Now, after we run the tests, all the files and directories created during the test run have been removed.

$ find test/mix/tmp/
find: ‘test/mix/tmp/’: No such file or directory

Suppressing IO messages

Our mix task outputs a useful message when it runs, but we don’t want that to clutter up the test results. We can prevent those messages by “capturing” the output with the ExUnit.CaptureIO module.

Add import ExUnit.CaptureIO to the top of our test file. Then, in our tests, we can pass our call to the mix task as a function to capture_io()

capture_io(fn -> Mix.Tasks.Build.run([]) end)

When we run our tests now, there are no more IO messages cluttering up the test results.

$ docker run --rm -w /opt -v $PWD:/opt hw_dev mix test
...
Finished in 0.02 seconds (0.02s async, 0.00s sync)
1 doctest, 2 tests, 0 failures

Final form

Here’s what the final draft of our test looks like.

test/mix/tasks/build_test.exs

defmodule Mix.Tasks.BuildTest do
  use ExUnit.Case

  import ExUnit.CaptureIO

  setup do
    output_dir = "public"
    original = Application.fetch_env!(:static_site, :output_directory)
    Application.put_env(:static_site, :output_directory, output_dir)

    on_exit(fn ->
      Application.put_env(:static_site, :output_directory, original)
      File.rm_rf!(tmp_path())
    end)

    [output_dir: output_dir]
  end

  test "creates output file", %{output_dir: output_dir} do
    File.mkdir_p!(tmp_path("build"))
    File.cd!(tmp_path("build"))

    File.write("index.html", """
    <!-- Auto-generated fixture -->
    <!DOCTYPE html>
    <html>
      <body>hello world</body>
    </html>
    """)

    capture_io(fn -> Mix.Tasks.Build.run([]) end)

    assert File.exists?(Path.join(output_dir, "index.html"))

    File.cd!(__DIR__)
  end

  defp tmp_path, do: Path.expand("../tmp", __DIR__)
  defp tmp_path(extension), do: Path.join(tmp_path(), extension)
end

Conclusion

This test seemed trivial at first, but increased in complexity quickly. We had to set some of our configuration in application environment variables, change the configuration temporarily before a test run and then change it back after, clean up test artifacts after the run, and capture IO messages that were generated during it. That’s enough going on that we thought it would be a good topic for a post. Cheers and happy coding!

Back to posts