Introduction

A short while ago, we published our latest version of a pandoc installer Hex package, modeled after the existing tailwind and esbuild packages. In this post, we will show how we used our pandoc package to add our current markdown publishing solution to the Phoenix Framework.

1. Generate a new Phoenix project and add the pandoc dependency

Let’s start with a new project. We will use the --no-ecto option because we don’t need a database for this project.

$ mix phx.new hello --no-ecto

Next we add pandoc as a dependency to our mix.exs file.

{:pandoc, "~> 0.3", only: :dev}

Then we fetch our dependencies.

$ mix deps.get

2. Configure pandoc

Because the goal is to have multiple documents, the name of the output file will depend on the name of the input file. For this reason, we cannot simply use a static value for the --output option. To deal with this problem, pandoc accepts a function for the args config key that allows us to set the output filename dynamically for each document.

if config_env() != :prod do
  # Configure pandoc (the version is required)
  config :pandoc,
    version: "3.6.1",
    hello: [
      args: fn extra_args ->
        {_, [input_file], _} = OptionParser.parse(extra_args, switches: [])
        ~w(--output=../priv/static/posts/#{Path.rootname(input_file)}.html)
      end,
      cd: Path.expand("../documents", __DIR__)
    ]
end

Because anonymous functions are not supported in releases, we can wrap our config in a if config_env() != :prod conditional, since we’ll only convert markdown to HTML at build time.

Next, we create the directory where our converted HTML documents will live.

$ mkdir priv/static/posts

And we add our new directory to .gitignore so that Git doesn’t save the HTML output of our documents.

# Ignore documents that are produced by pandoc.
/priv/static/posts/

Lastly, we add our pandoc watcher to the endpoint so that, in development, any changes to our documents’ markdown will reflect in the browser in real-time.

config :hello, HelloWeb.Endpoint,
  # ...
  watchers: [
    # ...
    pandoc: {Pandoc, :watch, [:hello]}
  ]

To make sure everything is working, we can give it a quick test:

$ mkdir documents
$ echo "# hello" > documents/hello.md
$ mix pandoc hello hello.md
... [debug] Downloading pandoc from ...
$ cat priv/static/posts/hello.html
<h1 id="hello">hello</h1>

3. Add new document aliases to mix.exs

Now that we have Pandoc installed, and a Mix task to convert a document from markdown to HTML, we need a way to call it on all the documents in the documents directory. We can do this by adding a new alias in mix.exs that will scan the directory for files and call the Pandoc Mix task on each.

defp aliases do
  [
    setup: [
      "deps.get",
      "assets.setup",
      "assets.build",
      "documents.setup",
      "documents.build"
    ],
    # ...
    "documents.setup": ["pandoc.install --if-missing"],
    "documents.build": &pandoc/1,
    "statics.deploy": ["assets.deploy", "documents.build"]
  ]
end

defp pandoc(_) do
  config = Application.get_env(:pandoc, :hello)
  cd = config[:cd] || File.cwd!()

  cd
  |> File.cd!(fn ->
    Enum.filter(File.ls!(), &(File.stat!(&1).type != :directory))
  end)
  |> Enum.each(&Mix.Task.rerun("pandoc", ["hello", &1]))
end

Now when we run mix setup, Pandoc will convert all the files in our documents directory to markup and place the output in priv/static/posts.

We also added a statics.deploy alias so we’ll only have to run a single task before we build a release.

4. Add context, controller, templates, and routes

Now that we have our documents in priv/static/posts in HTML format, we need a way to render them.

We start with a struct that will hold the values for each post.

defmodule Hello.Documents.Post do
  defstruct [:id, :path, :body]
end

Next, we add our Documents context that will be responsible for fetching a list of all posts as well as each individual post.

We have chosen to name our documents using hyphens (-) to separate words. Our post ids, then, will be the document filename minus the file extension suffix. So, a file documents/this-is-the-first-post.md will have an id of this-is-the-first-post and a URI that looks like https://example.org/posts/this-is-the-first-post.

defmodule Hello.Documents do
  @moduledoc """
  The Documents context.
  """

  alias Hello.Documents.Post

  def list_posts do
    "documents/*"
    |> Path.wildcard()
    |> Enum.map(fn path ->
      %Post{
        id: path |> Path.rootname() |> Path.basename(),
        path: path
      }
    end)
  end

  def get_post!(id) do
    post = Enum.find(list_posts(), fn post -> post.id == id end)

    body =
      :hello
      |> :code.priv_dir()
      |> Path.join("static/posts/#{id}.html")
      |> File.read!()

    %{post | body: body}
  end
end

Our posts controller and view are pretty standard.

defmodule HelloWeb.PostController do
  use HelloWeb, :controller

  alias Hello.Documents

  def index(conn, _params) do
    posts = Documents.list_posts()
    render(conn, :index, posts: posts)
  end

  def show(conn, %{"id" => id}) do
    post = Documents.get_post!(id)
    render(conn, :show, post: post)
  end
end

defmodule HelloWeb.PostHTML do
  use HelloWeb, :html

  embed_templates "post_html/*"
end

Our index and show templates are pretty similar to what the Phoenix phx.gen.html HTML generator spits out. We take full advantage of the UI core components that Phoenix provides.

<.header>
  Listing Posts
</.header>

<.table id="posts" rows={@posts} row_click={&JS.navigate(~p"/posts/#{&1}")}>
  <:col :let={post} label="id"><%= post.id %></:col>
  <:action :let={post}>
    <div class="sr-only">
      <.link navigate={~p"/posts/#{post}"}>Show</.link>
    </div>
  </:action>
</.table>

<.header>
  <%= @post.id %>
  <:subtitle>This is a post from your markdown documents.</:subtitle>
</.header>

<%= raw(@post.body) %>

<.back navigate={~p"/posts"}>Back to posts</.back>

Finally, we add the routes we will need, with only the index and show actions being necessary.

scope "/", HelloWeb do
  pipe_through :browser

  resources "/posts", PostController, only: [:index, :show]

  get "/", PageController, :home
end

Let’s create a couple of documents to see how our app renders them.

$ touch documents/{hello-there.md,welcome-to-our-demo-app.md}

And now we visit localhost:4000/posts.

Let’s add some markdown content to documents/hello-there.md so we can see how the show template looks.

# hello

This is a paragraph.

    This is a code block.

> This is a block quote.

## this is a heading

- this is
- a list
- of items

When we visit localhost:4000/posts/hello-there, it looks like this:

Our document’s content is visible, but it’s missing any styling. We will fix this by adding the typography plugin to Tailwind’s config file.

5. Style the markdown content

Let’s add @tailwindcss/typography to the plugins list in tailwind.config.js.

plugins: [
  // ...
  require("@tailwindcss/typography")
]

Then we’ll add Tailwind’s prose class to our post’s HTML content.

<div class="prose">
  <%= raw(@post.body) %>
</div>

After restarting our app, we can visit the post show page again, and this time our content is styled appropriately.

6. Auto-reload the browser when markdown content changes

Since our pandoc file-watcher converts documents to HTML whenever changes are detected, all we need to do to have the changes update in our browser in real-time is add the static files path to the live_reload config in config/dev.exs.

config :hello, HelloWeb.Endpoint,
  live_reload: [
    interval: 1000,
    debounce: 200,
    patterns: [
      # ...
      ~r"priv/static/posts/.*(html)$"
    ]
  ]

Now we can edit the markdown in our documents and, as soon as we save our changes, the new content should appear in our browser.

7. Handle draft documents

It would be very helpful if we had a place to keep draft documents that are visible to us during development but absent in a production release. With a few changes to lib/hello/documents.ex we can do just that.

First, let’s make a directory for draft documents.

$ mkdir documents/_drafts

So our published documents will live in the top-level documents directory, and our draft documents will live in the subdirectory documents/_drafts.

The first change to the list_posts/1 function in documents.ex adds a conditional depending on Mix.env() for what directories to scan. In production it will only scan documents/* for files, while in development it will scan all subdirectories with documents/**/*.

The second change required to list_posts/1 is to filter out directories, since Path.wildcard/1 will include the directory _drafts in with the list with files.

def list_posts do
  "documents"
  |> Path.join(if(Mix.env() != :prod, do: "**/*", else: "*"))
  |> Path.wildcard()
  |> Enum.filter(&(File.stat!(&1).type != :directory))
  |> Enum.map(fn path ->
  # ...

Now we just need to update the get_post!/1 function. In order to make sure that we never accidentally publish draft documents, we will convert them on the fly as necessary and will store their content in memory rather that writing to disk. Since the pandoc profile default outputs conversion results to stdout, we can use that profile instead of hello when running the conversion and simply capture the results with ExUnit.CaptureIO.capture_io/1.

def get_post!(id) do
  # ...
  body =
    if "_drafts" in Path.split(post.path) do
      ExUnit.CaptureIO.capture_io(fn ->
        Mix.Task.rerun("pandoc", ["default", post.path])
      end)
    else
      :hello
      |> :code.priv_dir()
      |> Path.join("static/posts/#{id}.html")
      |> File.read!()
    end
  # ...
end

Lastly, since we are not writing draft documents to disk, we need to add another pattern for Phoenix’s LiveReloader to reload the browser when draft content changes.

config :hello, HelloWeb.Endpoint,
  live_reload: [
    patterns: [
      # ...
      ~r"priv/static/posts/.*(html)$",
      ~r"documents/_drafts/.*(md)$"
    ]
  ]

8. Release

When we next release our application, we simply need to run mix statics.deploy to build assets and documents first, and then run the release command.

$ mix statics.deploy
$ MIX_ENV=prod mix release

Conclusion

That’s it! This solution allows us to write our posts in simple markdown and see them converted to HTML automatically with Phoenix and Pandoc. Furthermore, we are able to use Phoenix’s powerful template language HEEx to make writing HTML faster and easier, so we can focus more on content and less on development.

We hope this post was as useful to others as it has been to us. We are always appreciative of any feed back readers would like to share with us. Thanks for reading and happy coding!