Stork search with Franklin

Stork is a fairly new end-to-end solution for client based web search. We're using it here because it's very simple to use – the client side JavaScript is hosted in a CDN and it turns out that we also have a stork_jll already built for us (on Linux, at least).

Generating the search index

Of course, Stork needs a search index to do its work. It's pretty easy to generate that from the HTML pages produced by Franklin:

using Gumbo, AbstractTrees, stork_jll, TOML, Franklin

include("utils.jl")

const pageroot = joinpath(@__DIR__, "__site")

# Find the title on each page and push filepath, URL path, 
# and title into the filelist we give to the stork binary later
function add_to_filelist!(filelist, uri_path, filepath)
    content = read(filepath, String)

    html = Gumbo.parsehtml(content)

    title = ""
    for el in AbstractTrees.PreOrderDFS(html.root)
        if el isa Gumbo.HTMLElement
            if Gumbo.tag(el) == :title
                title = Gumbo.text(el)
                break
            end
        end
    end

    push!(
        filelist, 
        Dict(
            "path" => filepath, 
            "url" => uri_path, 
            "title" => title
        )
    )
end

function build_index(prepath = "")
    filelist = []

    # iterate over all index.htmls in __site
    for (root, _, files) in walkdir(pageroot)
        for file in files
            file == "index.html" || continue
            path = joinpath(root, file)
            # exclude tags dir
            uri_path = string("/", replace(
                relpath(dirname(path), pageroot), 
                "\\" => "/")
            )

            # we only want posts to be searchable
            occursin("/posts/", uri_path) || continue

            add_to_filelist!(filelist, prefix_prepath(uri_path, prepath), path)
        end
    end

    stork_config = Dict(
        "input" => Dict(
            "base_directory" => pageroot,
            # this is what defines searchable content
            "html_selector" => ".franklin-content",
            # Franklin puts the footer into .franklin-content,
            # so we exclude it here
            "exclude_html_selector" => ".page-foot, .tags-list",
            "files" => filelist,
        ),
        "output" => Dict(
            # Stork redirects to the nearest #fragment if 
            # this is set
            "save_nearest_html_id" => true,
            # this determines the amount of context Stork
            # saves in the index
            "excerpt_buffer" => 4,
        ),
    )

    config_path = joinpath(pageroot, "stork.config.toml")
    index_path = joinpath(pageroot, "stork.st")
    open(config_path, "w") do io
        TOML.print(io, stork_config)
    end

    if !stork_jll.is_available()
        error("Stork is not available on this platform.")
    end

    run(`$(stork_jll.stork()) build --input $config_path --output $index_path`)
    return index_path
end

Frontend

Now we just need to add a bit of scaffolding to our HTML template – typically you'd want to insert the search bar into the navigation, located in _layout/header.html for this theme. We add

<li class="search">
    <input data-stork="stork" class="stork-input" placeholder="Search..."/>
    <div data-stork="stork-output" class="stork-output"></div>
</li>

to our navigation and the following JS to the page footer via a hfun:

function hfun_stork()
  return """
  <script src="https://files.stork-search.net/releases/v1.5.0/stork.js"></script>
  <script>
      stork.register("stork", "$(prefix_prepath("stork.st"))");
  </script>
  """
end

Why do we need a hfun here? Well, Franklin doesn't expand templates in <script> tags, so we cannot just paste that code into our foot.html verbatim.

The prefix_prepath function adds the page prefix to a page-absolute link and works like this:

function prefix_prepath(path, prepath=Franklin.globvar("prepath"; default = ""))
  path = lstrip(path, '/')
  return isempty(prepath) ? string("/", path) : "/$(prepath)/$(path)"
end

Then let's sprinkle a bit of style on top (custom, because I don't like any of the included themes):

li.search {
  margin-left: auto;
  padding: 0.7rem 1.2rem;
  position: relative;
  overflow: visible;

  @media (max-width: $body-mobile-width) {
    padding-top: 0;
    margin-left: 0;
  }

  input {
    background-color: #222426;
    color: #eee;
    font-size: .8rem;
    padding: 3px 6px;
    outline: none;
    border: 1px solid #555;
    border-radius: 3px;
    
    width: 300px;
    @media (max-width: $body-mobile-width) {
      width: calc(100vw - 3rem);
    }
  }

  .stork-close-button {
    display: inline;
    position: absolute;
    right: 1.2rem;
    background: none;
    border: none;
    color: #aaa;
    padding: 6px;
    cursor: pointer;
  }

  .stork-progress {
    height: 4px;
    background: #555;
    margin-top: -3px;
  }

  .stork-output {
    position: absolute;
    background-color: #222426;
    border: 1px solid #555;
    z-index: 3;

    width: 300px;
    @media (max-width: $body-mobile-width) {
      width: calc(100vw - 3rem);
    };
    
    font-size: .8rem;

    margin-top: .5rem;
    margin-right: 1.2rem;
    border-radius: 3px;
    right: 0;

    .stork-message {
      font-size: 0.7rem;
      display: block;
      text-align: left;
      background: #202227;
      padding: 0.2em 0.5em;
      border-bottom: 1px solid #555;

      &:last-child {
        border-bottom: none;
      }
    }

    .stork-results {
      .stork-result {
        display: block;
        text-align: left;
        padding: .25em .5em;
        border-bottom: 1px solid #555;
        width: 100%;
        
        &:last-child {
          border-bottom: none;
        }

        a {
          padding: .25em 0;

          .stork-title {
            font-weight: bold;
          }
        }

        p {
          margin: 0;
        }

        .stork-highlight {
          color: #eee;
          background-color: inherit;
          text-decoration: underline dotted;
        }

        .stork-excerpt-container {
          font-size: .8em;
          margin-top: 0.25em;

          .stork-excerpt {
            margin-bottom: .5em;
          }
        }

        &:hover, &.selected {
          background-color: rgb(47, 50, 58);
        }
      }

      max-height: 60vh;
      overflow-y: scroll;
    }

    .stork-attribution {
      font-size: 0.7rem;
      display: block;
      text-align: right;
      background: #1c1e22;
      padding: 0.2em 0.5em;
      border-top: 1px solid #555;

      a {
        display: inline;
        padding: 0;
      }
    }

    display: none;
    &.stork-output-visible {
      display: block;
    }
  }
}

Deployment

All that's left now is to create the search index before deploying to e.g. gh-pages. It's easiest to tweak the default Deploy.yml and just include our search index generator script after the page is built:

jobs:
  build-and-deploy:
    ...
    - run: julia -e '
            prepath = "TinyStruggles";
            using Pkg; Pkg.activate("."); Pkg.instantiate();
            using NodeJS; run(`$(npm_cmd()) install highlight.js`);
            using Franklin;
            optimize(; prepath = prepath);
            include("stork.jl");
            build_index(prepath)'