Artifacts!

By: Josh Day

Re-posted from: https://www.juliafordatascience.com/artifacts/

Artifacts!

In this article we'll cover what artifacts are and how to use them, both as a package user and a package developer.

What is an Artifact?

In Julia, an artifact is simply a fixed/immutable something that a package needs (that isn't Julia code).  This could be an executable binary, a text file, or any other kind of immutable data.  

The "rules" for artifacts are the following:

  • Artifacts must be immutable.
  • Artifacts must be a (optionally g-zipped) tarfile (ends with .tar or .tar.gz).
  • Artifacts must be publicly available on the internet.

What's the Point of Artifacts?

To understand the usefulness of artifacts, let's take a look at some alternatives:

❌ Storing binary data directly in the git repo.

This is bad practice because git is only meant for tracking changes in text files.  A change to a binary file will require git to save an entirely new version of the file.

❌ Using a deps/build.jl script.

Before artifact support was added to Pkg, this was how you included artifacts along with a package.  When you install a package, this script will run (if it exists).  The script can include downloading files the package needs.  

Downsides to this approach are:

  • If two packages use the same artifact, they will both download their own copy.
  • Packages that use this method are incompatible with PackageCompiler.jl.
  • It is tricky to get platform-specific dependencies working properly.

In other words, Pkg's artifact support allows you to:

✅ Avoid bloat in git repos.
✅ Make source code immutable (a package directory's state isn't changed by a build.jl script).
✅ Have packages that share dependencies.
✅ Use PackageCompiler.
✅ Install platform-specific dependencies in a more robust way.

Artifacts as a Package User

As a user, you don't need to make yourself concerned with artifacts.  Whenever a package is installed on your machine, Julia's package manager will automatically download the artifacts it needs!

Artifacts as a Package Developer

As a developer, we highly recommend using the ArtifactUtils package.  See the docs for more info, but here we'll walk through creating a package from scratch that needs an artifact.

1) Let's make a package called Hello

using PkgTemplates

t = Template()

generate(t, "Hello")
c

2) Navigate to the root directory of Hello

path = joinpath(homedir(), ".julia", "dev", "Hello")

cd(path)

3) Create what we need

  • We'll write a file called "hello.txt" into a temporary directory with the contents "Hello!  I am an artifact!".
dir = mktempdir()

file = touch(joinpath(dir, "hello.txt"))

open(file, "w") do io 
    write(io, "Hello!  I am an artifact!")
end

4) Get the ID for our artifact

using ArtifactUtils

artifact_id = artifact_from_directory(dir)

5) Upload the artifact somewhere

gist = upload_to_gist(artifact_id)

6) Create an Artifacts.toml in your Hello package directory

add_artifact!("Artifacts.toml", "hello_artifact", gist)

7) Use the artifact in the package

  • Every package can have an __init__ function that runs right after the package is loaded.  In this case, we'll use __init__ to print out the contents of our "hello.txt" artifact.
sourcecode = """
module Hello 

using Artifacts 

function __init__()
    path = joinpath(artifact"hello_artifact", "hello.txt")
    
    println(read(path,String))
end

end #module
"""

open(io -> write(io, sourcecode), joinpath("src", "Hello.jl"), "w")

8) Add the Artifacts dependency and you're done!

using Pkg

# activate the Hello project
Pkg.activate(".")  

# Make sure we add the Artifact dependency.
Pkg.add("Artifacts")  

# get all of Hello's dependencies including artifacts
Pkg.instantiate()

🎉 Hey it works!

julia> using Hello
Hello!  I am an artifact!

🚀 That's It!

You now know the basics of Julia's artifact system!

Still confused about something?  Did we miss anything important?  Let us know on Twitter at @JuliaForDataSci!

Enjoying Julia For Data Science?  Please share us with a friend and follow us on Twitter at @JuliaForDataSci.

Want to write an article for Julia For Data Science?  Get in touch!  juliafordatascience@gmail.com

Julia For Data Science Numbers:

Hey, this section is new.  We thought it would be fun to include some statistics about Julia For Data Science.  We'll include our member/follower numbers in each post from here on out.

  • Newsletter members: 166
  • Twitter followers: 836