Julia Helps

Programmers are always learning. You learn how to use new APIs, new functions, new types. You learn why your code doesn’t work.

Julia has a lot of built-in tools to help you navigate, learn, and debug. You can use these in the REPL and in normal code.

In this post, I split these functions and macros up based on what they help you do: understand functions, examine types, navigate the type hierarchy, or debug code.

Exploring New Functions

If you’re in the REPL, you’re probably playing with something new, trying to make it work. You use new-to-you functions, which means the question “how do I use this function?” comes up frequently.

methods

The most basic answer to this question is to list the method signatures for the function in question. You can often guess what a method does just from the arguments’ names and types. If your problem is passing the arguments in the wrong order, this will solve it.

In [2]: 
methods(open)
Out[2]:
# 4 methods for generic function "open":
open(fname::String,rd::Bool,wr::Bool,cr::Bool,tr::Bool,ff::Bool) at io.jl:316
open(fname::String) at io.jl:327
open(fname::String,mode::String) at io.jl:330
open(f::Function,args...) at io.jl:340

help

While you can get a lot from a verb and the list of nouns/types it works on, a hand-written description of what the function does it even better. We can get those at the REPL, too. The output of help is the exactly the same as the online function documentation; they’re generated from the same source files.

Currently, help will only work for functions in the base libraries. For packages, you’ll have to read their documentation online. However, there is good coverage for functions in base.

In [1]: 
help(open)

Loading help data...
Base.open(file_name[, read, write, create, truncate, append]) -> IOStream

   Open a file in a mode specified by five boolean arguments. The
   default is to open files for reading only. Returns a stream for
   accessing the file.

Base.open(file_name[, mode]) -> IOStream

   Alternate syntax for open, where a string-based mode specifier is
   used instead of the five booleans. The values of "mode"
   correspond to those from "fopen(3)" or Perl "open", and are
   equivalent to setting the following boolean groups:

   +------+-----------------------------------+
   | r    | read                              |
   +------+-----------------------------------+
   | r+   | read, write                       |
   +------+-----------------------------------+
   | w    | write, create, truncate           |
   +------+-----------------------------------+
   | w+   | read, write, create, truncate     |
   +------+-----------------------------------+
   | a    | write, create, append             |
   +------+-----------------------------------+
   | a+   | read, write, create, append       |
   +------+-----------------------------------+

Base.open(f::function, args...)

   Apply the function "f" to the result of "open(args...)" and
   close the resulting file descriptor upon completion.

   **Example**: "open(readall, "file.txt")"

Because help is so useful, there’s even a short hand for it. You can use ? to call the help function:

In [2]: 
?help


 Welcome to Julia. The full manual is available at

    http://docs.julialang.org

 To get help, try help(function), help("@macro"), or help("variable").
 To search all help text, try apropos("string"). To see available functions,
 try help(category), for one of the following categories:

  "Getting Around"
  "All Objects"
  "Types"
  "Generic Functions"
  "Syntax"
  "Iteration"
  "General Collections"
  "Iterable Collections"
  "Indexable Collections"
  "Associative Collections"
  "Set-Like Collections"
  "Dequeues"
  "Strings"
  "I/O"
  "Network I/O"
  "Text I/O"
  "Multimedia I/O"
  "Memory-mapped I/O"
  "Mathematical Operators"
  "Mathematical Functions"
  "Data Formats"
  "Numbers"
  "BigFloats"
  "Random Numbers"
  "Arrays"
  "Combinatorics"
  "Statistics"
  "Signal Processing"
  "Numerical Integration"
  "Parallel Computing"
  "Distributed Arrays"
  "System"
  "C Interface"
  "Errors"
  "Tasks"
  "Events"
  "Reflection"
  "Internals"
  "Collections and Data Structures"
  "Constants"
  "Filesystem"
  "Graphics"
  "Linear Algebra"
  "BLAS Functions"
  "Package Manager Functions"
  "Profiling"
  "Sorting and Related Functions"
  "Sparse Matrices"
  "Unit and Functional Testing"

Exploring New Types

Julia is a dynamically typed language where you can talk about types; types are first class values. Just as for functions, there are tools for helping you understand what you can do with unfamiliar types.

typeof

If you have a value, but aren’t sure of its type, you can use typeof.

In [11]: 
typeof(5.0)
Out[11]:
Float64

The type that represents types is DataType. typeof is not just printing out a name; it is returning the type as a value.

In [12]: 
typeof(Float64)
Out[12]:
DataType

methods

Types in Julia define special constructor functions of the same name as the type, like in OO languages. For other functions, you use the methods function to find out what combinations of arguments it can take; this also works for type constructors.

In [15]: 
methods(Dict)
Out[15]:
# 6 methods for generic function "Dict":
Dict() at dict.jl:300
Dict{K,V}(ks::AbstractArray{K,N},vs::AbstractArray{V,N}) at dict.jl:302
Dict{K,V}(ks::(K...,),vs::(V...,)) at dict.jl:306
Dict{K}(ks::(K...,),vs::(Any...,)) at dict.jl:307
Dict{V}(ks::(Any...,),vs::(V...,)) at dict.jl:308
Dict(ks,vs) at dict.jl:303

names

Sometimes, you’ll get a new type and want to know not just what methods are already defined, but the structure of the type itself. Types in Julia are like records or structs in other languages: they have named properties. names will list the name of each property. These are Symbols instead of Strings because identifiers (variable names, etc) are distinct.

In [4]: 
names(IOStream)
Out[4]:
3-element Array{Any,1}:
 :handle
 :ios   
 :name

types

You can also get the types of the fields. They are stored in the types field of a DataType, as a tuple of DataTypes in the same order as the names returned by names.

In [3]: 
IOStream.types
Out[3]:
(Ptr{None},Array{Uint8,1},String)

methodswith

Once you have a value and know its type, you want to know what can be done with it.

When you want to shell out to other programs from Julia, you create Cmds. Creating one is easy — just put in backticks what you’d type at the command line: `echo hi`. However, creating a Cmd doesn’t actually run it: you just get a value.

What can you do with your Cmd? Just ask methodswith, which prints out method signatures for all methods that take that type.

In [2]: 
methodswith(Cmd)

spawn(pc::Union(ProcessChain,Bool),cmd::Cmd,stdios::(Union(File,IOStream,FileRedirect,Ptr{None},AsyncStream),Union(File,IOStream,FileRedirect,Ptr{None},AsyncStream),Union(File,IOStream,FileRedirect,Ptr{None},AsyncStream)),exitcb::Union(Function,Bool),closecb::Union(Function,Bool)) at process.jl:324
ignorestatus(cmd::Cmd) at process.jl:131
show(io::IO,cmd::Cmd) at process.jl:32
detach(cmd::Cmd) at process.jl:133
readdir(cmd::Cmd) at deprecated.jl:19
setenv{S<:Union(ASCIIString,UTF8String)}(cmd::Cmd,env::Array{S<:Union(ASCIIString,UTF8String),N}) at process.jl:135
setenv(cmd::Cmd,env::Associative{K,V}) at process.jl:136

In the case of Cmd, that’s not so helpful. I still don’t see a way to run my Cmd. 🙁

However, that’s not all you can do with a Cmd or the methodswith function. Passing true as the second argument will show all of the methods that take Cmd or any of it’s super types. (Be prepared for a very long list for most types.)

In [1]: 
methodswith(Cmd,true)

spawn(pc::Union(ProcessChain,Bool),cmd::Cmd,stdios::(Union(File,IOStream,FileRedirect,Ptr{None},AsyncStream),Union(File,IOStream,FileRedirect,Ptr{None},AsyncStream),Union(File,IOStream,FileRedirect,Ptr{None},AsyncStream)),exitcb::Union(Function,Bool),closecb::Union(Function,Bool)) at process.jl:324
spawn(pc::Union(ProcessChain,Bool),cmds::AbstractCmd,args...) at process.jl:370
spawn(cmds::AbstractCmd,args...) at process.jl:371
ignorestatus(cmd::Cmd) at process.jl:131
readall(cmd::AbstractCmd) at process.jl:437
readall(cmd::AbstractCmd,stdin::AsyncStream) at process.jl:437
|(a::AbstractCmd,b::AbstractCmd) at deprecated.jl:19
show(io::IO,cmd::Cmd) at process.jl:32
success(cmd::AbstractCmd) at process.jl:472
readsfrom(cmds::AbstractCmd) at process.jl:409
readsfrom(cmds::AbstractCmd,stdin::AsyncStream) at process.jl:411
detach(cmd::Cmd) at process.jl:133
<(a::AbstractCmd,b::String) at deprecated.jl:19
writesto(cmds::AbstractCmd,stdout::AsyncStream) at process.jl:417
writesto(cmds::AbstractCmd) at process.jl:420
&(left::AbstractCmd,right::AbstractCmd) at process.jl:138
run(cmds::AbstractCmd,args...) at process.jl:452
>>(src::AbstractCmd,dest::String) at process.jl:151
>(a::Union(File,IOStream,FileRedirect,AsyncStream),b::AbstractCmd) at deprecated.jl:19
>(a::String,b::AbstractCmd) at deprecated.jl:19
>(a::AbstractCmd,b::Union(File,IOStream,FileRedirect,AsyncStream)) at deprecated.jl:19
>(a::AbstractCmd,b::String) at deprecated.jl:19
eachline(cmd::AbstractCmd,stdin) at process.jl:400
eachline(cmd::AbstractCmd) at process.jl:406
readdir(cmd::Cmd) at deprecated.jl:19
readbytes(cmd::AbstractCmd) at process.jl:428
readbytes(cmd::AbstractCmd,stdin::AsyncStream) at process.jl:428
readandwrite(cmds::AbstractCmd) at process.jl:423
setenv{S<:Union(ASCIIString,UTF8String)}(cmd::Cmd,env::Array{S<:Union(ASCIIString,UTF8String),N}) at process.jl:135
setenv(cmd::Cmd,env::Associative{K,V}) at process.jl:136
.>(src::AbstractCmd,dest::AbstractCmd) at process.jl:140
.>(src::AbstractCmd,dest::Union(File,IOStream,FileRedirect,AsyncStream)) at process.jl:145
.>(src::AbstractCmd,dest::String) at process.jl:150

As you can see, most of the relevant methods are defined for AbstractCmd rather than Cmd. You can also see both the relevant execution functions (run,readall,readsfrom,writesto,readandwrite,etc) and the redirection ones (|,&,>,>>,etc). (Julia parses the Cmd and execs the process itself, so there’s no shell involved; instead, you use Julia code for redirection and globs. For more on Cmd see these blog posts or the manual.)

Exploring the Type Hierarchy

In Julia, types are not just individual, unconnected values. They are organized into a hierarchy, as in most languages.

super

Each type has one supertype; you can find out what it is by using the super function.

The type heirarchy is a connected graph: you can follow a path of supertypes up from any node to Any (whose supertype is Any). Let’s do that in code, starting from Float64.

In [5]: 
super(Float64)
Out[5]:
FloatingPoint
In [7]: 
super(FloatingPoint)
Out[7]:
Real
In [8]: 
super(Real)
Out[8]:
Number
In [9]: 
super(Number)
Out[9]:
Any
In [10]: 
super(Any)
Out[10]:
Any

subtypes

We can also go in the other direction. Let’s see what the subtypes of Any are.

In [17]: 
subtypes(Any)
Out[17]:
182-element Array{Any,1}:
 AbstractArray{T,N}                                                                                   
 AbstractCmd                                                                                          
 AbstractRNG                                                                                          
 Algorithm                                                                                            
 Any                                                                                                  
 Associative{K,V}                                                                                     
 AsyncWork                                                                                            
 Available                                                                                            
 BoundingBox                                                                                          
 Box                                                                                                  
 BuildInfo                                                                                            
 BuildStep                                                                                            
 CPUinfo                                                                                              
 ⋮                                                                                                    
 Vec2                                                                                                 
 VersionInterval                                                                                      
 VersionNumber                                                                                        
 VersionSet                                                                                           
 VersionWeight                                                                                        
 WeakRef                                                                                              
 Worker                                                                                               
 Zip                                                                                                  
 c_CholmodDense{T<:Union(Complex{Float64},Complex{Float32},Float64,Float32)}                          
 c_CholmodFactor{Tv<:Union(Complex{Float64},Complex{Float32},Float64,Float32),Ti<:Union(Int64,Int32)} 
 c_CholmodSparse{Tv<:Union(Complex{Float64},Complex{Float32},Float64,Float32),Ti<:Union(Int64,Int32)} 
 c_CholmodTriplet{Tv<:Union(Complex{Float64},Complex{Float32},Float64,Float32),Ti<:Union(Int64,Int32)}

subtypes is returing actual instances of DataType, which can be passed back into itself.

In [23]: 
subtypes(Real)
Out[23]:
4-element Array{Any,1}:
 FloatingPoint       
 Integer             
 MathConst{sym}      
 Rational{T<:Integer}
In [27]: 
subtypes(subtypes(subtypes(Real)[2])[end-1])
Out[27]:
5-element Array{Any,1}:
 Int128
 Int16 
 Int32 
 Int64 
 Int8

issubtype

Some interesting type relations span more than a single generation. Stepping around using super and subtypes makes exploring these by hand tedious.

issubtype is a function to tell you if its first argument is a descendent of its second argument. One reason this is useful is that if you have a method to handle the second argument, then you don’t have to worry if there’s an implementation for the first — it will use the implementation for the closest type ancestor that has one.

In [29]: 
issubtype(Int,Integer)
Out[29]:
true
In [31]: 
issubtype(Float64,Real)
Out[31]:
true
In [32]: 
issubtype(Any,DataType)
Out[32]:
false

Debugging

Once you’ve written your code, the built-in tools can continue to help you as you make it work correctly. Rather than showing you what’s available, these tools help you see what your code is actually doing.

Better print statement debugging with @show

While Julia has an interactive debugging package, it also has a @show macro that makes println debugging easier and more useful.

The show macro does two things:

  1. Print out a representation of the expression and the value it evaluates to
  2. Return that value
In [9]: 
@show 2 + 2

+(2,2) => 4
Out[9]:
4

That second thing is important for embeding @show‘s in the middle of expressions. Because it returns the resulting value, it is equivalent to the original expression.

In [28]: 
x = 5
y = x + @show x * 2

*(x,2) => 10
Out[28]:
15

@which

Something is going wrong in your code. When you read it, it looks fine: you’re definitely calling the right function. But when you run it, something is obviously wrong.

Which method is getting called there? Is that implementation correct?

The @which macro will take a function call and tell you not only what method would be called, but also give you a file name and line number.

In [13]: 
@which 2 + 2

+(x::Int64,y::Int64) at int.jl:41
In [15]: 
@which 2 + 2.0

+(x::Number,y::Number) at promotion.jl:148
In [19]: 
@which 'h' + 2

+(x::Char,y::Integer) at char.jl:30

Each of the above examples are methods in the base library, which means that you can either clone julia or look in your source install — look in the base folder for a file of the name @which indicates. For non-base code, it gives a file path rather than just a name.

macroexpand

Writing macros tends to be a bit complex and sometimes issues of macro hygeine can be difficult to predict from looking at the code of your macro. Just running the macro on some expressions doesn’t always help; you want to see what code the macro application results in.

In Julia, you can do that. You give macroexpand a quoted expression using your macro; it will return that expression with the macro transformations applied.

In [31]: 
macroexpand(:(@show 2+2))
Out[31]:
quote 
    $(Base).println("+(2,2) => ",$(Base).repr(begin  # show.jl, line 91:
                value#82 = +(2,2)
            end))
    value#82
end

You can also use macroexpand to see what other macros actually do. For example, a not uncommon pattern is to have the actual implementation in a normal function, while the macro allows uses to pass in unquoted expressions.

In [34]: 
macroexpand(:(@which 2+2))
Out[34]:
:($(Base).which(+,2,2))