Decision trees have played a significant role in data mining and machine learning since the 1960′s. They generate white-box classification and regression models which can be used for feature selection and sample prediction. The transparency of these models is a big advantage over black-box learners, in that the models are easy to understand and interpret, and that they can be readily extracted and implemented into any programming language (with nested if-else statements) for use in production environments. Furthermore, decision trees require almost no data preparation (i.e. normalization) and can handle both numerical and nominal/categorical data. Decision trees can also be … Keep reading
Tag Archives: Julia
Julia Helps
Programmers are always learning. You learn how to use new APIs, new functions, new types. You learn why your code doesn’t work.
Julia has a lot of built-in tools to help you navigate, learn, and debug. You can use these in the REPL and in normal code.
In this post, I split these functions and macros up based on what they help you do: understand functions, examine types, navigate the type hierarchy, or debug code.
Exploring New Functions¶
If you’re in the REPL, you’re probably playing with something new, trying to make it work. You use new-to-you functions, which means the question “how do I use this function?” comes up frequently.
methods¶
The most basic answer to this question is to list the method signatures for the function in question. You can often guess what a method does just from the arguments’ names and types. If your problem is passing the arguments in the wrong order, this will solve it.
In [2]:
methods(open)
Out[2]:
help¶
While you can get a lot from a verb and the list of nouns/types it works on, a hand-written description of what the function does it even better. We can get those at the REPL, too. The output of help is the exactly the same as the online function documentation; they’re generated from the same source files.
Currently, help will only work for functions in the base libraries. For packages, you’ll have to read their documentation online. However, there is good coverage for functions in base.
In [1]:
help(open)
Because help is so useful, there’s even a short hand for it. You can use ? to call the help function:
In [2]:
?help
Exploring New Types¶
Julia is a dynamically typed language where you can talk about types; types are first class values. Just as for functions, there are tools for helping you understand what you can do with unfamiliar types.
typeof¶
If you have a value, but aren’t sure of its type, you can use typeof.
In [11]:
typeof(5.0)
Out[11]:
The type that represents types is DataType. typeof is not just printing out a name; it is returning the type as a value.
In [12]:
typeof(Float64)
Out[12]:
methods¶
Types in Julia define special constructor functions of the same name as the type, like in OO languages. For other functions, you use the methods function to find out what combinations of arguments it can take; this also works for type constructors.
In [15]:
methods(Dict)
Out[15]:
names¶
Sometimes, you’ll get a new type and want to know not just what methods are already defined, but the structure of the type itself. Types in Julia are like records or structs in other languages: they have named properties. names will list the name of each property. These are Symbols instead of Strings because identifiers (variable names, etc) are distinct.
In [4]:
names(IOStream)
Out[4]:
types¶
You can also get the types of the fields. They are stored in the types field of a DataType, as a tuple of DataTypes in the same order as the names returned by names.
In [3]:
IOStream.types
Out[3]:
methodswith¶
Once you have a value and know its type, you want to know what can be done with it.
When you want to shell out to other programs from Julia, you create Cmds. Creating one is easy — just put in backticks what you’d type at the command line: `echo hi`. However, creating a Cmd doesn’t actually run it: you just get a value.
What can you do with your Cmd? Just ask methodswith, which prints out method signatures for all methods that take that type.
In [2]:
methodswith(Cmd)
In the case of Cmd, that’s not so helpful. I still don’t see a way to run my Cmd. 🙁
However, that’s not all you can do with a Cmd or the methodswith function. Passing true as the second argument will show all of the methods that take Cmd or any of it’s super types. (Be prepared for a very long list for most types.)
In [1]:
methodswith(Cmd,true)
As you can see, most of the relevant methods are defined for AbstractCmd rather than Cmd. You can also see both the relevant execution functions (run,readall,readsfrom,writesto,readandwrite,etc) and the redirection ones (|,&,>,>>,etc). (Julia parses the Cmd and execs the process itself, so there’s no shell involved; instead, you use Julia code for redirection and globs. For more on Cmd see these blog posts or the manual.)
Exploring the Type Hierarchy¶
In Julia, types are not just individual, unconnected values. They are organized into a hierarchy, as in most languages.
super¶
Each type has one supertype; you can find out what it is by using the super function.
The type heirarchy is a connected graph: you can follow a path of supertypes up from any node to Any (whose supertype is Any). Let’s do that in code, starting from Float64.
In [5]:
super(Float64)
Out[5]:
In [7]:
super(FloatingPoint)
Out[7]:
In [8]:
super(Real)
Out[8]:
In [9]:
super(Number)
Out[9]:
In [10]:
super(Any)
Out[10]:
subtypes¶
We can also go in the other direction. Let’s see what the subtypes of Any are.
In [17]:
subtypes(Any)
Out[17]:
subtypes is returing actual instances of DataType, which can be passed back into itself.
In [23]:
subtypes(Real)
Out[23]:
In [27]:
subtypes(subtypes(subtypes(Real)[2])[end-1])
Out[27]:
issubtype¶
Some interesting type relations span more than a single generation. Stepping around using super and subtypes makes exploring these by hand tedious.
issubtype is a function to tell you if its first argument is a descendent of its second argument. One reason this is useful is that if you have a method to handle the second argument, then you don’t have to worry if there’s an implementation for the first — it will use the implementation for the closest type ancestor that has one.
In [29]:
issubtype(Int,Integer)
Out[29]:
In [31]:
issubtype(Float64,Real)
Out[31]:
In [32]:
issubtype(Any,DataType)
Out[32]:
Debugging¶
Once you’ve written your code, the built-in tools can continue to help you as you make it work correctly. Rather than showing you what’s available, these tools help you see what your code is actually doing.
Better print statement debugging with @show¶
While Julia has an interactive debugging package, it also has a @show macro that makes println debugging easier and more useful.
The show macro does two things:
- Print out a representation of the expression and the value it evaluates to
- Return that value
In [9]:
@show 2 + 2
Out[9]:
That second thing is important for embeding @show‘s in the middle of expressions. Because it returns the resulting value, it is equivalent to the original expression.
In [28]:
x = 5
y = x + @show x * 2
Out[28]:
@which¶
Something is going wrong in your code. When you read it, it looks fine: you’re definitely calling the right function. But when you run it, something is obviously wrong.
Which method is getting called there? Is that implementation correct?
The @which macro will take a function call and tell you not only what method would be called, but also give you a file name and line number.
In [13]:
@which 2 + 2
In [15]:
@which 2 + 2.0
In [19]:
@which 'h' + 2
Each of the above examples are methods in the base library, which means that you can either clone julia or look in your source install — look in the base folder for a file of the name @which indicates. For non-base code, it gives a file path rather than just a name.
macroexpand¶
Writing macros tends to be a bit complex and sometimes issues of macro hygeine can be difficult to predict from looking at the code of your macro. Just running the macro on some expressions doesn’t always help; you want to see what code the macro application results in.
In Julia, you can do that. You give macroexpand a quoted expression using your macro; it will return that expression with the macro transformations applied.
In [31]:
macroexpand(:(@show 2+2))
Out[31]:
You can also use macroexpand to see what other macros actually do. For example, a not uncommon pattern is to have the actual implementation in a normal function, while the macro allows uses to pass in unquoted expressions.
In [34]:
macroexpand(:(@which 2+2))
Out[34]:
Julia Calling Python Calling Julia…
Julia is a young programming language.
This means that its native libraries are immature.
We are in a time when Julia is a mature enough as a language that it is out-pacing its libraries.
One way to use mature libraries from a young language is to borrow them from another language.
In this case, we’ll be borrowing from Python.
(Julia can also easily wrap libraries from C or Fortran.
In fact, this capability was important in combination with Python’s great C-interface to make calling Python from Julia do-able.)
Python from Julia: PyCall.jl
Calling Python from Julia requires PyCall.
You can get it using Pkg.add("PyCall").
Let’s start with a “Hello, World” scale example.
using PyCall
pyeval("2+2") #=> 4
pyeval("str(5)") #=> "5"
# doing things by hand, for fun :)
math = pyimport(:math) #=> PyObject <module 'math'>
pycall(math["sin"],Float64,1) #=> 0.8414709848078965
Using Python Libraries from Julia
For a quick practical example of Python libraries filling in Julia’s current gaps, we can use Python’s matplotlib for graphing.
If you also install matplotlib,
you can run this fun code snippet I found on the mailing list.
If you run it, it will pop up a window with a graph of a nice dotted, squiggley, red line.
using PyCall
@pyimport pylab
x = linspace(0,2*pi,1000); y = sin(3*x + 4*cos(2*x));
pylab.plot(x, y; color="red", linewidth=2.0, linestyle="--")
pylab.show()
pylab is the name for the Python module that includes matplotlib and numpy
in a single namespace, according to matplotlib’s docs;
it is the recommended interface to matplotlib.
@pyimport is a Julia macro that is equivalent to a using statement on the Julia equivalent of the Python module.
What this means is that we can now use pylab as if it’s a Julia module, which is what the dot-notation (pylab.plot) is taking advantage of.
We use two functions from pylab: plot and show.
Notice that we don’t have to do anything special when we invoke them:
they look just like Julia functions, right down to the keyword arguments.
Julia from Python: the julia module
This is the less polished direction;
it’s not really intended for public consumption yet.
To get the Python module that you need to call Julia, git clone https://github.com/JuliaLang/IJulia.jl.
Then, inside the IJulia.jl/python/ folder, run python setup.py install. (you may need sudo on that last command)
First, the opening incantation:
import julia
j = julia.Julia()
The second line will take a while to run; it’s starting and setting up Julia.
Now, you’ll be able to call Julia from the Python REPL.
For example:
j.run("2+2") #=> 4
j.run("sin(pi)") #=> 1.2246467991473532e-16
j.run("x = 5") #=> 5
j.run("x += 2") #=> 7
j.run("x") #=> 7
Let’s be best friends: Mutual Recursion.
This section is a code example.
We’ll start at the top, and then wander down through the implementation.
You can find all the code on github.
If you clone that repo, you can open a Python REPL up inside that folder and run the following:
import pyiseven
pyiseven.even(5)
As one would hope, you’ll get False back.
The import line probably took a really long time to load; the even function seems pretty straight forward given its name and behavior.
But, let’s take a look at pyiseven.py anyway.
import julia
j = julia.Julia()
def even(x):
j.run("using IsOdd")
return not j.run("IsOdd.odd(" + str(x) + ")")
Oh no!
This looks terribly inefficient: even if Julia is 20x faster than Python, calling another programming language to figure out if your number is odd is a bad plan.
Maybe we should take a look at IsOdd.jl:
module IsOdd
using PyCall
@pyimport pyiseven
function odd(x)
if x == 1
true
elseif x == 0
false
else
pyiseven.even(x-1)
end
end
end
Oh no! It gets worse!
We make another function call back into Python’s even function on every call to Julia’s odd.
That means that it takes two function calls (and two trips between languages) for each increment of even‘s argument.
(In fact, if you call pyiseven.even(201) (or greater), then the stack explodes, unfortunately.)
If you find that your production code is too slow because you’re using mutual recursion between nine different languages, blame Dan Luu for this terrible idea.
tl;dr
You should really check out PyCall, which does a great job of translating between Python and Julia values and modules.
If you’ve been using IPython notebook, then you should also try IJulia notebook. (Same frontend, different language). You may have noticed that the IJulia project’s repo is the origin of that Python julia module.