Newsletter November 2019

Julia Computing at the American Conference on Pharmacometrics: Julia Computing presented Pumas at the American Conference on Pharmacometrics (ACoP). Pumas is an artificial intelligence engine for pharmaceutical modeling and simulation in Julia. Julia Computing’s Chris Rackauckas and University of Maryland, Baltimore (UMB)’s Vijay Ivaturi were the winners of the International Society of Pharmacometrics (ISoP) Mathematical and Computational Sciences (MCS) Special Interest Group (SIG) Poster Award for Bayesian-Koopman Techniques for Optimization of Intervention With Respect to Uncertainty in PuMaS.jl.

Live Online Instructor-Led Julia Training: Sign up now for live instructor-led online courses taught by Julia Computing instructors. Each course is 4 hours per day for two days, for a total of 8 hours of instruction per course.

Course Schedule Cost for 8 hours of live online instruction from Julia Computing instructors
Introduction to Julia Day 1: Wed Nov 13 from 11 am – 3 pm ET Day 2: Thurs Nov 14 from 11 am – 3 pm ET $250
Introduction to Machine Learning and Artificial Intelligence Using Julia Day 1: Wed Nov 20 from 11 am – 3 pm ET Day 2: Thurs Nov 21 from 11 am – 3 pm ET $500
Parallel Computing in Julia Day 1: Tues Nov 26 from 11 am – 3 pm ET Day 2: Wed Nov 27 from 11 am – 3 pm ET $500
Register & Pay

Julia Enterprise Users: Enterprise use of Julia continues to grow. Please contact us for more information about Julia Computing enterprise solutions including JuliaSure, JuliaTeam and JuliaRun.

Julia Computing’s Alan Edelman Wins Institute of Electrical and Electronics Engineers (IEEE) Sidney Fernbach Award: Alan Edelman won the Sidney Fernbach Award for “outstanding breakthroughs in high-performance computing, linear algebra, and computational science and for contributions to the Julia programming language.” Alan is co-creator of Julia, co-founder and Chief Scientist at Julia Computing, director of the Julia Lab at MIT and Professor of Applied Mathematics at MIT. Alan will accept the award at SC19 in Denver, Colorado on November 19.

From Tree Leaves to Galaxies: Keno Fischer, Julia Computing co-founder and CTO (Tools) discussed Julia and Julia Computing with Robin.ly. The full video interview is available here.

JuliaTeam on AWS Marketplace: JuliaTeam is available for purchase on Amazon Web Services (AWS) Marketplace. JuliaTeam is an enterprise solution from Julia Computing that makes it easy for enterprise users to install and manage public and private packages, adhere to enterprise governance policies, deploy and scale applications, manage licenses and set up continuous integration.

Julia Day in New York: Julia Computing hosted a successful Julia Day in New York at Google’s New York headquarters on October 7. More than 50 attendees joined a discussion with Julia co-creators and Julia Computing co-founders Viral Shah and Stefan Karpinski, Conning Managing Director David Weiss and Elton Pereira from State Street / BestX.

Julia Computing Webinars: Julia Computing invites you to join two upcoming Webinars. Each Webinar is one hour in length, including question and answer, and led by Julia Computing’s Dr. Matt Bauman. There is no cost to participate.

Webinar Schedule Registration
Machine Learning with Julia
Dr. Matt Bauman, Julia Computing		 Tues Nov 19
12:00 pm – 1:00 pm ET		 Click here to register
Private Package Management and Governance with Julia
Dr. Matt Bauman, Julia Computing		 Thurs Nov 28
12:00 pm – 1:00 pm ET		 Click here to register

Other Upcoming Julia Computing Events

Julia Computing will be participating in a number of upcoming conferences. Please contact us if you would like to meet with us at any of these events.

  1. Denver: SC19 with Alan Edelman (Julia Computing) Nov 17-22
  2. London: Open Data Science Conference (ODSC) with Avik Sengupta (Julia Computing) Nov 19-22
  3. Montreal: Node + JS Interactive with Jameson Nash (Julia Computing) Dec 11-12

JuliaCon 2020: JuliaCon 2020 will take place July 27-31 at ISCTE – Instituto Universitário de Lisboa (ISCTE-IUL) in Lisbon, Portugal. Stay tuned for more details.

JuliaBox (Free) Ends – JuliaBox (Paid) Starts at Just $7 per Month for Academic Users: In January, we notified the Julia community that Julia’s growth was making free JuliaBox unsustainable, and that we would sunset the free version of JuliaBox. As a result, the free version of JuliaBox ended on Oct 31, 2019. We are grateful to all JuliaBox users – especially our paid users. If you want to continue using JuliaBox, please sign up for the paid version. For academic users, the cost starts at just $7 per month. If you think your organization, academic department or university should purchase JuliaBox, please contact the decision-maker at your organization, academic department or university, and explain to that person how and why JuliaBox is important to your work. Pricing information is available online, including the 50% academic discount. For other questions about purchasing a paid subscription to JuliaBox, including setting up a new account for your organization and invoicing, please contact us.

Julia and Julia Computing in the News

  • InsideHPC: Julia Computing Chief Scientist Alan Edelman Wins Prestigious IEEE Sidney Fernbach Award
  • MIT News: Alan Edelman recognized with 2019 IEEE Computer Society Sidney Fernbach Award
  • HPCWire: JuliaTeam Available on Amazon Web Services Marketplace
  • DevClass: AWS Tempts Open Source Projects with Promo Credit Sweeteners
  • HostReview: Which Machine Learning Frameworks To Try In 2019-2020
  • SIAM News: Scientific Machine Learning – How Julia Employs Differentiable Programming to Do it Best

Julia Blog Posts

Upcoming Julia Events

Recent Julia Events

Julia Meetup Groups: There are 37 Julia Meetup groups worldwide with 8,725 members. If there’s a Julia Meetup group in your area, we hope you will consider joining, participating and helping to organize events. If there isn’t, we hope you will consider starting one.

Julia Jobs, Fellowships and Internships

Do you work at or know of an organization looking to hire Julia programmers as staff, research fellows or interns? Would your employer be interested in hiring interns to work on open source packages that are useful to their business? Help us connect members of our community to great opportunities by sending us an email, and we’ll get the word out.

There are more than 300 Julia jobs currently listed on Indeed.com, including jobs at Accenture, Airbus, Amazon, AstraZeneca, AT&T, Barnes & Noble, BlackRock, Capital One, CBRE, Charles River Analytics, Citigroup, Comcast, Conde Nast, Cooper Tire & Rubber, Disney, Dow Jones, Facebook, Gallup, Genentech, General Electric, Google, Huawei, Ipsos, Johnson & Johnson, KPMG, Lockheed Martin, Match, McKinsey, NBCUniversal, Netflix, Nielsen, Novartis, OKCupid, Opendoor, Oracle, Pandora, Peapod, Pfizer, Raytheon, Spectrum, Wells Fargo, Zillow, Brown, BYU, Caltech, Dartmouth, Emory, Harvard, Johns Hopkins, Louisiana State University, Massachusetts General Hospital, MIT, Penn State, Princeton, UC Davis, University of Chicago, University of Delaware, University of Kentucky, UNC-Chapel Hill, USC, University of Virginia, Argonne National Laboratory, Federal Reserve Bank, Lawrence Berkeley National Laboratory, Los Alamos National Laboratory, National Renewable Energy Laboratory, Oak Ridge National Laboratory, Pacific Northwest National Laboratory, State of Wisconsin and many more.

Contact Us: Please contact us if you wish to:

  • Purchase or obtain license information for Julia products such as JuliaSure, JuliaTeam, or JuliaRun
  • Obtain pricing for Julia consulting projects for your organization
  • Schedule Julia training for your organization
  • Share information about exciting new Julia case studies or use cases
  • Spread the word about an upcoming conference, workshop, training, hackathon, meetup, talk or presentation involving Julia
  • Partner with Julia Computing to organize a Julia meetup, conference, workshop, training, hackathon, talk or presentation involving Julia
  • Submit a Julia internship, fellowship or job posting

About Julia and Julia Computing

Julia is the fastest high performance open source computing language for data, analytics, algorithmic trading, machine learning, artificial intelligence, and other scientific and numeric computing applications. Julia solves the two language problem by combining the ease of use of Python and R with the speed of C++. Julia provides parallel computing capabilities out of the box and unlimited scalability with minimal effort. Julia has been downloaded more than 11 million times and is used at more than 1,500 universities. Julia co-creators are the winners of the 2019 James H. Wilkinson Prize for Numerical Software. Julia has run at petascale on 650,000 cores with 1.3 million threads to analyze over 56 terabytes of data using Cori, one of the ten largest and most powerful supercomputers in the world.

Julia Computing was founded in 2015 by all the creators of Julia to develop products and provide professional services to businesses and researchers using Julia.

The Emergent Features of JuliaLang: Part II – Traits

By: Lyndon White

Re-posted from: https://invenia.github.io/blog/2019/11/06/julialang-features-part-2/

Introduction

This blog post is based on a talk originally given at a Cambridge PyData Meetup, and also at a London Julia Users Meetup.

This is the second of a two-part series of posts on the Emergent Features of JuliaLang.
That is to say, features that were not designed, but came into existence from a combination of other features.
While Part I was more or less a grab-bag of interesting tricks,
Part II (this post), is focused solely on traits, which are, in practice, much more useful than anything discussed in Part I.
This post is fully indepedent from Part I.

Traits

Inheritance can be a gun with only one bullet, at least in the case of single inheritance like Julia uses.
Single-inheritance means that, once something has a super-type, there can’t be another.
Traits are one way to get something similar to multiple inheritance under such conditions.
They allow for a form of polymorphism,
that is orthogonal to the inheritence tree, and can be implemented using functions of types.
In an earlier blog post this is explained using a slightly different take.

Sometimes people claim that Julia doesn’t have traits, which is not correct.
Julia does not have syntactic sugar for traits, nor does it have the ubiqutious use of traits that some langauges feature.
But it does have traits, and in fact they are even used in the standard library.
In particular for iterators, IteratorSize and IteratorEltype, and for several other interfaces.

These are commonly called Holy traits, not out of religious glorification, but after Tim Holy.
They were originally proposed to make StridedArrays extensible.
Ironically, even though they are fairly well established today, StridedArray still does not use them.
There are on-going efforts
to add more traits to arrays, which one day will no doubt lead to powerful and general BLAS-type functionality.

There are a few ways to implement traits, though all are broadly similar.
Here we will focus on the implementation based on concrete types.
Similar things can be done with Type{<:SomeAbstractType} (ugly, but flexible),
or even with values if they are of types that constant-fold (like Bool),
particularly if you are happy to wrap them in Val when dispatching on them (an example of this will be shown later).

There are a few advantages to using traits:

  • Can be defined after the type is declared (unlike a supertype).
  • Don’t have to be created up-front, so types can be added later (unlike a Union).
  • Otherwise-unrelated types (unlike a supertype) can be used.

A motivating example from Python: the AsList function

In Python TensorFlow, _AsList is a helper function:

def _AsList(x):
    return x if isinstance(x, (list, tuple)) else [x]

This converts scalars to single item lists, and is useful because it only needs code that deals with lists.
This is not really idiomatic python code, but TensorFlow uses it in the wild.
However, _AsList fails for numpy arrays:

>>> _AsList(np.asarray([1,2,3]))
[array([1, 2, 3])]

This can be fixed in the following way:

def _AsList(x):
    return x if isinstance(x, (list, tuple, np.ndarray)) else [x]

But where will it end? What if other packages want to extend this?
What about other functions that also depend on whether something is a list or not?
The answer here is traits, which give the ability to mark types as having particular properties.
In Julia, in particular, dispatch works on these traits.
Often they will compile out of existence (due to static dispatch, during specialization).

Note that at some point we have to document what properties a trait requires (e.g. what methods must be implemented).
There is no static type-checker enforcing this, so it may be helpful to write a test suite for anything that has a trait which checks that it works properly.

The Parts of Traits

Traits have a few key parts:

  • Trait types: the different traits a type can have.
  • Trait function: what traits a type has.
  • Trait dispatch: using the traits.

To understand how traits work, it is important to understand the type of types in Julia.
Types are values, so they have a type themselves: DataType.
However, they also have the special pseudo-supertype Type, so a type T acts like T<:Type{T}.

typeof(String) === DataType
String isa Type{String} === true
String isa Type{<:AbstractString} === true

We can dispatch on Type{T} and have it resolved at compile time.

Trait Type

This is the type that is used to attribute a particular trait.
In the following example we will consider a trait that highlights the properties of a type for statistical modeling.
This is similar to the MLJ ScientificTypes.jl, or the StatsModel schema.

abstract type StatQualia end

struct Continuous <: StatQualia end
struct Ordinal <: StatQualia end
struct Categorical <: StatQualia end
struct Normable <: StatQualia end

Trait Function

The trait function takes a type as input, and returns an instance of the trait type.
We use the trait function to declare what traits a particular type has.
For example, we can say things like floats are continuous, booleans are categorical, etc.

statqualia(::Type{<:AbstractFloat}) = Continuous()
statqualia(::Type{<:Integer}) = Ordinal()

statqualia(::Type{<:Bool}) = Categorical()
statqualia(::Type{<:AbstractString}) = Categorical()

statqualia(::Type{<:Complex}) = Normable()

Using Traits

To use a trait we need to re-dispatch upon it.
This is where we take the type of an input, and invoke the trait function on it to get objects of the trait type, then dispatch on those.

In the following example we are going to define a bounds function, which will define some indication of the range of values a particular type has.
It will be defined on a collection of objects with a particular trait, and it will be defined differently depending on which statqualia they have.

using LinearAlgebra

# This is the trait re-dispatch; get the trait from the type
bounds(xs::AbstractVector{T}) where T = bounds(statqualia(T), xs)

# These functions dispatch on the trait
bounds(::Categorical, xs) = unique(xs)
bounds(::Normable, xs) = maximum(norm.(xs))
bounds(::Union{Ordinal, Continuous}, xs) = extrema(xs)

Using the above:

julia> bounds([false, false, true])
2-element Array{Bool,1}:
 false
 true

julia> bounds([false, false, false])
1-element Array{Bool,1}:
 false

julia> bounds([1,2,3,2])
(1, 3)

julia> bounds([1+1im, -2+4im, 0+-2im])
4.47213595499958

We can also extend traits after the fact: for example, if we want to add the
property that vectors have norms defined, we could define:

julia> statqualia(::Type{<:AbstractVector}) = Normable()
statqualia (generic function with 6 methods)

julia> bounds([[1,1], [-2,4], [0,-2]])
4.47213595499958

Back to AsList

First, we define the trait type and trait function:

struct List end
struct Nonlist end

islist(::Type{<:AbstractVector}) = List()
islist(::Type{<:Tuple}) = List()
islist(::Type{<:Number}) = Nonlist()

The we define trait dispatch:

aslist(x::T) where T = aslist(islist(T), x)
aslist(::List, x) = x
aslist(::Nonlist, x) = [x]

This allows aslist to work as expected.

julia> aslist(1)
1-element Array{Int64,1}:
 1

julia> aslist([1,2,3])
3-element Array{Int64,1}:
 1
 2
 3

julia> aslist([1])
1-element Array{Int64,1}:
 1

As discussed above this is fully extensible.

Dynamic dispatch as fallback.

All the traits discussed so far have been fully-static, and they compile away.
We can also write runtime code, at a small runtime cost.
The following makes a runtime call to hasmethod to look up if the given type
has an iterate method defined.
(There are plans to make hasmethod compile time.
But for now it can only be done at runtime.)
Similar code to this can be used to dispatch on the values of objects.

islist(T) = hasmethod(iterate, Tuple{T}) ? List() : Nonlist()

We can see that this works on strings, as it does not wrap the following into an array.

julia> aslist("ABC")
"ABC"

Traits on functions

We can also attach traits to functions, because functions are instances of singleton types,
e.g. foo::typeof(foo).
We can use this idea for declarative input transforms.

As an example, we could have different functions expect the arrangement of observations to be different.
More specifically, different machine learning models might expect the inputs be:

  • Iterator of Observations.
  • Matrix with Observations in Rows.
  • Matrix with Observations in Columns.

This isn’t a matter of personal perference or different field standards;
there are good performance-related reasons to choose one the above options depending on what operations are needed.
As a user of a model, however, we should not have to think about this.

The following examples use LIBSVM.jl, and DecisionTree.jl.
In practice we can just use the MLJ interface instead,
which takes care of this kind of thing.

In this demonstration of the simple use of a few ML libraries, first we need some basic functions to deal with our data.
In particular we want to define get_true_classes which as it would naturally be written would expect an iterator of observations.

using Statistics
is_large(x) = mean(x) > 0.5
get_true_classes(xs) = map(is_large, xs)

inputs = rand(100, 1_000);  # 100 features, 1000 observations
labels = get_true_classes(eachcol(inputs));

For simplicity, we will simply test on our training data,
which should be avoided in general (outside of simple validation that training is working).

The first library we will consider is LIBSVM, which expect the data to be a matrix with 1 observation per column.

using LIBSVM
svm = svmtrain(inputs, labels)

estimated_classes_svm, probs = svmpredict(svm, inputs)
mean(estimated_classes_svm .== labels)

Next, lets try DecisionTree.jl, which expects data to be a matrix with 1 observation per row.

using DecisionTree
tree = DecisionTreeClassifier(max_depth=10)
fit!(tree, permutedims(inputs), labels)

estimated_classes_tree = predict(tree, permutedims(inputs))
mean(estimated_classes_tree .== labels)

As we can see above, we had to know what each function needed, and use eachcol and permutedims to modify the data.
The user should not need to remember these details; they should simply be encoded in the program.

Traits to the Rescue

We will attach a trait to each function that needed to rearrange its inputs.
There is a more sophisticated version of this,
which also attaches traits to inputs saying how the observations are currently arranged, or lets the user specify.
For simplicity, we will assume the data starts out as a matrix with 1 observations per column.

We are considering three possible ways a function might like its data to be arranged.
Each of these will define a different trait type.

abstract type ObsArrangement end

struct IteratorOfObs <: ObsArrangement end
struct MatrixColsOfObs <: ObsArrangement end
struct MatrixRowsOfObs <: ObsArrangement end

We can encode this information about each function expectation into a trait,
rather than force the user to look it up from the documentation.

# Out intial code:
obs_arrangement(::typeof(get_true_classes)) = IteratorOfObs()

# LIBSVM
obs_arrangement(::typeof(svmtrain)) = MatrixColsOfObs()
obs_arrangement(::typeof(svmpredict)) = MatrixColsOfObs()

# DecisionTree
obs_arrangement(::typeof(fit!)) = MatrixRowsOfObs()
obs_arrangement(::typeof(predict)) = MatrixRowsOfObs()

We are also going to attach some simple traits to the data types to say whether or not they contain observations.
We will use value types for this,
rather than fully declare the trait types, so we can just skip straight to declaring the trait functions:

# All matrices contain observations
isobs(::AbstractMatrix) = Val{true}()

# It must be iterator of vectors, else it doesn't contain observations
isobs(::T) where T = Val{eltype(T) isa AbstractVector}()

Next, we can define model_call: a function which uses the traits to decide how
to rearrange the observations before calling the function, based on the type of the function and on the type of the argument.

function model_call(func, args...; kwargs...)
    return func(maybe_organise_obs.(func, args)...; kwargs...)
end

# trait re-dispatch: don't rearrange things that are not observations
maybe_organise_obs(func, arg) = maybe_organise_obs(func, arg, isobs(arg))
maybe_organise_obs(func, arg, ::Val{false}) = arg
function maybe_organise_obs(func, arg, ::Val{true})
    organise_obs(obs_arrangement(func), arg)
end

# The heavy lifting for rearranging the observations
organise_obs(::IteratorOfObs, obs_iter) = obs_iter
organise_obs(::MatrixColsOfObs, obsmat::AbstractMatrix) = obsmat

organise_obs(::IteratorOfObs, obsmat::AbstractMatrix) = eachcol(obsmat)
function organise_obs(::MatrixColsOfObs, obs_iter)
    reduce(hcat, obs_iter)
end

function organise_obs(::MatrixRowsOfObs, obs)
    permutedims(organise_obs(MatrixColsOfObs(), obs))
end

Now, rather than calling things directly, we can use model_call,
which takes care of rearranging things.
Notice that the code no longer needs to be aware of the particular cases for each library, which makes things much easier for the end-user: just use model_call and don’t worry about how the data is arranged.

inputs = rand(100, 1_000);  # 100 features, 1000 observations
labels = model_call(get_true_classes, inputs);

using LIBSVM
svm = model_call(svmtrain, inputs, labels)

estimated_classes_svm, probs = model_call(svmpredict, svm, inputs)
mean(estimated_classes_svm .== labels)

using DecisionTree
tree = DecisionTreeClassifier(max_depth=10)
model_call(fit!, tree, inputs, labels)

estimated_classes_tree = model_call(predict, tree, inputs)
mean(estimated_classes_tree .== labels)

Conclusion: JuliaLang is not magic

In this series of posts we have seen a few examples of how certain features can give rise to other features.
Unit syntax, Closure-based Objects, Contextual Compiler Passes, and Traits, all just fall out of the combination of other features.
We have also seen that Types turn out to be very powerful, especially with multiple dispatch.