This week in Julia: October 25

By: This week in julia

Re-posted from: http://thisweekinjulia.github.io/julia/2014/10/25/October-25.html

The maintenence release Julia 0.3.2 was posted on Wednesday. A total of 70 commits were meticulously backported from the master development branch, bringing lots of bugfixes, documentation updates, and even a few performance improvements to the 0.3 release.

Major breaking changes on 0.4

The function call operator is now overloadable (#8712)! This has lots of wide-ranging implications… some of which I’m sure haven’t even been realized yet. Here’s what it already means:

  • It makes type constructors first class methods of the call function. You can still create constructor methods with the old function-like T(x,y,z) = ...; syntax, or you can add methods to call directly (e.g., call(::Type{A},x,y,z) = ...;). While I’m not familiar with the internals, it apparently simplifies a lot of the code surrounding type constructor generation. This “big new feature” resulted in a net decrease of 10 lines of code. Impressive!
  • There’s now a fallback method (by default) for all types such that calling T(args...) means convert(T, args...)::T (39dfd92). This means that you can now call Int(2.0) with the leading capital!
  • This is paving the way towards possibly deprecating the lowercase int functions (#1470: int, Int and box). One question is what to do with its elementwise methods (e.g., int([1.0,2.0,3.0]) to convert an Array{Float64} to an Array{Int}), but with one small optimization (2cb98a0), map(Int,[1.0,2.0,3.0]) is now just as fast as the old lowercase method. This is an interesting case; typically map is slower than explicit for loops since it cannot specialize on the function value, but when called with a type it can emit specialized code for that constructor.
  • This same trick is exploited in lots of the map/reduce code, using dummy Functor types to represent specific functions to allow method specialization and inlining. But with call overloading, those ‘dummy’ Functor types just got a lot smarter. Instead of using a secondary helper function to actually evaluate each function, you can overload the call operator for each Functor directly so the type behaves just like its desired function. It makes the Base reduction code a little bit simpler and much nicer to read. (#8790)
  • It also obviates the need for some functions like inf and nan, which are now deprecated. Previously, you would call inf(T) to get the representation of infinity in type T. With this change, however, you can simply type the reverse: T(Inf). (#8776)

I’m sure lots of this is still in flux, but this turned out to be a bigger and more exciting change than I expected. It has also broken lots of packages (although that may just be the result of a few dependencies breaking and cascading through the ecosystem), so in the short term beware.

Other 0.4 project updates

  • Work to allow static compilation and caching of modules (to reduce package load time dramatically) continues. A big backend change was merged (static compile part 3, #8656) in preparation for work on the user interace (as proposed part 4, #8745). There are still some questions about the underlying behaviors, but it’s exciting to see precompilation moving forward.
  • The documentation system also keeps marching along, now in a new PR (#8791).

Package ecosystem

  • A few weeks ago, Tim Holy proposed an alternative to multiple inheritance that has since become known as the “Tim Holy Traits Trick” (THTT) (comment on #2345). In short, it uses functions to dispatch on traits instead of using an inheritance tree to dispatch on ancestors. At its core, though, it allows for multiple interfaces. Mauro Werder took this idea and ran, putting together the Traits.jl package which uses macros to formalize interface definition, implementation, and dispatch. It’s not officially registered, but I think it’s all rather clever and worth a mention.

This week in Julia: October 18

By: This week in julia

Re-posted from: http://thisweekinjulia.github.io/julia/2014/10/18/October-18.html

Major breaking changes

  • As anticipated from last week’s integer arithmetic change, the usage of itrunc to truncate integers into a specific type has been removed. Since this is intrinsically a modular arithmetic calculation, rem is now the way to do this, with the idiomatic syntax being 1234 % Uint8 == 0xd2. (#8648)

New proposals

Several new proposals (juleps) were submitted this week.

  • Given that convert is one of the most heavily used functions in Julia, a renewed conversation about using x as Foo as sugar for convert(Foo, x)::Foo was started in #8710.
  • @StefanKarpinski put together a coherent story for how APL-style indexing and lazy transpose behaviors could be combined to solve some of the current inconsistencies (like the double-transpose of a vector not being a vector, and vector dot products being one-element arrays). #4774

0.4 project updates

  • @JeffBezanson managed to get the call-overload branch passing tests! A few questions remain before it can get merged, though. #8712.
  • And a postscript from last week: an intrepid group has managed to successfully build julia with MSVC. I missed including it last week, but I think it’s worthy of a note. See #7761 for details.

Package ecosystem

Sample image

October 10

By: This week in julia

Re-posted from: http://thisweekinjulia.github.io/julia/2014/10/10/October-10.html

There is so much new in the past two weeks! In the future, I’ll try to keep it to one-week posts, but I think all the major changes since my first shot at this on reddit deserve mention.

Major breaking changes

  • Integer arithmetic is now type-preserving
    #8420! Additionally,
    integer conversions (e.g., calling uint8(1234)) will now error if the
    argument doesn’t fit in the new type. The current way to properly truncate
    an integer to a smaller type is via itrunc(Uint8, 1234), but that may be
    changing very soon (#8646).
    Similarly, uint(-1) is also now an error.
  • The Dict literal syntax [a=>b,c=>d] has been deprecated and is replaced
    with Dict(a=>b,c=>d). {a=>b} is replaced with Dict{Any,Any}(a=>b).
    (K=>V)[...] is replaced with Dict{K,V}(...).
    The new syntax has many advantages: all of its components are first-class,
    it generalizes to other types of containers, it is easier to guess how to
    specify key and value types, and the syntaxes for empty and pre-populated
    dicts are synchronized. As part of this change, => is parsed as a normal
    operator, and Base defines it to construct Pair objects (#8521).
  • The any-typed array literal syntax {1,2,3} has also been deprecated. Taken together with the
    dict syntax change above, this means that curly braces will soon be available
    for an awesomer new syntax construct! #8578
  • An empty pair of square brackets [] now constructs an empty Any array
    instead of an array of None. This means that you can now push! elements into []. (#8493)

Standard library improvements

  • deepcopy now recurses through immutable types and makes copies of their mutable fields (#8560). In general, calling deepcopy on an object should generally have the same effect as serializing and then deserializing it.

Performance improvements

  • The julia REPL now magically starts up in half the time! (#8528)
  • Other things that got speed boosts: gcd (#8410), and comparisons with BigInts and BigFloats (#8512).

Upcoming changes and discussions

Package ecosystem

  • The pkg.julialang.org site now has a snazzy new ecosystem pulse page. It shows all the recent changes in the last week, with convenient links to the new and updated packages.
  • Tucked away in an innocent little thread about build systems is this gem: @nolta managed to convert the amos fortran library to julia. But that’s not even the coolest part… he wrote a little fortran-to-julia transpiler (in two passes, pass0.sh and pass1.jl to get the job done. And, amazingly, it is within 1.2x of the original fortran speed. Just goes to show how powerful @goto can really be!

Julia style string literal interpolation in R

By: Francis Smart

Re-posted from: http://www.econometricsbysimulation.com/2014/10/julia-style-string-literal.html

I feel like a sculptor who has been using the same metal tools for the last four years and happened to have looked at my comrades and found them sporting new, sleek electric tools. Suddenly all of the hard work put into maintaining and adapting my metal tools ends up looking like duck tape and bubble gum patches.

I hate to say it but I feel that I have become somewhat infatuated with Julia. And infatuation is the right word. I have not yet committed the time to fully immerse myself in the language, yet everything I know about it makes me want to learn more. The language is well known for its mind-blowingly speed accomplished through just-in-time compiling. It also has many features which enhance the efficiency and readability of its code (see previous post, note the documentation has greatly improved since posting).

However, though I very much want to, I cannot entirely switch my coding needs from R into Julia. This is primarily due to my ongoing usage of packages such as RStudio’s “Shiny” and the University of Cambridge’s server side software for building adaptive tests, “Concerto”. And so with regret I will resign my Julia coding to probably a minor portion of my programming needs.

That does not mean however that I can’t make some small changes to make R work more like Julia. To this end I have programmed a small function p which will replace string literals identified as “Hello #(name), how are you?” with their values being evaluated. If there are nested parenthesizes then it is necessary to close the literal with “)#”, for example “c=#(b^(1+a))#”.

# Julia like text concaction function.
p <- function(..., sep="", esc="#") { 
  # Change escape characters by specifying esc.
  # Break the input values into different strings cut at '#('
  x <- paste(..., sep=sep)
  x <- unlist(strsplit(x, paste0(esc,"("), fixed = TRUE))
 
  # The first element is never evaluated.
  out <- x[1]
  # Check if x has been split.
  if (length(x)>1) for (i in 2:length(x)) {
    y <- unlist(strsplit(x[i], paste0(")",esc), fixed = TRUE))
    if (x[i]==y[1])
      y <- unlist(regmatches(x[i], regexpr(")", x[i]),
                             invert = TRUE))
    out <- paste0(out, eval(parse(text=y[1])), y[-1])
  }
  out
}
 
name="Bob"
height=72
weight=230
 
# Let's see it in action
p(sep=" ", "Hello #(name).",
  "My record indicates you are #(height) inches tall and weigh #(weight) pounds.",
  "Your body mass index is #(round(703*weight/height^2,1))#") 
# [1] "Hello Bob. My record indicates you are 72 inches tall and weigh 230 pounds. 
# Your body mass index is 31.2" 
 
# The other nice thing about the p function is that it can be used to concat
# strings as a shortcut for paste0.
p("QRS","TUV")
# [1] "QRSTUV"

Created by Pretty R at inside-R.org

Thank you SO community for your help.

Julia: The “Distributions” Package

By: Francis Smart

Re-posted from: http://www.econometricsbysimulation.com/2014/10/julia-distributions-package.html

This is a follow up to my post from a few days ago exploring random number generation in Julia’s base system.  In this post I will explore the ‘distributions’ package.

You can find the excellent documentation on the “Distributions” package at:
http://distributionsjl.readthedocs.org/en/latest/index.html

# First let’s set the current directory
cd(“C:/Dropbox/Econometrics by Simulation/2014-10-October/”)

# This post uses the following distributions
using Distributions
using Gadfly

# I have got to say that I love the way Julia handles distributions
# as I discovered through this post.

# The Distributions package gives trenendous power to the user by
# providing a common framework to apply various function.

# For instance let’s say you want to draw 10 draws from a Binomial(n=10, p=.25) distribution
rand(Binomial(10, .25), 1, 10)
#  4  3  0  5  1  3  5  2  2  1

# Looks pretty standard right? Well, what if we want the mean?
mean(Binomial(10, .25))
# 2.5

# mode, skewness, kurtosis, median?
a = Binomial(10, .25)
println(“mode:”, mode(a), ” skewness:”, skewness(a),
        ” kurtosis:”, kurtosis(a), ” median:”, median(a))
# mode:3 skewness:0.3651483716701107 kurtosis:-0.06666666666666667 median:3
 

# Cool huh?

# Let’s see how we can use Gadfly to easily plot some distributions:
# First we generate the plot (assign it to a ‘p’ for later drawing to disk)
#  In order to plot the different CDFs I will use an ‘anonymous’ function defined:
#  argument -> f(argument)
p=plot([x -> cdf(Normal(5,5^.5), x),
      x -> cdf(Gamma(3,1), x),
      x -> cdf(Exponential(2), x)]
      , 0, 10)

 # Write the graph to disk
draw(PNG(“2014-10-06CDF.png”, 24cm, 12cm), p)

g = plot([x -> pdf(Normal(5,2), x),
      x -> pdf(Gamma(3,1), x),
      x -> pdf(Exponential(2), x)]
      , 0, 10)

draw(PNG(“2014-10-06PDF.png”, 24cm, 12cm), g)