Julia 0.5 Release Announcement

By: Julia Developers

Re-posted from: http://feedproxy.google.com/~r/JuliaLang/~3/tyDIHJxQsDY/julia-0.5-release

After over a year of development, the Julia community is proud to announce
the release of version 0.5 of the Julia language and standard library.
This release contains major language refinements and numerous standard library improvements.
A long list of changes is available in the NEWS log found in our main repository, with a summary reproduced below.
A separate blog post detailing some of the highlights of the new release has also been posted.

We’ll be releasing regular bugfix backports from the 0.5.x line, which is recommended for users requiring a stable language and API.
Major feature work is ongoing on master for 0.6-dev.

The Julia ecosystem continues to grow, and there are now over one thousand registered packages!
The third annual JuliaCon took place in Cambridge, MA in the summer of 2016, with an exciting line up of talks and keynotes.
Most of them are available to view.

Binaries are available from the main download page or visit JuliaBox to try this release from the comfort of your browser. Happy Coding!

Notable compiler and language changes:

  • The major focus of this release has been the ability to write fast functional code, removing the earlier performance penalty for anonymous functions and closures.
    This has been achieved via each function and closure now being its own type, and the captured variables of a closure are fields of its type.
    All functions, including anonymous functions, are now generic and support all features.

  • Experimental support for multi threading.

  • All dimensions indexed by scalars are now dropped, whereas previously only trailing scalar dimensions would be omitted from the result.
    This is a major breaking changes, but has been made to make the indexing rules much more consistent.

  • Generator expressions now can create iterators that are computed only on demand.

  • Experimental support for arrays whose indexing starts from values other than 1. Standard Julia arrays are still 1-based, but external packages can implement array types with indexing from arbitrary indices.

  • Major simplification of the string types, unifying ASCIIString and UTF8String as String, as well as moving types and functions related to different encodings out of the standard library.

  • Package operations now use the libgit2 library rather than shelling out to command line git. This makes these calls to package related functions much faster, and more reliable, especially on Windows.

  • And many many more changes and improvements…

Ports

Julia now runs on the ARM and Power architectures, making it possible to use it on the widest variety of hardware, from the smallest embedded machines to the largest HPC systems. Porting a language to a new architecture is never easy, so special thanks to the people who made it possible. Part of the work to create the Power port was supported by IBM, for which we are grateful.

Developing with Julia

The Julia debugger, Gallium, is now ready to use. It allows for a full, multi language debug experience, debugging Julia and C code with ease. The debugger is also integrated with Juno, the Julia IDE that is now fully featured and ready to use.

WordPress to Jekyll: A 30x Speedup

By: randyzwitch - Articles

Re-posted from: http://randyzwitch.com/wordpress-jekyll-30x-speedup/

About a month ago, I switched this blog from WordPress hosted on Bluehost to Jekyll on GitHub Pages. I suspected moving to a static website would be faster than generated HTML via PHP, and it is certainly cheaper (GitHub Pages is “free”). But it wasn’t until I needed a dataset for doing some dataset visualization development that I realize how much of an improvement it has been!

Packages, Packages, Packages

With the release of v0.5 of Julia, I’ve been working (less) on updating my packages and making new packages (more), because making new stuff is more fun than maintaining old stuff! One of the packages I’ve been building is for the ECharts visualization library (v3) from Baidu. While Julia doesn’t necessarily need another visualization library, visualization is something I’m interested in and learning is easier when you’re solving problems you like. And since the world doesn’t need another Iris example, I decided to share some real world website performance data 🙂

Line Chart

One of the first features I developed for ECharts.jl was X-Y charts, which I posit is the most common chart type in business. One thing that is great about the underlying ECharts JavaScript library is that interactivity is really easy to achieve:

using ECharts, DataFrames

#Read in data
df = readtable("/assets/data/website_time_data.csv")

#Make data two different series that overlap, so endpoint touches
df[:pre] = [(x[1] <= "2016-09-06" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]
df[:post] = [(x[1] >= "2016-09-06" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]

#Graph code
l = line(df[:date], hcat(df[:pre], df[:post]))
l.ec_width = 800
seriesnames!(l, ["loadtime_ms", "post"])
colorscheme!(l, palette = ("acw", "FlatUI"))
yAxis!(l, name = "Load time in ms")
title!(l, text = "randyzwitch.com",
          subtext = "Switching from WordPress on Bluehost to Jekyll on GitHub (2016/09/06)")
toolbox!(l, chartTypes = ["bar", "line"])
slider!(l)

Even though I switched to Jekyll on WordPress on 9/6/2016, it appears that the page cache for Google Webmaster Tools didn’t really expire until 9/12/2016 or so. At the average case, the load time went from 1128ms to 38ms! Of course, this isn’t really a fair comparison, as presumably GitHub Pages runs on much better hardware than the cheap Bluehost hosting I have, and I didn’t reimplement most of the garbage I had on the WordPress version of the blog. But from a user-experience standpoint, good lord what an improvement!

Box Plots

Want to test out further functionality, here are some box plots of the load time variation:

using ECharts, DataFrames

#Read in data
df = readtable("/Users/randyzwitch/Desktop/website_load_time.csv")
df[:pre] = [(x[1] <= "2016-09-06" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]
df[:post] = [(x[1] >= "2016-09-12" ? x[2] : nothing) for x in zip(df[:date], df[:loadtime_ms])]

#Remove nulls
pre = [x for x in df[:pre] if x != nothing]
post = [x for x in df[:post] if x != nothing]

#Graph code
b = box([pre, post], names = ["WordPress", "Jekyll"])
b.ec_width = 800
colorscheme!(b, palette = ("acw", "VitaminC"))
yAxis!(b, name = "Load time in ms", nameGap = 50, min = 0)
title!(b, text = "randyzwitch.com",
           subtext = "Switching from WordPress on Bluehost to Jekyll on GitHub (2016/09/06)")
toolbox!(b)

Usually, a box plot comparison that is as smushed as the Jekyll plot vs the WordPress one would be a poor visualization, but in this case I think it actually works. The load time for the Jekyll version of this blog is so quick and so consistent that it barely registers as an outlier if it were WordPress! It’s crazy to think that the -1.5 * IQR time for WordPress is the mean/median/min load time of Jekyll.

Where To Go Next?

This blog post is really just an interesting finding from my experience moving to Jekyll on GitHub. As it stands now, ECharts.jl is stil in pre-METADATA mode. Right now, I assume that this would be a useful enough package to submit to METADATA some day, but I guess that depends on how much further I get smoothing the rough edges. If there are people who are interested in cleaning up this package further, I’d absolutely love to collaborate.

Pkg Update: Your Window to the Julia Package Ecosystem

By: ChrisRackauckas

Re-posted from: http://www.pkgupdate.com/?p=18

Introduction to Pkg Update

Welcome to Pkg Update. This new blog focuses on the Julia package ecosystem: how packages/organizations are changing, what new projects you should be aware of, and how Base issues are propagating through the package-sphere. I hope that this will be a nice summary of the vast changes that happen throughout the Julia ecosystem. I want this to be a resource which both package users and package developers could read to get a general understanding of the big movements in the community. The goal is to link together issues/PRs/discussions from the the vast number of repositories to give a coherent idea of what changes to anticipate and congratulate. As Base continues to slim (for those who don’t know, there’s a trend for Base to become slimmer for a leaner install, with more functionality provided by the Julia organizations such as JuliaMath), I hope this helps you stay up to date with where everything is going!

While I try my best to be active throughout the Julia-sphere, this is not something that can be done alone. Please feel free to contact me via email, Twitter, or Gitter to give a summary of what’s going on in your organization / area of expertise. Also, if anyone is willing to join the Pkg Update team, I would be happy to have you! I am looking for correspondents to represent different Julia organizations and to help summarize the changes.

That said, here’s a quick highlight of some recent updates both users and developers might need to know.

JuliaMath: Exciting Developments in LibM.jl

I have to give a shoutout to @musm for his incredible work in Libm.jl. This project started with a discussion on whether Julia should continue to use OpenLibm (Libm is the math library which supplies the approximations for common functions like “sin” and “exp”). From this discussion the ambitious idea of creating a pure-Julia Libm was put forward, and Libm.jl has continued with the momentum.

Sleef is a C library notable for it’s use of SIMD. Libm.jl is largely based on a port of that code to Julia, it also has some added safety and correctness for edge cases over the original code. While the ported code is based on the non-SIMD parts of the C source, it reliably takes advantage f SIMD hardware none-the-less — thanks to LLVM autovectorisation . The code is particularly vectorisation friendly, as it uses next-to-no branches or table lookups Most of the benchmarks for this implementation are within 2x of C programs calling Sleef. In practice, because of how lining and FFI works, this means that using the julia Sleef implementation is significantly faster than ccalling the C sleef library. Not to mention safer.

Other libraries (Musl and Cephes) now have partial ports to Julia as well. There is work being done to use the strategies of these libraries together to build an even better Libm which might be dubbed Amal.jl. This library, being a Julia implementation, will have much broader support for the type system, including native implementations for Float16 and quad-precision floats, leading to a much more robust mathematical foundation.

JuliaStats API Decision: Degrees of Freedom as “dof”

This issue and the associated PR went through rather quickly. It highlighted the fact that until now, degrees of freedom in the stats packages had a tendency to use “df”, which is a naming conflict with using “df” as an instance for a DataFrame (a very common usage). The decision was to change the naming for degrees of freedom to dof. Since this change is in StatsBase, you can expect that this convention will propagate throughout the statistics stack: “df” for dataframes and “dof” for degrees of freedom.

JuliaDiffEq: Upcoming Sundials API Update

Sundials.jl, the interface to the popular ODE suite, has recently undergone a large overhaul to help improve the wrapper and to stop memory leaks. This is being combined with major changes to the “simplified interface” and an upgrade in the Sundials version. Thus the upcoming tag will be a minor release with some breaking but exciting improvements, including the ability to use the Sundials ARKODE solvers.

JunoLab: The Plot Pane is Taking Off

With the Julia v0.5 release, Juno released a new version of their stack which included an updated plot pane. The developers were rallied and many plotting packages now support the plot pane. This includes support from Plots.jl, meaning that the plots will plot in the plot pane if the backend is compatible. While Plotly is not compatible with the plot pane, a recent PR has added plot pane compatibility to PlotlyJS. PyPlot and GR are compatible with the plot pane. Also, compatibility with Gadfly has been added. This means that, at least on the master version of the Juno stack and the plotting packages, the plot pane is all working! For regular users you may want to wait for this all to be released, but if you’re brave, it’s already ready for testing and I have been making good use of it. Note that in Plots.jl you can still use the `gui()` command to open up the non-plot pane window. This can be useful since some plotting backends (like PyPlot) are not interactive in the plot pane.

JuliaML Manifesto Released, and Major Progress Ensues

If you haven’t read Tom Brelof’s JuliaML manifesto, I would take a good look at it. He gives an introduction to JuliaML, the new machine learning based Julia organization, and shows how the Learn ecosystem is leading to an easy-to-use but highly extendable API. If you’re interested in machine learning, this is an organization to start following. Lots of progress is happening over here.

New Package Highlights

Lastly, I would like to finish off each post by highlighting some newer and promising projects. Since this is the first post and there’s lots to say, I picked a few.

ParallelDataTransfer.jl

(Disclaimer: this is one of my own libraries) ParallelDataTransfer.jl has hit the top of the 14-day change chart on Julia Pulse, rapidly picking up steam since it’s new release. The package is pretty simple: it offers a easy-to-use API for performing safe data transfers between processes. The README also shows how to use the library within type-stable functions, giving an easy way to construct highly parallel Julia algorithms.

Query.jl

Another package showing large movement on the Julia Pulse is Query.jl. This package allows you to query from Julia data sources, including any iterable, DataFrames, and Arrays. It even allows for queries through DataStreams sources, effectively connecting the queries to CSVs and SQLite databases. It’s not hard to see the promising future that has.

RNG.jl

A Google Summer of Code project which has caught my attention is RNG.jl. This package does exactly what you’d expect: it implements a bunch of random number generators. The benchmarks show that there are some promising RNGs which are both fast and suitable for parallel usage. I could see this library’s RNGs as becoming a staple for many Julia packages.

That’s the End of the First Post!

Let me know what you think of this new blog. I tried to cover as wide of a range as I could, but to get a better sense of the even larger community, I will need your help. Let me know if you’re willing to be an organization correspondent who can write one of many (multiple) dispatches, or if you don’t have the time, feel free to submit stories when possible.