Julia features in Intel’s Parallel Universe Magazine

By: Julia Computing, Inc.

Re-posted from: http://juliacomputing.com/blog/2017/07/25/julia-in-intel-parallel-universe.html

Parallel Universe is Intel’s quarterly publication, known to cover stories on notable and latest innovations in the field of software development, from high performance computing to threading hybrid applications.

Julia received a special mention in the Editor’s (Henry A. Gabb, Senior Principal Engineer at Intel Corporation) letter of Issue 29, the latest publication that came out this week. Henry talks about using Julia and how it gave him startling performance gains, not just with numerically intensive applications, but also with string manipulation applications. He summed this positive mention up saying that Julia has his attention.

The main story on Julia, co-authored by Julia Computing’s Viral Shah, Ranjan Anantharaman and Prof. Alan Edelman, begins on page 23 of the magazine, and comprehensively covers an overview of the language and a summary of it’s powerful features. It also goes on to talk about how Julia solves the “two language problem”, about parallelization in Julia, and ends on a high with coverage on Project Celeste, a joint project of Julia Computing, Lawrence Berkeley Labs, Intel and UC Berkeley researchers, an ambitiously compute intensive project aiming to catalog a digital atlas of the universe.

The link to the magazine is available here

About Henry A. Gabb

Henry A. Gabb, Senior Principal Engineer at Intel Corporation, is a longtime high-performance and parallel
computing practitioner and has published numerous articles on parallel programming. He was editor/coauthor of
“Developing Multithreaded Applications: A Platform Consistent Approach” and was program manager of the Intel/
Microsoft Universal Parallel Computing Research Centers.

About Julia Computing and Julia

Julia Computing was founded in 2015 by the co-creators of the Julia language to provide support to businesses and researchers who use Julia.

Julia is the fastest modern high performance open source computing language for data and analytics. It combines the functionality and ease of use of Python, R, Matlab, SAS and Stata with the speed of Java and C++. Julia delivers dramatic improvements in simplicity, speed, scalability, capacity and productivity. Julia provides parallel computing capabilities out of the box and literally infinite scalability with minimal effort. With more than 1 million downloads and +161% annual growth, Julia adoption is growing rapidly in finance, energy, robotics, genomics and many other fields.

Julia calling C: A more minimal example

By: perfectionatic

Re-posted from: http://perfectionatic.org/?p=470

Earlier I presented a minimal example of Julia calling C. It mimics how one would go about writing C code, wrapping it a library and then calling it from Julia. Today I came across and even more minimal way of doing that while reading an excellent blog on Julia’s syntactic loop fusion. Associated with the blog was notebook that explores the matter further.

Basically, you an write you C in a string and pass it directly to the compiler. It goes something like

C_code= """
       double mean(double a, double b) {
         return (a+b) / 2;
const Clib=tempname()
open(`gcc -fPIC -O3 -xc -shared -o $(Clib * "." * Libdl.dlext) -`, "w") do f
     print(f, C_code)

The tempname function generate a unique temporary file path. On my Linux system Clib will be string like "/tmp/juliaivzRkT". That path is used to generate a library name "/tmp/juliaivzRkT.so" which will then used in the ccall:

julia> x=ccall((:mean,Clib),Float64,(Float64,Float64),2.0,5.0)

This approach would be be recommended if are writing anything sophisticated in C. However, it fun to experiment with for short bits of C code that you might like to call from Julia. Saves you the hassle of creating a Makefile, compiling, etc…

Julia in ecology: why multiple dispatch is good

By: Timothée Poisot

Re-posted from: https://armchairecology.blog/2017/07/10/julia-in-ecology-why-multiple-dispatch-is-good/

In what is going to be the most technical note so far, I will try to reflect on a few years of using the Julia programming language for computational ecology projects. In particular, I will discuss how multiple dispatch changed my life (for the better), and how it can be used to make ecological analyses streamlined. I will most likely add a few entries to this series during the fall, leading up to a class I will give in the winter.

But what is multiple dispatch?

Imagine a recipe that calls for onions, and you have left in the cupboard is shallots. You know that shallots are little delicate bundles of gustative pleasure, and so you cook them differently (butter and half a teaspoon of sugar), extra gently. And when it’s done, you add them to the rest of the ingredients. This is multiple dispatch.

In computer terms now, we can express this  recipe as the following pseudocode:

function cook(x::Onion)
   return fry(x, butter)

function cook(x::Shallot)
   return roast(x, butter, sugar)

If x is an onion, then we fry it. If it is a shallot, we roast it. The important point is that the interface is the same: no matter what x is, we can cook it.

And where is the ecology in that?

20170709_114207Let’s talk about species interaction networks! One of the things that has been bugging me for a while was that we have no good, common interface to analyze them. There are a variety of packages that are either specific to some types of networks, or specific to some measures, or (worth) both. Because there are many different types of ecological networks.

Or are there? In EcologicalNetwork.jl, I reduced them to a combination of two factors. Are they bipartite or unipartite, and are they quantitative, probabilistic, or deterministic.

In Julia, this can be explained by a number of types and unions of types, and this hierarchy allows to create a number of functions that have the same name, but behave in the correct way based on their input. For example, the number of species in a network is calculated differently if it is bipartite or unipartite:

function richness(N::Bipartite)
   return sum(size(N.A))

function richness(N::Unipartite)
   return size(N.A, 1)

Where this becomes more interesting, is when we start chaining functions. For example, we can take an empirical network, generate the probabilistic version for a null model, then generate replicates, and finally measure the nestedness on every replicate:

using EcologicalNetwork
ollerton() |> null2 |> nullmodel .|> (x) -> nodf(x)[1]

This lines takes advantage of the fact that each function will take the “right” decision based on the type of its input. Specifically, it goes this way: the empirical network is a bipartite and deterministic one. The null2 function generates a probabilistic network which is also bipartite. This is passed to nullmodel, which will generate a number of bipartite deterministic networks, all of them are then passed through  the nodf function to measure their nestedness.

And the resulting pipeline is also clear to read, and expresses what we want to do (how we do it is determined based on the types). As a consequence, we can have a much more general package for network analysis.

But why does this matter?

Because, in short, it lets us (and yes, there are other paradigms that let us do the same thing) express what we want to do. A good example would be measuring the diversity of an ecological community. Let’s say we have a site by species matrix, and this matrix has presence/absence data. We can measure diversity as the number of species as the sum of each row:

function diversity(x::Array{Bool,2})
   return sum(A, 2)

But if we have quantitative information, then we may want to apply Pielou’s measure on each row instead:

function diversity(x::Array{Number,2})
   return mapslices(pielou, x, 2)

In the case where we have a phylogenetic tree, then what about using PD?

function diversity(x::Array{Number,2}, t::PhyloTree)
 return mapslices(n -> pd(n, t), x, 2)

And so on and so forth. In all of these situations, we know that the same concept (diversity) means different things as a function of the context – and for this reason, we want to do different things.

I like  this approach because it lets me focus on the intent of what I want to do. The (still young) EcoJulia project led by Michael Krabbe Borregaard is an attempt to use some of the niftiest features of Julia to develop general interfaces to some types of ecological data. This is something I am really excited to see happen.