Tag Archives: coding

Vim as IDE for Julia

By: serhatcevikel

Re-posted from: http://www.serhatcevikel.com/?p=56

Despite being introduced recently (in 2012), Julia is growing in popularity as a general programming language and especially as a powerful tool in data analytics. Julia combines the productivity of interpreted languages such as R and Python and the efficiency of compiled languages such as C, C++ and Fortran and hence overcomes the “two language problem”.  Benhcmarks show that Julia performance is quite near that of C with the help of its powerful JIT compiler. The package base is quickly keeping up pace with the large bases of more established languages. The compiled assembly code of Julia code can be easily viewed from within its interactive shell (and its low level efficiency can be witnessed).

There are several ways to edit and run Julia code. The Jupyter notebook “IJulia” is the easiest one to deploy:

julia> Pkg.add("IJulia")
julia> using IJulia
julia> notebook(detached=true)

These commands will start a Julia kernel inside the default browser, and will persist even if Julia shell is exited. The notebook is good for prototyping short code snippets and executing them as block “cells”. However for larger and more complex coding tasks, an integrated development environment (IDE) is preferred.  “Getting Started with Julia”,  a comprehensive beginner’s resource for the language  by Ivo Balbaert, mentions the IDE options available for Julia as Julia Studio, Sublime-IJulia and Juno.

GUI based IDE’s usually absorb a large portion of the valuable CPU cycles and RAM that would otherwise be available for resource hungry data analytics tasks. For those coders who devote scarce computer resources to data tasks, lightweight solutions are preferred. Vim is among the most popular lightweight text editors and is highly versatile in a variety of languages due to its large base of plugins/extensions. For a coder who gets used to the practical key bindings, Vim acts as a natural extension of the body.

Although there are some extensions to be installed for Julia, for those who are used to work with Vim editor, there is not yet an integrated extension such as Nvim-R that combines Vim and R in a seamless manner. However several open-source extensions and applications can be combined to create a working environment for Julia through which you can edit Julia code with syntax highlighting and execute selected portions of the code inside the interactive shell.

As also mentioned in “Getting Started with Julia”, julia-vim plugin is a good starting point, but we have to combine it with some other resources to create an efficient environment.  The prerequisites for the recipe are:

Most of the prerequsites are available in repositories of mainstream Linux distributions or can be downloaded and installed from the source code repositories following the links.

We need the terminal multiplexer so that, both the code editor and interactive shell can be viewed at the same time ,  julia-vim plugin for Julia syntax highlighting in Vim and vim-slime plugin to send the Julia code from the Vim editor to interactive shell to be interpreted/compiled (not only for Julia, but any language with a REPL shell).

The default terminal multiplexer for vim-slime is GNU Screen and the below scripts will utilize it.  However the option can be changed to Tmux easily by:

let g:slime_target = "tmux"

command from the Vim editor or appending to .vimrc file.

After the installations, we first create the steps that GNU screen will follow when invoked from a shell script:

layout new
split -s
screen 0
exec vim
focus next
screen 1
exec julia
focus next
layout save default

We will save these steps into a file (named “screenjulia” in my example) that will be invoked by the -c argument of screen command in order to override the default configuration and create a new layout for the session. The file creates a new layout, splits the screen session horizontally, names the upper region as “0” and executes vim inside (the julia filename will be appended by the script), names the lower region as “1” and executes Julia inside.

Due to difficulties to pass arguments to the custom configuration file, the optional filename argument will be inserted by the script, if provided:

#!/bin/bash
# vim-julia-slime binder
sed -i "s/^exec vim.*$/exec vim $1/" /path/to/screenjulia
screen -S 1 -c /path/to/screenjulia

The sed command inserts the Julia filename if provided by the optional $1 argument to the end of “exec vim” line in the configuration file “screenjulia” (replace “/path/to/screenjulia” with your actual path). The default session name for screen is “1”.

In order to enable the Julia syntax highlighting feature, the filename must have a .jl extension. To enable these features, it is useful to add the following lines into the .vimrc file:

filetype plugin indent on
syntax on

Also enabling line numbers will simplify the line selection in visual mode, so it should also be added to the .vimrc file:

set nu

To use this script as a custom command, it should be made executable and a symlink pointing to the file should be created inside a location that is declared in the $PATH environment variable of your .bashrc or /etc/environment (such as /usr/bin) (replace /path/to/script with your actual path to the script file):

$ chmod +x /path/to/juliascriptname
$ sudo ln -s /path/to/juliascriptname /usr/bin/juliascriptname

Now, invoking the scriptname (replace it with any name you’d like for your command) command from anywhere will initiate the following layout:

$ juliascriptname juliafilename.jl

The code lines are selected in visual mode in Vim and sent by C-c, C-c command (insert mode should be exited) – simply holding CTRL and double tapping c. In each session, the first time this sequence is entered, Vim will prompt for screen session and window name. The default values as defined in our script are 1 and 1. Anytime you can change these values with C-c, v or :SlimeConfig.

The visual mode works by v for character-wise and V for line-wise selections. Without any selection, C-c, C-c binding will send the current block. vip command also selects the current block or paragraph. To be more precise in the line range of code, you should enter VXgg where X is the number of last line to be selected (set nu option in .vimrc file is important in this sense)
With a small code sample for demonstration purposes, selecting first 3 lines with V3gg from Vim and sending to Julia window by C-c, C-c result in:

The switch between panes you should use C-a [tab] and the complete layout can be exited through screen by C-a \ binding.

With an Nvim-R like plugin, vim + Julia can be integrated better with autocompletion and an object browser.

Revisiting emulated OOP behaviour and multiple dispatch in Julia

By: Terence Copestake

Re-posted from: https://thenewphalls.wordpress.com/2014/06/02/revisiting-emulated-oop-behaviour-and-multiple-dispatch-in-julia/

In an earlier post, I explored one approach to emulating bundling functionality with the data on which it operates, akin to object methods in OOP languages such as C# and PHP. A comment posted by Matthew Browne questioned whether this approach was compatible with Julia’s multiple dispatch.

This is something I thought about at the time of writing the original article, but I had assumed it wouldn’t be possible due to the way in which the anonymous functions are assigned to variables i.e. assigning one definition would overwrite the previous. However, Matthew’s question prompted me to reconsider – and after some brief experimentation and some small alterations, I found that there is indeed a way to maintain compatibility with multiple dispatch.

Below is an updated example type definition:

type MDTest
    method::Function

    function MDTest()
        this = new()

        function TestFunction(input::String)
            println(input)
        end

        function TestFunction(input::Int64)
            println(input * 10)
        end

        this.method = TestFunction

        return this
    end
end

The theory is basically the same, with the constructor assigning the methods to their respective fields within the type. The difference is in how the functions are defined and assigned.

On lines 7 and 11, methods are defined with different argument types. These methods could be defined outside of the type definition without error, but defining them within the constructor has the advantage of not polluting the global scope.

On line 15, the function is assigned to its field using some slightly different syntax, which allows both methods to be called.

With this, the example code below:

test = MDTest()

test.method("String")

test.method(5)

Produces the output:

String
50

Another advantage to this approach is the absence of anonymous functions – which, according to benchmarks and GitHub issues, have significantly worse performance compared to named functions.

Julia variable gotchas

By: Terence Copestake

Re-posted from: http://thenewphalls.wordpress.com/2014/04/07/julia-variable-gotchas/

As is typical for many languages, assigning one variable to another in Julia does not create a copy of the variable data, but rather a reference to the existing data. However, I learned the hard way whilst working on the CGI module* that Julia does not currently support a copy-on-write mechanism for collections.

Take the example code below:

n = [ 1, 2, 3 ]

m = n

As expected, m becomes a reference to the collection referenced by n. Working with any number of mainstream languages, one might expect a copy to be made of the data referenced by n if either n or m is modified, for example:

n = [ 1, 2, 3 ]

m = n

push!(n, 4)

# Expect n = [ 1, 2, 3, 4] and m = [ 1, 2, 3 ]

This is not the case for Julia. When the array pointed to by n is modified, m maintains its reference to that same array, giving both a value of [ 1, 2, 3, 4 ].

Problems in the wild

I encountered this quirk when working with binary data and UTF-8 strings.

n = Uint8[ 0x32, 0x33, 0x34, 0x61 ]

m = utf8(n)

empty!(n)

Having created a string using the utf8 function, I wanted to empty the original byte array to free those resources. After a few minutes of trying to figure out how a bounds error had crept in to my app, I narrowed it down to this deletion of the byte array.

Digging deeper into the Julia source, the utf8 function is just an alias for a conversion function.

utf8(x) = convert(UTF8String, x)
...
convert(::Type{UTF8String}, a::Array{Uint8,1}) = is_valid_utf8(a) ? UTF8String(a) : ...

You can see here that passing an array of Uint8 bytes to utf8() creates an instance of UTF8String with the Uint8 array as its data. The type definition for UTF8String is:

immutable UTF8String <: String
    data::Array{Uint8,1}
end

As was covered above, the UTF8String’s data field will be only a reference to the collection passed to the utf8 function. If that collection is modified in any way at any point during the program’s runtime, so too will be the returned string.

In closing

It seems that the solution at this time is to explicitly use the copy or deepcopy functions, where copies of data are required by the program logic.

The issue is explored in this Google Groups thread. If I’ve understood correctly, the gist of it is that Julia makes this sacrifice for the sake of performance. As this is a feature wanted by many, there’s a possibility of it being implemented in a later version of the language.

* Write-up to follow at a later date