Category Archives: Julia

Some comments on DataFrames 1.0 release

By: Blog by Bogumił Kamiński

Re-posted from: https://bkamins.github.io/julialang/2021/04/24/dataframes.html

Introduction

DataFrames.jl has just got a 1.0 release.
The major question we should answer following this is:

What consequences for users does this have?

The answer is pretty boring (but significant): you can expect that we will
not introduce any braking changes till 2.0 release. The point is that we judge
that the package is mature enough that 2.0 release will not happen soon.
In consequence it is safe to use DataFrames.jl in production code that is expected
not to be updates over longer periods.

The other aspect of calling DataFrames.jl mature enough is that we believe that
the package and its ecosystem have a good performance, so you should be able
to safely use it in your data science project without getting sudden timing
hiccups.

In consequence in this post I want to cover two things. First is what is
deprecated in DataFrames.jl 1.0 release (which means it might get removed or
changed in some 1.x release). The second is some simple performance comparisons.

This post was written under Julia 1.6 and DataFrames.jl 1.0.1.

Deprecations

indicator keyword argument in joins

We have a new functionality of vcat in DataFrames.jl 1.0.
Start with an example:

julia> using DataFrames

julia> df1 = DataFrame(a=1:3)
3×1 DataFrame
 Row │ a
     │ Int64
─────┼───────
   1 │     1
   2 │     2
   3 │     3

julia> df2 = DataFrame(a=4:6)
3×1 DataFrame
 Row │ a
     │ Int64
─────┼───────
   1 │     4
   2 │     5
   3 │     6

julia> vcat(df1, df2, source=:source)
6×2 DataFrame
 Row │ a      source
     │ Int64  Int64
─────┼───────────────
   1 │     1       1
   2 │     2       1
   3 │     3       1
   4 │     4       2
   5 │     5       2
   6 │     6       2

julia> vcat(df1, df2, source=:source => ["df1", "df2"])
6×2 DataFrame
 Row │ a      source
     │ Int64  String
─────┼───────────────
   1 │     1  df1
   2 │     2  df1
   3 │     3  df1
   4 │     4  df2
   5 │     5  df2
   6 │     6  df2

As you can see you can pass source keyword argument to get an identifier of
source data frame for every row in the resulting data frame.

We have a similar functionality for joins. Here is an example:

julia> outerjoin(df1, df2, on=:a, source=:source)
6×2 DataFrame
 Row │ a      source
     │ Int64  String
─────┼───────────────────
   1 │     1  left_only
   2 │     2  left_only
   3 │     3  left_only
   4 │     4  right_only
   5 │     5  right_only
   6 │     6  right_only

As you can see the keyword argument used here is source. In the past this
keyword argument was called indicator, which was not very discoverable and now
it would be not consistent wthi vcat either. Therefore indicator keyword
argument is deprecated in favor of source. It will be removed in 2.0 release,
as keeping both keyword arguments is mostly harmless (so you have a lot of time
to clean up your codes).

Broadcasting assignment behavior

This is a super tricky deprecation as it is currently not printed
(because it cannot be under Julia 1.6).

Here is an example of deprecated functionality:

~$ julia --depwarn=yes
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0 (2021-03-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using DataFrames

julia> df = DataFrame(a=1:3)
3×1 DataFrame
 Row │ a
     │ Int64
─────┼───────
   1 │     1
   2 │     2
   3 │     3

julia> df.a .= 'x'
3-element Vector{Int64}:
 120
 120
 120

julia> df
3×1 DataFrame
 Row │ a
     │ Int64
─────┼───────
   1 │   120
   2 │   120
   3 │   120

In short, as you can see, df.col = value syntax works in-place under Julia 1.6.
In this post I have discussed in detail why we decided to change it.
But one of the major reasons is to allow for:

julia> df.b .= 1
ERROR: ArgumentError: column name :b not found in the data frame; existing most similar names are: :a

Let us switch to Julia nightly for a second:

$ ./julia --depwarn=yes
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0-DEV.999 (2021-04-24)
 _/ |\__'_|_|_|\__'_|  |  Commit 1474566ffc* (0 days old master)
|__/                   |


julia> using DataFrames

julia> df = DataFrame(a=1:3)
3×1 DataFrame
 Row │ a
     │ Int64
─────┼───────
   1 │     1
   2 │     2
   3 │     3

julia> df.a .= 'x'
┌ Warning: In the future this operation will allocate a new column instead of performing an in-place assignment.
│   caller = top-level scope at REPL[3]:1
└ @ Core REPL[3]:1
3-element Vector{Int64}:
 120
 120
 120

julia> df.b .= 1
3-element Vector{Int64}:
 1
 1
 1

julia> df
3×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │   120      1
   2 │   120      1
   3 │   120      1

As you can see df.a .= 'x' is deprecated but df.b .= 1 is allowed.

The situation is very unfortunate, as we are unable to print deprecation
warnings under Julia 1.6. There are two deprecated cases:

  • df.a .= value into an existing column of a DataFrame currently is in-place,
    and will replace a column in the future;
  • dfv.a .= value will be disallowed if dfv is a SubDataFrame;

The good thing is that the first change (replace instead of in-place) will not
affect almost any user workflows (except for the case I have given above, when
by doing df.a .= 'x' you got 120 in the column; in the future you will get
'x', as most likely you wanted). The second change (error for SubDataFrame)
will be easy to fix, as it will throw an error.

In the future (when Julia 1.7 is commonly used), either in 1.x or 2.0 release
of DataFrames.jl these deprecations will be turned into the target functionality.

Performance

To show performance changes I decided to look for some examples of data.table
performance tests prepared by Jan Gorecki (Jan is a data.table guru and
data.table is a golden standard for in-memory data wrangling) that I have not
checked earlier when developing the 1.0 release (to have a fair comparison).

For this I have found this post that gave a link to these two Stack
Overflow questions answered by Jan using data.table: question 1 and
question 2 (one is for split-apply-combine, and the other is for joins).
I have changed the size of the data a bit to make a single run of the test
on data.table be around 1 second. Both for data.table and DataFrames.jl
I use 4 threads.

I decided to rewrite these two examples under DataFrames.jl and compare timings
of:

  • DataFrames.jl 1.0.1
  • data.table 1.14.0 (to see a competitive comparison)
  • DataFrames.jl 0.22.7 (to see the difference in the performance)

(we are on R 4.0.5 and back on Julia 1.6)

Below I share reproducible details but the short conclusion is:

  • we have improved;
  • we are competitive with data.table (at least on the tests that Jan Goercki
    used in the Stack Overflow examples).

(still we have a lot of work to do to improve performance post 1.0 release)

Split-apply-combine tests

We start with data.table timing:

> library(data.table)
data.table 1.14.0 using 4 threads (see ?getDTthreads).  Latest news: r-datatable.com
> library(microbenchmark)
> n = 5e7
> k = 5e5
> x = runif(n)
> grp = sample(k, n, TRUE)
> dt = setnames(setDT(list(x, grp)), c("x","grp"))
> microbenchmark(dt[, .(sum(x), .N), grp], times=10)
Unit: milliseconds
                     expr      min       lq     mean   median       uq      max
 dt[, .(sum(x), .N), grp] 928.9709 976.8374 1087.029 1095.204 1199.951 1220.086
 neval
    10
>

Now DataFrames.jl 1.0.1:

julia> using DataFrames

julia> using BenchmarkTools

julia> n = 50_000_000;

julia> k = 500_000;

julia> x = rand(n);

julia> grp = rand(1:k, n);

julia> df = DataFrame(x=x, grp=grp);

julia> @benchmark combine(groupby($df, :grp), :x => sum, nrow)
BenchmarkTools.Trial:
  memory estimate:  397.73 MiB
  allocs estimate:  634
  --------------
  minimum time:     593.295 ms (3.19% GC)
  median time:      609.667 ms (2.59% GC)
  mean time:        622.208 ms (2.04% GC)
  maximum time:     668.141 ms (6.77% GC)
  --------------
  samples:          9
  evals/sample:     1

Finally DataFrames.jl 0.22.7:

julia> using DataFrames

julia> using BenchmarkTools

julia> n = 50_000_000;

julia> k = 500_000;

julia> x = rand(n);

julia> grp = rand(1:k, n);

julia> df = DataFrame(x=x, grp=grp);

julia> @benchmark combine(groupby($df, :grp), :x => sum, nrow)
BenchmarkTools.Trial:
  memory estimate:  1.26 GiB
  allocs estimate:  225
  --------------
  minimum time:     7.936 s (0.00% GC)
  median time:      7.936 s (0.00% GC)
  mean time:        7.936 s (0.00% GC)
  maximum time:     7.936 s (0.00% GC)
  --------------
  samples:          1
  evals/sample:     1

Join tests

First goes data.table:

> library(microbenchmark)
> library(data.table)
data.table 1.14.0 using 4 threads (see ?getDTthreads).  Latest news: r-datatable.com
> n = 5e6
> df1 = data.frame(x=sample(n,n), y1=rnorm(n))
> df2 = data.frame(x=sample(n,n), y2=rnorm(n))
> dt1 = as.data.table(df1)
> dt2 = as.data.table(df2)
> microbenchmark(dt1[dt2, nomatch=NULL, on = "x"], times=10)
Unit: milliseconds
                               expr      min       lq    mean   median       uq
 dt1[dt2, nomatch = NULL, on = "x"] 893.9817 905.4534 924.142 914.2142 940.4294
      max neval
 993.6209    10
> microbenchmark(dt2[dt1, on = "x"], times=10)
Unit: milliseconds
               expr      min      lq     mean   median      uq      max neval
 dt2[dt1, on = "x"] 845.7094 884.172 885.5079 889.8894 891.272 903.6629    10
> microbenchmark(dt1[dt2, on = "x"], times=10)
Unit: milliseconds
               expr      min      lq     mean   median       uq      max neval
 dt1[dt2, on = "x"] 848.3348 874.032 878.4387 880.2858 885.1598 894.9579    10
> microbenchmark(merge(dt1, dt2, by = "x", all = TRUE), times=10)
Unit: seconds
                                  expr      min       lq     mean   median
 merge(dt1, dt2, by = "x", all = TRUE) 1.924328 1.931986 1.957051 1.954956
       uq      max neval
 1.981632 2.002862    10

Now DataFrames.jl 1.0.1:

julia> using DataFrames, Random

julia> using BenchmarkTools

julia> n = 5_000_000;

julia> df1 = DataFrame(x=shuffle(1:n), y1=randn(n));

julia> df2 = DataFrame(x=shuffle(1:n), y2=randn(n));

julia> @benchmark innerjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  228.90 MiB
  allocs estimate:  248
  --------------
  minimum time:     957.133 ms (0.00% GC)
  median time:      974.750 ms (0.13% GC)
  mean time:        975.144 ms (0.65% GC)
  maximum time:     992.020 ms (2.63% GC)
  --------------
  samples:          6
  evals/sample:     1

julia> @benchmark leftjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  234.26 MiB
  allocs estimate:  312
  --------------
  minimum time:     1.061 s (0.00% GC)
  median time:      1.071 s (0.00% GC)
  mean time:        1.073 s (0.41% GC)
  maximum time:     1.084 s (0.00% GC)
  --------------
  samples:          5
  evals/sample:     1

julia> @benchmark rightjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  234.26 MiB
  allocs estimate:  312
  --------------
  minimum time:     980.800 ms (0.00% GC)
  median time:      1.003 s (0.36% GC)
  mean time:        1.001 s (0.80% GC)
  maximum time:     1.013 s (2.63% GC)
  --------------
  samples:          6
  evals/sample:     1

julia> @benchmark outerjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  239.63 MiB
  allocs estimate:  332
  --------------
  minimum time:     1.081 s (0.00% GC)
  median time:      1.096 s (0.00% GC)
  mean time:        1.097 s (0.41% GC)
  maximum time:     1.106 s (0.66% GC)
  --------------
  samples:          5
  evals/sample:     1

Finally DataFrames.jl 0.22.7:

julia> using DataFrames, Random

julia> using BenchmarkTools

julia> n = 5_000_000;

julia> df1 = DataFrame(x=shuffle(1:n), y1=randn(n));

julia> df2 = DataFrame(x=shuffle(1:n), y2=randn(n));

julia> @benchmark innerjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  674.36 MiB
  allocs estimate:  218
  --------------
  minimum time:     4.788 s (0.55% GC)
  median time:      4.795 s (0.44% GC)
  mean time:        4.795 s (0.44% GC)
  maximum time:     4.803 s (0.32% GC)
  --------------
  samples:          2
  evals/sample:     1

julia> @benchmark leftjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  755.43 MiB
  allocs estimate:  223
  --------------
  minimum time:     4.975 s (0.36% GC)
  median time:      4.976 s (0.27% GC)
  mean time:        4.976 s (0.27% GC)
  maximum time:     4.978 s (0.18% GC)
  --------------
  samples:          2
  evals/sample:     1

julia> @benchmark rightjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  760.20 MiB
  allocs estimate:  237
  --------------
  minimum time:     5.077 s (0.39% GC)
  median time:      5.077 s (0.39% GC)
  mean time:        5.077 s (0.39% GC)
  maximum time:     5.077 s (0.39% GC)
  --------------
  samples:          1
  evals/sample:     1

julia> @benchmark outerjoin($df1, $df2, on=:x)
BenchmarkTools.Trial:
  memory estimate:  769.73 MiB
  allocs estimate:  228
  --------------
  minimum time:     5.148 s (0.18% GC)
  median time:      5.148 s (0.18% GC)
  mean time:        5.148 s (0.18% GC)
  maximum time:     5.148 s (0.18% GC)
  --------------
  samples:          1
  evals/sample:     1

Conclusions

In summary let me highlight some of the development objectives after 1.0 release:

  • API improvements (e.g. better stack/unstack functionality, having in-place
    joins, extensions of transformation minilanguage).
  • Review of the whole package for performance bottlenecks and using
    multi-threading in more places.
  • Resolving ecosystem integration performance bottlenecks (especially against
    CSV.jl and Arrow.jl). Here one big challenge is thinking of something that
    would reduce GC strain of having millions of strings stored in the DataFrame
    (which is a typical situation in many data science workflows).
  • Documentation improvements.

Happy data wrangling with DataFrames.jl 1.0!

Getting Started with Julia Lang

By: Fabian Becker

Re-posted from: https://geekmonkey.org/getting-started-with-julia-lang/

Getting Started with Julia Lang

During my early studies and later in my career I had the opportunity to hone my engineering skills in a variety of different programming languages. I started in my early teenage years with QBasic and then progressed on to more advanced languages.

In open source projects I've largely worked on JavaScript (node.js) and PHP codebases, while my professional career has taken me to Java, Ruby (on Rails) and finally Python. Little bits of exposure left and right to newer languages like Go and Rust, but the above are the main languages I've done actual work in.

Over the years I have worked on a number of optimization problems that required the use of machine learning algorithms and libraries or the plotting of graphs. The go-to language for this has naturally been Python with its large collection of data science packages of the like of numpy, scipy, matplotlib and not to mention pytorch and tensorflow.

Now there's nothing wrong with any of the above. Python is a solid language with a long history and its grown to become the language of choice for data science. For me though, Python never felt quite right. The Zen of Python makes you think the language is opinionated and has clear structure to it. Oftentimes I found the opposite to be true. Where in Ruby (on Rails) "convention over configuration" rules the thinking Python leaves most up to the developer.

Popular web frameworks like Django and Flask come with short examples on how to build an application but leave a lot unanswered when your application starts growing. Where do routes live? Where do I put my models? As a result every project will look differently making it inherently difficult to jump into new codebases.

In a similar vein Python has the basic building blocks for OOP and functional programming, but stops there. Private variables don't really exist, static methods and class methods require the use of decorators, lambda is but a crippled version of implementations in other languages like Ruby and JavaScript, map is a built-in function while reduce was demoted to the functools package. Even though the language technically supports these paradigms you will have to look hard to find them embraced in the popular packages.

Now you could argue that I'm comparing apples with oranges and to some extend I am. After all I'm comparing frameworks and not languages. To me though both go hand in hand. You can't have a language without the packages that make up said language. Just like a human language is influenced and evolves through its speakers, a programming language and the culture around it is defined by its community and ecosystem.

To be clear though – preferences differ and I'm not arguing that Python is a bad language, quite the contrary, it has built up a large following due to it being easy to learn and having a large ecosystem. It's just not a language that satisfies me because I care about structure and ultimately that's what made me look elsewhere.

Meet Julia

Julia goes back to 2009 and was started to create a fast high-level language. It wasn't until 2012 for Julia to be first announced and publicly released and version 1.0 landed 2018.

Julia has a dynamic type system, built-in package manager, multiple dispatch, performance approaching that of C and an ability to natively interface with other languages like Python. It's designed for distributed computing and makes it trivial to run algorithms on GPUs.

The ability to natively and easily interface with other languages sticks out to me. Take a look at the following example which uses PyCall to call some Python code from within Julia:

using PyCall
math = pyimport("math")
math.sin(math.pi / 4) # returns ≈ 1/√2 = 0.70710678...
Import Python's math module in Julia (Source)

The beauty is that you can very easily mix Python and Julia this way. Python's entire ecosystem including all its great libraries are available while you can also explore Julia's ecosystem. In fact Plots.jl, a plotting library for Julia can use PyPlot as one of its backends.

A neat feature of Julia is that you can use LaTeX shortcuts in the editor. For example when you type \sqrt<TAB> most Julia editors and the Julia REPL will automatically turn this sequence into the square-root sign: √. In this case √ is also aliased to the sqrt function.  This makes Julia great for mathematical applications where you can use mathematical notation in your programs.

Getting Started with Julia Lang

Julia lends itself to scientific applications and machine learning. Tools like Flux.jl

While it's true that Julia is heavily used in research it doesn't mean that you can't also build run-of-the-mill web applications. Frameworks like Genie

Installing Julia

The easiest way to install Julia is to download the compatible package for your OS on the official website. Personally I tend to use asdf-vm as it allows me to install and run multiple versions of my favourite languages simultaneously.

With asdf-vm installing Julia becomes as easy as:

$ asdf plugin add julia
$ asdf install julia 1.6.0

REPL

Julia's REPL is different from what you may be used to from other languages. It comes bundled with a package manager and several different modes.

The main shortcuts that you'll want to remember are:

  • ] gets you into package mode, from here you can add orupdate packages and manage projects
  • ? gets you into help mode. Julia is a well documented language and the help mode will allow you to find explanations and examples for all the built-ins
  • ; is how you can enter shell mode allowing you to use a bash-like shell without leaving the Julia interpreter

Installing packages

Installing packages is straight forward. Press ] to go into package mode and then add $PackageName to install a package named $PackageName. Note that this will install packages in the globally.

A package you'll definitely want to install globally is called Revise.jl.

Revise.jl

Revise allows you to hot reload code while working in the REPL. This is especially useful in Julia since the REPL takes a little longer to start than in other languages. With Revise you can edit code in your IDE and have it reflected in the REPL immediately.

Getting Started with Julia Lang

Editors

The two main editors for Julia are Juno and Julia for VSCode. Julia for VSCode is relatively young and was introduced at the last JuliaCon in 2020. Juno is based on Atom which has seen some decline in usage over time.

Introduction to VS Code for Julia at JuliaCon 2020

Since I'm already using VSCode for all my other development using the Julia for VSCode extension was an easy decision. It has all the features present in Juno.

Conclusion

Julia is a fascinating language with a growing community and a solid package ecosystem. While the ecosystem isn't as vast as Python's, it is trivial to call out to Python, R, C and other languages.

Julia's package manager is fast, easy to use. It's a breath of fresh air when compared to the shenanigans one has to deal with in Python with pip, setuptools, conda, poetry, et.al.

The lack of object-oriented programming features is hardly noticeable once you are comfortable with multiple dispatch. In fact Julia's simplicity and well thought out type system make it easy to reason about the objects you're dealing with in code. On top of that, I find Julia very easy to learn and well positioned to attract developers, data scientists and researchers who traditionally picked Python.

Getting Started with Julia Lang

By: Fabian Becker

Re-posted from: https://geekmonkey.org/getting-started-with-julia-lang/

Getting Started with Julia Lang

During my early studies and later in my career I had the opportunity to hone my engineering skills in a variety of different programming languages. I started in my early teenage years with QBasic and then progressed on to more advanced languages.

In open source projects I've largely worked on JavaScript (node.js) and PHP codebases, while my professional career has taken me to Java, Ruby (on Rails) and finally Python. Little bits of exposure left and right to newer languages like Go and Rust, but the above are the main languages I've done actual work in.

Over the years I have worked on a number of optimization problems that required the use of machine learning algorithms and libraries or the plotting of graphs. The go-to language for this has naturally been Python with its large collection of data science packages of the like of numpy, scipy, matplotlib and not to mention pytorch and tensorflow.

Now there's nothing wrong with any of the above. Python is a solid language with a long history and its grown to become the language of choice for data science. For me though, Python never felt quite right. The Zen of Python makes you think the language is opinionated and has clear structure to it. Oftentimes I found the opposite to be true. Where in Ruby (on Rails) "convention over configuration" rules the thinking Python leaves most up to the developer.

Popular web frameworks like Django and Flask come with short examples on how to build an application but leave a lot unanswered when your application starts growing. Where do routes live? Where do I put my models? As a result every project will look differently making it inherently difficult to jump into new codebases.

In a similar vein Python has the basic building blocks for OOP and functional programming, but stops there. Private variables don't really exist, static methods and class methods require the use of decorators, lambda is but a crippled version of implementations in other languages like Ruby and JavaScript, map is a built-in function while reduce was demoted to the functools package. Even though the language technically supports these paradigms you will have to look hard to find them embraced in the popular packages.

Now you could argue that I'm comparing apples with oranges and to some extend I am. After all I'm comparing frameworks and not languages. To me though both go hand in hand. You can't have a language without the packages that make up said language. Just like a human language is influenced and evolves through its speakers, a programming language and the culture around it is defined by its community and ecosystem.

To be clear though – preferences differ and I'm not arguing that Python is a bad language, quite the contrary, it has built up a large following due to it being easy to learn and having a large ecosystem. It's just not a language that satisfies me because I care about structure and ultimately that's what made me look elsewhere.

Meet Julia

Julia goes back to 2009 and was started to create a fast high-level language. It wasn't until 2012 for Julia to be first announced and publicly released and version 1.0 landed 2018.

Julia has a dynamic type system, built-in package manager, multiple dispatch, performance approaching that of C and an ability to natively interface with other languages like Python. It's designed for distributed computing and makes it trivial to run algorithms on GPUs.

The ability to natively and easily interface with other languages sticks out to me. Take a look at the following example which uses PyCall to call some Python code from within Julia:

using PyCall
math = pyimport("math")
math.sin(math.pi / 4) # returns ≈ 1/√2 = 0.70710678...
Import Python's math module in Julia (Source)

The beauty is that you can very easily mix Python and Julia this way. Python's entire ecosystem including all its great libraries are available while you can also explore Julia's ecosystem. In fact Plots.jl, a plotting library for Julia can use PyPlot as one of its backends.

A neat feature of Julia is that you can use LaTeX shortcuts in the editor. For example when you type \sqrt<TAB> most Julia editors and the Julia REPL will automatically turn this sequence into the square-root sign: √. In this case √ is also aliased to the sqrt function.  This makes Julia great for mathematical applications where you can use mathematical notation in your programs.

Getting Started with Julia Lang

Julia lends itself to scientific applications and machine learning. Tools like Flux.jl

While it's true that Julia is heavily used in research it doesn't mean that you can't also build run-of-the-mill web applications. Frameworks like Genie

Installing Julia

The easiest way to install Julia is to download the compatible package for your OS on the official website. Personally I tend to use asdf-vm as it allows me to install and run multiple versions of my favourite languages simultaneously.

With asdf-vm installing Julia becomes as easy as:

$ asdf plugin add julia
$ asdf install julia 1.6.0

REPL

Julia's REPL is different from what you may be used to from other languages. It comes bundled with a package manager and several different modes.

The main shortcuts that you'll want to remember are:

  • ] gets you into package mode, from here you can add orupdate packages and manage projects
  • ? gets you into help mode. Julia is a well documented language and the help mode will allow you to find explanations and examples for all the built-ins
  • ; is how you can enter shell mode allowing you to use a bash-like shell without leaving the Julia interpreter

Installing packages

Installing packages is straight forward. Press ] to go into package mode and then add $PackageName to install a package named $PackageName. Note that this will install packages in the globally.

A package you'll definitely want to install globally is called Revise.jl.

Revise.jl

Revise allows you to hot reload code while working in the REPL. This is especially useful in Julia since the REPL takes a little longer to start than in other languages. With Revise you can edit code in your IDE and have it reflected in the REPL immediately.

Getting Started with Julia Lang

Editors

The two main editors for Julia are Juno and Julia for VSCode. Julia for VSCode is relatively young and was introduced at the last JuliaCon in 2020. Juno is based on Atom which has seen some decline in usage over time.

Introduction to VS Code for Julia at JuliaCon 2020

Since I'm already using VSCode for all my other development using the Julia for VSCode extension was an easy decision. It has all the features present in Juno.

Conclusion

Julia is a fascinating language with a growing community and a solid package ecosystem. While the ecosystem isn't as vast as Python's, it is trivial to call out to Python, R, C and other languages.

Julia's package manager is fast, easy to use. It's a breath of fresh air when compared to the shenanigans one has to deal with in Python with pip, setuptools, conda, poetry, et.al.

The lack of object-oriented programming features is hardly noticeable once you are comfortable with multiple dispatch. In fact Julia's simplicity and well thought out type system make it easy to reason about the objects you're dealing with in code. On top of that, I find Julia very easy to learn and well positioned to attract developers, data scientists and researchers who traditionally picked Python.