Using Julia for Data Science

By: Posts on Cleyton Farias

Re-posted from: https://cleytonfar.github.io/posts/using-julia-for-data-science/

In recent years, data science has become a huge attractive field with the
profession of data scientist topping the list of the best jobs in America.
And with all the hype that the field produces, one might ask: what does it take do
be a data scientist?

Well… that’s a good question. First of all, there are a lot of requirements. But one
of the most important ones is to learn how to work with data sets. And I am not
talking about playing with spreadsheets. I am talking about working with some
real programming language to get the job done with any datasets, no matter how
huge it is.

Hence, this post is the first part of a series about working with tabular data
with the Julia programming language. Of course, the aim of this post is not only
to give you a quick introduction to the language, but also present to you how
you can easily install and work rightaway with datasets with Julia.

Why Julia?

Glad you asked! Julia is a high level programming language released in 2012 by a
team of MIT researchers. Since its beginning, the aim was to solve the so called
two-language programing problem: easy to use functionalities of interpretable languages
(Python, R, Matlab) vs high performance of compiled languages (C, C++, Fortran).
According to its creators:

We want a language that’s open source, with a liberal license.
We want the speed of C with the dynamism of Ruby.
We want a language that’s homoiconic, with true macros like Lisp, but with obvious,
familiar mathematical notation like Matlab. We want something as usable for
general programming as Python, as easy for statistics as R, as natural for string
processing as Perl, as powerful for linear algebra as Matlab, as good at gluing
programs together as the shell.
Something that is dirt simple to learn, yet keeps the most serious hackers happy.
We want it interactive and we want it compiled. — julialang.org

Hence, Julia was born. Combining the JIT (Just In Time) compiler and
Julia’s multiple-dispatch system plus the fact that its codebase is written
entirely in native language, Julia gives birth to the popular phrase in the
community:

“Walks like Python, runs like C.”

Installing Julia

To play around with Julia there are some options. One obvious way is to download
the official binaries from the site for your
specific plataform (Windows, macOS, Linux, etc). At the
time of this writting, the Current stable release is v1.1.0 and the
Long-term support release is v1.0.3. Once you downloaded and execute
the binaries, you will see the following window:



Another options is to use Julia in the browser on JuliaBox.com
with Jupyter notebooks. No installation is required – just point your browser
there, login and start playing around.

Installing Packages

All the package management in Julia is performed by the Pkg package. To
install a given package we use Pkg.add("package_name"). In this tutorial we
are going to use some packages that are not pre-installed with Julia. To install
them, do the following:

using Pkg
Pkg.add("DataFrames")
Pkg.add("DataFramesMeta")
Pkg.add("CSV")

We installed three packages: DataFrames (which is the subject of this post),
DataFramesMeta (we will use some of its functionalities) and CSV (to read
and write CSV files).

Of course there is more about package management in Julia than I just showed.
A great introduction is presented in this video by Jane
Harriman. For more advanced usage, please refer to the official documentation.

Introduction to DataFrames in Julia

In Julia, tablular data is handled using the DataFrames package. Other packages
are commonly used to read/write data into/from Julia such as CSV.

A data frame is created using the DataFrame() function:

using DataFrames 
foo = DataFrame();
foo 
## 0×0 DataFrame

To use the functionalities of the package, let’s create some random data. I will
use the rand() function to generate random numbers to create an array 100 x 10
and convert it to a data frame:

foo = DataFrame(rand(100, 10));
foo 
## 100×10 DataFrame. Omitted printing of 4 columns
## │ Row │ x1         │ x2       │ x3        │ x4       │ x5        │ x6       │
## │     │ Float64    │ Float64  │ Float64   │ Float64  │ Float64   │ Float64  │
## ├─────┼────────────┼──────────┼───────────┼──────────┼───────────┼──────────┤
## │ 1   │ 0.0193136  │ 0.466228 │ 0.790475  │ 0.805074 │ 0.51182   │ 0.201707 │
## │ 2   │ 0.986082   │ 0.33719  │ 0.309992  │ 0.117098 │ 0.792606  │ 0.102682 │
## │ 3   │ 0.00366356 │ 0.323071 │ 0.685271  │ 0.596414 │ 0.847368  │ 0.105035 │
## │ 4   │ 0.297846   │ 0.136907 │ 0.726739  │ 0.569452 │ 0.922995  │ 0.846519 │
## │ 5   │ 0.73245    │ 0.208294 │ 0.353801  │ 0.448741 │ 0.185897  │ 0.496741 │
## │ 6   │ 0.209719   │ 0.114021 │ 0.0662264 │ 0.463682 │ 0.628582  │ 0.130653 │
## │ 7   │ 0.341692   │ 0.608349 │ 0.946541  │ 0.589161 │ 0.418321  │ 0.295541 │
## ⋮
## │ 93  │ 0.714495   │ 0.661317 │ 0.954527  │ 0.209581 │ 0.107941  │ 0.233787 │
## │ 94  │ 0.680497   │ 0.101874 │ 0.872371  │ 0.596457 │ 0.669133  │ 0.740674 │
## │ 95  │ 0.909319   │ 0.182776 │ 0.343387  │ 0.142707 │ 0.0140866 │ 0.791679 │
## │ 96  │ 0.642578   │ 0.949993 │ 0.380511  │ 0.96358  │ 0.878766  │ 0.270409 │
## │ 97  │ 0.605148   │ 0.240233 │ 0.144059  │ 0.545245 │ 0.0463105 │ 0.188397 │
## │ 98  │ 0.0907523  │ 0.334278 │ 0.288403  │ 0.519876 │ 0.267965  │ 0.552448 │
## │ 99  │ 0.751681   │ 0.289301 │ 0.488135  │ 0.382877 │ 0.320208  │ 0.999445 │
## │ 100 │ 0.856248   │ 0.577105 │ 0.588476  │ 0.435958 │ 0.0163749 │ 0.337817 │

Maybe you have noticed the “;” at the end of a command. It turns out that in Julia,
contrary to many other languages, everything is an expression, so it will return
a result. Hence, to turn off this return, we must include the “;” at the end of
each command.

To get the dimension of a data frame, we can use the size() function. Also,
similarly to R programming language, nrow() and ncol() are available to
get the number of rows and columns, respectively:

size(foo)
## (100, 10)
nrow(foo)
## 100
ncol(foo)
## 10

Another basic task when working with datasets is to to get the names of each
variable contained in the table. We use the names() function to get the column
names:

names(foo)
## 10-element Array{Symbol,1}:
##  :x1 
##  :x2 
##  :x3 
##  :x4 
##  :x5 
##  :x6 
##  :x7 
##  :x8 
##  :x9 
##  :x10

To get a summary of the dataset in general, we can use the function describe():

describe(foo)
## 10×8 DataFrame. Omitted printing of 2 columns
## │ Row │ variable │ mean     │ min        │ median   │ max      │ nunique │
## │     │ Symbol   │ Float64  │ Float64    │ Float64  │ Float64  │ Nothing │
## ├─────┼──────────┼──────────┼────────────┼──────────┼──────────┼─────────┤
## │ 1   │ x1       │ 0.502457 │ 0.00190391 │ 0.508102 │ 0.993014 │         │
## │ 2   │ x2       │ 0.461593 │ 0.0143797  │ 0.465052 │ 0.949993 │         │
## │ 3   │ x3       │ 0.4659   │ 0.0180212  │ 0.409124 │ 0.978917 │         │
## │ 4   │ x4       │ 0.503142 │ 0.0130052  │ 0.508707 │ 0.986293 │         │
## │ 5   │ x5       │ 0.518394 │ 0.00177395 │ 0.502389 │ 0.994104 │         │
## │ 6   │ x6       │ 0.486075 │ 0.00543681 │ 0.475648 │ 0.999445 │         │
## │ 7   │ x7       │ 0.490961 │ 0.00366989 │ 0.482302 │ 0.996092 │         │
## │ 8   │ x8       │ 0.503405 │ 0.0180501  │ 0.525201 │ 0.985918 │         │
## │ 9   │ x9       │ 0.507343 │ 0.0327247  │ 0.533176 │ 0.990731 │         │
## │ 10  │ x10      │ 0.468541 │ 0.00622055 │ 0.470003 │ 0.996703 │         │

Note that there is a message indicating the omission of some columns. This is the
default behavior of Julia. To avoid this feature, we use the show() function
as follows:

show(describe(foo), allcols = true)

Manipulating Rows:

Subset rows in Julia can be a little odd in the beginning, but once you get used to, it becomes
more logical. For example, suppose we want the rows where x1 is above its average.
We could this as follows:

## Loading the Statistics package:
using Statistics
## Creating the conditional:
cond01 = foo[:x1] .>= mean(foo[:x1]);
## Subsetting the rows:
foo[cond01, :] 
## 51×10 DataFrame. Omitted printing of 4 columns
## │ Row │ x1       │ x2       │ x3        │ x4       │ x5        │ x6        │
## │     │ Float64  │ Float64  │ Float64   │ Float64  │ Float64   │ Float64   │
## ├─────┼──────────┼──────────┼───────────┼──────────┼───────────┼───────────┤
## │ 1   │ 0.986082 │ 0.33719  │ 0.309992  │ 0.117098 │ 0.792606  │ 0.102682  │
## │ 2   │ 0.73245  │ 0.208294 │ 0.353801  │ 0.448741 │ 0.185897  │ 0.496741  │
## │ 3   │ 0.716057 │ 0.325789 │ 0.193415  │ 0.813209 │ 0.232703  │ 0.314502  │
## │ 4   │ 0.538082 │ 0.932279 │ 0.101212  │ 0.363205 │ 0.979265  │ 0.274936  │
## │ 5   │ 0.693567 │ 0.78976  │ 0.123106  │ 0.566847 │ 0.492958  │ 0.798202  │
## │ 6   │ 0.794447 │ 0.405418 │ 0.0521367 │ 0.587886 │ 0.922298  │ 0.211156  │
## │ 7   │ 0.664186 │ 0.432662 │ 0.0431839 │ 0.810072 │ 0.963643  │ 0.678182  │
## ⋮
## │ 44  │ 0.85821  │ 0.484308 │ 0.899559  │ 0.754818 │ 0.252699  │ 0.0590497 │
## │ 45  │ 0.714495 │ 0.661317 │ 0.954527  │ 0.209581 │ 0.107941  │ 0.233787  │
## │ 46  │ 0.680497 │ 0.101874 │ 0.872371  │ 0.596457 │ 0.669133  │ 0.740674  │
## │ 47  │ 0.909319 │ 0.182776 │ 0.343387  │ 0.142707 │ 0.0140866 │ 0.791679  │
## │ 48  │ 0.642578 │ 0.949993 │ 0.380511  │ 0.96358  │ 0.878766  │ 0.270409  │
## │ 49  │ 0.605148 │ 0.240233 │ 0.144059  │ 0.545245 │ 0.0463105 │ 0.188397  │
## │ 50  │ 0.751681 │ 0.289301 │ 0.488135  │ 0.382877 │ 0.320208  │ 0.999445  │
## │ 51  │ 0.856248 │ 0.577105 │ 0.588476  │ 0.435958 │ 0.0163749 │ 0.337817  │

What if we want two conditionals? For example, we want the same condition as before
and/or the rows where x2 is greater than or equal its average? Now things
become trickier. Let’s check how we could do this:

## Creating the second conditional:
cond02 = foo[:x2] .>= mean(foo[:x2]);
## Subsetting cond01 AND cond02:
foo[.&(cond01, cond02), :]
## 25×10 DataFrame. Omitted printing of 4 columns
## │ Row │ x1       │ x2       │ x3        │ x4        │ x5        │ x6        │
## │     │ Float64  │ Float64  │ Float64   │ Float64   │ Float64   │ Float64   │
## ├─────┼──────────┼──────────┼───────────┼───────────┼───────────┼───────────┤
## │ 1   │ 0.538082 │ 0.932279 │ 0.101212  │ 0.363205  │ 0.979265  │ 0.274936  │
## │ 2   │ 0.693567 │ 0.78976  │ 0.123106  │ 0.566847  │ 0.492958  │ 0.798202  │
## │ 3   │ 0.567098 │ 0.747233 │ 0.589314  │ 0.0677154 │ 0.630238  │ 0.357654  │
## │ 4   │ 0.976991 │ 0.648552 │ 0.32794   │ 0.36951   │ 0.846276  │ 0.117798  │
## │ 5   │ 0.553247 │ 0.615375 │ 0.122955  │ 0.440636  │ 0.283713  │ 0.734161  │
## │ 6   │ 0.849795 │ 0.703195 │ 0.232944  │ 0.668432  │ 0.686921  │ 0.788872  │
## │ 7   │ 0.530801 │ 0.825475 │ 0.644381  │ 0.15488   │ 0.669306  │ 0.151317  │
## ⋮
## │ 18  │ 0.665926 │ 0.943121 │ 0.438038  │ 0.921251  │ 0.82234   │ 0.761529  │
## │ 19  │ 0.987506 │ 0.946972 │ 0.0462434 │ 0.67867   │ 0.731762  │ 0.482322  │
## │ 20  │ 0.862284 │ 0.886346 │ 0.694874  │ 0.0166389 │ 0.386215  │ 0.527352  │
## │ 21  │ 0.855198 │ 0.650342 │ 0.0321678 │ 0.723076  │ 0.449779  │ 0.0364525 │
## │ 22  │ 0.85821  │ 0.484308 │ 0.899559  │ 0.754818  │ 0.252699  │ 0.0590497 │
## │ 23  │ 0.714495 │ 0.661317 │ 0.954527  │ 0.209581  │ 0.107941  │ 0.233787  │
## │ 24  │ 0.642578 │ 0.949993 │ 0.380511  │ 0.96358   │ 0.878766  │ 0.270409  │
## │ 25  │ 0.856248 │ 0.577105 │ 0.588476  │ 0.435958  │ 0.0163749 │ 0.337817  │
## Subsetting cond01 OR cond02:
foo[.|(cond01, cond02), :]
## 77×10 DataFrame. Omitted printing of 4 columns
## │ Row │ x1        │ x2       │ x3        │ x4       │ x5        │ x6        │
## │     │ Float64   │ Float64  │ Float64   │ Float64  │ Float64   │ Float64   │
## ├─────┼───────────┼──────────┼───────────┼──────────┼───────────┼───────────┤
## │ 1   │ 0.0193136 │ 0.466228 │ 0.790475  │ 0.805074 │ 0.51182   │ 0.201707  │
## │ 2   │ 0.986082  │ 0.33719  │ 0.309992  │ 0.117098 │ 0.792606  │ 0.102682  │
## │ 3   │ 0.73245   │ 0.208294 │ 0.353801  │ 0.448741 │ 0.185897  │ 0.496741  │
## │ 4   │ 0.341692  │ 0.608349 │ 0.946541  │ 0.589161 │ 0.418321  │ 0.295541  │
## │ 5   │ 0.413762  │ 0.644062 │ 0.495503  │ 0.96149  │ 0.249137  │ 0.592854  │
## │ 6   │ 0.129374  │ 0.663032 │ 0.0180212 │ 0.280431 │ 0.887136  │ 0.329406  │
## │ 7   │ 0.716057  │ 0.325789 │ 0.193415  │ 0.813209 │ 0.232703  │ 0.314502  │
## ⋮
## │ 70  │ 0.85821   │ 0.484308 │ 0.899559  │ 0.754818 │ 0.252699  │ 0.0590497 │
## │ 71  │ 0.714495  │ 0.661317 │ 0.954527  │ 0.209581 │ 0.107941  │ 0.233787  │
## │ 72  │ 0.680497  │ 0.101874 │ 0.872371  │ 0.596457 │ 0.669133  │ 0.740674  │
## │ 73  │ 0.909319  │ 0.182776 │ 0.343387  │ 0.142707 │ 0.0140866 │ 0.791679  │
## │ 74  │ 0.642578  │ 0.949993 │ 0.380511  │ 0.96358  │ 0.878766  │ 0.270409  │
## │ 75  │ 0.605148  │ 0.240233 │ 0.144059  │ 0.545245 │ 0.0463105 │ 0.188397  │
## │ 76  │ 0.751681  │ 0.289301 │ 0.488135  │ 0.382877 │ 0.320208  │ 0.999445  │
## │ 77  │ 0.856248  │ 0.577105 │ 0.588476  │ 0.435958 │ 0.0163749 │ 0.337817  │

In Julia, instead of the syntax condition1 & condition2, which is more common in
other programming languages, we use &(condition1, condition2) or
|(condition1, condition2) operators to perform multiple conditional
filtering.

Now, let’s say you have a DataFrame and you want to append rows to it.
There are a couple of ways of doing data. The first one is to use the [data1; data2]
syntax:

## Creating a DataFrame with 3 rows and 5 columns:
x = DataFrame(rand(3, 5));
## Let's add another line using [dataset1; dataset2] syntax:
[ x ; DataFrame(rand(1, 5)) ]
## 4×5 DataFrame
## │ Row │ x1       │ x2        │ x3        │ x4       │ x5        │
## │     │ Float64  │ Float64   │ Float64   │ Float64  │ Float64   │
## ├─────┼──────────┼───────────┼───────────┼──────────┼───────────┤
## │ 1   │ 0.722487 │ 0.0930212 │ 0.146     │ 0.486439 │ 0.0892853 │
## │ 2   │ 0.640469 │ 0.5902    │ 0.667832  │ 0.882527 │ 0.766987  │
## │ 3   │ 0.094589 │ 0.805257  │ 0.291809  │ 0.582878 │ 0.704144  │
## │ 4   │ 0.18066  │ 0.187027  │ 0.0440521 │ 0.077637 │ 0.884914  │

We could get the same result using the vcat() function. According to the
documentation, vcat() performs concatenation along dimension 1, which means
it will concatenate rows. The syntax would be:

## taking the first 2 lines and append with the third one:
vcat(x[1:2, :] , x[3, :])

Another way to do that is using the function append!(). This function will append
a new row to the last row in a given DataFrame. Note that the column names must
match exactly.

## Column names matches
append!(x, DataFrame(rand(1, 5)))
## 4×5 DataFrame
## │ Row │ x1       │ x2        │ x3        │ x4       │ x5        │
## │     │ Float64  │ Float64   │ Float64   │ Float64  │ Float64   │
## ├─────┼──────────┼───────────┼───────────┼──────────┼───────────┤
## │ 1   │ 0.722487 │ 0.0930212 │ 0.146     │ 0.486439 │ 0.0892853 │
## │ 2   │ 0.640469 │ 0.5902    │ 0.667832  │ 0.882527 │ 0.766987  │
## │ 3   │ 0.094589 │ 0.805257  │ 0.291809  │ 0.582878 │ 0.704144  │
## │ 4   │ 0.492341 │ 0.823765  │ 0.0731187 │ 0.123074 │ 0.264452  │

Note that if the column names between two DataFrames do not match , the append!()
function is going to throw an error. Although this kind of behavior is important
when we want to control for possible side effects, we might also prefer to not worry about
this and “force” the append procedure. In order to do this we can make use of
the push!() function.

## providing an Array:
push!(x, rand(ncol(x)))
## 5×5 DataFrame
## │ Row │ x1       │ x2        │ x3        │ x4       │ x5        │
## │     │ Float64  │ Float64   │ Float64   │ Float64  │ Float64   │
## ├─────┼──────────┼───────────┼───────────┼──────────┼───────────┤
## │ 1   │ 0.722487 │ 0.0930212 │ 0.146     │ 0.486439 │ 0.0892853 │
## │ 2   │ 0.640469 │ 0.5902    │ 0.667832  │ 0.882527 │ 0.766987  │
## │ 3   │ 0.094589 │ 0.805257  │ 0.291809  │ 0.582878 │ 0.704144  │
## │ 4   │ 0.492341 │ 0.823765  │ 0.0731187 │ 0.123074 │ 0.264452  │
## │ 5   │ 0.632829 │ 0.357564  │ 0.09631   │ 0.198201 │ 0.924137  │
## providing an dictionary:
push!(x, Dict(:x1 => rand(),
              :x2 => rand(),
              :x3 => rand(),
              :x4 => rand(),
              :x5 => rand()))
## 6×5 DataFrame
## │ Row │ x1       │ x2        │ x3        │ x4       │ x5        │
## │     │ Float64  │ Float64   │ Float64   │ Float64  │ Float64   │
## ├─────┼──────────┼───────────┼───────────┼──────────┼───────────┤
## │ 1   │ 0.722487 │ 0.0930212 │ 0.146     │ 0.486439 │ 0.0892853 │
## │ 2   │ 0.640469 │ 0.5902    │ 0.667832  │ 0.882527 │ 0.766987  │
## │ 3   │ 0.094589 │ 0.805257  │ 0.291809  │ 0.582878 │ 0.704144  │
## │ 4   │ 0.492341 │ 0.823765  │ 0.0731187 │ 0.123074 │ 0.264452  │
## │ 5   │ 0.632829 │ 0.357564  │ 0.09631   │ 0.198201 │ 0.924137  │
## │ 6   │ 0.234059 │ 0.530488  │ 0.0448796 │ 0.565734 │ 0.262909  │

As we can see, this function also accepts that we give a dictionary or an array
to append to a DataFrame.

So, there are at least 4 methods to add rows to a DataFrame. Which one to use?
Let’s see how fast it is each function:

using BenchmarkTools
@btime [x ; DataFrame(rand(1, 5))];
@btime vcat(x, DataFrame(rand(1, 5)));
@btime append!(x, DataFrame(rand(1, 5)));
@btime push!(x, rand(1, 5));

Manipulating Columns:

One of the first things we would want to do when working with a dataset is selecting
some columns. In Julia, the syntax of selecting columns in DataFrames is similar to the one
used in Matlab/Octave. For instance, we can make use of the “:” symbol to represent
that we want all columns (or all rows) and/or a sequence of them:

## Taking all rows of the first 2 columns:
foo[:, 1:2]
## 100×2 DataFrame
## │ Row │ x1         │ x2       │
## │     │ Float64    │ Float64  │
## ├─────┼────────────┼──────────┤
## │ 1   │ 0.0193136  │ 0.466228 │
## │ 2   │ 0.986082   │ 0.33719  │
## │ 3   │ 0.00366356 │ 0.323071 │
## │ 4   │ 0.297846   │ 0.136907 │
## │ 5   │ 0.73245    │ 0.208294 │
## │ 6   │ 0.209719   │ 0.114021 │
## │ 7   │ 0.341692   │ 0.608349 │
## ⋮
## │ 93  │ 0.714495   │ 0.661317 │
## │ 94  │ 0.680497   │ 0.101874 │
## │ 95  │ 0.909319   │ 0.182776 │
## │ 96  │ 0.642578   │ 0.949993 │
## │ 97  │ 0.605148   │ 0.240233 │
## │ 98  │ 0.0907523  │ 0.334278 │
## │ 99  │ 0.751681   │ 0.289301 │
## │ 100 │ 0.856248   │ 0.577105 │
## Taking the first 10 rows of all columns:
foo[1:10, :]
## 10×10 DataFrame. Omitted printing of 4 columns
## │ Row │ x1         │ x2       │ x3        │ x4       │ x5       │ x6       │
## │     │ Float64    │ Float64  │ Float64   │ Float64  │ Float64  │ Float64  │
## ├─────┼────────────┼──────────┼───────────┼──────────┼──────────┼──────────┤
## │ 1   │ 0.0193136  │ 0.466228 │ 0.790475  │ 0.805074 │ 0.51182  │ 0.201707 │
## │ 2   │ 0.986082   │ 0.33719  │ 0.309992  │ 0.117098 │ 0.792606 │ 0.102682 │
## │ 3   │ 0.00366356 │ 0.323071 │ 0.685271  │ 0.596414 │ 0.847368 │ 0.105035 │
## │ 4   │ 0.297846   │ 0.136907 │ 0.726739  │ 0.569452 │ 0.922995 │ 0.846519 │
## │ 5   │ 0.73245    │ 0.208294 │ 0.353801  │ 0.448741 │ 0.185897 │ 0.496741 │
## │ 6   │ 0.209719   │ 0.114021 │ 0.0662264 │ 0.463682 │ 0.628582 │ 0.130653 │
## │ 7   │ 0.341692   │ 0.608349 │ 0.946541  │ 0.589161 │ 0.418321 │ 0.295541 │
## │ 8   │ 0.308353   │ 0.428978 │ 0.914878  │ 0.84873  │ 0.440174 │ 0.310166 │
## │ 9   │ 0.413762   │ 0.644062 │ 0.495503  │ 0.96149  │ 0.249137 │ 0.592854 │
## │ 10  │ 0.129374   │ 0.663032 │ 0.0180212 │ 0.280431 │ 0.887136 │ 0.329406 │

Also, we can select a column by using its name as a symbol or using the “.” operator:

## take the column x1 using "." operator:
foo.x1
## 100-element Array{Float64,1}:
##  0.019313572390828426
##  0.9860824001880526  
##  0.003663562628546835
##  0.2978463233159676  
##  0.7324498468154668  
##  0.2097185474768264  
##  0.34169153867123714 
##  0.30835315833846444 
##  0.41376236563754887 
##  0.1293737178707406  
##  ⋮                   
##  0.8582099391004219  
##  0.7144949034554522  
##  0.6804966837971145  
##  0.9093192833587018  
##  0.6425780404716646  
##  0.6051475800989663  
##  0.09075227070455938 
##  0.7516814773623635  
##  0.8562478916762768
## Take the column using "x1" as a symbol:
foo[:x1]
## 100-element Array{Float64,1}:
##  0.019313572390828426
##  0.9860824001880526  
##  0.003663562628546835
##  0.2978463233159676  
##  0.7324498468154668  
##  0.2097185474768264  
##  0.34169153867123714 
##  0.30835315833846444 
##  0.41376236563754887 
##  0.1293737178707406  
##  ⋮                   
##  0.8582099391004219  
##  0.7144949034554522  
##  0.6804966837971145  
##  0.9093192833587018  
##  0.6425780404716646  
##  0.6051475800989663  
##  0.09075227070455938 
##  0.7516814773623635  
##  0.8562478916762768

Notice that the return will be an Array. To select one or more column and return
them as a DataFrame type, we use the double brackets syntax:

using DataFramesMeta
## take column x1 as DataFrame
@linq foo[[:x1]] |> first(5)
## 5×1 DataFrame
## │ Row │ x1         │
## │     │ Float64    │
## ├─────┼────────────┤
## │ 1   │ 0.0193136  │
## │ 2   │ 0.986082   │
## │ 3   │ 0.00366356 │
## │ 4   │ 0.297846   │
## │ 5   │ 0.73245    │
## Take column x1 an x2:
@linq foo[[:x1, :x2]] |> first(5)
## 5×2 DataFrame
## │ Row │ x1         │ x2       │
## │     │ Float64    │ Float64  │
## ├─────┼────────────┼──────────┤
## │ 1   │ 0.0193136  │ 0.466228 │
## │ 2   │ 0.986082   │ 0.33719  │
## │ 3   │ 0.00366356 │ 0.323071 │
## │ 4   │ 0.297846   │ 0.136907 │
## │ 5   │ 0.73245    │ 0.208294 │

There are some new things here. The first() function aims to just show the first
lines of our dataset. Similarly, last() performs the same, but showing us the last
lines. Also, you may have noticed the use of the “|>” operator. This is the
pipe symbol in Julia. If you are familiar with R programming language, it
works similarly to the “%>%” operator from magrittr package, but with some
limitations. For example, we can not pipe to a specific argument in a
subsequent function, so that’s why the use of @linq from DataFramesMeta
package. For now just take these commands for granted. In another post I will show
how to use the functionalities of the metaprogramming tools for DataFrames.

Another trivial task we can perform with column is to add or alter columns in a
DataFrame. For example, let’s create a new column which will be a sequence between
1 and until 100 by 0.5:

## To create a sequence, use the function range():
foo[:new_column] = range(1, step = 0.5, length = nrow(foo));
foo[:, :new_column]
## 100-element Array{Float64,1}:
##   1.0
##   1.5
##   2.0
##   2.5
##   3.0
##   3.5
##   4.0
##   4.5
##   5.0
##   5.5
##   ⋮  
##  46.5
##  47.0
##  47.5
##  48.0
##  48.5
##  49.0
##  49.5
##  50.0
##  50.5

We can also add column using the insertcols!() function. The syntax allow us to
specify in which position we want to add the column in the DataFrame:

## syntax: insert!(dataset, position, column_name => array)
insertcols!(foo, 2, :new_colum2 => range(1, step = 0.5, length = nrow(foo)));
first(foo, 3)
## 3×12 DataFrame. Omitted printing of 6 columns
## │ Row │ x1         │ new_colum2 │ x2       │ x3       │ x4       │ x5       │
## │     │ Float64    │ Float64    │ Float64  │ Float64  │ Float64  │ Float64  │
## ├─────┼────────────┼────────────┼──────────┼──────────┼──────────┼──────────┤
## │ 1   │ 0.0193136  │ 1.0        │ 0.466228 │ 0.790475 │ 0.805074 │ 0.51182  │
## │ 2   │ 0.986082   │ 1.5        │ 0.33719  │ 0.309992 │ 0.117098 │ 0.792606 │
## │ 3   │ 0.00366356 │ 2.0        │ 0.323071 │ 0.685271 │ 0.596414 │ 0.847368 │

Note the use of the “!” in insertcols!() function. This means that the function
is altering the object in memory rather than in a “virtual copy” that later needs
to be assigned to a new variable. This is a behavior that can be used in other function
as well.

Ok… But what if you want to do the opposite? that is, to remove a column?
Well… it is just as easy as to add it. Just use the deletecols!() function:

deletecols!(foo, [:new_column, :new_colum2])
## 100×10 DataFrame. Omitted printing of 4 columns
## │ Row │ x1         │ x2       │ x3        │ x4       │ x5        │ x6       │
## │     │ Float64    │ Float64  │ Float64   │ Float64  │ Float64   │ Float64  │
## ├─────┼────────────┼──────────┼───────────┼──────────┼───────────┼──────────┤
## │ 1   │ 0.0193136  │ 0.466228 │ 0.790475  │ 0.805074 │ 0.51182   │ 0.201707 │
## │ 2   │ 0.986082   │ 0.33719  │ 0.309992  │ 0.117098 │ 0.792606  │ 0.102682 │
## │ 3   │ 0.00366356 │ 0.323071 │ 0.685271  │ 0.596414 │ 0.847368  │ 0.105035 │
## │ 4   │ 0.297846   │ 0.136907 │ 0.726739  │ 0.569452 │ 0.922995  │ 0.846519 │
## │ 5   │ 0.73245    │ 0.208294 │ 0.353801  │ 0.448741 │ 0.185897  │ 0.496741 │
## │ 6   │ 0.209719   │ 0.114021 │ 0.0662264 │ 0.463682 │ 0.628582  │ 0.130653 │
## │ 7   │ 0.341692   │ 0.608349 │ 0.946541  │ 0.589161 │ 0.418321  │ 0.295541 │
## ⋮
## │ 93  │ 0.714495   │ 0.661317 │ 0.954527  │ 0.209581 │ 0.107941  │ 0.233787 │
## │ 94  │ 0.680497   │ 0.101874 │ 0.872371  │ 0.596457 │ 0.669133  │ 0.740674 │
## │ 95  │ 0.909319   │ 0.182776 │ 0.343387  │ 0.142707 │ 0.0140866 │ 0.791679 │
## │ 96  │ 0.642578   │ 0.949993 │ 0.380511  │ 0.96358  │ 0.878766  │ 0.270409 │
## │ 97  │ 0.605148   │ 0.240233 │ 0.144059  │ 0.545245 │ 0.0463105 │ 0.188397 │
## │ 98  │ 0.0907523  │ 0.334278 │ 0.288403  │ 0.519876 │ 0.267965  │ 0.552448 │
## │ 99  │ 0.751681   │ 0.289301 │ 0.488135  │ 0.382877 │ 0.320208  │ 0.999445 │
## │ 100 │ 0.856248   │ 0.577105 │ 0.588476  │ 0.435958 │ 0.0163749 │ 0.337817 │

Now suppose that you do not want to delete a colum, but just change its name.
For this task, I am afraid there is a very difficult function to remember
the name: rename(). The syntax is as follows:

## rename(dataFrame, :old_name => :new_name)
rename(foo, :x1 => :A1, :x2 => :A2)
## 100×10 DataFrame. Omitted printing of 4 columns
## │ Row │ A1         │ A2       │ x3        │ x4       │ x5        │ x6       │
## │     │ Float64    │ Float64  │ Float64   │ Float64  │ Float64   │ Float64  │
## ├─────┼────────────┼──────────┼───────────┼──────────┼───────────┼──────────┤
## │ 1   │ 0.0193136  │ 0.466228 │ 0.790475  │ 0.805074 │ 0.51182   │ 0.201707 │
## │ 2   │ 0.986082   │ 0.33719  │ 0.309992  │ 0.117098 │ 0.792606  │ 0.102682 │
## │ 3   │ 0.00366356 │ 0.323071 │ 0.685271  │ 0.596414 │ 0.847368  │ 0.105035 │
## │ 4   │ 0.297846   │ 0.136907 │ 0.726739  │ 0.569452 │ 0.922995  │ 0.846519 │
## │ 5   │ 0.73245    │ 0.208294 │ 0.353801  │ 0.448741 │ 0.185897  │ 0.496741 │
## │ 6   │ 0.209719   │ 0.114021 │ 0.0662264 │ 0.463682 │ 0.628582  │ 0.130653 │
## │ 7   │ 0.341692   │ 0.608349 │ 0.946541  │ 0.589161 │ 0.418321  │ 0.295541 │
## ⋮
## │ 93  │ 0.714495   │ 0.661317 │ 0.954527  │ 0.209581 │ 0.107941  │ 0.233787 │
## │ 94  │ 0.680497   │ 0.101874 │ 0.872371  │ 0.596457 │ 0.669133  │ 0.740674 │
## │ 95  │ 0.909319   │ 0.182776 │ 0.343387  │ 0.142707 │ 0.0140866 │ 0.791679 │
## │ 96  │ 0.642578   │ 0.949993 │ 0.380511  │ 0.96358  │ 0.878766  │ 0.270409 │
## │ 97  │ 0.605148   │ 0.240233 │ 0.144059  │ 0.545245 │ 0.0463105 │ 0.188397 │
## │ 98  │ 0.0907523  │ 0.334278 │ 0.288403  │ 0.519876 │ 0.267965  │ 0.552448 │
## │ 99  │ 0.751681   │ 0.289301 │ 0.488135  │ 0.382877 │ 0.320208  │ 0.999445 │
## │ 100 │ 0.856248   │ 0.577105 │ 0.588476  │ 0.435958 │ 0.0163749 │ 0.337817 │

We could also add the “!” to the rename() function to alter the DataFrame
in memory.

Let’s talk about missing values:

Missing values are represented in Julia with missing value. When an array
contains missing values, it automatically creates an appropriate union type:

x = [1.0, 2.0, missing]
## 3-element Array{Union{Missing, Float64},1}:
##  1.0     
##  2.0     
##   missing
typeof(x)
## Array{Union{Missing, Float64},1}
typeof.(x)
## 3-element Array{DataType,1}:
##  Float64
##  Float64
##  Missing

To check if a particular element in an array is missing, we use the ismissing()
function:

ismissing.([1.0, 2.0, missing])
## 3-element BitArray{1}:
##  false
##  false
##   true

It is important to notice that missing comparison produces missing as a result:

missing == missing

isequal and === can be used to produce the results of type Bool:

isequal(missing, missing)
## true
missing === missing
## true

Other functions are available to work with missing values. For instance, suppose
we want an array with only non-missing values, we use the skipmissing() function:

x |> skipmissing |> collect
## 2-element Array{Float64,1}:
##  1.0
##  2.0

Here, we use the collect() function as the skipmissing() returns an iterator.

To replace the missing values with some other value we can use the
Missings.replace() function. For example, suppose we want to change the missing
values by NaN:

Missings.replace(x, NaN) |> collect
## 3-element Array{Float64,1}:
##    1.0
##    2.0
##  NaN

We also can use use other ways to perform the same operation:

## Using coalesce() function:
coalesce.(x, NaN)
## 3-element Array{Float64,1}:
##    1.0
##    2.0
##  NaN
## Using recode() function:
recode(x, missing => NaN)
## 3-element Array{Float64,1}:
##    1.0
##    2.0
##  NaN

Until now, we have only talked about missing values in arrays. But what about missing
values in DataFrames? To start, let’s create a DataFrame with some missing values:

x = DataFrame(A = [1, missing, 3, 4], B = ["A", "B", missing, "C"])
## 4×2 DataFrame
## │ Row │ A       │ B       │
## │     │ Int64⍰  │ String⍰ │
## ├─────┼─────────┼─────────┤
## │ 1   │ 1       │ A       │
## │ 2   │ missing │ B       │
## │ 3   │ 3       │ missing │
## │ 4   │ 4       │ C       │

For some analysis, we would want only the rows with non-missing values. One way
to achieve this is making use of the completecases() function:

x[completecases(x), :]
## 2×2 DataFrame
## │ Row │ A      │ B       │
## │     │ Int64⍰ │ String⍰ │
## ├─────┼────────┼─────────┤
## │ 1   │ 1      │ A       │
## │ 2   │ 4      │ C       │

The completecases() function returns an boolean array with value true for
rows that have non-missing values and false otherwise. For those who are familiar
with R, this is the same behavior as the complete.cases() function from stats package.

Another option to return the rows with non-missing values of a DataFrame in Julia
is to use the dropmissing() function:

dropmissing(x)
## 2×2 DataFrame
## │ Row │ A     │ B      │
## │     │ Int64 │ String │
## ├─────┼───────┼────────┤
## │ 1   │ 1     │ A      │
## │ 2   │ 4     │ C      │

and again, for R users is the same behavior as na.omit() function.

Merging DataFrames:

Often, we need to combine two or more DataFrames together based on some common
column(s) among them. For example, suppose we have two DataFrames:

df1 = DataFrame(x = 1:3, y = 4:6)
## 3×2 DataFrame
## │ Row │ x     │ y     │
## │     │ Int64 │ Int64 │
## ├─────┼───────┼───────┤
## │ 1   │ 1     │ 4     │
## │ 2   │ 2     │ 5     │
## │ 3   │ 3     │ 6     │
df2 = DataFrame(x = 1:3, z = 'd':'f', new = 11:13)
## 3×3 DataFrame
## │ Row │ x     │ z    │ new   │
## │     │ Int64 │ Char │ Int64 │
## ├─────┼───────┼──────┼───────┤
## │ 1   │ 1     │ 'd'  │ 11    │
## │ 2   │ 2     │ 'e'  │ 12    │
## │ 3   │ 3     │ 'f'  │ 13    │

which have the column x in common. To merge these two tables, we use the
join() function:

join(df1, df2, on = :x)
## 3×4 DataFrame
## │ Row │ x     │ y     │ z    │ new   │
## │     │ Int64 │ Int64 │ Char │ Int64 │
## ├─────┼───────┼───────┼──────┼───────┤
## │ 1   │ 1     │ 4     │ 'd'  │ 11    │
## │ 2   │ 2     │ 5     │ 'e'  │ 12    │
## │ 3   │ 3     │ 6     │ 'f'  │ 13    │

That’s it!! We merge our DataFrames altogether. But that’s the default behavior of
the function. There is more to explore. Essentially, join() takes 4 arguments:

  • DataFrame 1
  • DataFrame 2
  • on = the column(s) to be the key in merging;
  • kind = type of the merge (left, right, inner, outer, …)

The kind argument specifies the type of join we are interested in performing.
The definition of each one is as follows:

  • Inner: The output contains rows for values of the key that exist
    in BOTH the first (left) and second (right) arguments to
    join;

  • Left: The output contains rows for values of the key that exist in
    the first (left) argument to join, whether or not that value
    exists in the second (right) argument;

  • Right: The output contains rows for values of the key that exist in
    the second (right) argument to join, whether or not that
    value exists in the first (left) argument;

  • Outer: The output contains rows for values of the key that exist in
    the first (left) OR second (right) argument to join;

and here are the “strange” ones:

  • Semi: Like an inner join, but output is restricted to columns from
    the first (left) argument to join;

  • Anti: The output contains rows for values of the key that exist in
    the first (left) but NOT in the second (right) argument to
    join. As with semi joins, output is restricted to columns
    from the first (left) argument.

If you are familiar with SQL or with the join functions from dplyr package in R,
it is the same concept.

To illustrate how the different kind of joins work, let’s create more DataFrames
to demonstrate each type of join:

Names = DataFrame(ID = [20, 40], Name = ["John Doe", "Jane Doe"])
## 2×2 DataFrame
## │ Row │ ID    │ Name     │
## │     │ Int64 │ String   │
## ├─────┼───────┼──────────┤
## │ 1   │ 20    │ John Doe │
## │ 2   │ 40    │ Jane Doe │
jobs = DataFrame(ID = [20, 60], Job = ["Lawyer", "Astronaut"])
## 2×2 DataFrame
## │ Row │ ID    │ Job       │
## │     │ Int64 │ String    │
## ├─────┼───────┼───────────┤
## │ 1   │ 20    │ Lawyer    │
## │ 2   │ 60    │ Astronaut │

In the Names and jobs DataFrame, we have the ID column as the key to perform the
join. But notice that the ID values are not equal between the DataFrames. Now
let’s perform the joins:

join(Names, jobs, on = :ID, kind = :inner)
## 1×3 DataFrame
## │ Row │ ID    │ Name     │ Job    │
## │     │ Int64 │ String   │ String │
## ├─────┼───────┼──────────┼────────┤
## │ 1   │ 20    │ John Doe │ Lawyer │
join(Names, jobs, on = :ID, kind = :left)
## 2×3 DataFrame
## │ Row │ ID    │ Name     │ Job     │
## │     │ Int64 │ String   │ String⍰ │
## ├─────┼───────┼──────────┼─────────┤
## │ 1   │ 20    │ John Doe │ Lawyer  │
## │ 2   │ 40    │ Jane Doe │ missing │
join(Names, jobs, on = :ID, kind = :right)
## 2×3 DataFrame
## │ Row │ ID    │ Name     │ Job       │
## │     │ Int64 │ String⍰  │ String    │
## ├─────┼───────┼──────────┼───────────┤
## │ 1   │ 20    │ John Doe │ Lawyer    │
## │ 2   │ 60    │ missing  │ Astronaut │
join(Names, jobs, on = :ID, kind = :outer)
## 3×3 DataFrame
## │ Row │ ID    │ Name     │ Job       │
## │     │ Int64 │ String⍰  │ String⍰   │
## ├─────┼───────┼──────────┼───────────┤
## │ 1   │ 20    │ John Doe │ Lawyer    │
## │ 2   │ 40    │ Jane Doe │ missing   │
## │ 3   │ 60    │ missing  │ Astronaut │

Semi and anti join have a more uncommon behavior. Semi join returns the rows
from the left which DO MATCH with the ID from the right:

join(Names, jobs, on = :ID, kind = :semi)
## 1×2 DataFrame
## │ Row │ ID    │ Name     │
## │     │ Int64 │ String   │
## ├─────┼───────┼──────────┤
## │ 1   │ 20    │ John Doe │

Anti join returns the rows from the left which DO NOT MATCH with
the ID from the right

join(Names, jobs, on = :ID, kind = :anti)
## 1×2 DataFrame
## │ Row │ ID    │ Name     │
## │     │ Int64 │ String   │
## ├─────┼───────┼──────────┤
## │ 1   │ 40    │ Jane Doe │

Split-Apply-Combine:

Some common tasks involve splitting the data into groups, applying some function
to each of these groups and gathering the results to analyze later on. This is the
split-apply-combine strategy described in the paper “The Split-Apply-Combine Strategy for Data analysis” written by Hadley Wickham, creator
of many R packages, including ggplot2 and dplyr.

The DataFrames package in Julia supports the Split-Apply-Combine strategy
through the by() function, which takes three arguments:

  • DataFrame;
  • one or more column names to split on;
  • a function or expression to apply to each subset;

To illustrate its usage, let’s make use of the RDatasets package, which gives
access to some preloaded well known datasets from R packages.

using RDatasets
foo = dataset("datasets", "iris");
first(foo, 5)
## 5×5 DataFrame
## │ Row │ SepalLength │ SepalWidth │ PetalLength │ PetalWidth │ Species      │
## │     │ Float64     │ Float64    │ Float64     │ Float64    │ Categorical… │
## ├─────┼─────────────┼────────────┼─────────────┼────────────┼──────────────┤
## │ 1   │ 5.1         │ 3.5        │ 1.4         │ 0.2        │ setosa       │
## │ 2   │ 4.9         │ 3.0        │ 1.4         │ 0.2        │ setosa       │
## │ 3   │ 4.7         │ 3.2        │ 1.3         │ 0.2        │ setosa       │
## │ 4   │ 4.6         │ 3.1        │ 1.5         │ 0.2        │ setosa       │
## │ 5   │ 5.0         │ 3.6        │ 1.4         │ 0.2        │ setosa       │

A trivial task is to find how many of each “Species” there are in the
dataset. One way to do this is to apply the Split-Apply-Combine strategy: split
the data into the Species column, apply the nrow() function to this
splitted dataset, and combine the results:

## Syntax: by(dataset, :name_column_to_split, name_function)
by(foo, :Species, nrow)
## 3×2 DataFrame
## │ Row │ Species      │ x1    │
## │     │ Categorical… │ Int64 │
## ├─────┼──────────────┼───────┤
## │ 1   │ setosa       │ 50    │
## │ 2   │ versicolor   │ 50    │
## │ 3   │ virginica    │ 50    │

We can also make use of anonymous function:

by(foo, :Species, x -> DataFrame(N = nrow(x)))
## 3×2 DataFrame
## │ Row │ Species      │ N     │
## │     │ Categorical… │ Int64 │
## ├─────┼──────────────┼───────┤
## │ 1   │ setosa       │ 50    │
## │ 2   │ versicolor   │ 50    │
## │ 3   │ virginica    │ 50    │

One of the advantages of using anonymous function inside the by() function is
that we can format the resulted output and apply as many function as we want:

## Applying the count, mean and standard deviation function:
by(foo, :Species, x -> DataFrame(N = nrow(x),
                                 avg_PetalLength = mean(x[:PetalLength]),
                                 std_PetalWidth = std(x[:PetalWidth])))
## 3×4 DataFrame
## │ Row │ Species      │ N     │ avg_PetalLength │ std_PetalWidth │
## │     │ Categorical… │ Int64 │ Float64         │ Float64        │
## ├─────┼──────────────┼───────┼─────────────────┼────────────────┤
## │ 1   │ setosa       │ 50    │ 1.462           │ 0.105386       │
## │ 2   │ versicolor   │ 50    │ 4.26            │ 0.197753       │
## │ 3   │ virginica    │ 50    │ 5.552           │ 0.27465        │

Another way to use the Split-Apply-Combine strategy is implementing the
aggregate() function, which also takes three arguments:

  • DataFrame;
  • one or more column names to split on;
  • one or more function to be applied ON THE COLUMNS NOT USED TO SPLIT.

The difference between by() and aggregate() function is that in the
latter, the function(s) will be applied to each column not used in
the split part.

For instance, let’s say you want the average of each colum for each Species.
Instead of using by() with an anonymous function and writing the name of all columns
we can do:

aggregate(foo, :Species, [mean])
## 3×5 DataFrame. Omitted printing of 1 columns
## │ Row │ Species      │ SepalLength_mean │ SepalWidth_mean │ PetalLength_mean │
## │     │ Categorical… │ Float64          │ Float64         │ Float64          │
## ├─────┼──────────────┼──────────────────┼─────────────────┼──────────────────┤
## │ 1   │ setosa       │ 5.006            │ 3.428           │ 1.462            │
## │ 2   │ versicolor   │ 5.936            │ 2.77            │ 4.26             │
## │ 3   │ virginica    │ 6.588            │ 2.974           │ 5.552            │

Note that Julia only display output that fits the screen. Pay
attention to the message “Omitted printing of 1 columns”. To
overcome this, use the show() as advised before.

Reading and Writting CSV files:

Last but not least, let’s see how to read and write CSV files into/from Julia.
Although this is not exactly handled by the DataFrames package, the task of
reading/writing CSV files are so natural when working with DataFrame that I will
show you the basics.

To read/write CSV files, we use the CSV package. To demonstrate its usage,
let’s work with the iris dataset and write a CSV file to a local computer. Then,
we read it back.

So, first we are going to write the foo object (which contains the iris dataset)
to a CSV file. To do this we will use the CSV.write() function. Some useful
arguments in CSV.write are:

  • delim : the file’s delimeter. Default ‘,’;
  • header : boolean whether to write the colnames from source;
  • colnames : provide colnames to be written;
  • append : bool to indicate if it to append data;
  • missingstring : string that indicates how missing values will be represented.
    using CSV
    CSV.write("iris.csv", foo, missingsstring = "NA")

To read a CSV file, we use the CSV.read(). Some useful arguments are:

  • delim : a Char or String that indicates how columns are delimited in a file’s delimeter. Default ‘,’;
  • decimal : a Char indicating how decimals are separated in
    floats. Default ‘.’ ;
  • limit : indicates the total number of rows to read;
  • header : provide manually the names of the columns;
  • types : a Vector or Dict of types to be used for column types.
iris = CSV.read("iris.csv")

It is important to note that when loading in a DataFrame from a CSV, all columns
allow Missing by default.

This is the basics of reading/writting CSV files in Julia. To get more details
refers to the official documentation.

Conclusion:

This post was a very small introduction to the DataFrames packages in Julia.
After reading this post you will be able to read CSV datasets and perform some
tasks with the data at hand.

In the following posts, we will explore more advanced tricks to perform data
wrangling and exploratory data analysis. At each step we are going to build
knowledge to completely use Julia to perform data analysis for any problem that
you might face.