Category Archives: Julia

Understanding the Julia Type System

By: Justyn Nissly

Re-posted from: https://blog.glcs.io/julia-type-system

In the programming landscape, type systems traditionally fall into two categories: static and dynamic.
In static languages such as C, C++, Rust, and Go,
computations are performed before run time to determine types and the values of those types.

Pearl, Ruby, PHP, Python, and JavaScript are languages that use a dynamic type system.
In these languages, nothing is known until runtime.
Now you may be thinking, where does Julia fit on this landscape? or
how does Julias type system work?
That is exactly what we will cover in this post!

Julias Type System

Where exactly does Julia fall along the type system landscape?
Is it static? Is it dynamic? Is it some strange amalgam of both?
According to Julia’s documentation,
it is dynamic, nominative, and parametric.
So, what does that mean?

It means Julia is a dynamic language but with a powerful twist.
You get all the features of a dynamic type system while also gaining some of the advantages of static type systems.
Julia does this by allowing you to indicate that certain values are of specific types.

By default, Julia will allow values to be of any type if the type is not explicitly stated.
This allows you to write functions without ever explicitly using types; however, explicitly declared types can be added as needed to help improve human readability and to help catch errors.

Now that we have the introduction out of the way, lets get our hands dirty and see how we use types in Julia.

Type Declarations

The first thing to get familiar with is how to declare types in Julia.
To declare a type (also called type annotations), we use the :: operator on a variable or expression in a program.
When the :: operator is applied to a variable, it is read as is an instance of.
For example, if you were to type var::Integer you would read this as var is an instance of Integer.

Lets look at an example in the REPL:

julia> test = (1+1)::Integer
2
julia> typeof(test)
Int64

You can see in the example above that
we declared the result of the expression (1+1) to be an instance of Integer
but when we checked the type using the typeof() function, it returns Int64.
The reason we see the type as Int64 rather than Integer is because Julia chooses a default primitive type (more on this later) for that value.
For integers, Julia will choose Int64 on a 64-bit computer or Int32 on a 32-bit computer.
Furthermore,
typeof(variable_name) will always return what is called a concrete type.
A concrete type is a type where values can be created by the compiler.
Variables can’t be an abstract type so any time you check the type of a variable you will get some concrete type.
In our example, Integer is an abstract type (we will cover these types later) that is a supertype of the concrete type Int64.
If you go to the REPL you will see that test isa Integer returns true even though using typeof(test) returns Int64.
This is because of the hierarchy of types in Julia. We will cover this later.

Adding types also works on function declarations:

julia> function multiply_two_numbers(x,y)::Float64
           return x*y
       end
multiply_two_numbers (generic function with 1 method)

Now let’s run this function with two integer values and see what gets returned:

julia> x = multiply_two_numbers(2,4)
8.0

julia> typeof(x)
Float64

You notice it returns a Float64 even though we passed it two integers?
This is because we have told Julia that the function should ALWAYS return a Float64 regardless of what the values in the function are.

One thing that we should note is that type annotations are generally not used much.
Using type annotations does not improve performance unless there is a “type-unstable” function, but we won’t cover that in this article. You can learn more about type-stability here and here.

Abstract Types

Abstract types function differently than primitive types. (More on those later.)
They function as placeholders for groups of related types.
They can’t be used to create objects,
but they are essential for organizing different types into a hierarchy.
Think of them as labels that help Julia understand which types are related to each other.

Lets look at an example from the Julia documentation.

Julias hierarchy for numeric values is actually built off of abstract types:

abstract type Number end
abstract type Real          <: Number end
abstract type AbstractFloat <: Real end
abstract type Integer       <: Real end
abstract type Signed        <: Integer end
abstract type Unsigned      <: Integer end

You can see that Number is an abstract type (a direct descendant of the Any type). Then Real is a subtype of Number,
Integer is a subtype of Real,
and Signed is a subtype of Integer.
That means the hierarchy looks like this:

Abstract Types Graphs

We can write a function to see the hierarchy of any type:

function show_type_tree(T, level=0)
    println("\t" ^ level, T)
    for t in subtypes(T)
        show_type_tree(t, level+1)
    end
end

Now you know that there is a hierarchy,
you might want to know what the point of it is.
Let’s look at our earlier example of the multiply_two_numbers() function:

julia> function multiply_two_numbers(x,y)::Float64
           return x*y
       end
multiply_two_numbers (generic function with 1 method)

Lets see what happens when with give the function two Integers:

julia> multiply_two_numbers(2::Int,2::Int)
4.0

Now lets try an integer and a float:

julia> multiply_two_numbers(2::Int,2.3::Float64)
4.6

One more example, we will do one as a float and one without asserting a type:

julia> multiply_two_numbers(1.6::Float64,54)
86.4

Notice how these examples still work without errors?
That is because of abstract types!
Because we created multiply_two_numbers() without declaring a type for the parameters,
Julia will default them to the Any type and that means we can assign any of the number types that are in the hierarchy as subtypes of Any

Let’s look at the hierarchy again to see how this works
Big Type Graph

Since we didnt explicitly define the type of the parameters for the function,
we can send it any type of numeric value and the function can work on it.
We are not limited to sending only Int64 or a Float64.
That is the real power of abstract types!
It allows us to create very generic code that works on various types within a category
without having to create a function for each type.

Primitive Types

Primitive types are similar to Abstract types in that they are part of a hierarchy of types,
but they are different in that they are concrete types.
The data contained in a primitive type is simply the bits in memory.
One interesting thing about primitive types in Julia is that Julia will let you define your own.

primitive type name bits end
primitive type name <: supertype bits end

Interestingly enough, Julias own default primitive types are all defined in Julia itself using the syntax above.

primitive type Float16 <: AbstractFloat 16 end
primitive type Float32 <: AbstractFloat 32 end
primitive type Float64 <: AbstractFloat 64 end

primitive type Bool <: Integer 8 end
primitive type Char <: AbstractChar 32 end

primitive type Int8    <: Signed 8 end
primitive type Int16   <: Signed 16 end
primitive type Int32   <: Signed 32 end
primitive type Int64   <: Signed 64 end
primitive type Int128  <: Signed 128 end

primitive type UInt8   <: Unsigned 8 end
primitive type UInt16  <: Unsigned 16 end
primitive type UInt32  <: Unsigned 32 end
primitive type UInt64  <: Unsigned 64 end
primitive type UInt128 <: Unsigned 128 end

Just like Abstract types, if you leave off a supertype when you declare the primitive,
the primitive will have Any as its direct supertype.
One thing to note when declaring your own primitive types, according to Julias documentation,
it is usually preferred that you wrap an existing primitive type in a new composite type rather than creating your own primitive.
Fortunately for us, this is a great segway into our next topic: Composite Types!

Composite Types

Composite types are Julia’s implementation of what other languages call structs, records, or objects.
Interestingly enough, composite types are declared very similar to structs in C

julia> struct Foo
           bar
           baz::Int
           qux::Float64
       end

Just like we saw earlier, if you leave the type annotation off of a field in your composite type, it will be defaulted to the Any type.

Just like structs in C or Objects in Java, composite types are a very powerful part of the language. We will have a more detailed post on composite types later, but if you would like to know more in the meantime, check out the official Julia documentation

Type of Types

In the last few sections we covered abstract types, primitive types, and we touched on composite types.
Let’s see what happens when we check the type of some of these types

julia> typeof(Number)
DataType

julia> typeof(Int64)
DataType

julia> typeof(AbstractFloat)
DataType

julia> typeof(Char)
DataType

You will notice that they all return the same thing, DataType.
The reason for this is the shared properties that these types have.
Abstract, primitive, and composite types all have names.
They have a supertype that is explicitly declared.
All those types are explicitly declared.
Since these properties are shared amongst the different types we covered,
Julia represents each instance of these types as the same concept internally.

Summary

In this post, we covered Abstract types, Primitive types, and Composite types.
These are the building blocks of the Julia type system.
You can see the power of the structure of Julia’s type system.
You can create generic code that works on various types within a category
without having to create a function for each type.
This simplifies your code and makes it easier to understand and maintain long-term.
If you want more in-depth information about types you can visit Julia’s official documentation to learn more.

Additional Links

Understanding the Julia Type System

By: Justyn Nissly

Re-posted from: https://glcs.hashnode.dev/julia-type-system

In the programming landscape, type systems traditionally fall into two categories: static and dynamic.In static languages such as C, C++, Rust, and Go,computations are performed before run time to determine types and the values of those types.

Pearl, Ruby, PHP, Python, and JavaScript are languages that use a dynamic type system.In these languages, nothing is known until runtime.Now you may be thinking, where does Julia fit on this landscape? orhow does Julias type system work?That is exactly what we will cover in this post!

Julias Type System

Where exactly does Julia fall along the type system landscape?Is it static? Is it dynamic? Is it some strange amalgam of both?According to Julia’s documentation,it is dynamic, nominative, and parametric.So, what does that mean?

It means Julia is a dynamic language but with a powerful twist.You get all the features of a dynamic type system while also gaining some of the advantages of static type systems.Julia does this by allowing you to indicate that certain values are of specific types.

By default, Julia will allow values to be of any type if the type is not explicitly stated.This allows you to write functions without ever explicitly using types; however, explicitly declared types can be added as needed to help improve human readability and to help catch errors.

Now that we have the introduction out of the way, lets get our hands dirty and see how we use types in Julia.

Type Declarations

The first thing to get familiar with is how to declare types in Julia.To declare a type (also called type annotations), we use the :: operator on a variable or expression in a program.When the :: operator is applied to a variable, it is read as is an instance of.For example, if you were to type var::Integer you would read this as var is an instance of Integer.

Lets look at an example in the REPL:

julia> test = (1+1)::Integer2julia> typeof(test)Int64

You can see in the example above thatwe declared the result of the expression (1+1) to be an instance of Integerbut when we checked the type using the typeof() function, it returns Int64.The reason we see the type as Int64 rather than Integer is because Julia chooses a default primitive type (more on this later) for that value.For integers, Julia will choose Int64 on a 64-bit computer or Int32 on a 32-bit computer.Furthermore,typeof(variable_name) will always return what is called a concrete type.A concrete type is a type where values can be created by the compiler.Variables can’t be an abstract type so any time you check the type of a variable you will get some concrete type.In our example, Integer is an abstract type (we will cover these types later) that is a supertype of the concrete type Int64.If you go to the REPL you will see that test isa Integer returns true even though using typeof(test) returns Int64.This is because of the hierarchy of types in Julia. We will cover this later.

Adding types also works on function declarations:

julia> function multiply_two_numbers(x,y)::Float64           return x*y       endmultiply_two_numbers (generic function with 1 method)

Now let’s run this function with two integer values and see what gets returned:

julia> x = multiply_two_numbers(2,4)8.0julia> typeof(x)Float64

You notice it returns a Float64 even though we passed it two integers?This is because we have told Julia that the function should ALWAYS return a Float64 regardless of what the values in the function are.

One thing that we should note is that type annotations are generally not used much.Using type annotations does not improve performance unless there is a “type-unstable” function, but we won’t cover that in this article. You can learn more about type-stability here and here.

Abstract Types

Abstract types function differently than primitive types. (More on those later.)They function as placeholders for groups of related types.They can’t be used to create objects,but they are essential for organizing different types into a hierarchy.Think of them as labels that help Julia understand which types are related to each other.

Lets look at an example from the Julia documentation.

Julias hierarchy for numeric values is actually built off of abstract types:

abstract type Number endabstract type Real          <: Number endabstract type AbstractFloat <: Real endabstract type Integer       <: Real endabstract type Signed        <: Integer endabstract type Unsigned      <: Integer end

You can see that Number is an abstract type (a direct descendant of the Any type). Then Real is a subtype of Number,Integer is a subtype of Real,and Signed is a subtype of Integer.That means the hierarchy looks like this:

Abstract Types Graphs

We can write a function to see the hierarchy of any type:

function show_type_tree(T, level=0)    println("\t" ^ level, T)    for t in subtypes(T)        show_type_tree(t, level+1)    endend

Now you know that there is a hierarchy,you might want to know what the point of it is.Let’s look at our earlier example of the multiply_two_numbers() function:

julia> function multiply_two_numbers(x,y)::Float64           return x*y       endmultiply_two_numbers (generic function with 1 method)

Lets see what happens when with give the function two Integers:

julia> multiply_two_numbers(2::Int,2::Int)4.0

Now lets try an integer and a float:

julia> multiply_two_numbers(2::Int,2.3::Float64)4.6

One more example, we will do one as a float and one without asserting a type:

julia> multiply_two_numbers(1.6::Float64,54)86.4

Notice how these examples still work without errors?That is because of abstract types!Because we created multiply_two_numbers() without declaring a type for the parameters,Julia will default them to the Any type and that means we can assign any of the number types that are in the hierarchy as subtypes of Any

Let’s look at the hierarchy again to see how this worksBig Type Graph

Since we didnt explicitly define the type of the parameters for the function,we can send it any type of numeric value and the function can work on it.We are not limited to sending only Int64 or a Float64.That is the real power of abstract types!It allows us to create very generic code that works on various types within a categorywithout having to create a function for each type.

Primitive Types

Primitive types are similar to Abstract types in that they are part of a hierarchy of types,but they are different in that they are concrete types.The data contained in a primitive type is simply the bits in memory.One interesting thing about primitive types in Julia is that Julia will let you define your own.

primitive type name bits endprimitive type name <: supertype bits end

Interestingly enough, Julias own default primitive types are all defined in Julia itself using the syntax above.

primitive type Float16 <: AbstractFloat 16 endprimitive type Float32 <: AbstractFloat 32 endprimitive type Float64 <: AbstractFloat 64 endprimitive type Bool <: Integer 8 endprimitive type Char <: AbstractChar 32 endprimitive type Int8    <: Signed 8 endprimitive type Int16   <: Signed 16 endprimitive type Int32   <: Signed 32 endprimitive type Int64   <: Signed 64 endprimitive type Int128  <: Signed 128 endprimitive type UInt8   <: Unsigned 8 endprimitive type UInt16  <: Unsigned 16 endprimitive type UInt32  <: Unsigned 32 endprimitive type UInt64  <: Unsigned 64 endprimitive type UInt128 <: Unsigned 128 end

Just like Abstract types, if you leave off a supertype when you declare the primitive,the primitive will have Any as its direct supertype.One thing to note when declaring your own primitive types, according to Julias documentation,it is usually preferred that you wrap an existing primitive type in a new composite type rather than creating your own primitive.Fortunately for us, this is a great segue into our next topic: Composite Types!

Composite Types

Composite types are Julia’s implementation of what other languages call structs, records, or objects.Interestingly enough, composite types are declared very similar to structs in C

julia> struct Foo           bar           baz::Int           qux::Float64       end

Just like we saw earlier, if you leave the type annotation off of a field in your composite type, it will be defaulted to the Any type.

Just like structs in C or Objects in Java, composite types are a very powerful part of the language. We will have a more detailed post on composite types later, but if you would like to know more in the meantime, check out the official Julia documentation

Type of Types

In the last few sections we covered abstract types, primitive types, and we touched on composite types.Let’s see what happens when we check the type of some of these types

julia> typeof(Number)DataTypejulia> typeof(Int64)DataTypejulia> typeof(AbstractFloat)DataTypejulia> typeof(Char)DataType

You will notice that they all return the same thing, DataType.The reason for this is the shared properties that these types have.Abstract, primitive, and composite types all have names.They have a supertype that is explicitly declared.All those types are explicitly declared.Since these properties are shared amongst the different types we covered,Julia represents each instance of these types as the same concept internally.

Summary

In this post, we covered Abstract types, Primitive types, and Composite types.These are the building blocks of the Julia type system.You can see the power of the structure of Julia’s type system.You can create generic code that works on various types within a categorywithout having to create a function for each type.This simplifies your code and makes it easier to understand and maintain long-term.If you want more in-depth information about types you can visit Julia’s official documentation to learn more.

Do you understand the basics of Julia’s type system?Move on to thenext post to learn about basic data structures in Julia!Or,feel free to take a lookat our other Julia tutorial posts!

Additional Links

Getting ready for JuliaCon Local Eindhoven 2023

By: Blog by Bogumił Kamiński

Re-posted from: https://bkamins.github.io/julialang/2023/10/13/juliacon.html

Introduction

I am really excited that soon we will have a Julia conference in Europe.
The JuliaCon Local Eindhoven 2023 will take place on December 1, 2023 in Eindhoven.

I thought that as nice preparation for this event it would be useful to share
some tips that can be useful for DataFrames.jl users.
Frequently, when working with tables, one has to process time series data.
I want to share a few examples how such data can be stored in DataFrames.jl.

The post was written under Julia 1.9.2 and DataFrames.jl 1.6.1.

Setting up the stage

Let us start with a data frame that has 4 columns and 1000 rows representing consecutive measurements of some data.
I also add the :period column keeping track of the moment at which the measurement was made.

julia> using DataFrames

julia> using Random

julia> Random.seed!(1234);

julia> df1 = DataFrame(rand(1000, 4), :auto);

julia> df1.period = 1:1000;

julia> df1
1000×5 DataFrame
  Row │ x1         x2         x3        x4         period
      │ Float64    Float64    Float64   Float64    Int64
──────┼───────────────────────────────────────────────────
    1 │ 0.579862   0.473374   0.366288  0.771849        1
    2 │ 0.411294   0.176997   0.652475  0.938484        2
    3 │ 0.972136   0.676856   0.900155  0.390673        3
    4 │ 0.0149088  0.177963   0.374975  0.694063        4
  ⋮   │     ⋮          ⋮         ⋮          ⋮        ⋮
  997 │ 0.424295   0.78012    0.577681  0.274228      997
  998 │ 0.481554   0.0441882  0.700181  0.317677      998
  999 │ 0.401528   0.658903   0.438309  0.211217      999
 1000 │ 0.58985    0.909857   0.354735  0.442829     1000
                                          992 rows omitted

Reshaping data: wide to long

The first operation we want to do is reshaping the data from wide to long format,
in which we will have three columns: the period, the variable name, and the variable value.
To achieve this we can use the stack function:

julia> df2 = stack(df1, Not(:period))
4000×3 DataFrame
  Row │ period  variable  value
      │ Int64   String    Float64
──────┼─────────────────────────────
    1 │      1  x1        0.579862
    2 │      2  x1        0.411294
    3 │      3  x1        0.972136
    4 │      4  x1        0.0149088
  ⋮   │   ⋮        ⋮          ⋮
 3997 │    997  x4        0.274228
 3998 │    998  x4        0.317677
 3999 │    999  x4        0.211217
 4000 │   1000  x4        0.442829
                   3992 rows omitted

Reshaping data: long to wide

Now let us perform the reverse operation, but this time put :period as columns.
We can do it using the unstack function:

julia> df3 = unstack(df2, :variable, :period, :value)
4×1001 DataFrame
 Row │ variable  1         2         3         4          5         6        ⋯
     │ String    Float64?  Float64?  Float64?  Float64?   Float64?  Float64? ⋯
─────┼────────────────────────────────────────────────────────────────────────
   1 │ x1        0.579862  0.411294  0.972136  0.0149088  0.520355  0.639562 ⋯
   2 │ x2        0.473374  0.176997  0.676856  0.177963   0.670122  0.042361
   3 │ x3        0.366288  0.652475  0.900155  0.374975   0.387857  0.779594
   4 │ x4        0.771849  0.938484  0.390673  0.694063   0.724383  0.130453
                                                           995 columns omitted

Nesting columns

Another way to reshape this data is to put all periods in a single cell of a data frame.
We can do it in two ways.

The first is to keep them in columns :x1, :x2, :x3, and :x4. Here are two ways
how you can do it starting from df1 or df2:

julia> combine(df1, Not(:period) .=> Ref, renamecols=false)
1×4 DataFrame
 Row │ x1                                 x2                                 ⋯
     │ Array…                             Array…                             ⋯
─────┼────────────────────────────────────────────────────────────────────────
   1 │ [0.579862, 0.411294, 0.972136, 0…  [0.473374, 0.176997, 0.676856, 0…  ⋯
                                                             2 columns omitted
julia> unstack(df2, [], :variable, :value, combine=collect)
1×4 DataFrame
 Row │ x1                                 x2                                 ⋯
     │ Array…?                            Array…?                            ⋯
─────┼────────────────────────────────────────────────────────────────────────
   1 │ [0.579862, 0.411294, 0.972136, 0…  [0.473374, 0.176997, 0.676856, 0…  ⋯
                                                             2 columns omitted

Alternatively, we could want to have two columns, the first with variable names
and the second with the vectors. Again, let me show two ways to do it starting
from df2 and df3 data frames:

julia> df4 = combine(groupby(df2, :variable), :value => Ref, renamecols=false)
4×2 DataFrame
 Row │ variable  value
     │ String    SubArray…
─────┼─────────────────────────────────────────────
   1 │ x1        [0.579862, 0.411294, 0.972136, 0…
   2 │ x2        [0.473374, 0.176997, 0.676856, 0…
   3 │ x3        [0.366288, 0.652475, 0.900155, 0…
   4 │ x4        [0.771849, 0.938484, 0.390673, 0…

julia> combine(df3, :variable, AsTable(Not(:variable)) => ByRow(identity∘collect) => :value)
4×2 DataFrame
 Row │ variable  value
     │ String    Array…
─────┼─────────────────────────────────────────────
   1 │ x1        Union{Missing, Float64}[0.579862…
   2 │ x2        Union{Missing, Float64}[0.473374…
   3 │ x3        Union{Missing, Float64}[0.366288…
   4 │ x4        Union{Missing, Float64}[0.771849…

These examples are slightly more complicated so let me explain them.

In the first one we use Ref to protect a vector against expansion into multiple columns
(note that no copying of data happens here, to perform a copy you should write Ref∘copy).

In the second one the AsTable(Not(:variable)) source column selector produces a NamedTuple.
However, Julia users probably are aware that with 1000 entries such a NamedTuple would take a long
time to compile. For this reason we are using an optimization that DataFrames.jl provides.
If we pass a function of a form some_function∘collect then this compilation step is avoided.
In our case the function is just identity as we are happy with producing a vector (which collect already returns).

Unnesting columns

As a last example let us show how to expand the df4 data frame back into multiple columns.
This is easy. Just write:

julia> select(df4, :variable, :value => AsTable)
4×1001 DataFrame
 Row │ variable  x1        x2        x3        x4         x5        x6       ⋯
     │ String    Float64   Float64   Float64   Float64    Float64   Float64  ⋯
─────┼────────────────────────────────────────────────────────────────────────
   1 │ x1        0.579862  0.411294  0.972136  0.0149088  0.520355  0.639562 ⋯
   2 │ x2        0.473374  0.176997  0.676856  0.177963   0.670122  0.042361
   3 │ x3        0.366288  0.652475  0.900155  0.374975   0.387857  0.779594
   4 │ x4        0.771849  0.938484  0.390673  0.694063   0.724383  0.130453
                                                           995 columns omitted

Since we used AsTable as target column names specifier we got auto-generated column names.

Conclusions

I hope you liked the examples I presented today and learned something from them.

I am sure that if you attend JuliaCon Local Eindhoven 2023 you are going to
see a lot of highly informative and inspirational talks there!