Category Archives: Julia

Exploring Julia 1.10 – Key Features and Updates

By: Great Lakes Consulting

Re-posted from: https://blog.glcs.io/julia-1-10

A new version of the Julia programming languagewas just released!Version 1.10 is now the latest stable version of Julia.

This release is a minor release,meaning it includes language enhancementsand bug fixesbut should also be fully compatiblewith code written in previous Julia versions(from version 1.0 and onward).

In this post,we will check out some of the features and improvementsintroduced in this newest Julia version.Read the full post,or click on the links belowto jump to the features that interest you.

Improved Latency, or Getting Started Faster

Julia 1.10 has improved latency,which means you can get started faster.

Two sources of latencyhistorically have been slow in Julia:package loadingand just-in-time code compilation.A classic example where this latency was readily noticeablewas when trying to create a plot;consequently,this latency often is calledthe time to first plot (TTFP),or how long one has to waitbefore seeing a plot.

Note that the TTFP issue exists in the first placebecause Julia was designedwith a trade-off in mind:by taking the time to compile a functionthe first time it is called,subsequent calls to the functioncan run at speeds comparable to C.This, however, leads to increased latencyon the first call.

Recent Julia versions have been tackling this issue,and Julia 1.10 further improves latency.

Below is a screenshot of a slide sharedduring the State of Julia talk at JuliaCon 2023.It shows how the time it takesto load Plots.jland then call plotdecreases when moving from Julia 1.8to Julia 1.9and then to Julia 1.10(in this case, Julia 1.10wasn’t released yet,so the alpha version was used).

Improved latency

I saw similar results on my computercomparing Julia 1.9.4 to Julia 1.10.0-rc1(the first release candidate of Julia 1.10):

# Julia 1.9.4julia> @time using Plots  1.278046 seconds (3.39 M allocations: 194.392 MiB, 10.10% gc time, 6.28% compilation time: 89% of which was recompilation)julia> @time display(plot(1:10))  0.365514 seconds (246.08 k allocations: 16.338 MiB, 58.76% compilation time: 10% of which was recompilation)# Julia 1.10.0-rc1julia> @time using Plots  0.713279 seconds (1.42 M allocations: 97.684 MiB, 3.30% gc time, 15.26% compilation time: 86% of which was recompilation)julia> @time display(plot(1:10))  0.257097 seconds (247.72 k allocations: 17.621 MiB, 6.29% gc time, 81.56% compilation time: 9% of which was recompilation)

It’s amazing how much latencyhas been improved!

Better Error Messages

Julia 1.10 now uses JuliaSyntax.jlas the default parser,replacing the old Lisp-based parser.

Having a new parserdoesn’t change how the language runs,but the new parserdoes improve error messages,enabling easier debuggingand creating a lower barrier to entryfor new Julia users.

As an example,consider the following buggy code:

julia> count = 0;julia> for i = 1:10           count++       end

Can you spot the error?

Julia 1.9 gives the following error message:

ERROR: syntax: unexpected "end"

Julia 1.10 gives the following:

ERROR: ParseError:# Error @ REPL[2]:3:1    count++end  invalid identifier

There are at least three improvementsto the error message:

  1. The file location of the offending tokenis prominently displayed.(REPL[2]:3:1 meansthe second REPL entry,the third line,and the first character.This would be replacedwith a file path and line and character numbersif the code were run in a file.)
  2. The specific offending tokenis pointed out with some context.
  3. It is now clear that an identifier(i.e., a variable name)was expectedafter count++.(Note that ++ is a user-definableinfix operator in Julia;so just as a + end is an error,so too is count ++ end.)

Improved error messagesare certainly a welcome addition!

Multithreaded Garbage Collection

Part of Julia’s garbage collectionis now parallelizedin Julia 1.10,resulting in faster garbage collection.

Below is a screenshot of a slide sharedduring the State of Julia talk at JuliaCon 2023.It shows the percentage of timea piece of code spentdoing garbage collectionin different Julia versions(here the master branch is a pre-release version of Julia 1.10).The takeaway is that using threadsdecreased garbage collection time!

Faster garbage collection

The parallelization is implementedusing threads,and the number of threadsavailable for garbage collectioncan be specified when starting Juliawith the command line argument --gcthreads.For example,to use four threads for garbage collection:

julia --gcthreads=4

By default,--gcthreads is halfthe total number of threadsJulia is started with.

Experiment with different numbersof garbage collection threadsto see what works bestfor your code.

Timing Package Precompilation

Timing how long individual packages take to precompileis now easily achieved withPkg.precompile(timing = true).

In Julia 1.9,Pkg.precompile reported just the overall time precompilation took:

julia> using Pkg; Pkg.precompile()Precompiling project...  20 dependencies successfully precompiled in 91 seconds. 216 already precompiled.

Pkg.precompile()(without the timing option)behaves the same in Julia 1.10.But now there is the optionto report the precompilation timefor individual packages:

julia> using Pkg; Pkg.precompile(timing = true)Precompiling project...  19850.9 ms   DataFrames   2858.4 ms   Flux  26206.5 ms   Plots  3 dependencies successfully precompiled in 49 seconds. 235 already precompiled.

Now it is easyto see what packagesprecompile faster than others!

Broadcasting Defined for CartesianIndex

Julia 1.10 now defines broadcastingfor CartesianIndex objects.

A CartesianIndex is a wayto represent an indexinto a multidimensional arrayand can be useful forworking with loops over arrays of arbitrary dimensionality.

Suppose we define the following:

julia> indices = [CartesianIndex(2, 3), CartesianIndex(4, 5)];julia> I = CartesianIndex(1, 1);

In Julia 1.9,attempting to broadcast over a CartesianIndex(for example, indices .+ I)resulted in the following error:

ERROR: iteration is deliberately unsupported for CartesianIndex.

With broadcasting defined,where previously we would have to wrapthe CartesianIndex in a Tuple(e.g., indices .+ (I,)),now the following works:

julia> indices .+ I2-element Vector{CartesianIndex{2}}: CartesianIndex(3, 4) CartesianIndex(5, 6)

Summary

In this post,we learned aboutsome of the new featuresand improvementsintroduced in Julia 1.10.Curious readers cancheck out the release notesfor the full list of changes.

What are you most excited aboutin Julia 1.10?Let us know in the comments below!

Additional Links

Exploring Julia 1.10 – Key Features and Updates

By: Steven Whitaker

Re-posted from: https://glcs.hashnode.dev/julia-1-10

A new version of the Julia programming languagewas just released!Version 1.10 is now the latest stable version of Julia.

This release is a minor release,meaning it includes language enhancementsand bug fixesbut should also be fully compatiblewith code written in previous Julia versions(from version 1.0 and onward).

In this post,we will check out some of the features and improvementsintroduced in this newest Julia version.Read the full post,or click on the links belowto jump to the features that interest you.

If you are new to Julia(or just need a refresher),feel free to check out our Julia tutorial series,beginning with how to install Julia and VS Code.

Improved Latency, or Getting Started Faster

Julia 1.10 has improved latency,which means you can get started faster.

Two sources of latencyhistorically have been slow in Julia:package loadingand just-in-time code compilation.A classic example where this latency was readily noticeablewas when trying to create a plot;consequently,this latency often is calledthe time to first plot (TTFP),or how long one has to waitbefore seeing a plot.

Note that the TTFP issue exists in the first placebecause Julia was designedwith a trade-off in mind:by taking the time to compile a functionthe first time it is called,subsequent calls to the functioncan run at speeds comparable to C.This, however, leads to increased latencyon the first call.

Recent Julia versions have been tackling this issue,and Julia 1.10 further improves latency.

Below is a screenshot of a slide sharedduring the State of Julia talk at JuliaCon 2023.It shows how the time it takesto load Plots.jland then call plotdecreases when moving from Julia 1.8to Julia 1.9and then to Julia 1.10(in this case, Julia 1.10wasn’t released yet,so the alpha version was used).

Improved latency

I saw similar results on my computercomparing Julia 1.9.4 to Julia 1.10.0-rc1(the first release candidate of Julia 1.10):

# Julia 1.9.4julia> @time using Plots  1.278046 seconds (3.39 M allocations: 194.392 MiB, 10.10% gc time, 6.28% compilation time: 89% of which was recompilation)julia> @time display(plot(1:10))  0.365514 seconds (246.08 k allocations: 16.338 MiB, 58.76% compilation time: 10% of which was recompilation)# Julia 1.10.0-rc1julia> @time using Plots  0.713279 seconds (1.42 M allocations: 97.684 MiB, 3.30% gc time, 15.26% compilation time: 86% of which was recompilation)julia> @time display(plot(1:10))  0.257097 seconds (247.72 k allocations: 17.621 MiB, 6.29% gc time, 81.56% compilation time: 9% of which was recompilation)

It’s amazing how much latencyhas been improved!

Better Error Messages

Julia 1.10 now uses JuliaSyntax.jlas the default parser,replacing the old Lisp-based parser.

Having a new parserdoesn’t change how the language runs,but the new parserdoes improve error messages,enabling easier debuggingand creating a lower barrier to entryfor new Julia users.

As an example,consider the following buggy code:

julia> count = 0;julia> for i = 1:10           count++       end

Can you spot the error?

Julia 1.9 gives the following error message:

ERROR: syntax: unexpected "end"

Julia 1.10 gives the following:

ERROR: ParseError:# Error @ REPL[2]:3:1    count++end  invalid identifier

There are at least three improvementsto the error message:

  1. The file location of the offending tokenis prominently displayed.(REPL[2]:3:1 meansthe second REPL entry,the third line,and the first character.This would be replacedwith a file path and line and character numbersif the code were run in a file.)
  2. The specific offending tokenis pointed out with some context.
  3. It is now clear that an identifier(i.e., a variable name)was expectedafter count++.(Note that ++ is a user-definableinfix operator in Julia;so just as a + end is an error,so too is count ++ end.)

Improved error messagesare certainly a welcome addition!

Multithreaded Garbage Collection

Part of Julia’s garbage collectionis now parallelizedin Julia 1.10,resulting in faster garbage collection.

Below is a screenshot of a slide sharedduring the State of Julia talk at JuliaCon 2023.It shows the percentage of timea piece of code spentdoing garbage collectionin different Julia versions(here the master branch is a pre-release version of Julia 1.10).The takeaway is that using threadsdecreased garbage collection time!

Faster garbage collection

The parallelization is implementedusing threads,and the number of threadsavailable for garbage collectioncan be specified when starting Juliawith the command line argument --gcthreads.For example,to use four threads for garbage collection:

julia --gcthreads=4

By default,--gcthreads is halfthe total number of threadsJulia is started with.

Experiment with different numbersof garbage collection threadsto see what works bestfor your code.

Timing Package Precompilation

Timing how long individual packages take to precompileis now easily achieved withPkg.precompile(timing = true).

In Julia 1.9,Pkg.precompile reported just the overall time precompilation took:

julia> using Pkg; Pkg.precompile()Precompiling project...  20 dependencies successfully precompiled in 91 seconds. 216 already precompiled.

Pkg.precompile()(without the timing option)behaves the same in Julia 1.10.But now there is the optionto report the precompilation timefor individual packages:

julia> using Pkg; Pkg.precompile(timing = true)Precompiling project...  19850.9 ms   DataFrames   2858.4 ms   Flux  26206.5 ms   Plots  3 dependencies successfully precompiled in 49 seconds. 235 already precompiled.

Now it is easyto see what packagesprecompile faster than others!

Broadcasting Defined for CartesianIndex

Julia 1.10 now defines broadcastingfor CartesianIndex objects.

A CartesianIndex is a wayto represent an indexinto a multidimensional arrayand can be useful forworking with loops over arrays of arbitrary dimensionality.

Suppose we define the following:

julia> indices = [CartesianIndex(2, 3), CartesianIndex(4, 5)];julia> I = CartesianIndex(1, 1);

In Julia 1.9,attempting to broadcast over a CartesianIndex(for example, indices .+ I)resulted in the following error:

ERROR: iteration is deliberately unsupported for CartesianIndex.

With broadcasting defined,where previously we would have to wrapthe CartesianIndex in a Tuple(e.g., indices .+ (I,)),now the following works:

julia> indices .+ I2-element Vector{CartesianIndex{2}}: CartesianIndex(3, 4) CartesianIndex(5, 6)

Summary

In this post,we learned aboutsome of the new featuresand improvementsintroduced in Julia 1.10.Curious readers cancheck out the release notesfor the full list of changes.

What are you most excited aboutin Julia 1.10?Let us know in the comments below!

Additional Links

Evolution of DataFrames.jl load time

By: Blog by Bogumił Kamiński

Re-posted from: https://bkamins.github.io/julialang/2023/12/29/dfload.html

Introduction

Julia 1.10 has just been released. You can find a summary
of the improvements in this post. Seeing the release
I was curious how the DataFrames.jl load time changed in it.

In this post I want to show you how long it takes to install
and to load DataFrames.jl.

Experiment setup

The tests were performed on a laptop with i7-1250U processor, 16 GB of RAM and under Windows 11 Pro.

I tested the following Julia versions: 1.6.7, 1.7.3, 1.8.5, 1.9.4, 1.10.0 (the current release),
and additionally the latest development version 1.11.0-DEV.1165.

For all setups I have cleaned all Julia temporary files and performed two operations:

  • installation fo DataFrames.jl using using Pkg; Pkg.add("DataFrames") operation; here I collected the reported total precompilation time;
  • loading DataFrames.jl using @time using DataFrames as the only operation in a fresh Julia session; here I recorded load time and memory used.

The version of DataFrames.jl used in tests is 1.6.1.

Experiment results

I have collected the results of my test in the following table:

Julia version Pkg.add("DataFrames") @time using DataFrames
1.6.7 55 s. 1.19 s., 2.64 M allocations: 190 MiB
1.7.3 44 s. 1.20 s., 2.63 M allocations: 187 MiB
1.8.5 43 s. 2.04 s., 4.76 M allocations: 338 MiB
1.9.4 79 s. 1.23 s., 1.55 M allocations: 92 MiB
1.10.0 60 s. 0.79 s., 579 k allocations: 44 MiB
1.11.0-DEV.1165 72 s. 0.54 s., 542 k allocations: 35 MiB

Conclusions

The major things that we can conclude are the following:

  • If we excluded Julia 1.8 the load time of DataFrames.jl constantly goes down with newer versions; the same applies to memory usage.
  • The precompilation time increases; in general you have to wait for around one minute to get all dependencies of DataFrames.jl compiled. Fortunately, this is a one-time cost.

In summary: Julia 1.10 brings a significant decrease of DataFrames.jl load time. Also we can see that the upcoming Julia 1.11 can be expected to be even faster. This is great news.

Happy New Year!