By: Dean Markwick's Blog -- Julia

Re-posted from: https://dm13450.github.io/2023/09/27/Easy-Reinforcement-Learning-and-The-Multi-Armed-Bandits.html

This is another draft that’s been sitting on my laptop and I was sitting on the Eurostar on the way to TradeTech and thought I’d try and formalise it into a blog post. This is all about reinforcement learning and a basic model that can be easily implemented in Julia. This post is me walking through and implementing the 2nd chapter of Reinforcement Learning: An Introduction.

Enjoy these types of posts? Then sign up for my newsletter.

Reinforcement learning is a pillar of machine learning and it combines the use of data and learning how to make a better decision automatically. One of the basic models in reinforcement learning is the multi-armed bandit. A bit of an anachronistic name, but the single-armed bandit refers to a casino game where you pull the lever (or push a button), some cassettes roll round and you might win a prize.

The multi-armed bandit is an extension to this type of game and means we have different levers we can pull that lead to a different reward. The reward depends on the lever pulled.

This simple mental model is surprisingly applicable to lots of different problems and it can act as a good approximation to whatever you are trying to solve. For example, let’s use an advertising example. You have multiple adverts that you display to try and get people to click through to your website. Each time a page loads you can load one advertisement, you then record how many people click on that advert and use that to decide which advert to show next. With each page load you decide, do I show the most succesful advert so far or try a new advert to see how that performs? Over time you will find out which advert performs the best and show that as much as possible to get as many clicks.

A Simple Bandit

Imagine we have a multi-armed bandit machine, where we pull a lever and get a reward. The reward depends on the lever pulled, how do we learn what the best lever is?

First let’s build our bandit. We will have 5 levers and the reward will be a sample from a normal distribution where each lever will have a random mean and standard deviation.

using Plots, StatsPlots
using Distributions

nLevers = 5

rewardMeans = rand(Normal(0, 3), nLevers)
rewardSD = rand(Gamma(2, 2), nLevers)

hcat(rewardMeans, rewardSD)

5×2 Matrix{Float64}:
 -4.7724   5.88533
 -4.60967  0.627556
 -5.96987  1.14465
  8.96919  3.80253
  2.11311  4.84983

These are the parameters of our levers in our bandit, so lets look at the distribution of the rewards.

density(rand(Normal(rewardMeans[1], rewardSD[1]), 1000), label = "Lever 1")

for i in 2:nLevers
    density!(rand(Normal(rewardMeans[i], rewardSD[i]), 1000), label = "Lever " * string(i))
end
plot!()

So our levers giving us a sample from a normal distribution is illustrated above. The 4th lever looks like the best as it has the most likely chance of getting a positive value and has the wider tail too. As we are talking about rewards, large positive values are better.

So given we have a process of pulling a lever and getting a reward, how do we learn what the best lever is and importantly as quickly as possible?

Like all good statistics problems, we start with the most basic model and start pulling levers randomly.

The Random Strategy

Just pull a random lever every time. Nothing is being learned here though and we are just demonstrating how the problem setup works. With each play we generate a random integer that corresponds to the lever, pull the lever (draw a random normal variable with mean/deviation of that lever), record what lever was pulled and the reward amount. Then repeat several times.

function random_learner(rewardMeans, rewardSD, nPlays)

    nLevers = length(rewardMeans)
    
    selectedLever = zeros(Int64, nPlays)
    rewards = zeros(nPlays)

    cumSelection = zeros(Int64, nLevers)
    cumRewards = zeros(nLevers)
    
    optimalChoice = Array{Bool}(undef, nPlays)

    bestLever = findmax(rewardMeans)[2]
    
    for i = 1:nPlays
    
        selectedLever[i] = rand(1:nLevers)
        
        optimalChoice[i] = selectedLever[i] == bestLever
        
        rewards[i] = rand(Normal(rewardMeans[selectedLever[i]], rewardSD[selectedLever[i]]))
    
        cumSelection[selectedLever[i]] += 1
        cumRewards[selectedLever[i]] += rewards[i]
    
    end
    return selectedLever, rewards, cumSelection, cumRewards, optimalChoice
end

We run this learner for 1,000 steps and look at the number of times each lever is pulled.

randomStrat = random_learner(rewardMeans, rewardSD, 1000);

histogram(randomStrat[1], label = "Number of Time Lever Pulled")

Each of the levers is pulled a roughly equal amount of times, with no learning, just randomly pulling.
Moving on, how do we learn?

Action Value Methods

Reinforcement learning is about balancing the explore/exploit set-up of the problem. We need to sample each of the levers and work out what kind of rewards they provide and then use that information to inform our next decision.

For each iteration, we randomly decide if we will pull any lever or do we use the old information to choose our best guess at the best lever. Our information in this case is the rolling average of the reward each time we pulled the lever. This is called a greedy learner. It’s just doing its best with what it knows and has no real ability to decide whether to explore a new lever.

The probability of choosing a random lever is called the learning rate (\(\eta\)) and controls how often we make the perceived optimal choice. A high value of \(\eta\) means lots of exploring (learning) and a low value restricts the learning and means we pull the (perceived) best lever each time. So if we had many levers and a low learning rate it is possible that we never find the globally optimal lever and instead just stick to the locally optimal lever, hence why it is called a greedy learner, it can get stuck.

function greedy_learner(rewardMeans, rewardSD, nPlays, eta)

    nLevers = length(rewardMeans)
    
    selectedLever = zeros(Int64, nPlays)
    rewards = zeros(nPlays)

    cumSelection = zeros(Int64, nLevers)
    cumRewards = zeros(nLevers)
    
    optimalChoice = Array{Bool}(undef, nPlays)
    
    bestLever = findmax(rewardMeans)[2]

    for i = 1:nPlays

        if rand() < eta
            selectedLever[i] = rand(1:nLevers)
        else 
            q = cumRewards ./ cumSelection
            q[isnan.(q)] .= 0
            selectedLever[i] = findmax(q)[2]
        end
        
        optimalChoice[i] = selectedLever[i] == bestLever
        
        rewards[i] = rand(Normal(rewardMeans[selectedLever[i]], rewardSD[selectedLever[i]]))

        cumSelection[selectedLever[i]] += 1
        cumRewards[selectedLever[i]] += rewards[i]

    end
    return selectedLever, rewards, cumSelection, cumRewards, optimalChoice
end

Again, we can run it for 1,000 steps and we set our learning rate to 0.5.

greedyStrat = greedy_learner(rewardMeans, rewardSD, 1000, 0.5)

histogram(greedyStrat[1], label = "Number of Time Lever Pulled", legend = :topleft)

This has done what we thought, it has selected the 4th lever that we thought looked the best from the distribution. So we’ve learned something, hooray!

Varying in the Learning Rate

The \(\eta\) parameter was set to 0.5 above, but how does varying change the outcome? To explore this we will do multiple runs of multiple plays of the game and also increase the number of levers. For each run, we will generate a new set of reward averages/standard deviations and run the random learner and the greedy learner with different \(\eta\).

nRuns = 2000
nPlays = 1000
nLevers = 10

optimalLevel = zeros(nRuns)

randomRes = Array{Tuple}(undef, nRuns)
greedyRes = Array{Tuple}(undef, nRuns)
greedyRes05 = Array{Tuple}(undef, nRuns)
greedyRes01 = Array{Tuple}(undef, nRuns)
greedyRes001 = Array{Tuple}(undef, nRuns)
greedyRes0001 = Array{Tuple}(undef, nRuns)


for i=1:nRuns
    rewardMeans = rand(Normal(0, 1), nLevers)
    rewardSD = ones(nLevers)
   
    randomRes[i] = random_learner(rewardMeans, rewardSD, nPlays)
    greedyRes[i] = greedy_learner(rewardMeans, rewardSD, nPlays, 0)
    greedyRes05[i] = greedy_learner(rewardMeans, rewardSD, nPlays, 0.5)
    greedyRes01[i] = greedy_learner(rewardMeans, rewardSD, nPlays, 0.1)
    greedyRes001[i] = greedy_learner(rewardMeans, rewardSD, nPlays, 0.01)
    greedyRes0001[i] = greedy_learner(rewardMeans, rewardSD, nPlays, 0.001)
    
    optimalLevel[i] = findmax(rewardMeans)[2]
    
end

For each of the runs we have the evolution of the reward, so we want to take the average of the reward on each time step and see how that evolves with each play of the game.

randomAvg = mapreduce(x-> x[2], +, randomRes) ./ nRuns
greedyAvg = mapreduce(x-> x[2], +, greedyRes) ./ nRuns
greedyAvg01 = mapreduce(x-> x[2], +, greedyRes01) ./ nRuns
greedyAvg09 = mapreduce(x-> x[2], +, greedyRes05) ./ nRuns
greedyAvg001 = mapreduce(x-> x[2], +, greedyRes001) ./ nRuns;
greedyAvg0001 = mapreduce(x-> x[2], +, greedyRes0001) ./ nRuns;

And plotting the average reward over time.

plot(1:nPlays, randomAvg, label="Random", legend = :bottomright, xlabel = "Time Step", ylabel = "Average Reward")
plot!(1:nPlays, greedyAvg, label="0")
plot!(1:nPlays, greedyAvg05, label="0.5")
plot!(1:nPlays, greedyAvg01, label="0.1")
plot!(1:nPlays, greedyAvg001, label="0.01")
plot!(1:nPlays, greedyAvg0001, label="0.001")

Good to see that all the greedy learners outperform the random learner, so that algorithm is doing something.
If we focus on the gready learners we see how the learning rates changes performances.

plot(1:nPlays, greedyAvg, label="0", legend=:bottomright, xlabel = "Time Step", ylabel = "Average Reward")
plot!(1:nPlays, greedyAvg01, label="0.1")
plot!(1:nPlays, greedyAvg001, label="0.01")
plot!(1:nPlays, greedyAvg0001, label="0.001")

This is an interesting result! When \(\eta = 0\) we see that it never reaches as high as the other learning rates. So when \(\eta = 0\) we never explore the other options, we just select what we think is the best one from history and never stray away from our beliefs. This ultimately hurts us because if we don’t get the best level on the first try then we are stuck in a suboptimal. Likewise, when the learning rate is very low, it doesn’t get much better, so this shows there is always value in exploring the options.

Philosophically, this shows that with any procedure you need to iterate through different configurations and explore the outcomes rather than sticking with what you believe is optimal.

scatter([0, 0.5, 0.1,0.01, 0.001], 
    map(x-> mean(x[750:1000]), [greedyAvg, greedyAvg05, greedyAvg01, greedyAvg001, greedyAvg0001]),
    xlabel="Learning Rate",
    ylabel = "Converged Reward", legend=:none)

The learning rate looks like it is optimal around 0.1. You can do a grid search to see how the overall behaviour changes in terms of both the speed of convergence to the final state and how good that final reward state is.

Speed it Up – Incremental Implementation

We can improve the above implementation by just saving memory and CPU cycles by doing ‘online learning’ of the rewards and using that to drive the selection. We create one matrix $$Q$, update it with the average reward of each lever and use the maximum of each iteration to select our lever if we are not exploring.

function greedy_learner_incremental(rewardMeans, rewardSD, nPlays, eta)

    nLevers = length(rewardMeans)
    
    selectedLever = zeros(Int64, nPlays)
    rewards = zeros(nPlays)

    cumSelection = zeros(Int64, nLevers)
    cumRewards = zeros(nLevers)

    Q = zeros((nPlays+1, nLevers))
    rewardsArray = zeros(nLevers)
    
    optimalChoice = Array{Bool}(undef, nPlays)
    
    bestLever = findmax(rewardMeans)[2]
    
    for i = 1:nPlays

        if rand() < eta
            selectedLever[i] = rand(1:nLevers)
        else 
            selectedLever[i] = findmax(Q[i,:])[2]
        end
        
        optimalChoice[i] = selectedLever[i] == bestLever
        
        reward = rand(Normal(rewardMeans[selectedLever[i]], rewardSD[selectedLever[i]]))
        rewards[i] = reward
        rewardsArray[selectedLever[i]] = reward
        
        cumSelection[selectedLever[i]] += 1
        cumRewards[selectedLever[i]] += reward

        Q[i+1, :] = Q[i, :] + (1/i) * (rewardsArray - Q[i,:])
        
    end
    return selectedLever, rewards, cumSelection, cumRewards, optimalChoice
end

Using the normal Julia benchmarking tools we can get a good idea if this rewrite has changed anything materially.

using BenchmarkTools

oldImp = @benchmark greedy_learner(rewardMeans, rewardSD, nPlays, 0.1)
newImp = @benchmark greedy_learner_incremental(rewardMeans, rewardSD, nPlays, 0.1)

judge(median(oldImp), median(newImp))

BenchmarkTools.TrialJudgement: 
  time:   -43.91% => improvement (5.00% tolerance)
  memory: -70.15% => improvement (1.00% tolerance)

It’s 50% faster and uses 70% less memory, so a good optimisation.

Conclusion

This is the basic intro to reinforcement learning but a good foundation for how to think about these problems. The main step is going from data to decisions and how to update the decisions you make each time. You need to make sure you explore the problem space as otherwise you never know how much better some other options might be.

By: Steven Whitaker

Re-posted from: https://blog.glcs.io/variables-and-functions

Variables and functions
are the building blocks
of any programmer’s code.
Variables allow computations to be reused,
and functions help keep code organized.

In this post,
we will cover some of the basics
of variables and functions
in Julia,
a relatively new,
free, and open-source programming language.
In particular,
we will discuss what a variable is,
what sorts of data
can be assigned to a variable,
and
how to define and use functions.

Variables

A variable is a label
used to refer to an object.

julia> a = 1
1

In the above code snippet,
we assigned a value of 1
to a variable called a.
Now we can use a in other expressions,
and the value of a (1 in this case)
will be used.

julia> a + 2
3

We can also reassign variables.

julia> a = 4
4

julia> a = "Hello"
"Hello"

julia> a = [1, 2, 3]
3-element Vector{Int64}:
 1
 2
 3

We can even use Unicode characters!
We can write many math symbols
by typing the corresponding LaTeX command
and then pressing <tab>.
Here,
we assign
(a Julia constant equal to \( \pi \)
and typed with \pi<tab>)
to
(typed with \theta<tab>).

julia>  = 
 = 3.1415926535897...

Variables Are Labels

One important thing to remember about variables
is that they are labels for data,
not the data itself.
Let’s illustrate what that means.
At this point,
a refers to a Vector.
We will create another variable, b,
and assign it the value of a.

julia> b = a
3-element Vector{Int64}:
 1
 2
 3

Now let’s change one of the elements of b.

julia> b[1] = 100; b
3-element Vector{Int64}:
 100
   2
   3

We didn’t change a,
so it should be the same as before, right?
Nope!

julia> a
3-element Vector{Int64}:
 100
   2
   3

What happened?
Remember, a is just a label
for some data (a Vector).
When we created b,
we created a new label
for the same data.
Both a and b refer
to the same data,
so modifying one modifies the other.

If you want b
to have the same values as a
but refer to different data,
use copy.

julia> b = copy(a)
3-element Vector{Int64}:
 100
   2
   3

julia> b[1] = 1; b
3-element Vector{Int64}:
 1
 2
 3

julia> a
3-element Vector{Int64}:
 100
   2
   3

Now that we know
how to create variables,
let’s learn about
some basic types of data
we can assign to variables.

Basic Types

Julia has many basic data types.

integer = 9000
floating_point = 3.14
boolean = true
imaginary = 1 + 2im
rational = 4//3
char = 'x'
str = "a string"
array = [1.0, 2.0]

Integers and floating-point numbers
can be expressed
with different numbers of bits.

Int64, Int32, Int16       # and more
Float64, Float32, Float16 # and more

By default,
integer numbers
(technically, literals)
are of type Int64 on 64-bit computers
or of type Int32 on 32-bit computers.
(Note that Int is shorthand for Int64 or Int32
for 64-bit or 32-bit computers, respectively.
Therefore, all integer literals are of type Int.)

On the other hand,
floating-point numbers (literals)
of the form 3.14 or 2.3e5
are of type Float64 on all computers,
while those of the form 2.3f5
are of type Float32.

To use different numbers of bits,
just use the appropriate constructor.

julia> Int16(20)
20

julia> Float16(1.2)
Float16(1.2)

Basic Operations

Now we will cover
some basic operations.
This is by no means an exhaustive list;
check out the Julia documentation
for more details.

# Math
addition = 1 + 2.0
subtraction = 1 - 1
multiplication = 3 * 4//3
division = 6 / 4
integer_division = 6  4 # Type \div<tab>
power = 2^7

# Boolean
not = !false
and = true && not
or = not || and

# Comparison
equality = addition == 1
greater = division > integer_division
chained = addition < subtraction <= power

# Strings
string_concatenation = "hi " * "there"
string_interpolation = "1 - 1 = $subtraction"
string_indexing = string_interpolation[5]
substring = string_concatenation[4:end]
parsing = parse(Int, string_indexing)

# Arrays
a = [1, 2, 3]
b = [4, 5, 6]
concat_vert = [a; b]
concat_horiz = [a b]
vector_indexing = b[2]
vector_range_indexing = b[1:2]
matrix_indexing = concat_horiz[2:3,1]
elementwise1 = a .+ 1
elementwise2 = a .- b

# Displaying
print(addition)
println(string_concatenation)
@show not
@info "some variables" power a

Function Basics

Some of the basic operations we saw above,
e.g., parse and print,
were functions.
As demonstrated above,
functions are called
using the following familiar syntax:

func()           # For no inputs
func(arg1)       # For one input
func(arg1, arg2) # For two inputs
# etc.

Note that just writing the function name
(i.e., without parentheses)
is valid syntax, but it is not a function call.
In this case,
the function name is treated essentially like a variable,
meaning, for example, it can be used as an input
to another function.

For example,
one way to compute the sum
of the absolute value
of an array of numbers
is as follows:

julia> sum(abs, [-1, 0, 1])
2

Here,
the function abs is not being called (by us)
but is used as an input to the function sum
as if it were a variable.

Function Vectorization

Often,
we have a function
that operates on a single input
that we want to apply
to every element of an array.
Julia provides a convenient syntax
to do so:
just add a dot (.).
For example,
the following takes the absolute value
of every array element:

julia> abs.([-1, 0, 1])
3-element Vector{Int64}
 1
 0
 1

There is also a function, map,
that does the same thing
in this example:

julia> map(abs, [-1, 0, 1])
3-element Vector{Int64}
 1
 0
 1

(Note, however,
that map and the dot syntax
are not always interchangeable.)

Defining Functions

When writing Julia code,
it is convenient
to place code inside of functions.
There are two main syntaxes
for creating a function.

1. Using the function keyword:

   function myfunc(x)
       return x + 1
   end
   

2. Using the assignment form:

   myfunc2(x, y) = x + y
   

Optional Arguments

Sometimes we want a function
to have optional inputs.
The syntax for specifying optional arguments is

function myfunc3(required, optional = "hello")
    println(required, optional)
end

Here,
optional is optional
and has a default value of "hello"
if not provided by the caller.

julia> myfunc3("say ")
say hello

julia> myfunc3("see you ", "later")
see you later

Keyword Arguments

Another way to specify optional arguments
is to use keyword arguments.
The syntax is almost the same
as regular optional arguments,
except we use a semicolon (;) instead of a comma (,).

function myfunc4(x; y = 3)
    return x * y
end

Here,
y is optional,
but to specify it
we need to use the keyword y.

julia> myfunc4(2)
6

julia> myfunc4(2; y = 10)
20

julia> myfunc4(2, 10)
ERROR: MethodError: no method matching myfunc4(::Int64, ::Int64)

When calling myfunc4
we can also use a comma
when specifying y.

julia> myfunc4(2, y = 1)
2

Returning Multiple Values

Sometimes we need a function
to return multiple values.
The way to do this in Julia
is to return a Tuple.
Here’s an example:

function plusminus1(x)
    return (x + 1, x - 1)
end

Then multiple variables can be assigned at once.

julia> (plus1, minus1) = plusminus1(1)
(2, 0)

julia> plus1
2

julia> minus1
0

Note that taking the output
of a function with multiple return values
and assigning it to a single variable
will assign that variable the whole Tuple of outputs.
The following code illustrates this
and shows how to return just one output:

julia> both = plusminus1(1);

julia> both
(2, 0)

julia> (one,) = plusminus1(1);

julia> one
2

(Note, however, that in this last case
the second output is still computed;
it is just immediately discarded,
so there are no savings in computation.)

Vectorizing a Function with Multiple Return Values

Vectorizing a function with multiple return values
requires a bit more work.
For this example,
we will use the sincos function
that computes the sine and cosine simultaneously.
We can still use the dot syntax,
but we might be tempted to try the following:

julia> (s, c) = sincos.([0, /2, ]);

julia> s
(0.0, 1.0)

julia> c
(1.0, 6.123233995736766e-17)

Here, s has the value of sincos(0),
not the value of sin.([0, /2, ])
like we might have expected.

Instead, we can do the following:

julia> sc = sincos.([0, /2, ])
3-element Vector{Tuple{Float64, Float64}}:
 (0.0, 1.0)
 (1.0, 6.123233995736766e-17)
 (1.2246467991473532e-16, -1.0)

julia> s = first.(sc)
3-element Vector{Float64}:
 0.0
 1.0
 1.2246467991473532e-16

julia> c = last.(sc)
3-element Vector{Float64}:
  1.0
  6.123233995736766e-17
 -1.0

(Note that instead of using first or last,
we could write it this way:
output_i = getindex.(sc, i).
This way also works for functions
that return more than two values.)

Summary

In this post,
we learned about what a variable is
and some basic data types.
We also learned about
how to define and use functions.

There is a lot more we could cover
about these topics,
so if you want to learn more,
check out the links below,
or write a comment below
letting us know what additional concepts or topics
you would like to see!

Understand variables and functions in Julia?
Move on to the
next post to learn how to master the Julia REPL!
Or,
feel free to take a look
at our other Julia tutorial posts!

Additional Links

• Julia documentation
  • For variables
  • For integer and floating-point numbers
  • For complex and rational numbers
  • For basic operations
  • For strings
  • For functions

By: Steven Whitaker

Re-posted from: https://glcs.hashnode.dev/variables-and-functions

Variables and functionsare the building blocksof any programmer’s code.Variables allow computations to be reused,and functions help keep code organized.

In this post,we will cover some of the basicsof variables and functionsin Julia,a relatively new,free, and open-source programming language.In particular,we will discuss what a variable is,what sorts of datacan be assigned to a variable,andhow to define and use functions.

Variables

A variable is a labelused to refer to an object.

julia> a = 11

In the above code snippet,we assigned a value of 1to a variable called a.Now we can use a in other expressions,and the value of a (1 in this case)will be used.

julia> a + 23

We can also reassign variables.

julia> a = 44julia> a = "Hello""Hello"julia> a = [1, 2, 3]3-element Vector{Int64}: 1 2 3

We can even use Unicode characters!We can write many math symbolsby typing the corresponding LaTeX commandand then pressing <tab>.Here,we assign (a Julia constant equal to \( \pi \)and typed with \pi<tab>)to (typed with \theta<tab>).

julia>  =  = 3.1415926535897...

Variables Are Labels

One important thing to remember about variablesis that they are labels for data,not the data itself.Let’s illustrate what that means.At this point,a refers to a Vector.We will create another variable, b,and assign it the value of a.

julia> b = a3-element Vector{Int64}: 1 2 3

Now let’s change one of the elements of b.

julia> b[1] = 100; b3-element Vector{Int64}: 100   2   3

We didn’t change a,so it should be the same as before, right?Nope!

julia> a3-element Vector{Int64}: 100   2   3

What happened?Remember, a is just a labelfor some data (a Vector).When we created b,we created a new labelfor the same data.Both a and b referto the same data,so modifying one modifies the other.

If you want bto have the same values as abut refer to different data,use copy.

julia> b = copy(a)3-element Vector{Int64}: 100   2   3julia> b[1] = 1; b3-element Vector{Int64}: 1 2 3julia> a3-element Vector{Int64}: 100   2   3

Now that we knowhow to create variables,let’s learn aboutsome basic types of datawe can assign to variables.

Basic Types

Julia has many basic data types.

integer = 9000floating_point = 3.14boolean = trueimaginary = 1 + 2imrational = 4//3char = 'x'str = "a string"array = [1.0, 2.0]

Integers and floating-point numberscan be expressedwith different numbers of bits.

Int64, Int32, Int16       # and moreFloat64, Float32, Float16 # and more

By default,integer numbers(technically, literals)are of type Int64 on 64-bit computersor of type Int32 on 32-bit computers.(Note that Int is shorthand for Int64 or Int32for 64-bit or 32-bit computers, respectively.Therefore, all integer literals are of type Int.)

On the other hand,floating-point numbers (literals)of the form 3.14 or 2.3e5are of type Float64 on all computers,while those of the form 2.3f5are of type Float32.

To use different numbers of bits,just use the appropriate constructor.

julia> Int16(20)20julia> Float16(1.2)Float16(1.2)

Basic Operations

Now we will coversome basic operations.This is by no means an exhaustive list;check out the Julia documentationfor more details.

# Mathaddition = 1 + 2.0subtraction = 1 - 1multiplication = 3 * 4//3division = 6 / 4integer_division = 6  4 # Type \div<tab>power = 2^7# Booleannot = !falseand = true && notor = not || and# Comparisonequality = addition == 1greater = division > integer_divisionchained = addition < subtraction <= power# Stringsstring_concatenation = "hi " * "there"string_interpolation = "1 - 1 = $subtraction"string_indexing = string_interpolation[5]substring = string_concatenation[4:end]parsing = parse(Int, string_indexing)# Arraysa = [1, 2, 3]b = [4, 5, 6]concat_vert = [a; b]concat_horiz = [a b]vector_indexing = b[2]vector_range_indexing = b[1:2]matrix_indexing = concat_horiz[2:3,1]elementwise1 = a .+ 1elementwise2 = a .- b# Displayingprint(addition)println(string_concatenation)@show not@info "some variables" power a

Function Basics

Some of the basic operations we saw above,e.g., parse and print,were functions.As demonstrated above,functions are calledusing the following familiar syntax:

func()           # For no inputsfunc(arg1)       # For one inputfunc(arg1, arg2) # For two inputs# etc.

Note that just writing the function name(i.e., without parentheses)is valid syntax, but it is not a function call.In this case,the function name is treated essentially like a variable,meaning, for example, it can be used as an inputto another function.

For example,one way to compute the sumof the absolute valueof an array of numbersis as follows:

julia> sum(abs, [-1, 0, 1])2

Here,the function abs is not being called (by us)but is used as an input to the function sumas if it were a variable.

Function Vectorization

Often,we have a functionthat operates on a single inputthat we want to applyto every element of an array.Julia provides a convenient syntaxto do so:just add a dot (.).For example,the following takes the absolute valueof every array element:

julia> abs.([-1, 0, 1])3-element Vector{Int64} 1 0 1

There is also a function, map,that does the same thingin this example:

julia> map(abs, [-1, 0, 1])3-element Vector{Int64} 1 0 1

(Note, however,that map and the dot syntaxare not always interchangeable.)

Defining Functions

When writing Julia code,it is convenientto place code inside of functions.There are two main syntaxesfor creating a function.

1. Using the function keyword:

   function myfunc(x)    return x + 1end

2. Using the assignment form:

   myfunc2(x, y) = x + y

Optional Arguments

Sometimes we want a functionto have optional inputs.The syntax for specifying optional arguments is

function myfunc3(required, optional = "hello")    println(required, optional)end

Here,optional is optionaland has a default value of "hello"if not provided by the caller.

julia> myfunc3("say ")say hellojulia> myfunc3("see you ", "later")see you later

Keyword Arguments

Another way to specify optional argumentsis to use keyword arguments.The syntax is almost the sameas regular optional arguments,except we use a semicolon (;) instead of a comma (,).

function myfunc4(x; y = 3)    return x * yend

Here,y is optional,but to specify itwe need to use the keyword y.

julia> myfunc4(2)6julia> myfunc4(2; y = 10)20julia> myfunc4(2, 10)ERROR: MethodError: no method matching myfunc4(::Int64, ::Int64)

When calling myfunc4we can also use a commawhen specifying y.

julia> myfunc4(2, y = 1)2

Returning Multiple Values

Sometimes we need a functionto return multiple values.The way to do this in Juliais to return a Tuple.Here’s an example:

function plusminus1(x)    return (x + 1, x - 1)end

Then multiple variables can be assigned at once.

julia> (plus1, minus1) = plusminus1(1)(2, 0)julia> plus12julia> minus10

Note that taking the outputof a function with multiple return valuesand assigning it to a single variablewill assign that variable the whole Tuple of outputs.The following code illustrates thisand shows how to return just one output:

julia> both = plusminus1(1);julia> both(2, 0)julia> (one,) = plusminus1(1);julia> one2

(Note, however, that in this last casethe second output is still computed;it is just immediately discarded,so there are no savings in computation.)

Vectorizing a Function with Multiple Return Values

Vectorizing a function with multiple return valuesrequires a bit more work.For this example,we will use the sincos functionthat computes the sine and cosine simultaneously.We can still use the dot syntax,but we might be tempted to try the following:

julia> (s, c) = sincos.([0, /2, ]);julia> s(0.0, 1.0)julia> c(1.0, 6.123233995736766e-17)

Here, s has the value of sincos(0),not the value of sin.([0, /2, ])like we might have expected.

Instead, we can do the following:

julia> sc = sincos.([0, /2, ])3-element Vector{Tuple{Float64, Float64}}: (0.0, 1.0) (1.0, 6.123233995736766e-17) (1.2246467991473532e-16, -1.0)julia> s = first.(sc)3-element Vector{Float64}: 0.0 1.0 1.2246467991473532e-16julia> c = last.(sc)3-element Vector{Float64}:  1.0  6.123233995736766e-17 -1.0

(Note that instead of using first or last,we could write it this way:output_i = getindex.(sc, i).This way also works for functionsthat return more than two values.)

Summary

In this post,we learned about what a variable isand some basic data types.We also learned abouthow to define and use functions.

There is a lot more we could coverabout these topics,so if you want to learn more,check out the links below,or write a comment belowletting us know what additional concepts or topicsyou would like to see!

Understand variables and functions in Julia?Move on to thenext post to learn how to master the Julia REPL!Or,feel free to take a lookat our other Julia tutorial posts!

Additional Links

• Julia documentation
  • For variables
  • For integer and floating-point numbers
  • For complex and rational numbers
  • For basic operations
  • For strings
  • For functions