By: Dean Markwick's Blog -- Julia

Re-posted from: https://dm13450.github.io/2024/12/17/Importance-Sampling-Reinforcement-Learning-and-Getting-More-From-The-Data-You-Have.html

A new paper hit my feed Choosing trading strategies in electronic execution using
importance sampling. I’ve only encountered sampling as part of a statistical computing course as part of my PhD, and I had never strayed away from Monte Carlo sampling, but this practical example provided an intuitive understanding of its importance and utility.

Enjoy these types of posts? Then you should sign up for my newsletter.

The key tenet of the paper is to use the data you have to evaluate a strategy you are considering without actually running the new strategy in production. In real life, changing something like these strategies can take a long time, with limited upside but unlimited downside if it all goes wrong.

This blog post will run through the paper and replicate the main themes in Julia. I believe the author is a Julia user too, I remember enjoying their JuliaCon talk about high-frequency covariance matrices – HighFrequencyCovariance: Estimating Covariance Matrices in Julia and the associated Julia package HighFrequencyCovariance.jl

The Execution Traders Problem

You are an execution trader with access to 4 different broker algorithms (algos) to execute your trade. With each trade you need to choose an algo and measure the trade’s overall slippage – the price you paid vs the price at the start of the order. You want to chose the best algo to ensure each of your trades gets the best price.

How do you chose what one to use? Do you have enough data to decide what one is the best one? Is any one algo better than the other? These are all difficult questions to answer but with some data on how the algos performs you should be able to use the data to help inform your decision.

We are trying to maximise the performance of each trade by choosing the correct algo. Our trade is described by a variable \(x\) and each algo performs differently depending on \(x\). The paper calls the performance ‘slippage’ but then tries to maximise the slippage which sounds weird to me – I always talk about minimising slippage! But that’s splitting hairs.

The performance of algo \(i\) is described by an analytical function with parameters \(\alpha _i, \beta _i\) plus some noise that depends on the duration of the trade \(d\) and the volatility \(\sigma\).

function expSlippage(x, alpha, beta)
   @. -alpha*(x - beta)^2 
end

function slippage(x, alpha, beta, d, sigma)
    expSlippage(x, alpha, beta) + rand(Normal(0, d*sigma/2))
end

The \(\alpha\)’s and \(\beta\)’s are simple constants set in the paper.

alphas = [5,10,15,20]
betas = [0.2, 0.4, 0.6, 0.8]

x = collect(0:0.01:1)
p = plot(xlabel = "x", ylabel = "Expected Slippage")
for i in eachindex(alphas)
   plot!(p, x, expSlippage(x, alphas[i], betas[i]), label = "Algo " * string(i), lw = 2) 
end
p

Here we can see where each algo is better for each \(x\). In reality, this is impossible to know or it might not even exist.

We are going to devise a rule of when we will select each trading algo:

• If \(x<0.5\) then we will randomly select Strategy 1 62.5% of the time and the others 12.5% of the time.

• If \(x>0.5\) then Strategy 3 62.5% and the others 12.5%.

function tradingRule(x)
    if x < 0.5
        return [0.625, 0.125, 0.125, 0.125]
    else 
        return [0.125, 0.125, 0.625, 0.125]
    end
end

Julia’s vectorisation makes it easy to simulate going through multiple trades.

x = rand(Uniform(), 100)
d = rand(Uniform(), 100)
stratProbs = tradingRule.(x)
strat = rand.(Categorical.(stratProbs))
stratProb = getindex.(stratProbs, strat)
slippageVal = slippage.(x, alphas[strat], betas[strat], d, 5)

res = DataFrame(x=x, d=d, strat=strat, stratProb=stratProb, prob=stratProb, slippage=slippageVal)
first(res, 3)

This is our ‘production data’ for 100 random trades. The aim of the game is to understand how good our trading rules are rather than trying to estimate how good the individual algos are.

Does our rule above do better than just randomly choosing an algo? This is where we can use importance sampling to take the 100 trades and specially weight them to assess a new trading rule.

Importance Sampling

Importance sampling is about using observed probabilities \(q\) and observations of a variable with different probabilities \(p\). In our case we want to calculate the expected slippage of a trading strategy given the observations we have of the current strategy.

\[\mathbb{E} [\text{Slippage}] = \frac{1}{N} \sum _i \text{Slipage}_i \frac{p_i(\text{New Strategy})}{q_i(\text{Current Strategy})}\]

\(q_i(\text{Current Strategy})\) is equal to the stratProb column in the dataframe and \(p_i\) is the probability we would have chosen the given algo under the new strategy.

For the importance sampling, we calculate the likelihood ratio using equal probabilities and then take the weighted average of the slippages.

res = @transform(res, :EqProb = 0.25)
res = @transform(res, :ratio = :EqProb ./ :stratProb)
@combine(res, :StratSlippage = mean(:slippage), :EqStratSlippage = mean(:slippage, Weights(:ratio)))

The average slippage for the 100 trades is worse (more negative) that the current strategy. This suggests that randomly choosing would perform worse.

Then plotting the average slippage across the orders.

res = @transform(res, :StratSlipapgeRolling = cumsum(:slippage) ./collect(1:length(:slippage)))
res = @transform(res, :EqSlipapgeRolling = cumsum(:slippage .* :ratio) ./cumsum(:ratio))

plot(res.StratSlipapgeRolling, label = "Production", lw =2)
plot!(res.EqSlipapgeRolling, label = "Equal Weighted", lw =2)

The timeseries of the slippage shows that the equally weighted strategy is worse, so gives us confidence in the current strategy. When we observe a bad outcome the likelihood ratio weights that outcome based on how different the probability is from the production strategy.

How can we use importance sampling to build better strategies?

Easy Reinforcement Learning and Expected Slippage

Each trade is described by \(x\). In this toy model that is just a number but in real life this could correspond to the size of the order, the asset, the time of day and any combination of variables. In the original paper they use the spread, volatility, order size relative to the ADV and duration as descriptive variables of a random dataset. I’m going to keep it simple and stick to \(x\) being just a single number.

We want to understand if a particular \(x\) means we should use algo \(i\). For this, we need to build an ‘expected slippage’ model where we use the historical \(x\) values and outcomes of using algo \(i\).

For the modelling part, we will use xgboost through MLJ.jl.

using MLJ
xgboostModel = @load XGBoostRegressor pkg=XGBoost verbosity = 0
xgboostmodel = xgboostModel(eval_metric=["rmse"]);

The inputs are \(x\) and an indicator of the chosen algo.

res2 = coerce(res[:,[:x, :strat, :slippage]], :strat=>Multiclass);

y, X = unpack(res2, ==(:slippage); rng=123);

encoder = ContinuousEncoder()
encMach = machine(encoder, X) |> fit!
X_encoded = MLJ.transform(encMach, X);

xgbMachine = machine(xgboostmodel, X_encoded, y)

evaluate!(xgbMachine,
          resampling=CV(nfolds = 6, shuffle=true),
          measures=[rmse, rsq],
          verbosity=0)

The overall regression gets an \(R^2\) of 0.5 on our 100 trade dataset – a decent model.

In this new simulation, we will fit the xgboost model on the trades to build up an expected slippage model with all the data we have so far. prepareData and fitSlippage transform the data and fit the model.

We will then use this model to predict the expected slippage (predictSlippage) for each algo and use that to selected what algo to use for a given trade.

function prepareData(x, strat, slippage)
    res = coerce(DataFrame(x=x, strat=strat, slippage=slippage), :strat=>Multiclass);
    y, X = unpack(res, ==(:slippage); rng=123);
    encoder = ContinuousEncoder()
    encMach = machine(encoder, X) |> fit!
    X_encoded = MLJ.transform(encMach, X);
    return X_encoded, y
end

function fitSlippage(x, strat, slippage, xgboostmodel)
    X_encoded, y = prepareData(x, strat, slippage)
    xgbMachine = machine(xgboostmodel, X_encoded, y)

    evaluate!(xgbMachine,
          resampling=CV(nfolds = 6, shuffle=true),
          measures=[rmse, rsq],
          verbosity=0)
    return (xgbMachine, encMach)
end

function predictSlippage(x, xgbMachine, encMachine)
    X_pred = DataFrame(x=x, strat = [1,2,3,4], slippage = NaN)
    X_pred = coerce(X_pred[:,[:x, :strat, :slippage]], :strat=>Multiclass)
    X_pred = MLJ.transform(encMach, X_pred)
    preds = MLJ.predict(xgbMachine, X_pred)
    return(preds)
end

function slippageToProb(preds)
    scores = exp.(preds) ./ sum(exp.(preds))
    p = ((0.9 .* scores) .+ 0.025) ./ sum((0.9 .* scores) .+ 0.025) 
    return p
end

The predicted slippage is then transformed into a probability using the softmax function (slippageToProb) which gives us a mapping of the real-valued estimated slippage onto a probability. We then sample which strategy to use from this probability. By adding an element of randomness into the algo selection we are making sure we can use the importance sampling framework to either change the model (xgboost to something else) or change how we build the probabilities (softmax to something else).

To simulate the problem we will start by randomly choosing a strategy for the first 200 runs. After this we will start using the xgboost regression model to predict the expected slippage of each strategy and use this to decide what strategy to use.

epsilon = 0.05
volatility = 5
N = 1000

x = zeros(N)
strat = zeros(N)
slippages = zeros(N)
d = zeros(N)
stratProb = zeros(N)

for i in 1:N
    xVal = rand(Uniform())
    dVal = rand(Uniform())

    if i > 200
        xgbMachine, encMachine = fitSlippage(x[1:i], strat[1:i], slippages[1:i], xgboostmodel)
        predCost = predictSlippage(xVal, xgbMachine, encMachine)
        stratProbs = slippageToProb(predCost)
    else
        stratProbs = [0.25, 0.25, 0.25, 0.25]
    end

    stratVal = rand(Categorical(stratProbs))
    slippageVal = slippage(xVal, alphas[stratVal], betas[stratVal], dVal, volatility)
    
    x[i] = xVal
    strat[i] = stratVal
    stratProb[i] = stratProbs[stratVal]
    slippages[i] = slippageVal
    d[i] = dVal
end

res = DataFrame(x=x, d=d, strat=strat, stratProb=stratProb, slippage=slippages)

Again, we output each strategy and the probability the strategy was used. We use the importance sampling approach to estimate the slippage for choosing an algo randomly to gives us a comparison to the xgboost method.

res = @transform(res, :EqProb = 0.25)
res = @transform(res, :EqRatio = :EqProb ./ :stratProb)
res = @transform(res, :StratSlipapgeRolling = cumsum(:slippage) ./collect(1:length(:slippage)))
res = @transform(res, :EqSlipapgeRolling = cumsum(:slippage .* :EqRatio) ./cumsum(:EqRatio));

plot(res.StratSlipapgeRolling[50:end], label = "Production")
plot!(res.EqSlipapgeRolling[50:end], label = "Equal Weighting")

For the first 200 trades we are just selecting randomly, so no difference in performance. Then afterwards we can see the XGBoost model starts to outperform as it learns what algo is better for each \(x\).
So whilst we have only run the XGBoost model in production it has shown it is doing better than random by using the importance sampling method.

Testing a New Model Without Running it in Production

The XGBoost model is doing well and out-performing an equal weighted model, but what if you wanted to change from XGBoost to something else? How can you build the case that this is something worth doing?

By constructing new probabilities of whether the strategy would be selected (new \(p_i\)’s) and with the current strategy probabilities (\(q_i\)’s) we can estimate the slippage of the new model without having to run any more trades.

With MLJ.jl we can create a new model and pass it into the functions to replicate running the strategy in production. This time we use a simple linear regression model with the same features. We run through the trades in the same order so there is no information leakage.

@load LinearRegressor pkg=MLJLinearModels

linreg = MLJLinearModels.LinearRegressor()

newProb = ones(N) * 0.25

for i in 1:(N-1)

    if i > 200
        linMachine, enchMachine = fitSlippage(res.x[1:i], res.strat[1:i], res.slippage[1:i], linreg)
        predSlippage = predictSlippage(res.x[i+1], linMachine, enchMachine)
        stratProbs = slippageToProb(predSlippage)
        newProbVal = stratProbs[Int(res.strat[i+1])]
        newProb[i] = newProbVal
    end
    
end

res[:, :LinearProb] = newProb

res = @transform(res, :LinearRatio = :LinearProb ./ :stratProb)
res = @transform(res, :LinearSlipapgeRolling = cumsum(:slippage .* :LinearRatio) ./cumsum(:LinearRatio))
plot(res.StratSlipapgeRolling[50:end], label = "Production")
plot!(res.EqSlipapgeRolling[50:end], label = "Equal Weighting")
plot!(res.LinearSlipapgeRolling[50:end], label = "Linear Model")

Adding the linear regression decision rule to the data gives us a way of assessing this new model without having to run it directly in production. We can see that the linear model is better than XGBoost and also better than the equal weighting.

A simple bootstrap of taking the average slippage for each strategy a random amount of times provides the simplest performance measure.

bs = mapreduce(x-> @combine(res[sample(201:nrow(res), nrow(res)-200), :], 
              :StratSlippage = mean(:slippage), 
              :EqStratSlippage = mean(:slippage, Weights(:EqRatio)),
              :LinearStratSlippage = mean(:slippage, Weights(:LinearRatio))),
			  vcat, 1:1000);

@combine(groupby(stack(bs), :variable), :avg = mean(:value), :sd = std(:value))

As its a toy problem, nothing of significance between the models – but both models do better than the random allocation.

Conclusion

Importance sampling gives you a way of getting more out of the current data and strategy you are using. By weighting the observations in a new way you can get an idea whether a new strategy is worth it or not.
By rethinking you current setup you can easily add a bit of randomness into decisions and use the importance sampling framework going forward.

By: Dean Markwick's Blog -- Julia

Re-posted from: https://dm13450.github.io/2024/09/19/Alpha-Capture-and-Acquired.html

People are never short of a trade idea. There is a whole industry of
researchers, salespeople and amateurs coming up with trading ideas and
making big calls on what stock will go up, what country will cut
interest rates and what the price of gold will do next. Alpha capture
is about systematically assessing ideas and working out who has
alpha and generates profitable ideas and who is just making it up as
they are going along.

Enjoy these types of posts? Then you should sign up for my newsletter.

Alpha capture started as a way of profiling a broker’s stock
recommendation. If you have 50 people recommending you 50 different
ideas, how do you know who is good? You’ll quickly run out of money if
you blindly follow all the recommendations that hit your
inbox. Instead, you need to profile each person’s idea and see
who on average can make good recommendations. Whoever is good at
picking stocks probably deserves more of your business.

It has since expanded that some hedge fund have internal desks that
are doing a similar analysis on their portfolio managers (PMs) to double
down on profitable bets and mitigate risks of all the PMs picking the
same stock. Picking stocks and managing a portfolio across many PMs
are two different skills and different departments at your modern
hedge fund.

A simple way to measure the alpha of a PM or broker recommendation
will be to see if the price of a stock they buy (or recommend) goes up
after the day they suggest it. Those with alpha would see their
picks move higher on a large enough sample and those without alpha
would average out to zero, some ideas would go higher, some ideas
lower, the net result being 0 alpha. If a PM has the opposite effect,
every stock they buy goes down they are a contrarian
indicator so take their idea and do the opposite!

Alpha Capture Systems: Past, Present, and Future
Directions
goes through the history of alpha capture and is a good short read
that inspired this blog post.

Basic Alpha Capture

What if we wanted to try our own Alpha Capture? We need some stock recommendations and a way of calculating what happens to the price after the recommendation. This is where the Acquired podcast comes in.

Acquired tells the stories and strategies of great companies (taken from their website). It’s a pretty popular podcast and each episode gets close to a million listeners. So this makes it an ideal Alpha Capture study – when they release an episode about a company does the stock price of that company go higher or lower on average?
If it were to go higher then each time an episode is released call your broker and go long the stock!

They aren’t explicitly recommending a stock by talking about
it, as they say in their intro. So it’s just a toy exercise to see if
there is any correlation between the stock price and the release date
of an episode.

To systematically test this we need to get a list of the episodes and calculate a ‘markout’ from each episode.

Collecting Podcast Data

The internet is a wonderful thing and each episode of Acquired is
available as a XML feed from transistor.fm. So doing some fun parsing
of XML I can get the full history of the podcast with each date
and title.

function parseEpisode(x)
  rawDate = first(simplevalue.(x[tag.(x) .== "pubDate"]))
  date = ZonedDateTime(rawDate, dateformat"eee, dd uuu yyyy HH:MM:ss z")

  Dict("title" => first(simplevalue.(x[tag.(x) .== "title"])),
       "date" =>date)
end

function parse_date(t)
   Date(string(split(t, "T")[1]))
end

url = "https://feeds.transistor.fm/acquired"

data = parse(Node, String(HTTP.get(url).body))

episodes = children(data[3][1])
filter!(x -> tag(x) == "item", episodes)
episodes = children.(episodes)

episodeData = parseEpisode.(episodes)

episodeFrame = vcat(DataFrame.(episodeData)...)
CSV.write("episodeRaw.csv", episodeFrame)

After writing the data to a CSV I need to somehow parse the episode
title into a stock ticker. This is a tricky task as the episode names
are human friendly not computer friendly. So time for our LLM
overlords to lend a hand a do the heavy lifting. I drop the CSV into
Perplexity and prompt it to add the relevant stock ticker to the
file. I then reread the CSV into my notebook.

episodeFrame = CSV.read("episodeTicker.csv", DataFrame)
episodeFrame.date = ZonedDateTime.(String.(episodeFrame.date), dateformat"yyyy-mm-ddTHH:MM:SS.sss-z")

vcat(first(@subset(episodeFrame, :stock_ticker .!= "-"), 4),
        last(@subset(episodeFrame, :stock_ticker .!= "-"), 4))

It’s done an ok job. Most of the episodes seem to correspond to the
right ticker but we can see it has hallucinated the RenTech stock
ticker as RNR. RenTech is a private company, no stock ticker and
instead, Perplexity has decided the RNR (a reinsurance company) is the
correct stock ticker. So not 100% accurate. Still, it has saved me a
good chunk of time and we can move on to getting the stock price data.

We want to measure the average price move of a stock after an episode is released. If Acquired had stock-picking skill, you expect the price to increase after the release of an episode as they are generally speaking positively about the various companies.

So using AlpacaMarkets.jl we get the stock price for the days before and the days after the episode. As AlpacaMarkets only has US stock data then only some of the episodes end up with a full dataset.

What is a Markout?

We calculate the percentage change relative to the episode date and then aggregate all the stock tickers together.

\[\text{Markout} = \frac{p – p_{\text{episode released}}}{p_{\text{episode released}}}\]

Acquired is about great companies so they choose to speak favourably about a company, therefore I think it’s a reasonable assumption that we expect the stock price to increase after everyone gets round to listening to it.
So once we aggregate all the episodes we should hopefully have
enough data to decide if this is true.

function getStockData(stock, startDate)
  prices = AlpacaMarkets.stock_bars(stock, "1Day", startTime=startDate - Month(1), limit=10000)[1]
  prices.date .= startDate
  prices.t = parse_date.(prices.t)
  prices[:, [:t, :symbol, :vw, :date]]
end

function calcMarkout(data)
   arrivalInd = findlast(data.t .<= data.date)
   arrivalPrice = data[arrivalInd, :vw]
   data.arrivalPrice .= arrivalPrice
   data.ts = [x.value for x in (data.t .- data.date)]
   data.markout = 1e4*(data.vw .- data.arrivalPrice) ./ data.arrivalPrice
   data
end

res = []

for row in eachrow(episodeFrame)
    
    try 
        stockData = getStockData(row.stock_ticker, Date(row.date))
        stockData = calcMarkout(stockData)
        append!(res, [stockData])
    catch e
        println(row.stock_ticker)
    end
end

res = vcat(res...)

With the data pulled we now aggregate by each day before and after the episode.

markoutRes = @combine(groupby(res, :ts), :n = length(:markout), 
                                         :avgMarkout = mean(:markout),
                                         :devMarkout = std(:markout))
markoutRes = @transform(markoutRes, :errMarkout = :devMarkout ./sqrt.(:n))

Always need error bars as this data gets noisy.

markoutResSub = @subset(markoutRes, :ts .<= 60, :n .>= 10)
plot(markoutResSub.ts, markoutResSub.avgMarkout, yerr=markoutResSub.errMarkout, 
     xlabel = "Days", ylabel = "Markout", title = "Acquired Alpha Capture", label = :none)
hline!([0], ls = :dash, color = "grey", label = :none)
vline!([0], ls = :dash, color = "grey", label = :none)

Not really a pattern. The majority of the error bars are intercepting zero after the podcast is released.
If you squint a little bit there seems to be a bit of a downward trend post-episode which would suggest they talk about a company at the peak of the stock price.

Beforehand there is a bit of positive momentum, again suggesting that
they release the podcast at the peak of the stock price. Now this is
even more of a stretch given there is only 1 podcast a month and it
takes more than 20 days to prepare an episode (I imagine!), so
more noise than signal.

markoutIndRes = @combine(groupby(res, [:symbol, :ts]), :n = length(:markout), 
                                         :avgMarkout = mean(:markout),
                                         :devMarkout = std(:markout))
markoutIndRes = @transform(markoutIndRes, :errMarkout = :devMarkout ./sqrt.(:n))

p = plot()
hline!(p, [0], ls = :dash, color = "grey", label = :none)
vline!(p, [0], ls = :dash, color = "grey", label = :none)
for sym in ["TSLA", "V", "META"]
   markoutResSub = sort(@subset(markoutIndRes, :symbol .== sym, :ts .<= 60, :n .>= 1), :ts)
    plot!(p, markoutResSub.ts, markoutResSub.avgMarkout, yerr=markoutResSub.errMarkout, 
     xlabel = "Days", ylabel = "Markout", title = "Acquired Alpha Capture", label = sym, lw =2) 
end
p

When we pull out 3 examples of episodes we can see the randomness and specifically the volatility of TSLA here.

Conclusion

From this, we would not put any specific weight on the stock
performance after an episode is released. There doesn’t appear to be
any statistical pattern to exploit. No alpha means no alpha
capture. It is a nice exercise though and has hopefully explained the
concept of a markout.

By: Dean Markwick's Blog -- Julia

Re-posted from: https://dm13450.github.io/2024/04/25/Currency-Hedging-and-Principal-Component-Analysis.html

Principal component analysis (PCA) reduces a dataset to its main
components. When we apply it to a dataset of different
currencies it helps us understand how each currency drives the overall
portfolio and what currency might be a common factor.

Enjoy these types of posts? Then sign up for my newsletter.

This post was inspired by a problem on the r/quant subreddit where someone posted their interview/take-home question.

A client is considering using SGD to (proxy) hedge their exposure to
a basket of other Asian currencies. Is this likely to be effective?
What analysis could you produce that would help inform their
decision? The client is a US Corporate. The client is exposed to
medium-term changes (say monthly) in the currency. The client has equal (USD equivalent) revenues in each Asian currency. We are not considering hedging costs for this analysis (spot-only component). The data for daily close spot values against USD for each pair is provided. Which currency pairs will it work better for? Would it work for an equally weighted currency portfolio? Would another (single) currency work better? Which correlations should we consider and how reliable are these?

This is an interesting question and not too dissimilar to the
occasional question I answer in my day job. So I thought I’d run through how I might answer it.

Getting FX Data

First, we need to get some data and I’ll be using Alphavantage to pull
daily closing prices of the different currencies. I’ll calculate the
log returns and save the data to cache it for future use. Plus
AlphaVantage only lets you make 25 calls a day, so each time I mucked
up I got locked out for the day – delaying the analysis. We have to
start from 2014 as this is the earliest common date across all
currencies.

function _pull_data(ccy)
    println(ccy)
    res = AlphaVantage.fx_daily("USD", ccy, outputsize="full", datatype="csv")
    res = DataFrame(Dict(:Date=>Date.(res[1][:, 1]), :c=>Float64.(res[1][:,5]), :ccy => ccy));
    res = sort(res, :Date)
    res = @transform(res, :LogReturn = [0; diff(log.(:c))])
    res
end

function pull_data(ccy)
    if isfile("$ccy.csv")
        res = CSV.read("$ccy.csv", DataFrame)
    else
        res = _pull_data(ccy)
        CSV.write("$ccy.csv", res)
    end
    res
end

ccys = ["JPY", "CNH", "SGD", "THB", "HKD", "KRW", "TWD"]
res = vcat(pull_data.(ccys)...);
res = sort(res, :Date)
res = @transform(groupby(res, :ccy), :LogReturn = [0; diff(log.(:c))])
res = @subset(res, :Date .>= Date("2014-11-24"))

Like all good blog posts, let’s start with the plot of the
cumulative returns. Only HKD stands out as something different given
its peg to USD.

p = plot(ylabel = "Cummulative Return")
for ccy in ccys
    plot!(p, res[res.ccy .== ccy, :].Date, cumsum(res[res.ccy .== ccy, :].LogReturn), label = ccy, lw = 2)
end
p

According to the problem, our client is long equal amounts of these
Asian currencies, so it makes sense to calculate the market returns by
taking the average return each day.

market = @combine(groupby(res, :Date), :LogReturn = mean(:LogReturn))
market[!, :ccy] .= "Market"
market[!, :c] .= NaN;

Which we add to the original plot.

p = plot!(p, market.Date, cumsum(market.LogReturn)
    label = "Market", color = "black", lw  = 2)

The client thinks that hedging with SGD alone is enough to protect
against the overall market returns. We can see from the graph that
this probably isn’t the case. But how do we recommend a better
approach?

First, we will start with the correlation in returns between the
different currencies. This will shed some light on how linked they
are and is also simple to explain to the client.

cr = cor(Matrix(modelData[:, [:JPY, :CNH, :SGD, :THB, :HKD, :KRW, :TWD]]))
heatmap(ccys, ccys, cr .> 0.5)

We use a heat-map, but only highlight when two currencies have a
correlation > 0.5, otherwise it’s a bit of a psychedelic nightmare.

We can see that HKD has a low correlation with most, KRW and SGD have
a high correlation between each other and KRW has a high correlation with the majority of
these currencies.
However, we will use the covariance matrix to analyse the best hedging portfolio rather than the correlation matrix.

Principal Component Analysis

Principal component analysis (or PCA) is a tool that tries to find a
common basis of variation in a matrix. It’s about transforming the
data into uncorrelated components through linear algebra.

For this we are using the covariance matrix, so now the diagonals are
the individual price series variances and the off-diagonals are the
covariances between two currencies. If this were a different problem
we might rescale the returns so they all had the same volatility but
this would mean applying leverage, which our hypothetical customer
probably wouldn’t be up for it.

We pull out the covariance matrix

modelData = dropmissing(unstack(res, :Date, :ccy, :LogReturn))
cm = cov(Matrix(modelData[:, [:JPY, :CNH, :SGD, :THB, :HKD, :KRW, :TWD]]))

The MultivariateStats.jl package has the functions for doing PCA and
the appropriate functions for pulling out the right data after fitting the
PCA model.

pcaRes = fit(PCA, cm; maxoutdim=3)

Firstly the weights of all the currencies for the three principal
components.

• PC1 shows the weights for each currency but is unnormalised. The key thing
  we can see here is that HKD is magnitudes smaller than the others.
• PC2 is long JPY and short all the others
• PC3 is long THB and short all the others

Then the explained variance of the three components.

The first component can explain 49% of the variance and then including
the second component 91% of the variance, with the final component
making up 5% to take it to 96% in total. This means that this dataset
can be broken down quite nicely into the two principal components and
this explains most of the variation.

The first principal component is commonly called the
‘market’ portfolio and represents the overall combined market dynamics
of the portfolio. The next portfolio (using the 2nd PC weights) is
uncorrelated to the market and thus more diversified to the overall
market.

In our problem then we can see that we are trying to come up with a
representation of the market and use that to decide how to hedge out
our currencies. So the first principal component is the most relevant.

We take these principal component weights and join them to the original
dataframe to start exploring what the market portfolio looks like.

evFrame = DataFrame(Dict(:ccy => String.([:JPY, :CNH, :SGD, :THB, :HKD, :KRW, :TWD]), 
          :ev1 => eigvecs(pcaRes)[:,1],
          :ev2 => eigvecs(pcaRes)[:,2]))
sort!(evFrame, :ev1)

res = leftjoin(res, dropmissing(evFrame), on = :ccy)

evFrame = sort(evFrame, :ev1);

Then plotting the weights by currency pair

bar(evFrame.ccy, evFrame.ev1 ./ sum(evFrame.ev1), label = "Eigen Weights")

These are the weights of the different currencies of the first eigen
portfolio. This combination of currencies is what we would recommend
if the client was exposed to a similar basket. The key points:

• The client is long these currencies through their business
• They short this portfolio and thus are market-neutral

We now calculate the returns of the eigen portfolios, the portfolio
that only uses the largest 2 (and 3) weights.

evPortfolios = @combine(groupby(res, :Date), 
         :ReturnEV1 = sum(:LogReturn .* :ev1) ./ sum(:ev1), 
         :ReturnEV2 = sum(:LogReturn .* :ev2) ./ sum(:ev2));

ccy2Portfolio = @combine(groupby(res[in.(res.ccy, Ref(["KRW", "JPY"])), :], :Date), 
         :Return2Ccy = sum(:LogReturn .* :ev1) ./ sum(:ev1));

ccy3Portfolio = @combine(groupby(res[in.(res.ccy, Ref(["KRW", "JPY", "SGD"])), :], :Date), 
         :Return3Ccy = sum(:LogReturn .* :ev1) ./ sum(:ev1));

And plotting these returns

plot(market.Date, cumsum(market.LogReturn), label = "Market", color = "black", lw = 2)
plot!(evPortfolios.Date,  cumsum(evPortfolios.ReturnEV1), label = "Eigen Portfolio", lw = 2)
plot!(ccy2Portfolio.Date,  cumsum(ccy2Portfolio.Return2Ccy), label = "2 Ccy", lw =2)
plot!(ccy3Portfolio.Date,  cumsum(ccy3Portfolio.Return3Ccy), label = "3 Ccy", lw = 2)

Then finally, looking at the correlation between these portfolios

• The Eigen Portfolio 1 is most correlated with the equal-weighted portfolio.
• With just KRW and JPY you get to a 93% correlation with the market.
• KRW, JPY and SGD gets you to a 95% with the market.

As expected Eigen portfolio 2 is the most uncorrelated with the
market.

Summary

So our final answer to the client would be:

• We have a proprietary portfolio (the market eigen portfolio) that you
  should hedge with – this will give you the best outcome.
• If you don’t want the full portfolio use a 60/40 ratio of KRW and
  JPY.
• SGD probably isn’t a great idea and will leave you exposed.

Now, we are assuming that these weightings are stable through time and
haven’t changed recently and are therefore valid for the future
returns too. We are ignoring transaction costs, KRW being an NDF and
more expensive to trade compared to a spot currency (like JPY) means
that this approach will break down if the client needs to hedge a
significant amount.