JuliaPro Featured in Danske Bank’s Business Analytics Challenge 2017

By: Julia Computing, Inc.

Re-posted from: http://juliacomputing.com/press/2017/02/21/danske.html

Copenhagen, Denmark – Danske Bank, Denmark’s largest bank, announced that JuliaPro will be available on Microsoft Azure’s Data Science Virtual Machine (DSVM) for participants in the Business Analytics Challenge 2017.

The Business Analytics Challenge 2017 is sponsored by Danske Bank, Microsoft and KMD. The competition is open to all undergraduate and master’s degree students in Denmark and the first prize is 75 thousand kroner. Registration is open until March 31.

This announcement comes two months after the release of JuliaPro and one month after JuliaPro launched on Microsoft Azure’s Data Science Virtual Machine (DSVM).

Viral Shah, Julia Computing CEO says, “We are thrilled that Julia adoption is accelerating so rapidly during the first quarter of 2017. In the last three months, we introduced the new JuliaPro and launched it on the world’s two largest cloud environments: Amazon’s AWS and Microsoft Azure’s Data Science Virtual Machine (DSVM). Julia Computing wishes the best of luck to all contestants in the Danske Bank Business Analytics Challenge 2017.”

About Julia Computing and Julia

Julia Computing (JuliaComputing.com) was
founded in 2015 by the co-creators of the Julia language to provide
support to businesses and researchers who use Julia.

Julia is the fastest modern high performance open source computing language for data and analytics. It combines the functionality and ease of use of Python, R, Matlab, SAS and Stata with the speed of Java and C++. Julia delivers dramatic improvements in simplicity, speed, capacity and productivity.

  1. Julia is lightning fast. Julia provides speed improvements up to
    1,000x for insurance model estimation, 225x for parallel
    supercomputing image analysis and 11x for macroeconomic modeling.

  2. Julia is easy to learn. Julia’s flexible syntax is familiar and
    comfortable for users of Python, R and Matlab.

  3. Julia integrates well with existing code and platforms. Users of
    Python, R, Matlab and other languages can easily integrate their
    existing code into Julia.

  4. Elegant code. Julia was built from the ground up for
    mathematical, scientific and statistical computing, and has advanced
    libraries that make coding simple and fast, and dramatically reduce
    the number of lines of code required – in some cases, by 90%
    or more.

  5. Julia solves the two language problem. Because Julia combines
    the ease of use and familiar syntax of Python, R and Matlab with the
    speed of C, C++ or Java, programmers no longer need to estimate
    models in one language and reproduce them in a faster
    production language. This saves time and reduces error and cost.

Employers looking to hire Julia programmers in 2017 include: Google, Apple, Amazon, Facebook, IBM, BlackRock, Capital One, PricewaterhouseCoopers, Ford, Oracle, Comcast, Massachusetts General Hospital, NaviHealth, Harvard University, Columbia University, Farmers Insurance, Pilot Flying J, Los Alamos National Laboratory, Oak Ridge National Laboratory and the National Renewable Energy Laboratory.

Julia users and partners include: Amazon, IBM, Intel, Microsoft, DARPA, Lawrence Berkeley National Laboratory, National Energy Research Scientific Computing Center (NERSC), Federal Aviation Administration (FAA), MIT Lincoln Labs, Moore Foundation, Nobel Laureate Thomas J. Sargent, Federal Reserve Bank of New York (FRBNY), Capital One, Brazilian National Development Bank (BNDES), BlackRock, Conning, Berkery Noyes, BestX, Path BioAnalytics, Invenia, AOT Energy, AlgoCircle, Trinity Health, Gambit, Augmedics, Tangent Works, Voxel8, UC Berkeley Autonomous Race Car (BARC) and many of the world’s largest investment banks, asset managers, fund managers, foreign exchange analysts, insurers, hedge funds and regulators.

Universities and institutes using Julia include: MIT, Caltech, Stanford, UC Berkeley, Harvard, Columbia, NYU, Oxford, NUS, UCL, Nantes, Alan Turing Institute, University of Chicago, Cornell, Max Planck Institute, Australian National University, University of Warwick, University of Colorado, Queen Mary University of London, London Institute of Cancer Research, UC Irvine, University of Kaiserslautern.

Julia is being used to: analyze images of the universe and research dark matter, drive parallel computing on supercomputers, diagnose medical conditions, provide surgeons with real-time imagery using augmented reality, analyze cancer genomes, manage 3D printers, pilot self-driving racecars, build drones, improve air safety, manage the electric grid, provide analytics for foreign exchange trading, energy trading, insurance, regulatory compliance, macroeconomic modeling, sports analytics, manufacturing and much, much more.

DynProg Class – Week 2

By: pkofod

Re-posted from: http://www.pkofod.com/2017/02/18/dynprog-class-week-2/

This post, and other posts with similar tags and headers, are mainly directed at the students who are following the Dynamic Programming course at Dept. of Economics, University of Copenhagen in the Spring 2017. The course is taught using Matlab, but I will provide a few pointers as to how you can use Julia to solve the same problems. So if you are an outsider reading this I welcome you, but you won’t find all the explanations and theory here. If you want that, you’ll have to come visit us at UCPH and enroll in the class!

This week we continue with a continuous choice model. This means we have to use interpolation and numerical optimization.

A (slightly less) simple model

Consider a simple continuous choice consumption-savings model:

\(V_t(M_t) = \max_{C_t}\sqrt{C_t}+\mathbb{E}[V_{t+1}(M_{t+1})|C_t, M_t]\)
subject to
\(M_{t+1} = M_t – C_t+R_t\\
C_t\leq M_t\\
C_t\in\mathbb{R}_+\)

where \(R_t\) is 1 with probability \(\pi\) and 0 with probability \(1-\pi\), \(\beta=0.9\), and \(\bar{M}=5\)

Last week the maximization step was merely comparing values associated with the different discrete choices. This time we have to do continuous optimization in each time period. Since the problem is now continuous, we cannot solve for all \(M_t\). Instead, we need to solve for \(V_t\) at specific values of \(M_t\), and interpolate in between. Another difference to last time is the fact that the transitions are stochastic, so we need to form (very simple) expectations as part of the Bellman equation evaluations.

Interpolation

It is of course always possible to make your own simple interpolation scheme, but we will use the functionality provided by the Interpolations.jl package. To perform interpolation, we need a grid to be used for interpolation \(\bar{x}\), and calculate the associated function values.

f(x) = (x-3)^2
x̄ = linspace(1,5,5)
fx̄ = f.()

Like last time, we remember that the dot after the function name and before the parentheses represent a “vectorized” call, or a broadcast – that is we call f on each element of the input. We now use the Interpolations.jl package to create an interpolant \(\hat{f}\).

using Interpolations
f̂ = interpolate((collect(),), fx̄, Gridded(Linear()))

We can now index into \(\hat{f}\) as if it was an array

[-3] #returns 16.0

We can also plot it

using Plots
plot()

which will output

Solving the model

Like last time, we prepare some functions, variables, and empty containers (Arrays)

# Model again
u(c) = sqrt(c)
T = 10; β = 0.9
π = 0.5 ;M₁ = 5
# Number of interpolation nodes
Nᵐ = 50 # number of grid points in M grid
Nᶜ  = 50 # number of grid points in C grid
 
M = Array{Vector{Float64}}(T)
V = Array{Any}(T)
C = Array{Any}(T)

The V and C arrays are allocated using the type “Any”. We will later look at how this can hurt performance, but for now we will simply do the convenient thing. Then we solve the last period

M[T] = linspace(0,M₁+T,Nᵐ)
C[T] = M[T]
V[T] = interpolate((M[T],), u.(C[T]), Gridded(Linear()))

This new thing here is that we are not just saving V[T] as an Array. The last element is the interpolant, such that we can simply index into V[T] as if we had the exact solution at all values of M (although we have to remember that it is an approximation). For all periods prior to T, we have to find the maximum as in the Bellman equation from above. To solve this reduced “two-period” problem (sum of utility today and discounted value tomorrow), we need to form expectations over the two possible state transitions given an M and a C today, and then we need to find the value of C that maximizes current value. We define the following function to handle this

# Create function that returns the value given a choice, state, and period
v(c, m, t, V) = u(c)*(π*V[t+1][m-c+1]+(1)*V[t+1][m-c])

Notice how convenient it is to simply index into V[t] using the values we want to evaluate tomorrow’s value function at. We perform the maximization using grid search on a predefined grid from 0 to the particular M we’re solving form. If we abstract away from the interpolation step, this is exactly what we did last time.

for t = T-1:-1:1
    M[t] = linspace(0,M₁+t,Nᵐ)
    C[t] = zeros(M[t])
    Vt = fill(-Inf, length(M[t]))
    for (iₘ, m) = enumerate(M[t])
        for c in linspace(0, m, Nᶜ)
            _v = v(c, m, t, V)
            if _v >= Vt[iₘ]
                Vt[iₘ] = _v
                C[t][iₘ] = c
            end
        end
    end
    V[t] = interpolate((M[t],), Vt, Gridded(Linear()))
end

Then we can plot the value functions to verify that they look sensible

Nicely increasing in time and in M.

Using Optim.jl for optimization

The last loop could just as well have been done using a proper optimization routine. This will in general be much more robust, as we don’t confine ourselves to a certain amount of C-values. We use the one of the procedures in Optim.jl. In Optim.jl, constrained, univariate optimization is available as either Brent’s method or Golden section search. We will use Brent’s method. This is the standard method, so an optimization call simply has the following syntax

using Optim
f(x) = x^2
optimize(f, -1.0, 2.0)

Unsurprisingly, this will return the global minimizer 0.0. However, if we constrain ourselves to a strictly positive interval

optimize(f, 1.0, 2.0)

we get a minimizer of 1.0. This is not the unconstrained minimizer of the square function, but it is minimizer given the constraints. Then, it should be straight forward to see how the grid search loop can be converted to a loop using optimization instead.

for t = T-1:-1:1
    update_M!(M, M₁, t, Nᵐ)
    C[t] = zeros(M[t])
    Vt = fill(-Inf, length(M[t]))
    for (iₘ, m) = enumerate(M[t])
        if m == 0.0
            C[t][iₘ] = m
            Vt[iₘ] = v(m, m, t, V)
        else
            res = optimize(c->-v(c, m, t, V), 0.0, m)
            Vt[iₘ] = -Optim.minimum(res)
            C[t][iₘ] = Optim.minimizer(res)
        end
    end
    V[t] = interpolate((M[t],), Vt, Gridded(Linear()))
end

If our agent has no resources at the beginning of the period, the choice set has only one element, so we skip the optimization step. We also have to negate the minimum to get the maximum we’re looking for. The main advantage of using a proper optimization routine is that we’re not restricting C to be in any predefined grid. This increases precision. If we look at the number of calls to “v” (using Optim.f_calls(res)), we see that it generally takes around 10-30 v calls. With only 10-30 grid points from 0 up to M, we would generally get an approximate solution of much worse quality.

Julia bits and pieces

This time we used a few different package from the Julia ecosystem: Plots.jl, Interpolations.jl, and Optim.jl. These are based on my personal choices (full disclosure: I’ve contributed to the former and the latter), but there are lots of packages to explore. Visit the JuliaLang discourse forum or gitter channel to discuss Julia and the various packages with other users.

Parallel Neural Styles on Video Powered by Azure

By: Julia Computing, Inc.

Re-posted from: http://juliacomputing.com/blog/2017/02/17/styles-vid.html

Neural Styles is an algorithm based on neural networks that is used to learn
artistic styles. It is commonly used in apps like Prisma to beautify images.
The algorithm extracts features from a “style image” and then applies it to a
“content image”.

In a previous blog post, we described
the model and outlined the training process. We have now scaled our
algorithm from individual images to video, using Julia’s built-in parallel primitives
that make it trivially easy it is to scale algorithms to multiple nodes.

We ran the transformation on an Azure Data Science VM. The Azure DSVM is a pre-built image with many data science libraries included. It contains an installation of JuliaPro, and includes over a 100 major Julia packages. It is by far the easiest way to get started with running Julia code on the Azure ecosystem.

Our VM came with an Intel(R) Xeon(R) CPU E5-2660 with
8 physical cores and 56 GB RAM. We first extracted the frames from the video with VideoIO.jl, and then processed them in parallel with
8 Julia worker processes. Julia has several high level primitives for distributed computing, and we used
the parallel map pmap function to split the images amongst the workers and
subsequently texturize them. For larger videos, exactly the same code would be used to run this on a cluster of multiple physical or virtual servers.

@everywhere function f(a)
        texturize(a, "fire", "style")
end

pmap(x -> f(x), frames_from_video)

After the parallel processing, we stitched the images back together via ffmpeg to form a video.

Results

This is the original video:

These are our results on running the video through two models: “fire” and “frost”.

  • Fire:

  • Frost: