The post Interfacing with Relational Database using MySQL.jl and PyMySQL appeared first on juliabloggers.com.

]]>Prior to the advent of computing, relational database can be thought of log books typically used for inventory and visitor’s time-in time-out. These books contain rows that define the item/transaction, and columns describing the features of each row. Indeed, these are the core attributes of any relational database. Unlike spreadsheets, which are used for handling small datasets, databases are mostly used for storing huge transactional data for later use. They run on a server and often at the backend of any user (client) interface such as websites and mobile applications. These applications communicate with database via processing servers (e.g. Flask and Django). The figure below illustrates the request and response communcations between client and servers.

As mentioned earlier, databases are meant to store data for *later use* — in the sense that we can use it as a response to client’s requests, such as viewing or data extraction for insights. In this article, we are interested in data extraction from the database. In particular, the objective is to illustrate how to send request to MySQL server both from Julia and Python.

To start with, we need to setup MySQL server in our machine. Click the following link to download and install the application.

Note that I recommend you to download the latest version of MySQL since the setup above is using the old version.

In order to appreciate what we are aiming in this article, we need to go through some basic SQL queries to understand what type of request to send to MySQL server. I’m using macOS, but the following should work on Windows as well. For macOS users, open the MySQL Server Shell by running `mysql -u root -p`

(hit `return` or `enter` , and type in your MySQL root password you specified during the installation setup from the previous section) in the terminal. For windows, try to look for it in the Start Menu.

From here, we are going to check the available databases in MySQL server. To do this, run the following:

Indeed, there are four out-of-the-box defined databases already, and we don’t want to touch that. Instead, we going to create our own database, let’s call it `tutorial`

. To do this, run the following codes:

The best thing about SQL syntax is that, everything is self-explanatory, except maybe for line 19, which simply confirmed that we are using `tutorial`

as our database.

Next is to create a table for our database, we are going to use the 2019 Philippine Senatorial Election results with columns: Last Name, First Name, Party, Votes. Further, for purpose of illustration, we are going to use the top 5 only.

The following codes will insert the top five senators from the 2019 Philippine election results.

To view the encoded data, we simply select all (`*`

) the columns from the table.

In this section, we are going to interface with MySQL server on Julia and Python. This is possible using MySQL.jl and PyMySQL libraries. To start with, install the necessary libraries as follows:

For this exercise, our goal is to save the NYC Flights data into the database and query it from Julia and Python.

I have a copy of the dataset on Github, and so the following code will download the said data:

In order for the client to send request to MySQL server, the user/client needs to connect to it using the credentials set in the installation.

Note that you need to have a strong password, and this configuration should not be exposed to the public. The above snippets are meant for illustration.

To test the connection, let’s send our first request — to show the tables in the database:

In Julia, the response is recieved as a MySQL.Query object and can be viewed using DataFrame. For Python, however, you will get a tuple object.

At this point, we can now create the table for our dataset. To do this, run the following:

As shown in the previous section, sending request to the server both in Julia and in Python is done by simply using a string of SQL queries as input to MySQL.jl and PyMySQL APIs. Hence, the `query`

object (referring to the Julia code) in line 3 above, simply automates the concatenation of SQL query for creating a table. Having said, you can of course write the query manually. To check if we have indeed created the table, run the following codes:

As you can see, we’ve created it already, but with no entry yet.

Finally, we are going to populate the table in the database by inserting the values row by row.

Notice from the above Julia code, the result of the `stmt`

is an SQL `INSERT`

query with placeholder values indicated by `?`

. The timed (`@time`

in Julia code) loop in line 9 above maps the values of the vector, one-to-one, to the elements (`?`

) of the tuple in `stmt`

. One major difference between these libraries is that, PyMySQL will not populate the table even after executing all sorts of SQL queries unless you commit it (`con.commit`

), as shown above. This is contrary to MySQL.jl which automatically commits every execution of the SQL queries. I do like the idea of having `con.commit`

in PyMySQL, since this avoids accidental deletion or modification in the database, thus adding a layer of security. To check if we have indeed populated the table, run the following:

To disconnect from the server, run `MySQL.disconnect(con)`

(Julia) or `con.close()`

(Python).

For the benchmark, I added a timelapse recorder in populating and reading the table in the previous section. The figure below summarizes the results.

The figure was plotted using Gadfly.jl. Install this package using `Pkg`

as described above (see the first code block under *MySQL Clients on Julia and Python* section), along with Cario.jl and Fontconfig.jl. The latter two packages are used to save the plot in PNG format. See the code below to reproduce:

The aim of this article was simply to illustrate the usage of MySQL.jl APIs in comparison to the PyMySQL. Hence, I would say both libraries have similarities in APIs (as expected) and are stable for the tasks. I should emphasize though that, I do like the `con.commit`

of PyMySQL since this adds a level of security, and I think this is a good addition to MySQL.jl in the future.

If you are impatient, here are the complete codes excluding the MySQL codes and the plots. These should work after installing the required libraries shown above:

- MySQL.jl Github Repo: https://github.com/JuliaDatabases/MySQL.jl
- PyMySQL Github Repo: https://github.com/PyMySQL/PyMySQL
- Flaticon: https://www.flaticon.com/

The post Interfacing with Relational Database using MySQL.jl and PyMySQL appeared first on juliabloggers.com.

]]>sparsity handling.

(Breaking with Deprecations) DiffEqGPU: GPU-based Ensemble Simulations

The MonteCarloProblem interface received an overhaul. First of all, the

interface has b...

The post DifferentialEquations.jl v6.7.0: GPU-based Ensembles and Automatic Sparsity appeared first on juliabloggers.com.

]]>Re-posted from: http://juliadiffeq.org/2019/07/05/AutomaticSparsity.html

Let’s just jump right in! This time we have a bunch of new GPU tools and

sparsity handling.

The `MonteCarloProblem`

interface received an overhaul. First of all, the

interface has been renamed to `Ensemble`

. The changes are:

`MonteCarloProblem`

->`EnsembleProblem`

`MonteCarloSolution`

->`EnsembleSolution`

`MonteCarloSummary`

->`EnsembleSummary`

**Specifying parallel_type has been deprecated** and a deprecation warning is

thrown mentioning this. So don’t worry: your code will work but will give

warnings as to what to change. Additionally,

is no longer necessary for any of this functionality

Now, `solve`

of a `EnsembleProblem`

works on the same dispatch mechanism as the

rest of DiffEq, which looks like `solve(monteprob,Tsit5(),EnsembleThreads())`

where the third argument is an ensembling algorithm to specify the

threading-based form. Code with the deprecation warning will work until the

release of DiffEq 7.0, at which time the alternative path will be removed.

See the updated ensembles page for more details

The change to dispatch was done for a reason: it allows us to build new libraries

specifically for sophisticated handling of many trajectory ODE solves without

introducing massive new dependencies to the standard DifferentialEquations.jl

user. However, many people might be interested in the first project to make

use of this: DiffEqGPU.jl.

DiffEqGPU.jl lets you define a problem, like an `ODEProblem`

, and then solve

thousands of trajectories in parallel using your GPU. The syntax looks like:

```
monteprob = EnsembleProblem(my_ode_prob)
solve(monteprob,Tsit5(),EnsembleGPUArray(),num_monte=100_000)
```

and it will return 100,000 ODE solves. **We have seen between a 12x and 90x speedup
depending on the GPU of the test systems**, meaning that this can be a massive

improvement for parameter space exploration on smaller systems of ODEs.

Currently there are a few limitations of this method, including that events

cannot be used, but those will be solved shortly. Additional methods for

GPU-based parameter parallelism are coming soon to the same interface. Also

planned are GPU-accelerated multi-level Monte Carlo methods for faster weak

convergence of SDEs.

Again, this is utilizing compilation tricks to take the user-defined `f`

and recompile it on the fly to a `.ptx`

kernel, and generating kernel-optimized

array-based formulations of the existing ODE solvers

Shashi Gowda (@shashigowda) implemented a sparsity detection algorithm which

digs through user-defined Julia functions with Cassette.jl to find out what

inputs influence the output. The basic version checks at a given trace, but

a more sophisticated version, which we are calling Concolic Combinatoric Analysis,

looks at all possible branch choices and utilizes this to conclusively build a

Jacobian whose sparsity pattern captures the possible variable interactions.

The nice part is that this functionality is very straightforward to use.

For example, let’s say we had the following function:

```
function f(dx,x,p,t)
for i in 2:length(x)-1
dx[i] = x[i-1] - 2x[i] + x[i+1]
end
dx[1] = -2x[1] + x[2]
dx[end] = x[end-1] - 2x[end]
nothing
end
```

If we want to find out the sparsity pattern of `f`

, we would simply call:

```
sparsity_pattern = sparsity!(f,output,input,p,t)
```

where `output`

is an array like `dx`

, `input`

is an array like `x`

, `p`

are possible parameters, and `t`

is a possible `t`

. The function will then

be analyzed and `sparsity_pattern`

will return a `Sparsity`

type of `I`

and `J`

which denotes the terms in the Jacobian with non-zero elements. By doing

`sparse(sparsity_pattern)`

we can turn this into a `SparseMatrixCSC`

with the

correct sparsity pattern.

This functionality highlights the power of Julia since there is no way to

conclusively determine the Jacobian of an arbitrary program `f`

using numerical

techniques, since all sorts of scenarios lead to “fake zeros” (cancelation,

not checking a place in parameter space where a branch is false, etc.). However,

by directly utilizing Julia’s compiler and the SSA provided by a Julia function

definition we can perform a non-standard interpretation that tells all of the

possible numerical ways the program can act, thus conclusively determining

all of the possible variable interactions.

Of course, you can still specify analytical Jacobians and sparsity patterns

if you want, but if you’re lazy…

See SparseDiffTools’s README for more details.

We are pleased to announce the `LinSolveGPUFactorize`

option which allows for

automatic offloading of linear solves to the GPU. For a problem with a large

enough dense Jacobian, using `linsolve=LinSolveGPUFactorize()`

will now

automatically perform the factorization and back-substitution on the GPU,

allowing for better scaling. For example:

```
using CuArrays
Rodas5(linsolve = LinSolveGPUFactorize())
```

This simply requires a working installation of CuArrays.jl. See

the linear solver documentation for more details.

We have been dabbling in allowing automated accelerator (GPU, multithreading,

distributed, TPU, etc.) offloading when the right hardware is detected and the

problem size is sufficient to success a possible speedup.

A working implementation exists as a PR for DiffEqBase

which would allow automated acceleration of linear solves in implicit DE solving.

However, this somewhat invasive of a default, and very architecture dependent,

so it is unlikely we will be releasing this soon. However, we are investigating

this concept in more detail in the AutoOffload.jl. If you’re interested in Julia-wide automatic acceleration,

please take a look at the repo and help us get something going!

Previous releases had only a pre-built GMRES implementation. However, as

detailed on the linear solver page,

we now have an array of iterative solvers readily available, including:

- LinSolveGMRES – GMRES
- LinSolveCG – CG (Conjugate Gradient)
- LinSolveBiCGStabl – BiCGStabl Stabilized Bi-Conjugate Gradient
- LinSolveChebyshev – Chebyshev
- LinSolveMINRES – MINRES

These are all compatible with matrix-free implementations of a

`AbstractDiffEqOperator`

.

Thanks to Yingbo Ma (@YingboMa), the exprb methods have been greatly improved.

Our current development is very much driven by the ongoing GSoC/JSoC projects,

which is a good thing because they are outputting some really amazing results!

Here’s some things to look forward to:

- Automated matrix-free finite difference PDE operators
- Surrogate optimization
- Jacobian reuse efficiency in Rosenbrock-W methods
- Native Julia fully implicit ODE (DAE) solving in OrdinaryDiffEq.jl
- High Strong Order Methods for Non-Commutative Noise SDEs
- GPU-Optimized Sparse (Colored) Automatic Differentiation
- Parallelized Implicit Extrapolation of ODEs

The post DifferentialEquations.jl v6.7.0: GPU-based Ensembles and Automatic Sparsity appeared first on juliabloggers.com.

]]>The post Variational Importance Sampling appeared first on juliabloggers.com.

]]>Re-posted from: https://cscherrer.github.io/post/variational-importance-sampling/

Lots of distributions are easy to evaluate (the density), but hard to sample. So when we need to sample such a distribution, we need to use some tricks. We’ll see connections between two of these: importance sampling and variational inference, and see a way to use them together for fast inference.

Importance sampling Importance sampling aims to make it easy to compute expected values. Say we have a distribution \(p\), and we’d like to compute the average of some function \(f\) of the distribution (or equivalently, the expected value of a "push-forward along \(f\)").

The post Variational Importance Sampling appeared first on juliabloggers.com.

]]>If you have a function f!(du,u) which has a Tridiagonal Jacobian, you could

calculate that Jacobian by mixing perturbations. For example, instead of doing

u .+ [epsilon...

The post DifferentialEquations.jl v6.6.0: Sparse Jacobian Coloring, Quantum Computer ODE Solvers, and Stiff SDEs appeared first on juliabloggers.com.

]]>Re-posted from: http://juliadiffeq.org/2019/06/24/coloring.html

If you have a function `f!(du,u)`

which has a Tridiagonal Jacobian, you could

calculate that Jacobian by mixing perturbations. For example, instead of doing

`u .+ [epsilon,0,0,0,0,0,0,0,...]`

, you’d do `u .+ [epsilon,0,0,epsilon,0,0,...]`

.

Because the `epsilons`

will never overlap, you can then decode this “compressed”

Jacobian into the sparse form. Do that 3 times and boom, full Jacobian in

4 calls to `f!`

no matter the size of `u`

! Without a color vector, this matrix

would take `1+length(u)`

`f!`

calls, so I’d say that’s a pretty good speedup.

This is called Jacobian coloring. `[1,2,3,1,2,3,1,2,3,...]`

are the colors in

this example, and places with the same color can be differentiated simultaneously.

Now, the DiffEqDiffTools.jl internals allow for passing a color vector into the

numerical differentiation libraries and automatically decompressing into a

sparse Jacobian. This means that DifferentialEquations.jl will soon be compatible

with this dramatic speedup technique. In addition, other libraries in Julia with

rely on our utility libraries, like Optim.jl, could soon make good use of this.

What if you don’t know a good color vector for your Jacobian? No sweat! The

soon to be released SparseDiffTools.jl repository has methods for automatically

generating color vectors using heuristic graphical techniques.

DifferentialEquations.jl will soon make use of this automatically if you specify

a sparse matrix for your Jacobian!

Note that the SparseDiffTools.jl repository also includes functions for calculating

the sparse Jacobians using color vectors and forward-mode automatic differentiation

(using Dual numbers provided by ForwardDiff.jl). In this case, the number of Dual

partials is equal to the number of colors, which can be dramatically lower than

the `length(u)`

(the dense default!), thereby dramatically reducing compile

and run time.

Stay tuned for the next releases which begin to auto-specialize everything

along the way based on sparsity structure. Thanks to JSoC student Pankaj (@pkj-m)

for this work.

Deepesh Thakur (@deeepeshthakur) continues his roll with stiff stochastic

differential equation solvers by implementing not 1 but 7 new high weak order

stiff SDE solvers. SROCK1 with generalized noise, SKSROCK, and a bunch of

variants of SROCK2. Benchmark updates will come soon, but I have a feeling

that these new methods may be by far the most stable methods in the library,

and the ones which achieve the lowest error in the mean solution most efficiently.

GSoC student Kanav Gupta (@kanav99) implemented a bot for the JuliaDiffEq

team that allows us to run performance regression benchmarks on demand with

preset Gitlab runners. Right now this has a dedicated machine for CPU and

parallelism performance testing, and soon we’ll have a second machine

up and running for performance testing on GPUs. If you haven’t seen the Julialang

blog post on this topic, please check it out!.

If you happen to have a quantum computer handy, hold your horses. `QuLDE`

from

QuDiffEq.jl is an ODE solver designed for quantum computers. It utilizes the

Yao.jl quantum circuit simulator to run, but once Yao.jl supports QASM then

this will compile to something compatible with (future) quantum computing

hardware. This means that, in order to enter the new age of computing, all

you have to do is change `solve(prob,Tsit5())`

to `solve(prob,QuLDE())`

and you’re

there. Is it practical? Who knows (please let us know). Is it cool? Oh yeah!

See the quantum ODE solver blog post for more details.

The commutative noise SDE solvers are now GPU-compatible thanks to GSoC student

Deepesh Thakur (@deeepeshthakur). The next step will be to implement high order

non-commutative noise SDE solvers and the associated iterated integral

approximations in a manner that is GPU-compatible.

DiffEqBenchmarks.jl and DiffEqTutorials.jl are now fully updated to a Weave.jl

form. We still need to fix up a few benchmarks, but it’s in a state that is ready

for new contributions.

The GBS extrapolation methods have gotten optimized, and they now are the one

of the most efficient methods at lower tolerances of the Float64 range for

non-stiff ODEs:

Thank you to Konstantin Althaus (@AlthausKonstantin) for contributing the first

version of this algorithm and GSoC student Saurabh Agarwal (@saurabhkgp21) for

adding automatic parallelization of the method.

This method will soon see improvements as multithreading will soon be improved

in Julia v1.2. The new PARTR features will allow our internal `@threads`

loop

to perform dynamic work-stealing which will definitely be a good improvement to

the current parallelism structure. So stay tuned: this will likely benchmark

even better in a few months.

Thanks to Yingbo Ma (@YingboMa) for making the internal `exp`

calls of the

exponential integrators non-allocating. Continued improvements to this category

of methods is starting to show promise in the area of semilinear PDEs.

JSoC student Langwen Huang (@huanglangwen) has added the Rosenbrock-W class of

methods to OrdinaryDiffEq.jl. These methods are like the Rosenbrock methods

but are able to reuse their W matrix for multiple steps, allowing the method

to scale to larger ODEs more efficiently. Since the Rosenbrock methods

benchmark as the fastest methods for small ODEs right now, this is an exciting

new set of methods which will get optimized over the course of the summer.

Efficient Jacobian reuse techniques and the ability to utilize the sparse

differentiation tooling are next on this project.

Our current development is very much driven by the ongoing GSoC/JSoC projects,

which is a good thing because they are outputting some really amazing results!

Here’s some things to look forward to:

- Higher order SDE methods for non-commutative noise
- Parallelized methods for stiff ODEs
- Integration of sparse colored differentiation into the differential equation solvers
- Jacobian reuse efficiency in Rosenbrock-W methods
- Exponential integrator improvements
- Native Julia fully implicit ODE (DAE) solving in OrdinaryDiffEq.jl
- Automated matrix-free finite difference PDE operators
- Surrogate optimization
- GPU-based Monte Carlo parallelism

The post DifferentialEquations.jl v6.6.0: Sparse Jacobian Coloring, Quantum Computer ODE Solvers, and Stiff SDEs appeared first on juliabloggers.com.

]]>The post Deep Learning: Exploring High Level APIs of Knet.jl and Flux.jl in comparison to Tensorflow-Keras appeared first on juliabloggers.com.

]]>When it comes to complex modeling, specifically in the field of deep learning, the go-to tool for most researchers is the Google’s TensorFlow. There are a number of good reason as to why, one of it is the fact that it provides both high and low level APIs that suit the needs of both beginners and advanced users, respectively. I have used it in some of my projects, and indeed it was powerful enough for the task. This is also due to the fact that TensorFlow is one of the most actively developed deep learning framework, with Bayesian inference or probabilistic reasoning as the recent extension (see TensorFlow Probability, another extension is the TensorFlow.js). While the library is written majority in C++ for optimization, the main API is served in Python for ease of use. This design works around the static computational graph that needs to be defined declaratively before executed. The static nature of this graph, however, led to difficulty on debugging the models since the codes are itself data for defining the computational graph. Hence, you cannot use a debugger to check the results of the models line by line. Thankfully, it’s 2019 already and we have a stable Eager Execution that allows users to immediately check the results of any TensorFlow operations. Indeed, this is more intuitive and more pythonic. In this article, however, we’ll attempt to explore, what else we have in 2019. In particular, let’s take look at Julia’s deep learning libraries and compare it to high level APIs of TensorFlow, i.e. Keras’ model specification.

As a language that leans towards numerical computation, it’s no surprise that Julia offers a number of choices for doing deep learning, here are the stable libraries:

- Flux.jl – The Elegant Machine Learning Stack.
- Knet.jl – Koç University deep learning framework.
- MLJ.jl – Julia machine learning framework by Alan Turing Institute.
- MXNet.jl – Apache MXNet Julia package.
- TensorFlow.jl – A Julia wrapper for TensorFlow.

Other related packages are maintained in JuliaML. For this article, we are going to focus on the usage of

Flux.jl and Knet.jl, and we are going to use the Iris dataset for classification task using Multilayer Perceptron. To start with, we need to install the following packages. I’m using Julia 1.1.0. and Python 3.7.3.

The Iris dataset is available in the RDatasets.jl Julia package and in Python’s Scikit-Learn. The following codes load the libraries and the data itself.

The random seed set above is meant for reproducibility as it will give us the same random initial values for model training. The `iris`

variable in line 11 (referring to Julia code) contains the data, and is a data frame with 150 × 5 dimensions, where the columns are: Sepal Length, Sepal Width, Petal Length, Petal Width, and Species. There are several ways to partition this data into training and testing datasets, one procedure is to do stratified sampling, with simple random sampling without replacement as the sampling selection within each stratum — the species. The following codes define the function for partitioning the data with the mentioned sampling design:

Extract the training and testing datasets using the function above as follows:

All three codes above extract `xtrn`

, the training data (feature) matrix of size 105 × 4 (105 observations by 4 features) dimensions; `ytrn`

, the corresponding training target variable with 105 × 1 dimension; `xtst`

, the feature matrix for testing dataset with 45 × 4 dimensions; and `ytst`

, the target variable with 45 × 1 dimension for testing dataset. Moreover, contrary to TensorFlow-Keras, Knet.jl and Flux.jl need further data preparation from the above partitions. In particular, Knet.jl takes minibatch object as input data for model training, while Flux.jl needs one-hot encoding for the target variables `ytrn`

and `ytst`

. Further, unlike Knet.jl which ships with minibatch function, Flux.jl gives the user the flexibility to create their own.

The model that we are going to use is a Multilayer Perceptron with the following architecture: 4 neurons for the input layer, 10 neurons for the hidden layer, and 3 neurons for the output layer. The first two layers contain bias, and the neurons of the last two layers are activated with Rectified Linear Unit (ReLU) and softmax functions, respectively. The diagram below illustrates the architecture described:

The codes below specify the model:

Coming from TensorFlow-Keras, Flux.jl provides Keras-like API for model specification, with `Flux.Chain`

as the counterpart for Keras’ `Sequential`

. This is different from Knet.jl where the highest level API you can get are the nuts and bolts for constructing the layers. Having said, however, `Flux.Dense`

is defined almost exactly as the Dense struct of the Knet.jl code above (check the source code here). In addition, since both Flux.jl and Knet.jl are written purely in Julia, makes the source codes under the hood accessible to beginners. Thus, giving the user a full understanding of not just the code, but also the math. Check the screenshots below for the distribution of the file types in the Github repos of the three frameworks:

From the above figure, it’s clear that Flux.jl is 100% Julia. On the other hand, Knet.jl while not apparent is actually 100% Julia as well. The 41.4% of Jupyter Notebooks and other small percentages account for the tutorials, tests and examples and not the source codes.

Finally, train the model as follows for 100 epochs:

The codes (referring to Julia codes) above save both loss and accuracy for every epoch into a data frame and then into a CSV file. These will be used for visualization. Moreover, unlike Flux.jl and Knet.jl which require minibatch preparation prior to training, TensorFlow-Keras specifies this on `fit`

method as shown above. Further, it is also possible to train the model in Knet.jl using a single function without saving the metrics. This is done as follows:

The Flux.jl code above simply illustrates the use of `Flux.@epochs`

macro for looping instead of the `for`

loop. The loss of the model for 100 epochs is visualized below across frameworks:

From the above figure, one can observe that Flux.jl had a bad starting values set by the random seed earlier, good thing Adam drives the gradient vector rapidly to the global minimum. The figure was plotted using Gadfly.jl. Install this package using `Pkg`

as described in the first code block, along with Cario.jl and Fontconfig.jl. The latter two packages are used to save the plot in PNG format, see the code below to reproduce:

The output of the model ends with a vector of three neurons. The index or location of the neurons in this vector defines the corresponding integer encoding, with 1st index as setosa, 2nd as versicolor, and 3rd as virginica. Thus, the codes below take the argmax of the vector to get the integer encoding for evaluation.

The figure below shows the traces of the accuracy during training:

TensorFlow took 25 epochs before surpassing 50% again. To reproduce the figure, run the following codes (make sure to load Gadfly.jl and other related libraries mentioned earlier in generating the loss plots):

At this point, we are going to record the training time of each framework.

The benchmark was done by running the above code repeatedly for about 10 times for each framework, I then took the lowest timestamp out of the results. In addition, before running the code for each framework, I keep a fresh start of my machine.

The code of the above figure is given below (make sure to load Gadfly.jl and other related libraries mentioned earlier in generating the loss plots):

In conclusion, I would say Julia is worth investing even for deep learning as illustrated in this article. The two frameworks, Flux.jl and Knet.jl, provide a clean API that introduces a new way of defining models, as opposed to the object-oriented approach of the TensorFlow-Keras. One thing to emphasize on this is the `for`

loop which I plainly added in training the model just to save the accuracy and loss metrics. The `for`

loop did not compromise the speed (though Knet.jl is much faster without it). This is crucial since it let’s the user spend more on solving the problem and less on optimizing the code. Further, between the two Julia frameworks, I find Knet.jl to be Julia + little-else, as described by Professor Deniz Yuret (the main developer), since there are no special APIs for Dense, Chains, etc., you have to code it. Although this is also possible for Flux.jl, but Knet.jl don’t have these out-of-the-box, it ships only with the nuts and bolts, and that’s the highest level APIs the user gets. Having said, I think Flux.jl is a better recommendation for beginners coming from TensorFlow-Keras. This is not to say that Knet.jl is hard, it’s not if you know Julia already. In addition, I do love the extent of flexibility on Knet.jl by default which I think is best for advanced users. Lastly, just like the different extensions of TensorFlow, Flux.jl is flexible enough that it works well with Turing.jl for doing Bayesian deep learning, which is a good alternative for TensorFlow Probability. For Neural Differential Equations, Flux.jl works well with DifferentialEquations.jl, checkout DiffEqFlux.jl.

In my next article, we will explore the low level APIs of Flux.jl and Knet.jl in comparison to the low level APIs of TensorFlow. One thing that’s missing also from the above exercise is the use of GPU for model training, and I hope to tackle this in future articles. Finally, I plan to test these Julia libraries on real deep learning problems, such as computer vision and natural language processing (checkout the workshop on these from JuliaCon 2018).

If you are impatient, here are the complete codes excluding the benchmarks and the plots. These should work after installing the required libraries shown above:

- Yuret, Deniz (2016). Knet: beginning deep learning with 100 lines of Julia. 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain.
- Innes, Mike (2018). Flux: Elegant machine learning with Julia. Journal of Open Source Software, 3(25), 602, https://doi.org/10.21105/joss.00602
- Abadi, Martin et al (2016). TensorFlow-Keras: A system for large-scale machine learning. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). p265–283.

The post Deep Learning: Exploring High Level APIs of Knet.jl and Flux.jl in comparison to Tensorflow-Keras appeared first on juliabloggers.com.

]]>into account when computing statistics?

Synthetic Gold mine.

In this tutorial, I illustrate how not taking coordinates of

samples into account can lead to overestimation of re...

The post Declustered statistics appeared first on juliabloggers.com.

]]>Re-posted from: https://juliohm.github.io/science/declust-stats/

Why it is important to take spatial coordinates of samples

into account when computing statistics?

In this tutorial, I illustrate how not taking coordinates of

samples into account can lead to overestimation of resources,

and how GeoStats.jl overcomes this very common issue:

The post Declustered statistics appeared first on juliabloggers.com.

]]>new features for performance in specific domains. Large ODEs, stiff SDEs, high

accuracy ODE solving, many callbacks, etc. are all specialized on and greatly

improved in ...

The post DifferentialEquations.jl v6.5.0: Stiff SDEs, VectorContinuousCallback, Multithreaded Extrapolation appeared first on juliabloggers.com.

]]>Re-posted from: http://juliadiffeq.org/2019/06/06/StiffSDEs.html

Well, we zoomed towards this one. In this release we have a lot of very compelling

new features for performance in specific domains. Large ODEs, stiff SDEs, high

accuracy ODE solving, many callbacks, etc. are all specialized on and greatly

improved in this PR.

The post DifferentialEquations.jl v6.5.0: Stiff SDEs, VectorContinuousCallback, Multithreaded Extrapolation appeared first on juliabloggers.com.

]]>At the time we made DiffEqFlux, we were the "first to the gate" for many of these differential equations types and left it as an open question for people to find a use for these tools. And judging by the Arxiv papers that went out days after NeurIPS submissions were due, it looks like people now have justified some machine learning use cases for them. There were two separate papers on neural ... READ MORE

The post Neural Jump SDEs (Jump Diffusions) and Neural PDEs appeared first on Stochastic Lifestyle.

The post Neural Jump SDEs (Jump Diffusions) and Neural PDEs appeared first on juliabloggers.com.

]]>Re-posted from: http://www.stochasticlifestyle.com/neural-jump-sdes-jump-diffusions-and-neural-pdes/

This is just an exploration of some new neural models I decided to jot down for safe keeping. DiffEqFlux.jl gives you the differentiable programming tools to allow you to use any DifferentialEquations.jl problem type (DEProblem) mixed with neural networks. We demonstrated this before, not just with neural ordinary differential equations, but also with things like neural stochastic differential equations and neural delay differential equations.

At the time we made DiffEqFlux, we were the “first to the gate” for many of these differential equations types and left it as an open question for people to find a use for these tools. And judging by the Arxiv papers that went out days after NeurIPS submissions were due, it looks like people now have justified some machine learning use cases for them. There were two separate papers on neural stochastic differential equations, showing them to be the limit of deep latent Gaussian models. Thus when you stick these new mathematical results on our existing adaptive high order GPU-accelerated neural SDE solvers, you get some very interesting and fast ways to learn some of the most cutting edge machine learning methods.

So I wanted to help you guys out with staying one step ahead of the trend by going to the next differential equations. One of the interesting NeurIPS-timed Arxiv papers was on jump ODEs. Following the DiffEqFlux.jl spirit, you can just follow the DifferentialEquations.jl tutorials on these problems, implement them, add a neural network, and it will differentiate through them. So let’s take it one step further and show an example of how you’d do that. I wanted to take a look at jump diffusions, or jump stochastic differential equations, which are exactly what they sound like. They are a mixture of these two methods. After that, I wanted to show how using some methods for stiff differential equations plus a method of lines discretization gives a way to train neural partial differential equations.

Instead of being fully defined by neural networks, I will also be showcasing how you can selectively make parts of a differential equation neuralitized and other parts pre-defined, something we’ve been calling mixed neural differential equations, so we’ll demonstrate a mixed neural jump stochastic differential equation and a mixed neural partial differential equation with fancy GPU-accelerated adaptive etc. methods. I’ll then leave as homework how to train a mixed neural jump stochastic partial differential equation with the fanciest methods, which should be easy to see from this blog post (so yes, that will be the MIT 18.337 homework). This blog post will highlight that these equations are all already possible within our framework, and will also show the specific places we see that we need to accelerate to really put these types of models into production.

To get to jump diffusions, let’s start with a stochastic differential equation. A stochastic differential equation is defined via

which is essentially saying that there is a deterministic term and a continuous randomness term driven by a Brownian motion. Theorems like Donsker’s theorem can be thought of as a generalization of the central limit theorem, saying that continuous stochastic processes of some large class can be reinterpreted as this kind of process (due to the Gaussian-ness of Brownian motion), so in some sense this is a very large encompassing class. If you haven’t seen the previous blog post which mentions how to define neural SDEs, please check that out now. Let’s start with a code that uses reverse-mode automatic differentiation through a GPU-accelerated high order adaptive SDE solver. The code looks like:

This just uses the diffeq_rd layer function to tell Flux to use reverse-mode AD (using Tracker.jl, unless you check out a bunch of weird Zygote.jl branches: wait for Zygote) and then trains the neural network using a discrete adjoint. While the previously posted example uses forward-mode, we have found that this is much much faster on neural SDEs, so if you’re trying to train them, I would recommend using this code instead (and I’ll get the examples updated).

Now to this equation let’s add jumps. A jump diffusion is defined like:

where are the jump terms. The jump terms differ from the Brownian terms because they are non-continuous: they are zero except at countably many time points where you “hit” the equation with an amount . The timing at which these occur is based on an internal rate of the jump .

Jump diffusions are important because, just as there is a justification for the universality of stochastic differential equations, there is a justification here as well. The Levy Decomposition says that essentially any Markov process can be decomposed into something of this form. They also form the basis for many financial models, because for example changing regimes into a recession isn’t gradual but rather sudden. Models like Merton’s model thus use these as an essential tool in quantitative finance. So let’s train a neural network on that!

What we have to do is define jump processes and append them onto an existing differential equation. The documentation shows how to use the different jump definitions along with their pros and cons, so for now we will use ContinuousRateJump. Let’s define a ContinuousRateJump which has a constant rate and a neural network that decides what the effect of the jump () will be. To do this, you’d simply put the neural network in there:

where dudt2 is another neural network, and then wrap that into a jump problem:

And of course you can make this fancier: just replace that rate 2.0 with another neural network, make the g(u,p,t) term also have a neural network, etc.: explore this as you wish and go find some cool stuff. Let’s just stick with this as our example though, but please go ahead and make these changes and allow DiffEqFlux.jl to help you to explore your craziest mathematical idea!

Now when you solve this, the jumps also occur along with the stochastic differential equation. To show what that looks like, let’s define a jump diffusion and solve it 100 times, taking its mean as our training data:

From the plot you can see wild discontinuities mixed in with an equation with continuous randomness. Just lovely.

A full code for training a neural jump diffusion thus is:

Notice how it’s almost exactly the same as the SDE code but with the definition of the jumps. You still get the same high order adaptive GPU-accelerated (choice of implicit, etc.) SDE solvers, but now to this more generalized class of problems. Using the GPU gives a good speedup in the neural network case, but slows it down quite a bit when generating the training data since it’s not very parallel. Finding out new ways to use GPUs is one thing I am interested in perusing here. Additionally, using a lower tolerance StackOverflows Tracker.jl, which is something we have fixed with Zygote.jl and will be coming to releases once Zygote.jl on the differential equation solvers is more robust. Lastly, the plotting with GPU-based arrays is wonky right now, we’ll need to make the interface a little bit nicer. However, this is a proof of concept that this stuff does indeed work, though it takes awhile to train it to a “decent” loss (way more than the number of repetitions showcased in here).

[Note: you need to add using CuArrays to enable the GPU support. I turned it off by default because I was training this on my dinky laptop :)]

Now let’s do a neural partial differential equation (PDE). We can start by pulling code from this older blog post on solving systems of stochastic partial differential equations with GPUs. Here I’m going to strip the stochastic part off, simply because I want to train this on my laptop before the flight ends, so again I’ll leave it as an exercise to do the same jump diffusion treatment to this PDE. Let’s start by defining the method of lines discretization for our PDE. If you don’t know what that is, please go read that blog post on defining SPDEs. What happens is the discretization gives you a set of ODEs to solve, which looks like:

The interesting part of this neural differential equation is the local/global aspect of parts. The mapslices call makes it so that way there’s a local nonlinear function of 3 variables applied at each point in space. While it keeps the neural network small, this currently does not do well with reverse-mode automatic differentiation or GPUs. That isn’t a major problem here because, since the neural network is kept small in this architecture, the number of parameters is also quite small. That said, reverse-mode AD will be required for fast adjoint passes, so this is still a work in progress / proof of concept, with a very specific point made (all that’s necessary here is overloads to make mapslices work well).

One point that really came out of this was the ODE solver methods. The ROCK2 method is much faster when generating the training data and when running diffeq_fd. It was a difference of 3 minutes with ROCK2 vs 40 minutes with BS3 (on the CPU), showing how specialized methods really are the difference between the problem being solvable or not. The standard implicit methods like Rodas5 aren’t performing well here either since the 30,000×30,000 dense matrix, and I didn’t take the time to specify sparsity patterns or whatnot to actually make them viable competitors. So for the lazy neural ODE use with sparsity, ROCK2 seems like a very interesting option. This is a testament to our newest GSoC crew’s results since it’s one of the newer methods implemented by our student Deepesh Thakur. There are still a few improvements that need to be made to make the eigenvalue estimates more GPU-friendly as well, making this performance result soon carry over to GPUs as well (currently, the indexing in this part of the code gives it trouble, so a PR is coming probably in a week or so). Lastly, I’m not sure what’s a good picture for these kinds of things, so I’m going to have to think about how to represent a global neural PDE fit.

Have fun with this. There are still some rough edges, for example plotting is still a little wonky because all of the automatic DiffEq solution plotting seems to index, so the GPU-based arrays don’t like that (I’ll update that soon now that it’s becoming a standard part of the workflow). Use it as starter code and find some cool stuff. Note that the examples shown here are not the only ones that are possible. This all just uses Julia’s generic programming and differentiable programming infrastructure in order to automatically generate code that is compatible with GPUs and automatic differentiation, so it’s impossible for me to enumerate all of the possible combinations. That means there’s plenty of things to explore. These are very early preliminary results, but shows that these equations are all possible. These examples show some places where we want to continue accelerating by both improving the methods and their implementation details. I look forward to doing an update with Zygote soon.

Christopher Rackauckas, Neural Jump SDEs (Jump Diffusions) and Neural PDEs, *The Winnower*6:e155975.53637 (2019). DOI:10.15200/winn.155975.53637

This post is open to read and review on The Winnower.

The post Neural Jump SDEs (Jump Diffusions) and Neural PDEs appeared first on Stochastic Lifestyle.

The post Neural Jump SDEs (Jump Diffusions) and Neural PDEs appeared first on juliabloggers.com.

]]>The post Vertex removal in LightGraphs appeared first on juliabloggers.com.

]]>Re-posted from: https://matbesancon.github.io/post/2019-05-30-vertex-safe-removal/

In various graph-related algorithms, a graph is modified through successive operations, merging, creating and deleting vertices. That’s the case for the Blossom algorithm finding a best matching in a graph and using contractions of nodes. In such cases, it can be useful to keep the same vertex numbers and delete only the vertex number being contracted.

LightGraphs.jl offers a set of abstractions, types and algorithms to get started with graphs.

The post Vertex removal in LightGraphs appeared first on juliabloggers.com.

]]>the run time types of its arguments. The Julia programming language uses

multiple dispatch as its main

paradigm, and this has been a central design concept of the language from its

inception.

I already talked about multiple dispatch when I showed a simple implementation

of the rock-paper-scissors game.

Multiple dispatch is useful not only to develop games, but also to write

mathematically correct code.

A key idea of multiple dispatch is that methods don’t belong to a single class,

but they are shared by all the arguments. Quoting from Julia

documentation:

does the addition operation in

`x + y`

belong to`x`

any more than it does to

`y`

? The implementation of a mathematical operator generally depends on the

types of all of its arguments. Even beyond mathematical operations, however,

multiple dispatch ends up being a powerful and convenient paradigm for

structuring and organizing programs.

An interesting example of how multiple dispatch allows one to more easily write

mathematically correct code is the division involving complex numbers. In

particular, the corner case of division by 0. But… is the 0 real or complex?

Indeed the answer is different depending on the nature of the 0. When you

divide a complex finite number by (positive) real 0, you’re computing the

limit:

and it is clear the direction in which you take the limit: the real axis. Thus,

for example, from the division we should expect as a result

, assuming that the limit is taken from the positive part of

the real axis. When the 0 is complex, instead, the direction of the limit in

the complex plane is not well-defined, and so the result of the division by a

complex zero should be undefined. This behaviour is also mandated by the

ISO/IEC 10967 standard (part 3,

section 5.2.5.5 “Fundamental complex floating point arithmetic”).

Let’s see how different programming languages behave with this operation.

```
>>> import numpy as np
>>> np.complex128(1) / 0
__main__:1: RuntimeWarning: divide by zero encountered in cdouble_scalars
__main__:1: RuntimeWarning: invalid value encountered in cdouble_scalars
(inf+nanj) # This is correct
>>> np.float64(1) / np.complex128(0)
__main__:1: RuntimeWarning: divide by zero encountered in true_divide
__main__:1: RuntimeWarning: invalid value encountered in true_divide
(inf+nanj) # Should be (nan+nanj)
>>> np.complex128(1) / np.complex128(0)
(inf+nanj) # Should be (nan+nanj)
>>> np.float64(1) / np.complex128(1)
(1+0j) # Should be (1-0j)
```

```
> complex(real = 1.0) / 0.0
[1] Inf+NaNi # This is correct
> 1.0 / complex(real = 0.0)
[1] Inf+NaNi # Should be NaN+NaNi
> complex(real = 1.0) / complex(real = 0.0)
[1] Inf+NaNi # Should be NaN+NaNi
> 1.0 / complex(real = 1.0)
[1] 1+0i # Should be 1-0i
```

```
octave:1> (1.0 + 0.0*i) / 0.0
warning: division by zero
ans = Inf
octave:2> 1.0 / (0.0 + 0.0*i)
warning: division by zero
ans = Inf
octave:3> (1.0 + 0.0*i) / (0.0 + 0.0*i)
warning: division by zero
ans = Inf
octave:4> 1.0 / (1.0 + 0.0*i)
ans = 1
```

```
#include <complex.h>
#include <stdio.h>
int main(void)
{
double complex c_zero = 0.0 + 0.0*I;
double complex c_one = 1.0 + 0.0*I;
double complex w1 = c_one / 0.0;
double complex w2 = 1.0 / c_zero;
double complex w3 = c_one / c_zero;
double complex w4 = 1.0 / c_one;
printf("(%g,%g) # Should be (inf,nan)\n", creal(w1), cimag(w1));
printf("(%g,%g) # Should be (nan,nan)\n", creal(w2), cimag(w2));
printf("(%g,%g) # Should be (nan,nan)\n", creal(w3), cimag(w3));
printf("(%g,%g) # Should be (1,-0)\n", creal(w4), cimag(w4));
return 0;
}
```

```
$ gcc -std=c99 test.c&&./a.out
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
$ icc -std=c99 test.c&&./a.out
test.c(7): warning #39: division by zero
double complex w1 = c_one / 0.0;
^
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
#include <complex>
#include <iostream>
int main(void)
{
std::complex<double> c_zero(0.0, 0.0);
std::complex<double> c_one(1.0, 0.0);
std::complex<double> w1 = c_one / 0.0;
std::complex<double> w2 = 1.0 / c_zero;
std::complex<double> w3 = c_one / c_zero;
std::complex<double> w4 = 1.0 / c_one;
std::cout << w1 << " # Should be (inf,nan)" << std::endl;
std::cout << w2 << " # Should be (nan,nan)" << std::endl;
std::cout << w3 << " # Should be (nan,nan)" << std::endl;
std::cout << w4 << " # Should be (1,-0)" << std::endl;
return 0;
}
```

```
$ g++ test.cpp&&./a.out
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
$ icc test.cpp&&./a.out
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
program main
implicit none
complex :: c_zero = (0.0, 0.0)
complex :: c_one = (1.0, 0.0)
double precision :: f_zero = 0.0
double precision :: f_one = 1.0
write (*, '(F0.0,"+i",F0.0,A)') c_one / f_zero, " # Should be Inf+iNaN"
write (*, '(F0.0,"+i",F0.0,A)') f_one / c_zero, " # Correct"
write (*, '(F0.0,"+i",F0.0,A)') c_one / c_zero, " # Correct"
write (*, '(F0.0,"+i",F0.0,A)') f_one / c_one, " # Should be 1.-i0."
endprogram main
```

```
$ gfortran test.f90&&./a.out
NaN+iNaN # Should be Inf+iNaN
NaN+iNaN # Should be NaN+iNaN
NaN+iNaN # Should be NaN+iNaN
1.+i0. # Should be 1.-i0.
```

```
$ ifort test.f90&&./a.out
Inf+iNaN # Should be Inf+iNaN
Inf+iNaN # Should be NaN+iNaN
Inf+iNaN # Should be NaN+iNaN
1.+i0. # Should be 1.-i0.
```

```
julia> complex(1) / 0
Inf + NaN*im # This is correct
julia> 1 / complex(0)
NaN + NaN*im # This is correct
julia> complex(1) / complex(0)
NaN + NaN*im # This is correct
julia> 1 / complex(1)
1.0 - 0.0im # This is correct
```

The difference between Julia and the other languages is that the division is

implemented in a different way depending on the type of the two operands. By

using the

`@which`

macro we can see which are the methods called in the examples above:

```
julia> @which complex(1) / 0
/(z::Complex, x::Real) in Base at complex.jl:324
julia> @which 1 / complex(0)
/(a::R, z::S) where {R<:Real, S<:Complex} in Base at complex.jl:323
julia> @which complex(1) / complex(0)
/(a::Complex{T}, b::Complex{T}) where T<:Real in Base at complex.jl:327
```

Thus each combination of real and complex arguments calls a different method.

This directly matches the different mathematical operations that should be

carried out.

The fact that in the other languages reviewed here we get the same result for

any combination of the types of the operands suggests that in those cases the

arguments are always silently converted to or treated as they were all complex,

thus losing the information about their original nature.

Multiple dispatch is nothing completely magic. One could achieve similar

results in Python with a bunch of `if`

s that check the types of the arguments,

but multiple dispatch keeps the different implementations neatly separated in

different methods, instead of putting the code for the different operations in a

single catch-em-all function. Also in C++ one could define different methods

for different combinations of the type of the arguments, with the difference

that in that case the dispatch would be dynamic only on the first argument (the

class instance, the receiver of the method) and static on the others, whereas

with multiple dispatch the dispatch is based on the runtime type of *all* the

arguments.

*Note*: this post is largely inspired by the discussion in issue

#22983 of the Julia

repository.

The post A case for multiple dispatch: complex division appeared first on juliabloggers.com.

]]>Re-posted from: http://giordano.github.io/blog/2019-05-26-complex-division/

Multiple dispatch is the ability to dispatch a function based on the number and

the run time types of its arguments. The Julia programming language uses

multiple dispatch as its main

paradigm, and this has been a central design concept of the language from its

inception.

I already talked about multiple dispatch when I showed a simple implementation

of the rock-paper-scissors game.

Multiple dispatch is useful not only to develop games, but also to write

mathematically correct code.

A key idea of multiple dispatch is that methods don’t belong to a single class,

but they are shared by all the arguments. Quoting from Julia

documentation:

does the addition operation in

`x + y`

belong to`x`

any more than it does to

`y`

? The implementation of a mathematical operator generally depends on the

types of all of its arguments. Even beyond mathematical operations, however,

multiple dispatch ends up being a powerful and convenient paradigm for

structuring and organizing programs.

An interesting example of how multiple dispatch allows one to more easily write

mathematically correct code is the division involving complex numbers. In

particular, the corner case of division by 0. But… is the 0 real or complex?

Indeed the answer is different depending on the nature of the 0. When you

divide a complex finite number by (positive) real 0, you’re computing the

limit:

and it is clear the direction in which you take the limit: the real axis. Thus,

for example, from the division we should expect as a result

, assuming that the limit is taken from the positive part of

the real axis. When the 0 is complex, instead, the direction of the limit in

the complex plane is not well-defined, and so the result of the division by a

complex zero should be undefined. This behaviour is also mandated by the

ISO/IEC 10967 standard (part 3,

section 5.2.5.5 “Fundamental complex floating point arithmetic”).

Let’s see how different programming languages behave with this operation.

```
>>> import numpy as np
>>> np.complex128(1) / 0
__main__:1: RuntimeWarning: divide by zero encountered in cdouble_scalars
__main__:1: RuntimeWarning: invalid value encountered in cdouble_scalars
(inf+nanj) # This is correct
>>> np.float64(1) / np.complex128(0)
__main__:1: RuntimeWarning: divide by zero encountered in true_divide
__main__:1: RuntimeWarning: invalid value encountered in true_divide
(inf+nanj) # Should be (nan+nanj)
>>> np.complex128(1) / np.complex128(0)
(inf+nanj) # Should be (nan+nanj)
>>> np.float64(1) / np.complex128(1)
(1+0j) # Should be (1-0j)
```

```
> complex(real = 1.0) / 0.0
[1] Inf+NaNi # This is correct
> 1.0 / complex(real = 0.0)
[1] Inf+NaNi # Should be NaN+NaNi
> complex(real = 1.0) / complex(real = 0.0)
[1] Inf+NaNi # Should be NaN+NaNi
> 1.0 / complex(real = 1.0)
[1] 1+0i # Should be 1-0i
```

```
octave:1> (1.0 + 0.0*i) / 0.0
warning: division by zero
ans = Inf
octave:2> 1.0 / (0.0 + 0.0*i)
warning: division by zero
ans = Inf
octave:3> (1.0 + 0.0*i) / (0.0 + 0.0*i)
warning: division by zero
ans = Inf
octave:4> 1.0 / (1.0 + 0.0*i)
ans = 1
```

```
#include <complex.h>
#include <stdio.h>
int main(void)
{
double complex c_zero = 0.0 + 0.0*I;
double complex c_one = 1.0 + 0.0*I;
double complex w1 = c_one / 0.0;
double complex w2 = 1.0 / c_zero;
double complex w3 = c_one / c_zero;
double complex w4 = 1.0 / c_one;
printf("(%g,%g) # Should be (inf,nan)\n", creal(w1), cimag(w1));
printf("(%g,%g) # Should be (nan,nan)\n", creal(w2), cimag(w2));
printf("(%g,%g) # Should be (nan,nan)\n", creal(w3), cimag(w3));
printf("(%g,%g) # Should be (1,-0)\n", creal(w4), cimag(w4));
return 0;
}
```

```
$ gcc -std=c99 test.c&&./a.out
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
$ icc -std=c99 test.c&&./a.out
test.c(7): warning #39: division by zero
double complex w1 = c_one / 0.0;
^
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
#include <complex>
#include <iostream>
int main(void)
{
std::complex<double> c_zero(0.0, 0.0);
std::complex<double> c_one(1.0, 0.0);
std::complex<double> w1 = c_one / 0.0;
std::complex<double> w2 = 1.0 / c_zero;
std::complex<double> w3 = c_one / c_zero;
std::complex<double> w4 = 1.0 / c_one;
std::cout << w1 << " # Should be (inf,nan)" << std::endl;
std::cout << w2 << " # Should be (nan,nan)" << std::endl;
std::cout << w3 << " # Should be (nan,nan)" << std::endl;
std::cout << w4 << " # Should be (1,-0)" << std::endl;
return 0;
}
```

```
$ g++ test.cpp&&./a.out
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
$ icc test.cpp&&./a.out
(inf,-nan) # Should be (inf,nan)
(inf,-nan) # Should be (nan,nan)
(inf,-nan) # Should be (nan,nan)
(1,0) # Should be (1,-0)
```

```
program main
implicit none
complex :: c_zero = (0.0, 0.0)
complex :: c_one = (1.0, 0.0)
double precision :: f_zero = 0.0
double precision :: f_one = 1.0
write (*, '(F0.0,"+i",F0.0,A)') c_one / f_zero, " # Should be Inf+iNaN"
write (*, '(F0.0,"+i",F0.0,A)') f_one / c_zero, " # Correct"
write (*, '(F0.0,"+i",F0.0,A)') c_one / c_zero, " # Correct"
write (*, '(F0.0,"+i",F0.0,A)') f_one / c_one, " # Should be 1.-i0."
endprogram main
```

```
$ gfortran test.f90&&./a.out
NaN+iNaN # Should be Inf+iNaN
NaN+iNaN # Should be NaN+iNaN
NaN+iNaN # Should be NaN+iNaN
1.+i0. # Should be 1.-i0.
```

```
$ ifort test.f90&&./a.out
Inf+iNaN # Should be Inf+iNaN
Inf+iNaN # Should be NaN+iNaN
Inf+iNaN # Should be NaN+iNaN
1.+i0. # Should be 1.-i0.
```

```
julia> complex(1) / 0
Inf + NaN*im # This is correct
julia> 1 / complex(0)
NaN + NaN*im # This is correct
julia> complex(1) / complex(0)
NaN + NaN*im # This is correct
julia> 1 / complex(1)
1.0 - 0.0im # This is correct
```

The difference between Julia and the other languages is that the division is

implemented in a different way depending on the type of the two operands. By

using the

`@which`

macro we can see which are the methods called in the examples above:

```
julia> @which complex(1) / 0
/(z::Complex, x::Real) in Base at complex.jl:324
julia> @which 1 / complex(0)
/(a::R, z::S) where {R<:Real, S<:Complex} in Base at complex.jl:323
julia> @which complex(1) / complex(0)
/(a::Complex{T}, b::Complex{T}) where T<:Real in Base at complex.jl:327
```

Thus each combination of real and complex arguments calls a different method.

This directly matches the different mathematical operations that should be

carried out.

The fact that in the other languages reviewed here we get the same result for

any combination of the types of the operands suggests that in those cases the

arguments are always silently converted to or treated as they were all complex,

thus losing the information about their original nature.

Multiple dispatch is nothing completely magic. One could achieve similar

results in Python with a bunch of `if`

s that check the types of the arguments,

but multiple dispatch keeps the different implementations neatly separated in

different methods, instead of putting the code for the different operations in a

single catch-em-all function. Also in C++ one could define different methods

for different combinations of the type of the arguments, with the difference

that in that case the dispatch would be dynamic only on the first argument (the

class instance, the receiver of the method) and static on the others, whereas

with multiple dispatch the dispatch is based on the runtime type of *all* the

arguments.

*Note*: this post is largely inspired by the discussion in issue

#22983 of the Julia

repository.

The post A case for multiple dispatch: complex division appeared first on juliabloggers.com.

]]>During my Geometr...

The post Bézier curves in Julia with animations appeared first on juliabloggers.com.

]]>Re-posted from: https://opensourc.es/blog/bezier-curve

I think many heard about Bézier curves but maybe some of you didn’t and I heard about it but wasn’t really sure what they are and how they work.

During my Geometric Modelling and Animations course in university we had some lectures on it and I did some coding for homeworks and also to understand a bit more about it. During the last days I published my animations on Twitter and asked whether you’re interested in a post. There was quite a bit feedback on that so here it is!

Let us start with a basic Bézier curve:

Before I show you what Bézier curves are we should probably have a short look what a basic curve is. A curve can be described by a parameterized description as:

\[

\mathbf{b}(t) = (x(t),y(t))^T \quad t_1 \leq t \leq t_2\]

whereas \(\mathbf{b}\) is a vector and \(x, y\) are polynomial functions like

\[

x(t) = a_0 + a_1t + a_2t^2 + \dots + a_nt^n\]

Now the idea is to compute the values on the curve from \(t_1 = 0\) to \(t_2 = 1\) using a different basis not simply \(t^0, \dots, t^n\).

For Bézier curves this basis are defined as:

\[

B_{i}^{n}(t) :=\left( \begin{array}{c}{n} \\ {i}\end{array}\right) t^{i}(1-t)^{n-i}, \quad 0 \leq i \leq n\]

and is called Bernstein basis.

Which looks a bit random for now but they have some interesting properties:

- \(B_{0}^{n}(t)+B_{1}^{n}(t)+\cdots+B_{n}^{n}(t)=1\)
- $B_0^n(0) = 1, B_n^n(1)=1

which means when we write down the complete formula to compute \(\mathbf{b}(t)\):

\[

\mathbf{b}(t) = \sum_{i=0}^n B_i^n \mathbf{b}_i\]

where \(\mathbf{b}_i\) are our control points. Now the second property just means that \(\mathbf{b}(0) = \mathbf{b}_0\) and \(\mathbf{b}(0) = \mathbf{b}_n\)

I think it would be nice to actually see the basis functions:

Now we can combine it with our control points to obtain:

Besides the change in color you probably don’t see a difference to the first plot even though that was plotted differently.

I know some of you are here for code so I’ll show you the bernstein code:

```
using Plots
# for the LaTeX labels in the legend
using LaTeXStrings
function compute_bernstein(i,n; steps=100)
return [binomial(n,i)*t^i*(1-t)^(n-i) for t in LinRange(0,1,steps)]
end
function compute_bernstein_poly(px,py; steps=100)
n = length(px)-1
bernsteins = [compute_bernstein(i,n) for i=0:n]
x_vals = [sum(px[k]*bernsteins[k][t] for k=1:n+1) for t=1:steps]
y_vals = [sum(py[k]*bernsteins[k][t] for k=1:n+1) for t=1:steps]
return x_vals, y_vals
end
function plot_with_bernstein(px,py; steps=100, subplot=1)
x_vals, y_vals = compute_bernstein_poly(px,py; steps=steps)
plot!(x_vals, y_vals, color=:blue, label="",subplot=subplot)
end
function main()
px = [0, 3, 7]
py = [2, 9, 3]
plot(;size=(700,500), axisratio=:equal, legendfont=font(13))
plot!(px, py, linetype=:scatter, label="control points")
plot_with_bernstein(px,py)
png("using_bernstein")
end
```

Pretty basic so far. Now combining the two and animate:

Actually I’m not too sure about how to interpret the colored animating part on the lower plot with the different bernstein polynomials but I think it looks interesting and I never saw that before. The three dots red, green and blue some up to the black in both ways.

Anyway I think the interesting things are still missing. First of all we only have 3 control points at…

The post Bézier curves in Julia with animations appeared first on juliabloggers.com.

]]>to the JuliaDiffEq package ecosystem. A lot of this release focuses on performance

features. The ability to use stiff ODE solvers on the GPU, with automated

tooling for matrix-f...

The post DifferentialEquations.jl v6.4.0: Full GPU ODE, Performance, ModelingToolkit appeared first on juliabloggers.com.

]]>Re-posted from: http://juliadiffeq.org/2019/05/09/GPU.html

This is a huge release. We should take the time to thank every contributor

to the JuliaDiffEq package ecosystem. A lot of this release focuses on performance

features. The ability to use stiff ODE solvers on the GPU, with automated

tooling for matrix-free Newton-Krylov, faster broadcast, better Jacobian

re-use algorithms, memory use reduction, etc. All of these combined give some

pretty massive performance boosts in the area of medium to large sized highly

stiff ODE systems. In addition, numerous robustness fixes have enhanced the

usability of these tools, along with a few new features like an implementation

of extrapolation for ODEs and the release of ModelingToolkit.jl.

Let’s start by summing up this release with an example.

Here’s a nice showcase of DifferentialEquations.jl: Neural ODE with batching on

the GPU (without internal data transfers) with high order adaptive implicit ODE

solvers for stiff equations using matrix-free Newton-Krylov via preconditioned

GMRES and trained using checkpointed adjoint equations. Few programs work

directly with neural networks and allow for batching, few utilize GPUs, few

have methods applicable to highly stiff equations, few allow for large stiff

equations via matrix-free Newton-Krylov, and finally few have checkpointed

adjoints. This is all done in a high level programming language. What does the

code for this look like?

```
using OrdinaryDiffEq, Flux, DiffEqFlux, DiffEqOperators, CuArrays
x = Float32[2.; 0.]|>gpu
tspan = Float32.((0.0f0,25.0f0))
dudt = Chain(Dense(2,50,tanh),Dense(50,2))|>gpu
p = destructure(model)
dudt_(u::TrackedArray,p,t) = restructure(model,p)(u)
dudt_(u::AbstractArray,p,t) = Flux.data(restructure(model,p)(u))
ff = ODEFunction(dudt_,jac_prototype = JacVecOperator(dudt_,x))
prob = ODEProblem(ff,x,tspan,p)
diffeq_adjoint(p,prob,KenCarp4();u0=x,
saveat=0.0:0.1:25.0,backsolve=false)
```

That is 10 lines of code, and we can continue to make it even more succinct.

Now, onto the release highlights.

Now not just the non-stiff ODE solvers but the stiff ODE solvers allow for

the initial condition to be a GPUArray, with the internal methods not

performing any indexing in order to allow for all computations to take place

on the GPU without data transfers. This allows for expensive right-hand side

calculations, like those in neural ODEs or PDE discretizations, to utilize

GPU acceleration without worrying about whether the cost of data

transfers will overtake the solver speed enhancements.

While the presence of broadcast throughout the solvers might worry one about

performance…

Yingbo Ma (@YingboMa) implemented a fancy broadcast wrapper that allows for

all sorts of information to be passed to the compiler in the differential

equation solver’s internals, making a bunch of no-aliasing and sizing assumptions

that are normally not possible. These change the internals to all use a

special `@..`

which turns out to be faster than standard loops, and this is the

magic that really enabled the GPU support to happen without performance

regressions (and in fact, we got some speedups from this, close to 2x in some

cases!)

One of the biggest performance-based features to be released is smarter linsolve

defaults. If you are using dense arrays with a standard Julia build, OpenBLAS

does not perform recursive LU factorizations which we found to be suboptimal

by about 5x in some cases. Thus our default linear solver now automatically

detects the BLAS installation and utilizes RecursiveFactorizations.jl to give

this speedup for many standard stiff ODE cases. In addition, if you passed a

sparse Jacobian for the `jac_prototype`

, the linear solver now automatically

switches to a form that works for sparse Jacobians. If you use an

`AbstractDiffEqOperator`

, the default linear solver automatically switches to

a Krylov subspace method (GMRES) and utilizes the matrix-free operator directly.

Banded matrices and Jacobians on the GPU are now automatically handled as well.

Of course, that’s just the defaults, and most of this was possible before but

now has just been made more accessible. In addition to these, the ability to

easily switch to GMRES was added via `LinSolveGMRES`

. Just add

`linsolve = LinSolveGMRES()`

to any native Julia algorithm with a swappable

linear solver and it’ll switch to using GMRES. In this you can pass options

for preconditioners and tolerances as well. We will continue to integrate this

better into our integrators as doing so will enhance the efficiency when

solving large sparse systems.

When using `GMRES`

, one does not need to construct the full Jacobian matrix.

Instead, one can simply use the directional derivatives in the direction of

`v`

in order to compute `J*v`

. This has now been put into an operator form

via `JacVecOperator(dudt_,x)`

, so now users can directly ask for this to

occur using one line. It allows for the use of autodifferentiation or

numerical differentiation to calculate the `J*v`

.

One of the nichest but nicest new features is DEStats. If you do `sol.destats`

then you will see a load of information on how many steps were taken, how many

`f`

calls were done, etc. giving a broad overview of the performance of the

algorithm. Thanks to Kanav Gupta (@kanav99) and Yingbo Ma (@YingboMa) for really

driving this feature since it has allowed for a lot of these optimizations to

be more thoroughly investigated. You can expect DiffEq development to

accelerate with this information!

One of the things which was noticed using DEStats was that the amount of Jacobians

and inversions that were being calculated could be severly reduced. Yingbo Ma (@YingboMa)

did just that, greatly increasing the performance of all implicit methods like

`KenCarp4`

showing cases in the 1000+ range where OrdinaryDiffEq’s native

methods outperformed Sundials CVODE_BDF. This still has plenty of room for

improvement.

Samuel Isaacson (@isaacson) has been instrumental in improving DiffEqBiological.jl

and its ability to handle large reaction networks. It can now parse the networks

much faster and can build Jacobians which utilize sparse matrices. It pairs

with his ParseRxns(???) library and has been a major source of large stiff

test problems!

We now have working examples of partial neural differential equations, which

are equations which have pre-specified portions that are known while others

are learnable neural networks. These also allow for batched data and GPU

acceleration. Not much else to say except let your neural diffeqs go wild!

Kanav Gupta (@kanav99) and Hendrik Ranocha (@ranocha) did amazing jobs at doing memory optimizations of

low-memory Runge-Kutta methods for hyperbolic or advection-dominated PDEs.

Essentially these methods have a minimal number of registers which are

theoretically required for the method. Kanav added some tricks to the implementation

(using a fun `=`

-> `+=`

overload idea) and Henrick added the `alias_u0`

argument

to allow for using the passed in initial condition as one of the registers. Unit

tests confirm that our implementations achieve the minimum possible number of

registers, allowing for large PDE discretizations to make use of

DifferentialEquations.jl without loss of memory efficiency. We hope to see

this in use in some large-scale simulation software!

Our `ContinuousCallback`

implementation now has increased robustness in double

event detection, using a new strategy. Try to break it.

New contributor Konstantin Althaus (@AlthausKonstantin) implemented midpoint

extrapolation methods for ODEs using Barycentric formulas and different a

daptivity behaviors. We will be investigating these methods for their

parallelizability via multithreading in the context of stiff and non-stiff ODEs.

ModelingToolkit.jl has now gotten some form of a stable release. A lot of credit

goes to Harrison Grodin (@HarrisonGrodin). While it has

already been out there and found quite a bit of use, it has really picked up

steam over the last year as a modeling framework suitable for the flexibility

DifferentialEquations.jl. We hope to continue its development and add features

like event handling to its IR.

While we are phasing out Sundials from our standard DifferentialEquations.jl

practice, the Sundials.jl continues to improve as we add more features to

benchmark against. Sundials’ J*v interface has now been exposed, so adding a

DiffEqOperator to the `jac_prototype`

will work with Sundials. `DEStats`

is

hooked up to Sundials, and now you can pass preconditioners to its internal

Newton-Krylov methods.

- Improved nonlinear solvers for stiff SDE handling
- More adaptive methods for SDEs
- Better boundary condition handling in DiffEqOperators.jl
- More native implicit ODE (DAE) solvers
- Adaptivity in the MIRK BVP solvers
- LSODA integrator interface
- Improved BDF

The post DifferentialEquations.jl v6.4.0: Full GPU ODE, Performance, ModelingToolkit appeared first on juliabloggers.com.

]]>The post A take on Benders decomposition in JuMP appeared first on juliabloggers.com.

]]>Re-posted from: https://matbesancon.github.io/post/2019-05-08-simple-benders/

Cracking Benders decomposition, one cut at a time.

The post A take on Benders decomposition in JuMP appeared first on juliabloggers.com.

]]>channel (request an invite

here!) prompted a lively discussion about

how to deal with pointers in Julia. As the history of the channel is ephemeral,

I wrote this post to keep note...

The post Getting the value of a pointer in Julia appeared first on juliabloggers.com.

]]>Re-posted from: http://giordano.github.io/blog/2019-05-03-julia-get-pointer-value/

Yesterday a question in the Julia’s Slack

channel (request an invite

here!) prompted a lively discussion about

how to deal with pointers in Julia. As the history of the channel is ephemeral,

I wrote this post to keep note of some interesting points.

In the C world, the operation of getting the value of a pointer is called

dereferencing. When

dealing with C libraries in Julia, we may need to get the value of a pointer

into a Julia object. The Julia manual is quite detailed about accessing data

in

pointers

and I warmly recommend reading that first.

Note that, usually, in Julia it’s not possible to “dereference” a pointer in the

C-sense, because most of the operations we’re going to see will *copy* the data

into Julia structures.

`unsafe_load`

is

the generic function to access the value of a pointer and copy it to a Julia

object. However, depending on the specific data types involved, there may be

different and more efficient solutions. For example, you can obtain the value

of the pointer to an array with

`unsafe_wrap`

,

while for strings there is

`unsafe_string`

.

Let’s have a look at a simple time-related example.

```
julia> result = Ref{Int64}(0)
Base.RefValue{Int64}(0)
julia> ccall((:time, "libc.so.6"), Int64, (Ptr{Int64},), result)
1556838715
julia> result
Base.RefValue{Int64}(1556838715)
```

By calling the `time`

function from the C standard library, we have saved in

`result`

the current number of seconds since the Unix epoch. We now want to

turn this number into an instance of the C struct

`tm`

. To do this, we can use the

`localtime`

function. However, if we want to later use the `tm`

struct in Julia

we need to define a Julia mutable structure with the same layout as the C one

and define the `show`

method to have a fancy output:

```
julia> mutable struct Ctm
sec::Cint
min::Cint
hour::Cint
mday::Cint
mon::Cint
year::Cint
wday::Cint
yday::Cint
isdst::Cint
end
julia> function Base.show(io::IO, t::Ctm)
print(io, t.year + 1900, "-", lpad(t.mon, 2, "0"), "-",
lpad(t.mday, 2, "0"), "T", lpad(t.hour, 2, "0"), ":",
lpad(t.min, 2, "0"), ":", lpad(t.sec, 2, "0"))
end
```

We are now going to use `unsafe_load`

to copy the result of the call to

`localtime`

into an instance of the `Ctm`

structure:

```
julia> localtime = ccall((:localtime, "libc.so.6"), Ptr{Ctm}, (Ptr{Int64},), result)
Ptr{Ctm} @0x00007f730fe9d300
julia> unsafe_load(localtime)
2019-04-03T00:11:55
julia> dump(unsafe_load(localtime))
Ctm
sec: Int32 55
min: Int32 11
hour: Int32 0
mday: Int32 3
mon: Int32 4
year: Int32 119
wday: Int32 5
yday: Int32 122
isdst: Int32 1
```

Of course, if you want to deal with dates you can use the

`Dates`

standard library

instead of playing with system calls. Have also a look at the other

time-related functions in the `Base.Libc`

module of Julia.

Now that we’ve seen how to use `unsafe_load`

, we can define the following

function to “dereference” a generic pointer to a Julia object (with the caveat

that we’re copying it’s value to the new object!):

```
julia> dereference(ptr::Ptr) = unsafe_load(ptr)
dereference (generic function with 1 method)
julia> dereference(T::DataType, ptr::Ptr) = unsafe_load(Ptr{T}(ptr))
dereference (generic function with 2 methods)
```

This `dereference`

function has two arguments: the type that will wrap the data,

and the input pointer. The first argument is optional, and defaults to the type

of the data pointed by `ptr`

. The first method is effectively an alias of the

plain `unsafe_load`

. The second method is interesting because it allows us to

copy the value of a pointer of a certain data type to an object of another type

with the same memory layout.

For example, let’s define a simple Julia mutable structure called `Bar`

, get

an instance of it, and its pointer:

```
julia> mutable struct Bar
x::UInt64
end
julia> b = Bar(rand(UInt64))
Bar(0x55139e3fd61e43a4)
julia> pointer_from_objref(b)
Ptr{Nothing} @0x00007f1f72eb7850
julia> ptr = Ptr{Bar}(pointer_from_objref(b))
Ptr{Bar} @0x00007f1f72eb7850
```

`pointer_from_objref`

gave us a pointer to `Nothing`

(that is, the C `void`

), we have then casted it

to a pointer to `Bar`

and assigned this pointer to `ptr`

. We now define another

data structure with the same memory layout as `Bar`

and use

`dereference`

to “dereference” `ptr`

as an instance of the new structure:

```
julia> mutable struct Foo
a::UInt16
b::UInt16
c::UInt32
end
julia> f = dereference(Foo, ptr)
Foo(0x43a4, 0xd61e, 0x55139e3f)
```

Note that

```
julia> UInt64(f.a) | UInt64(f.b) << 16 | UInt64(f.c) << 32
0x55139e3fd61e43a4
julia> b.x
0x55139e3fd61e43a4
```

as expected.

Remember that the term “dereference” is a stretch, because data is copied from

the pointer. In fact:

```
julia> dereference(Foo, ptr) |> pointer_from_objref
Ptr{Nothing} @0x00007f1f72a62440
julia> dereference(Foo, ptr) |> pointer_from_objref
Ptr{Nothing} @0x00007f1f72fb3840
julia> dereference(Bar, ptr) |> pointer_from_objref
Ptr{Nothing} @0x00007f1f730054e0
julia> dereference(Bar, ptr) |> pointer_from_objref
Ptr{Nothing} @0x00007f1f73007340
```

What we’ve seen here is nothing new for programmers already familiar with the C

language, but at the same time shows how easy and natural can be to communicate

from Julia with C programs.

If you feel really bold and creative, you can even abuse the `*`

operator to do

*very* weird things like:

```
julia> Base.:*(ptr::Ptr) = dereference(ptr)
julia> Base.:*(T::DataType, ptr::Ptr) = dereference(T, ptr)
julia> *(ptr)
Bar(0x55139e3fd61e43a4)
julia> Foo *ptr
Foo(0x43a4, 0xd61e, 0x55139e3f)
julia> Bar *ptr
Bar(0x55139e3fd61e43a4)
```

However, the `*`

operator shouldn’t be really used to mock C syntax is an

awkward way (we’re not actually dereferencing and the syntax `Foo *ptr`

is for

declaration of a pointer rather than actual dereferencing), so don’t tell people

that I told you to use this!

The post Getting the value of a pointer in Julia appeared first on juliabloggers.com.

]]>The post Random Forest Regression from scratch used on Kaggle competition appeared first on juliabloggers.com.

]]>Re-posted from: https://opensourc.es/blog/random-forest

I normally write some code and some days, weeks, month later when everything is done I write a blog about it. I think the blog is more structured that way but maybe it has less details and my errors in between aren’t that visible anymore (just if someone of you comment afterwards ) Thanks for that. It really helps improve my code and this blog.

In the past couple of weeks I wanted to understand random forest better and not just how to use them so I’ve built my own package.

For me it’s not satisfying enough to use a machine learning package and tune the parameters I really like to build stuff from scratch.

That’s basically the reason for this blog I see too many blogs about: “How to use random forest?”, “How to achieve an awesome accuracy for MNIST?” but as mentioned in my post about the latter there aren’t many who dive deeper into it and actually use it for real (well and blog about it).

I learn a lot doing this and also when I blog about it and especially when I learn through comments but it also takes quite some time to write these posts. If you’ve read multiple of my posts and learned a good amount please consider a donation via Patreon to keep it going.

Thank you very much!

Of course for everyone else I keep to continue writing and you can’t pay everyone on the internet just because you read a post from them. It’s more about whether you enjoy this blog for a longer time now and found something you didn’t elsewhere.

Back to the track. I’m using random forest for my current Kaggle Challenge about predicting earth quakes. I might publish an extra article about that one when it’s done but for now I want to use my Random Forest package on a different challenge which is more for fun (I mean I do the other for fun as well but Kaggle/LANL are paying money for the winners) whereas this one is purely for training.

It’s about predicting house prices based on some features.

This post is on different aspects

- Creating a julia package (basics)
- Explaining random forest
- Simple features first
- What can be improved?

I’ve only have my random forest code at the moment so my score in the competition might (and probably will) turn out quite bad. Anyway the goal in general is to learn random forest (by coding it) and somehow apply it.

First let’s create our `RandomForestRegression`

package. Starting julia in your favorite projects folder.

`(v1.1) pkg> generate RandomForestRegression`

That creates a folder `RandomForestRegression`

with:

```
src/
- RandomForestRegression.jl
Project.toml
```

and the `RandomForestRegression.jl`

looks like this:

```
module RandomForestRegression
greet() = print("Hello World!")
end # module
```

we later want to add some dependencies to it but for now we are done. Normally you should also add a test folder etc…

Probably you want to have a look at the official documentation: Creating…

The post Random Forest Regression from scratch used on Kaggle competition appeared first on juliabloggers.com.

]]>The post Variables are not values: types and expressions in mathematical optimization appeared first on juliabloggers.com.

]]>Re-posted from: https://matbesancon.github.io/post/2019-04-14-optimization-function-evaluation/

Some digging in representations for optimization modelling

The post Variables are not values: types and expressions in mathematical optimization appeared first on juliabloggers.com.

]]>The post Julia Data Science Tutorial: Working with DataFrames and CSV appeared first on juliabloggers.com.

]]>Re-posted from: https://www.techiediaries.com/julia-data-science-tutorial-dataframe-csv/

In this tutorial we'll take our first steps in data science with Julia programming language and we'll see how to work with a DataFrame and CSV. We'll learn about:

What's data science

How to Install Julia in Ubuntu

How to Install Packages in Julia

What is a DataFrame

Creating a Julia DataFrame from Vectors

Creating a Julia DataFrame from Dictionaries

Reading and Writing CSV Files

Writing a DataFrame to CSV File

Reading a CSV File

What is Data Science

First what's data science and what data scientists exactly do?

Wikipedia defines data science as:

A multi-disciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from structured and unstructured data.

Also from Wikipedia:

Data science is the same concept as data mining and big data: "use the most powerful hardware, the most powerful programming systems, and the most efficient algorithms to solve problems".

Data science comprises fields like statistics, data analysis, machine learning and their related methods.

Investopedia defines data science as:

As a field of Big Data geared toward providing meaningful information based on large amounts of complex data. Data science, or data-driven science, combines different fields of work in statistics and computation in order to interpret data for the purpose of decision making.

So what do you need to become a data scientist?

You first need to learn a programming language like Python or Julia.

Python is most known for its use in data science but Julia is also a promising language and one of the fatest growing programming languages for data science.

Julia is a high-level, high-performance and dynamic programming language for technical computing.

Data science is all about databases and large data sets. Julia has official tools for working with almost all databases using JDBC.jl and ODBC.jl drivers. In addition, it also integrates with the Hadoop ecosystem using Spark.jl, HDFS.jl, and Hive.jl.

Julia also provides tools, such as DataFrames, JuliaDB, Queryverse and JuliaGraphs, to work with multidimensional datasets quickly, perform aggregations, joins and preprocessing operations in parallel, and save them to disk in efficient formats.

You can also use packages from Python, R, C/Fortran, C++, and Java in your Julia code if you want to work with a specific library.

How to Install Julia in Ubuntu

Let's now see how to install Julia in Ubuntu. As of this writing, the current stable release is v1.1.0.

Open a new terminal and run the following commands to download Julia binary for Ubuntu from the offcial website and extract it:

$ cd ~

$ wget https://julialang-s3.julialang.org/bin/linux/x64/1.1/julia-1.1.0-linux-x86_64.tar.gz

$ tar xvfa julia-1.1.0-linux-x86_64.tar.gz

Next, you need to put the ~/julia-1.1.0/bin/ path in your system $PATH variable to be able to access the julia binary from any directory.

$ echo PATH=\$PATH:~/julia-1.1.0/bin/ >> ~/.profile

$ source ~/.profile

Now, you can run julia using the following command:

$ julia

This is the output of the program:

_

_ _ _(_)_ | Documentation: https://docs.julialang.org

(_) | (_) (_) |

_ _ _| |_ __ _ | Type “?” for help, “]?” for Pkg help.

| | | | | | |/ _` | |

| | |_| | | | (_| | | Version 1.1.0 (2019-01-21)

_/ |\__’_|_|_|\__’_| | Official https://julialang.org/ release

|__/ |

You can quit the program using exit() or press Ctrl-D.

Note: You can also run Julia in the browser on JuliaBox.com with Jupyter notebooks.

How to Install Packages in Julia

One important aspect of any programming language is package management. Julia provides the Pkg package for installing packages.

Note: Julia has more than 2000 Julia packages. See the list of the most used Julia packages.

You can install a specific package using the Pkg.add("package_name") command.

Let's see this by example. Head back to your terminal and run julia then run the following code:

julia> Pkg.add(“DataFrames”)

You will get the following error:

ERROR: UndefVarError: Pkg not defined

Stacktrace:

[1] top-level scope at none:0

That's because Pkg itself is a package in Julia so you need to import it using the using keyword:

julia> using Pkg

julia> Pkg.add(“DataFrames”)

This will download and install the DataFrames package and all its dependecies.

You can check the installed packages by calling Pkg.installed():

julia> Pkg.installed()

Dict{String,Union{Nothing, VersionNumber}} with 1 entry:

“DataFrames” => v”0.17.1″

Note: You can check the official docs for more information about how to work with packages in Julia.

What is a DataFrame?

A DataFrame is a 2-dimensional labeled data structure that can have columns of different types.

You can see a DataFrame as an Excel sheet.

You can load datasets in your DataFrame memory structure from other Julia buitin structures or persistent storage such as Excel, CSV and SQL database.

Let's consider this example tabular data contructed using Google Spreadsheets:

We have three columns – Name, Title and Salary and six rows.

Let's now see how we can construct a Julia DataFrame from this table.

Creating a Julia DataFrame from Vectors

From our example, we can see that each column can be seen as a Vector.

The first vector is Name = ["N1", "N2", "N3", "N4", "N5", "N6"]

The second vector is Title = ["Engineer", "Developer", "Designer", "DB Administrator", "Mobile Developer", "Data Scientist"]

The third vector is Salary = [20000, 30000, 25000, 26000, 30000, 40000]

Note: Just like Mathematics, In Julia a Vector is a special type of Matrix that has only one row (row matrix) or one column (column matrix).

This is also called a 1D-Array

Just like most programming languages you use arrays in Julia for creating ordered collections of elements. In Julia you can use the square brackets and commas to create arrays.

In Julia, you can use arrays for representing lists, vectors, tables, and matrices.

Head back to your Julia interpeter and run the following code

julia> using DataFrames

julia> df = DataFrame(Name = [“N1”, “N2”, “N3”, “N4”, “N5”, “N6”], Title = [“Engineer”, “Developer”, “Designer”, “DB Administrator”, “Mobile Developer”, “Data Scientist”], Salary = [20000, 30000, 25000, 26000, 30000, 40000])

We create a DataFrame from our previous vectors (or arrays).

You should get the following output:

6×3 DataFrame

│ Row │ Name │ Title │ Salary │

│ │ String │ String │ Int64 │

├─────┼────────┼──────────────────┼────────┤

│ 1 │ N1 │ Engineer │ 20000 │

│ 2 │ N2 │ Developer │ 30000 │

│ 3 │ N3 │ Designer │ 25000 │

│ 4 │ N4 │ DB Administrator │ 26000 │

│ 5 │ N5 │ Mobile Developer │ 30000 │

│ 6 │ N6 │ Data Scientist │ 40000 │

This means we created a DataFrame with six rows and three columns. It's exactly our table in the spreadsheet!

Creating a Julia DataFrame from Dictionaries

You can use Dict to create a dictionary in Julia. Given a single iterable argument, let's construct a Dict whose key-value pairs are taken from 2-tuples (key,value).

Examples

julia> Dict([(“A”, 1), (“B”, 2)])

Dict{String,Int64} with 2 entries:

“B” => 2

“A” => 1

Alternatively, a sequence of pair arguments may be passed.

julia> Dict(“A”=>1, “B”=>2)

Dict{String,Int64} with 2 entries:

“B” => 2

“A” => 1

Let's now see how we can contruct our previous DataFrame from our example table.

Let's first define our pairs:

The first pair is "Name" => ["N1", "N2", "N3", "N4", "N5", "N6"]

The second pair is "Title" => ["Engineer", "Developer", "Designer", "DB Administrator", "Mobile Developer", "Data Scientist"]

The third pair is "Salary" => [20000, 30000, 25000, 26000, 30000, 40000]

In you terminal run the following code to create a DataFrame from a Dict:

julia> df = DataFrame(Dict(“Name” => [“N1”, “N2”, “N3”, “N4”, “N5”, “N6”], “Title” => [“Engineer”, “Developer”, “Designer”, “DB Administrator”, “Mobile Developer”, “Data Scientist”], “Salary” => [20000, 30000, 25000, 26000, 30000, 40000]))

Again, we get the following output:

6×3 DataFrame

│ Row │ Name │ Salary │ Title │

│ │ String │ Int64 │ String │

├─────┼────────┼────────┼──────────────────┤

│ 1 │ N1 │ 20000 │ Engineer │

│ 2 │ N2 │ 30000 │ Developer │

│ 3 │ N3 │ 25000 │ Designer │

│ 4 │ N4 │ 26000 │ DB Administrator │

│ 5 │ N5 │ 30000 │ Mobile Developer │

│ 6 │ N6 │ 40000 │ Data Scientist │

Reading and Writing CSV Files

We can use ths CSV.jl package to read and write CSV files.

Head back to your julia terminal and import CSV.jl using the following code:

julia> using Pkg

Pkg.add(“CSV”)

This will download and install the CSV package and its dependencies.

Next, import the CSV methods using the following code:

julia> using CSV

Writing a DataFrame to CSV

Next, let's write our previous DataFrame to a data.csv file using the following code:

julia> CSV.write(“data.csv”, df)

“data.csv”

You should find the data.csv file in your current directory from which you invoked julia.

Reading a CSV File

You can read a CSV file using the CSV.read() method. Let's read our data.csv file back using the following code:

julia> CSV.read(“data.csv”)

The result is a DataFrame as follows:

6×3 DataFrame

│ Row │ Name │ Salary │ Title │

│ │ String⍰ │ Int64⍰ │ String⍰ │

├─────┼─────────┼────────┼──────────────────┤

│ 1 │ N1 │ 20000 │ Engineer │

│ 2 │ N2 │ 30000 │ Developer │

│ 3 │ N3 │ 25000 │ Designer │

│ 4 │ N4 │ 26000 │ DB Administrator │

│ 5 │ N5 │ 30000 │ Mobile Developer │

│ 6 │ N6 │ 40000 │ Data Scientist │

Conclusion

In this tutorial we introduced you to data science with the Julia programming language. We have seen what's data science and how to install Julia in Ubuntu then seen the definition of the DataFrame structure and how to create a DataFrame from our example tabular data using the DataFrames package.

The post Julia Data Science Tutorial: Working with DataFrames and CSV appeared first on juliabloggers.com.

]]>The post Picking different names with integer optimization appeared first on juliabloggers.com.

]]>Re-posted from: https://matbesancon.github.io/post/2019-04-07-name_distances/

I must admit I am not always the most talented at social events. One point I am especially bad at is remembering names, and it gets even harder when lots of people have similar or similar-sounding names. What if we could select a list of people with names as different from each other as possible?

First some definitions, different here is meant with respect to the Hamming distance of any two names.

The post Picking different names with integer optimization appeared first on juliabloggers.com.

]]>This is a brief writeup of a few of them.

Closures

Closures are when a function is created (normally via returning from anotehr function)

that references some variable in the e...

The post Julia Nomenclature appeared first on juliabloggers.com.

]]>Re-posted from: https://white.ucc.asn.au/2019/04/03/Julia-Nomenclature.html

These are some terms that get thrown around a lot by julia programmers.

This is a brief writeup of a few of them.

Closures are when a function is created (normally via returning from anotehr function)

that references some variable in the enclosing scope.

We say that it **closes over** those variables.

This closure closes over `count`

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

I use this to control early stopping when training neural networks.

This closes over `best_loss`

and `remaining_patience`

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

e.g. the following closes over `model`

```
function runall(dates)
model = Model()
pmap(dates) do the_day
simulate(model, the_day)
end
end
```

3 types:

- Multiprocessing / Distributed
- Multithreading / Shared Memory
- Asynchronous / Coroutines

- this is
`pmap`

,`remotecall`

,`@spawn`

. - Actually starts seperate julia process
- potentially on another machine
- Often has high communication overhead

- this is
`@threads`

- Also in julia 1.2 is coming
**PARTR** - Can be unsafe, care must always be taken to do things in a threadsafe way

- this is
`@async`

, and`@asyncmap`

- Does not actually allow two things to run at once, but allows tasks to take turns running
- Mostly safe
- Does not lead to speedup unless the “work” is done elsewhere
- e.g. in
`IO`

the time is spent filling network buffers / spinning up disks - e.g. if you are spawning extra process like with
`run`

time is spent in those processes.

- e.g. in

- If which method to call needs to be dicided at
**runtime**then it will be a**dynamic dispatch**- i.e. if it nees to be is decided by the
**values**of the input, or by**external**factors

- i.e. if it nees to be is decided by the
- If it can be decided at
**compile time**it will be a**static dispatch**- i.e. if it can be decided only by the
**types**of the input

- i.e. if it can be decided only by the

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

Closely related to Dynamic vs Static Dispatch

- If the
*return type*can decided at**compile time**then it will be a**type stable**- i.e. if the
**return type**is decided only by the**types**of the input

- i.e. if the
- If the
*return type*can’t decided until**run time**then it will be a**type unstable**- i.e. if the
**return type**is decided by the**values**of the input, or by**external**factors

- i.e. if the

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

If your package did not define the

**Function**(name); or- at least 1 of the argument
**types**

You are doing a **type piracy**, and this is a bad thing.

By doing type piracy you can break code in other models even if they don’t import your definitions.

**Input:**

**Output:**

we are going to call it `mapreduce`

because is is kind of mapping this reduction in magnitude.

And because this is a slightly forced example.

**Input:**

**Input:**

**Output:**

**Input:**

**Output:**

```
DimensionMismatch("arrays could not be broadcast to a common size")
Stacktrace:
[1] _bcs1 at ./broadcast.jl:438 [inlined]
[2] _bcs at ./broadcast.jl:432 [inlined]
[3] broadcast_shape at ./broadcast.jl:426 [inlined]
[4] combine_axes at ./broadcast.jl:421 [inlined]
[5] _axes at ./broadcast.jl:208 [inlined]
[6] axes at ./broadcast.jl:206 [inlined]
[7] combine_axes at ./broadcast.jl:422 [inlined]
[8] combine_axes at ./broadcast.jl:421 [inlined]
[9] instantiate at ./broadcast.jl:255 [inlined]
[10] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(*),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(sign),Tuple{Array{Int64,1}}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(-),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(abs),Tuple{Array{Int64,1}}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(abs),Tuple{Array{Function,1}}}}}}}) at ./broadcast.jl:753
[11] mapreduce(::Function, ::Function, ::Array{Int64,1}) at ./In[19]:3
[12] _sum at ./reducedim.jl:653 [inlined]
[13] _sum at ./reducedim.jl:652 [inlined]
[14] #sum#550 at ./reducedim.jl:648 [inlined]
[15] sum(::Array{Int64,1}) at ./reducedim.jl:648
[16] top-level scope at In[21]:1
```

Sometimes to make two packages work together,

you have to make them aware of each others types.

For example to implement

```
convert(::Type(DataFrame), axisarray::AxisArray)
```

where

`convert`

is from*Base*`DataFrame`

is from*DataFrames.jl*`AxisArray`

is from*AxisArrays.jl*

Then the only way to do this without **type piracy** is to do it either *DataFrames.jl* or *AxisArrays.jl*.

But that isn’t possible without adding a dependency which isn’t great.

So instead we have a **Glue Package**, eg, *DataFrameAxisArrayBuddies.jl*,

that adds this method.

It is piracy but it is fairly safe, since it is adding behavour to types that would normally be a method error as is. **Misdemenor type piracy.**

I would argue that this is a core part of polymorphism via composition.

In the following example, we construct `SampledVector`

,

which is a vector-like type that has fast access to the total so that it can quickly calculate the mean.

It is a **wrapper** of the Vector type,

and it **delegates** several methods to it.

Even though it overloads `Statistics.mean`

, and `push!`

, `size`

and `getindex`

from `Base`

,

we do not commit **type piracy**, as we alrways own one of the types – the `SampleVector`

.

**Input:**

**Output:**

**Input:**

**Input:**

`size`

and `getindex`

**Input:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

**Input:**

**Output:**

Traits as something that naturally falls out of functions that can be performed on types at compile time,

and on having multiple dispatch.

See previous post for details..

The post Julia Nomenclature appeared first on juliabloggers.com.

]]>The post Pentachoron appeared first on juliabloggers.com.

]]>Re-posted from: https://cormullion.github.io/blog/2019/04/03/pentachoron.html

This page contains quite a few animated GIFs. I apologise in advance if your browser or network connection doesn’t like them. It’s much easier to add a GIF than it is to embed a video into a GitHub pages blog, unfortunately.

I encountered the word “pentachoron” recently for the first time, and I thought it had to be something cool. Turns out it is, so I thought I’d have a go at drawing one…

Unfortunately, it’s a 4-dimensional object, which is going to tax my limited 2D skills. But I like being out of my depth.

The first job is to define a couple of types to help organize all the various points.

Here’s an immutable structure to hold the coordinates of a point in four dimensions. In four dimensions, the axes are usually called `x`

, `y`

, `z`

, and `w`

.

By adopting the AbstractArray as our stepmother type, we can pick up many useful behaviors. `size()`

and `getindex()`

can be taught how to handle Point4Ds. If you browse the official documentation for a while you’ll find out how to effectively tap in to many desirable pre-defined abilities that our new type can inherit.

While we’re here, let’s do three-dimensional points as well:

The primary task we have to address is how to convert a 4D point into a 2D point. Let’s start with the easier task: how to convert a 3D point into a 2D point, ie how can we draw a 3D shape on a flat surface?

Consider a simple cube. The front face and the back face could have the same X and Y coordinates, and vary only by their Z values.

(This is just a diagram, it’s not really 3D…)

So the idea is to project the cube from 3D to 2D by keeping the first two values, and multiplying or modifying them by the third value. We’ll make a `convert()`

function to do this:

`K`

is just a constant which provides a consistent value for depth:

Testing this quickly in Luxor.jl gives promising results:

It’s a simple type of perspective projection.

Using the same principle, let’s make a method for converting a 4D point.

Let’s hope it works.

We can combine these into a single utility function called `flatten()`

that takes a list of 4D points and double-maps them into a list of 2D points suitable for drawing.

To test this, we’ll define the vertices of a unit pentachoron:

(According to Wikipedia, other names for the pentachoron include:

- Regular 5-cell
- C5
- pentatope
- pentahedroid
- tetrahedral pyramid
- 4-simplex

I think *pentachoron* is the coolest, though.)

Here’s a list of “faces”, with each face defined by three of the vertices:

A quick test:

This isn’t very interesting, although I *think* it’s correct. To see a more appealing display, let’s make it dance…

You usually rotate 2D points about a 1D point. You usually rotate 3D points about a 2D line (often one of the XYZ axes). So logically you’d rotate 4D points with reference to a 3D plane. We’ll define some matrices that do 4D rotations relative to a plane defined by two of the X, Y, Z, and W axes.

As I’m thinking about it, the XY plane is typically the plane of the drawing surface. If you think of the XY plane as a computer screen straight in front of you, the XZ plane is parallel to your desk or floor, and the YZ plane is like a wall beside your desk on your right or left side.

But what about the XW plane? And the YW and ZW planes, for that matter? This is the mystery of 4D shapes: we can’t see these planes, we can only imagine their existence by watching shapes moving through and around them.

Here’s a function that generates a suitable rotation matrix for rotating a 4D point in the XY plane:

and here’s another one that rotates a 4D point in the XW plane:

Oh, it looks like we’re having a matrix party, so we might as well do the others while we’re here:

We’ll make a handy utility function that uses one of these matrix functions to rotate an array of 4D points `A`

in the plane defined by the matrix function:

So we can write `rotate4(A, XW(π/2))`

to rotate the array of points in `A`

in the XW plane by π/2. We can also write `rotate4(A, XW(π/2) * XY(π/2))`

to rotate the points first in the XW plane and then in the XY plane.

Notice that the `rotate4()`

function returns a new array, rather than modifying the original one. This probably has some disadvantages, but for now it means that we don’t have to keep track of mutating objects so carefully.

We’re now able to make an animation that rotates our pentachoron in one or more planes.

The `frame()`

function generates a single frame, using the framenumber (`eased_n`

provides a normalized version of the framenumber modulated by the easing function) to control the rotation between 0 and 2π. The scalefactor makes the unit shape big enough to fill the frame.

It’s helpful to color the faces very slightly, using `poly(..., :fill)`

-er.

And this function generates a series of frames and saves them to an animation:

This is a rotation in the “parallel to the desk my computer’s resting on” XZ plane.

It’s not a realistic rendering with hidden-surface removal—I wouldn’t know how to do that in 3D, let alone 4D. But it does make it slightly easier to follow some of the faces. Something to do one day might be to sort the “faces” to determine which are drawn first, but given the maths involved this is a challenge I’m dubious about tackling…

We can modify `frame()`

to allow various permutations of XY, XW, XZ, YZ, YW, and ZW rotations. Here’s a version that rotates in XZ, followed by YW:

These are starting to make me think of kneading machines for making pizza dough.

You can draw a few different models at once, placing each one in a separate cell:

This is showing 2D shadows of 3D shadows of 4D objects. Do these objects exist? I don’t know…

While the word “pentachoron” was unfamiliar to me until recently, the word “tesseract” is much more familiar. After all, didn’t the tesseract contain the Space Stone, one of the six Infinity Stones that apparently predate the universe and possess unlimited energy? (According to the scientists at Marvel Comics, at least…) It’s also the name of a 4-dimensional hypercube, one step up from the pentachoron.

Here are the vital numbers for the tesseract:

Replace `pentachoron`

everywhere by `tesseract`

in the above code, choosing the rotations to taste.

(Just for fun I temporarily switched the color scheme over to the Julia logo colors. You can see the 3D rendering problems more clearly, as well!)

Here’s a composite of four pairs of rotations:

It’s tempting to go further, but these web pages do get very big with all these GIFs. But just one more… I quite like this one, the *hexadecachoron*:

There’s lots more fun to be had (combining two or more different shapes is fun), but I’m worried about your network and my brain cells, so that’s enough animated GIFs for today.

[2018-04-03]

*This page was generated using Literate.jl.*

The post Pentachoron appeared first on juliabloggers.com.

]]>After writing that blog post the owner of the julia library for that a...

The post Speeding up my julia code for the hungarian method appeared first on juliabloggers.com.

]]>Re-posted from: https://opensourc.es/blog/hungarian-performance

A few days ago I had a look at my hungarian method project again. You can find the blog post here.

After writing that blog post the owner of the julia library for that algorithm commented on my post and made a huge performance improvement to his library to be much faster than my code where my goal of actually implementing it was to be faster to compete in this kaggle challenge. He also gave me some performance tips which I wanted to test.

You can of course just read this article for the performance improvements but if you want to know more about the hungarian method please read my previous post about it. Okay… a short version: The hungarian method is an algorithm which finds the minimal sum in a square matrix given the constraint that each row and each column has exactly one chosen value. In my example I had 5 kids and 5 gifts and each kid and each gift has a ranking for the gifts and kids respectively. I want to get the best kid <-> gift combination to make the kids, gifts as happy as possible. Yes it’s a bit weird but quite funny

Before I start I have to actually change some code of the hungarian method to be compatible with Julia v1.0 and not Julia v0.6 which I used back then.

Changes:

`seed = srand(...)`

to`using Random`

and then`seed = Random.seed!(...)`

`gc()`

to`Base.GC.gc()`

`A[A .> 50000] = 50000`

to`A[A .> 50000] .= 50000`

- in general in quite some places changes
`-`

to`.-`

or`+`

to`.+`

etc whenever the right hand side must be broadcasted.

- in general in quite some places changes
`find`

to`findall`

- here for 2D arrays you get
`CartesianIndex`

with`(y,x)`

instead of a single coordinate.

- here for 2D arrays you get

so this:

```
function get_matching(mat)
# create bipartite matching graph
zero_vals = find(x->x==0,mat)
ei = Vector{Int64}()
ej = Vector{Int64}()
for i in zero_vals
# zero based
zi = i-1
zr = zi % size(mat)[1]
push!(ej,zr+1) # row
push!(ei,div((zi-zr),size(mat)[1])+1) # col
end
matching = bipartite_matching(ones(Int64,length(ei)), ei, ej)
return matching
end
```

will be:

```
function get_matching(mat)
# create bipartite matching graph
zero_vals = findall(x->x==0,mat)
ei = Vector{Int64}()
ej = Vector{Int64}()
for yx in zero_vals
push!(ej,yx[1]) # row
push!(ei,yx[2]) # col
end
matching = bipartite_matching(ones(Int64,length(ei)), ei, ej)
return matching
end
```

which is actually much easier to read.

Now the current benchmark with the new version of the Hungarian Method library:

Matrix size | Lib seconds | Lib memory | Own seconds | Own memory |
---|---|---|---|---|

800 | 0.15 | 1.4MiB | 2.78 | 1.7GiB |

1000 | 0.2 | 2.1MiB | 4.57 | 2.1GiB |

2000 | 0.51 | 7.9MiB | 12.5 | 4.9GiB |

4000 | 1.22 | 31.2MiB | 29.4 | 10.6GiB |

8000 | 3.8 | 123.2MiB | 65.8 | 20.9GiB |

first of all the library is always faster and uses less memory and in both cases I mean much faster and way less memory.

The library is about 20 times faster and uses \(\approx 170\) times less memory. By memory I always refer to memory allocation during the process.

At no point of my algorithm is actually 20GiB used.

First I…

The post Speeding up my julia code for the hungarian method appeared first on juliabloggers.com.

]]>After writing that blog post the owner of the julia library for that a...

The post Speeding up my julia code for the hungarian method appeared first on juliabloggers.com.

]]>Re-posted from: http://opensourc.es/blog/hungarian-performance

A few days ago I had a look at my hungarian method project again. You can find the blog post here.

After writing that blog post the owner of the julia library for that algorithm commented on my post and made a huge performance improvement to his library to be much faster than my code where my goal…

The post Speeding up my julia code for the hungarian method appeared first on juliabloggers.com.

]]>The post Static lists in Julia appeared first on juliabloggers.com.

]]>Re-posted from: https://matbesancon.github.io/post/2019-03-30-static-list/

This post explores the possibility to build static lists in Julia, meaning lists for which the size is known at compile-time. This is inspired by a post on a Scala equivalent but will take different roads to see more than a plain port. Of course, this implementation is not that handy nor efficient but is mostly meant to explore the possibilities of the type system, especially a trick of using recursive types as values (replacing a dependent type system).

The post Static lists in Julia appeared first on juliabloggers.com.

]]>I found this a really nice idea but at that time I didn't ha...

The post Hang your code on the wall appeared first on juliabloggers.com.

]]>Re-posted from: https://opensourc.es/blog/codeposter

Several years ago I found commits.io which is a service for making a poster out of your GitHub code project.

I found this a really nice idea but at that time I didn’t have a project big enough to fill a poster. Now programming Juniper.jl I have enough but I think on their website you don’t have enough options to play with your poster. Can I make the background black?

What color does the code have? And why do I have to pay so much for it?

A few days ago I decided to build this kind for myself and make it open to the public of course. You can’t use it as easily on a website and order it directly but I think for people reading my blog it’s easy to set it up and maybe you can print the poster in your university or at a local service to save some money.

Okay for everyone who just wants to see the project and play with it: CodePoster on GitHub.

For the coders in you who want to know how to build something like that with Julia and maybe want to contribute:

There are several steps:

- Have your code lines on the canvas
- Put text on top of it
- Make it easily useable and customizable

First we have to read the project folder and extract the code lines:

Reading wise you basically need this code:

```
for (root, dirs, files) in walkdir(folder)
if !occursin(ignore,root)
for file in files
if occursin(ext_pattern, extension(file))
open(root*"/"*file) do file
for ln in eachline(file)
# do something
end
end
end
end
end
end
```

which recursively reads all files in your specified `folder`

and then checks whether the file should be ignored for example the `.git`

folder is probably nothing you want to have on your poster.

Additionally the user should be specify the `ext_pattern`

. For example making it possible to include or exclude the markdown files like `README.md`

.

For the actual drawing part I use Luxor.jl. In front of our reading loop we need to specify the `Canvas`

with:

```
Drawing(size_x, size_y, "code.png")
origin(Point(fsize,fsize))
background("black")
fontsize(fsize)
##################
# draw something #
##################
finish() # saves the file
```

later we check how the size of the canvas is specified by the user and talk more about the font size.

The origin function defines where we want to place the next text where basically the lower left corner of a char is the origin. (i.e `g`

has the origin higher than their lowest point as the lower part of `g`

is under the normal text line.)

For each line in our code we want to remove the whitespaces at the beginning and end as well as removing whitespaces from things like:

`time_limit :: Float64`

which can be achieved by:

```
# there should be a whitespace between two lines
stripped = strip(ln)*" "
stripped = replace(stripped, r"\s+" => " ")
```

In order to place the next line at the correct position we need…

The post Hang your code on the wall appeared first on juliabloggers.com.

]]>I found this a really nice idea but at that time I didn't ha...

The post Hang your code on the wall appeared first on juliabloggers.com.

]]>Re-posted from: http://opensourc.es/blog/codeposter

Several years ago I found commits.io which is a service for making a poster out of your GitHub code project.

I found this a really nice idea but at that time I didn’t have a project big enough to fill a poster. Now programming Juniper.jl I have enough but I think on their website you don’t have…

The post Hang your code on the wall appeared first on juliabloggers.com.

]]>