Author Archives: Christopher Rackauckas

Recent advancements in differential equation solver software

By: Christopher Rackauckas

Re-posted from: http://www.stochasticlifestyle.com/recent-advancements-in-differential-equation-solver-software/

This was a talk given at the Modelica Jubilee Symposium – Future Directions of System Modeling and Simulation.

Recent Advancements in Differential Equation Solver Software

Since the time of the ancient Fortran methods like dop853 and DASSL were created, many advancements in numerical analysis, computational methods, and hardware have accelerated computing. However, many applications of differential equations still rely on the same older software, possibly to their own detriment. In this talk we will describe the recent advancements being made in differential equation solver software, focusing on the Julia-based DifferentialEquations.jl ecosystem. We will show how high order Rosenbrock and IMEX methods have been proven advantageous over traditional BDF implementations in certain problem domains, and the types of issues that give rise to general performance characteristics between the methods. Extensions of these solver methods to adaptive high order methods for stochastic differential-algebraic and delay differential-algebraic equations will be demonstrated, and the potential use cases of these new solvers will be discussed. Acceleration and generalization of adjoint sensitivity analysis through source-to-source reverse-mode automatic differentiation and GPU-compatibility will be demonstrated on neural differential equations, differential equations which incorporate trainable latent neural networks into their derivative functions to automatically learn dynamics from data.

The post Recent advancements in differential equation solver software appeared first on Stochastic Lifestyle.

A Collection of Jacobian Sparsity Acceleration Tools for Julia

By: Christopher Rackauckas

Re-posted from: http://www.stochasticlifestyle.com/a-collection-of-jacobian-sparsity-acceleration-tools-for-julia/

Over the summer there have been a whole suite of sparsity acceleration tools for Julia. These are encoded in the packages:

The toolchain is showcased in the following blog post by Pankaj Mishra, the student who build a lot of the Jacobian coloring and decompression framework. Langwen Huang setup the fast paths for structured matrices (tridiagonal, banded, and block-banded matrices) and also integrated these tools with DifferentialEquations.jl. Shashi Gowda then setup a mechanism for automatically detecting the sparsity of Julia programs (!!!).

A tutorial using this workflow together is described in the SparseDiffTools.jl README. In summary, to use the tools you have the following flow:

  1. Find your sparsity pattern, Jacobian structure (i.e. Jacobian type), or automatically detect it with SparsityDetection.jl.
  2. Call `matrix_colors(A)` from SparseDiffTools.jl to get the `colorvec` for A. This is the vector that the differentiation tools need to have to exploit sparsity and reduce the total cost of generating the Jacobian.
  3. When calling `forwarddiff_color_jacobian` from SparseDiffTools.jl for sparse AD or `finite_difference_jacobian` from DiffEqDiffTools.jl for sparse finite differencing, pass the `colorvec` and `sparsity` (the sparsity pattern by either passing the sparse matrix or the structured matrix), then the differentiation tools will automatically accelerate to be fast for that kind of matrix
  4. When building the ODEFunction for the DifferentialEquations.jl ODE solver, pass the `colorvec` and `jac_prototype` and all internal functions will automatically specialize on the sparsity pattern and accelerate. If you pass a structured matrix, like a BandedMatrix, the color vector will be determined automatically, making those accelerations free.

Thus together the chain is: get the sparsity, get the colorvec, pass it to packages and boom you’re faster!

The Math of Sparsity

If you’re interested in how this all works, please take a look at the lecture notes for my course 18.337:

  1. Forward-Mode AD via High Dimensional Algebras (necessary backstory)
  2. Solving Stiff Ordinary Differential Equations (which explains the sparse AD story)

Additionally, take a look at this paper for an explanation of how you can do automatic sparsity detection of Julia packages.

Conclusion

It will be interesting to see how having an integrated platform for acceleration via sparsity effects a high level language, especially the automatic sparsity detection. It is fitting for Julia to have these tools since, given the focus on performance, this is the piece of math that is required to make your performance work really matter! This is a pervasive mechanism which lets you accelerate differentiation on your own, or directly give the differential equation solvers these to utilize (and it works with ODEs, SDEs, DAEs, DDEs, hybrid equations, etc.). We hope to integrate this with NLsolve.jl, and get the Hessian tools finished for Optim.jl and JuMP.jl. Also, JuMP.jl is getting a new and improved NLP interface which will utilize a lot of this behind the scenes automatically. Stay tuned.

The post A Collection of Jacobian Sparsity Acceleration Tools for Julia appeared first on Stochastic Lifestyle.

Developing Julia Packages

By: Christopher Rackauckas

Re-posted from: http://www.stochasticlifestyle.com/developing-julia-packages/

Have you ever wanted to develop your own package for the Julia programming language? Have you ever wanted to contribute a bug fix? Then this tutorial is for you! I will walk you through getting the community resources (Discourse and Slack) so that you can get help, get the Juno and GitKraken development environments going, and show all of the steps of building a package. In this video you will learn how to use modules, how to interactively update a package without recompiling, how to setup continuous integration testing, and how to get your package registered. In addition, I show how to “dev” a package to get a local copy to work on, and use this to give a bug fix to open a pull-request to fix an issue on an existing package.

Note: After “]dev”ing a package, the local version of “using Package” uses the package in your “~/.julia/dev/Package” folder!

The post Developing Julia Packages appeared first on Stochastic Lifestyle.