Category Archives: Julia

Julia and OpenFST a glue story

Julia as a Glue Language

Julia is a great language for scientific and technical programming.
It is more or all I use in my research code these days.
It gets a lot of attention for being great for scientific programming because of its:
great matrix syntax, high speed and optimisability, foreign function interfaces, range of scientific libraries, etc etc.
It has all that sure. (Though it is still in alpha, so many things are a bit broken at times.)
One things that is under-mentioned is how great it is as a “glue” language. Continue reading →

Julia, The New Tech Skill Companies Demand in 2017

New York, NY – What do the following companies and organizations all have in common?

Apple, Amazon, Facebook, BlackRock, Ford, Oracle, Comcast, Massachusetts General Hospital, Farmers Insurance, Los Alamos National Laboratory and the National Renewable Energy Laboratory

They are all looking to hire Julia programmers in 2017.

“In the last quarter of 2016 and already in the first quarter of 2017, there is an explosion in the number of job postings for skilled Julia programmers,” said Viral Shah, Julia Computing CEO. “While 2016 showed tremendous growth among early adopters, 2017 is shaping up to be the breakout year for Julia adoption.”

Julia is the fastest modern high performance open source computing language for data and analytics. It combines the functionality and ease of use of R and Python with the speed of Java and C++. Julia delivers dramatic improvements in simplicity, speed, capacity and productivity.

  1. Julia is lightning fast. Julia provides speed improvements up to 1,000x for insurance model estimation, 225x for parallel supercomputing image analysis and 11x for macroeconomic modeling.
  2. Julia is easy to learn. Julia’s flexible syntax is familiar and comfortable for users of Python and R.
  3. Julia integrates well with existing code and platforms. Users of Python, R and other languages can easily integrate their existing code into Julia.
  4. Elegant code. Julia was built from the ground up for mathematical, scientific and statistical computing, and has advanced libraries that make coding simple and fast, and dramatically reduce the number of lines of code required – in some cases, by 90% or more.
  5. Julia solves the two language problem. Because Julia combines the ease of use and familiar syntax of Python and R with the speed of C, C++ or Java, programmers no longer need to estimate models in one language and reproduce them in a faster production language. This saves time and reduces error and cost.

Julia users and partners include: Amazon, IBM, Intel, Microsoft, DARPA, Lawrence Berkeley National Laboratory, National Energy Research Scientific Computing Center (NERSC), Federal Aviation Administration (FAA), MIT Lincoln Labs, Moore Foundation, Nobel Laureate Thomas J. Sargent, Federal Reserve Bank of New York (FRBNY), Brazilian National Development Bank (BNDES), BlackRock, Conning, Berkery Noyes, BestX and many of the world’s largest investment banks, asset managers, fund managers, foreign exchange analysts, insurers, hedge funds and regulators. Julia is being used to analyze images of the universe and research dark matter, drive parallel computing on supercomputers, diagnose medical conditions, manage 3D printers, build drones, improve air safety, provide analytics for foreign exchange trading, insurance, regulatory compliance, macroeconomic modeling, sports analytics, manufacturing and much, much more.

Julia Computing was founded in 2015 by the co-creators of the Julia language to provide support to businesses and researchers who use Julia.

Building a Web App in Julia: DifferentialEquations.jl Online

By: Christopher Rackauckas

Re-posted from: http://www.stochasticlifestyle.com/building-web-app-julia-differentialequations-jl-online/

Web apps are all the rage because of accessibility. However, there are usually problems with trying to modify some existing software to be a web app: specifically user interface and performance. In order for a web application to perform well, it must be able to have a clean user interface with a “smart” backend which can automatically accommodate to the user’s needs, but also give back a nice looking response in under a second. This has typically limited the types of applications which can become responsive web applications.

Scientific software is one domain where user interface has been obstructive at best, and always in need of better performance. However, the programming language Julia has allowed us to build both an easy to use and highly performant ecosystem of numerical differential equation solvers over the last 8 months. Thus we had to ask the next question: can Julia be used as a viable backend for scientific web applications?

The answer is a definitive yes! Today I am announcing DifferentialEquations.jl Online, a web interface for DifferentialEquations.jl, a library of numerical methods for differential equations written in Julia. Using this web application, you can easily solve biological models, draw interactive 3D phase diagrams, and easily see what happens when you add stochasticity (randomness) to a model. While we restrict the user to a light computational load (maxes out at 1000 iterations), this serves as a good educational tool, a good tool for easily exploring models, and as a gateway to DifferentialEquations.jl.

If you like this work and would like to support it, please star the DifferentialEquations.jl repository. In this blog post I would like to share how we built this app and the lack of difficulties that we encountered.

Getting The Prototype Together

I took the idea over to Julia’s discourse forum where many people throw around some ideas for how to get this working. Alex Mellnik is the star of the show who developed a working prototype in what must have been a few hours using JuliaWebAPI.jl along with an AngularJS frontend (though he shortly after changed the backend to Mux.jl). The resulting code was short and simple enough that I was able to dive right into it.

Next we had to look around for how to host it. While we were at first were looking at using AWS Lambda, we eventually settled on using Heroku and deploying via Docker containers. This setup allows us to build an install of Julia the way we like and then just ship it over to the server. Major kudos to Alex for figuring out how to set this up. You can see how we develop and deploy by looking at our Github repository for the frontend and the repository for the backend.

Optimizing the Web Application

There are a few things to note about Julia and how we had to make the app. Specifically, Julia is a really interesting language because it allows the use of just-in-time (JIT) compilation in order to speedup functions. It does this by specializing functions to the arguments which they are given, and caching that function call for further uses. DifferentialEquations.jl is built around optimizing the numerical solvers for the most difficult differential equations you can throw at it. However, this means that it specializes and recompiles a new version of the solver functions for each ODE/SDE/etc. that you give it. In most cases, this is the right thing to do and on beefier computers has a startup cost of around 0.4 seconds. This form of hyper-specialization allows our solvers to routinely beat even the classic FORTRAN codes in performance benchmarks. However, for our web application we are restricting users to only 1000 iterations (and thus easy problems), so instead of brute performance we needed a fast response time. This kind of hyper-specialization caused some response time issues since the web server was quite slow at compiling (around 1 second).

But Julia is very flexible, and making it so that way it wouldn’t fully specialize and recompile was as easy as setting up a few new types. While the standard problem types that are generated for the differential equations are strongly typed in the function fields, we made a special set of types for handling these “quick problems” which only specify “f” as a Function (which is an abstract and not a concrete type, since each function is its own type). Those 44 lines were pretty much the only changes required to have Julia automatically optimize for quick problems and response time instead of long problems. The rest of the logic all calls the exact same functions as the normal types because it’s all handled via dispatches on the abstract type hierarchy. The result? Less than 100 lines of code were required to get average response times come out to be around 0.2 seconds (with plotting via Plots.jl taking around half of that time), which is more than responsive enough for a computationally heavy web application!

Conclusion

And that’s the main conclusion I wanted to share. Not only was it quick and easy to make this web application in Julia, but the resulting product is quick to respond and easy to use. Since it plugs into DifferentialEquations.jl, there really is not much special code required for the web server other than a few type definitions. However, those type definitions were all that was required to make Julia pretty much automatically optimize the resulting code for a completely different application. The result is both easy to maintain and easy to extend. I couldn’t be happier with this experience.

The post Building a Web App in Julia: DifferentialEquations.jl Online appeared first on Stochastic Lifestyle.