Newsletter 2017

We wanted to thank all Julia users and well wishers for the support and for being part of the Julia Community, and to give an update on some exciting developments for 2017:

  1. JuliaPro Release
  2. Julia by the Numbers
  3. Julia Jobs Growth
  4. Julia Development Update Calendar
  5. Julia in the News
  6. Julia Case Studies
  7. Contact Julia Computing

  1. JuliaPro Release

    If you haven’t already downloaded the new JuliaPro, please take
    a moment to do so now.

    JuliaPro comes in two flavors:

    • JuliaPro: The fast, free way to download and use Julia
      immediately on your desktop or laptop. Includes Julia compiler,
      debugger, profiler, integrated development environment,
      visualization and plotting and over 100 curated packages.

    • JuliaPro Enterprise: For installation on your desktop, laptop or
      enterprise server. Includes all the features of JuliaPro, plus Excel
      integration and support for $1,500 per user per year. Indemnity is
      available with a customer site license for an additional charge.

  2. Julia by the Numbers

    Julia use expanded dramatically in 2016, and 2017 is shaping up to be
    the year that Julia expands from early adopters into the mainstream.

Cumulative Number of Source As of Jan 1, 2016 As of Jan 1, 2017 Annual growth
Stack Overflow questions Stack Overflow, script 1,420 2,540 +79%
Registered packages pkg.julialang.org 690 1,190 +72%
GitHub stars across packages pkg.julialang.org 12,582 21,843 +74%
JuliaBox users juliabox.com 34k 58k +71%
Julia downloads S3stat, using S3 logs 346k 904k +161%
Published citations for Julia Paper ([1] & [2]) Google Scholar 143 316 +121%

[1] Julia: A Fast Dynamic Language for Technical Computing (2012)
[2] Julia: A Fresh Approach to Numerical Computing (2014)

  1. Julia Jobs Growth

    Demand for Julia programmers is increasing rapidly in finance,
    industry, science and technology.

    Employers looking to hire Julia programmers in 2017 include: Apple,
    Amazon, Facebook, BlackRock, Ford, Oracle, Comcast, Massachusetts
    General Hospital, Farmers Insurance, Los Alamos National Laboratory and
    the National Renewable Energy Laboratory.

    Indeed.com: Julia Job Postings and Julia Jobseeker Interest


  2. Julia Development Update Calendar

    2016

    • Julia 0.5, JuliaPro and JuliaPro Enterprise released, including
      Excel integration

    • Julia debugger, profiler and integrated development environment
      released

    • Julia became available on Amazon Web Services and AWS pronounced
      MXNet the “Framework of Choice” for deep learning using Julia and
      other languages

    2017 – Q1

    • JuliaFin launches, including Bloomberg integration, Excel
      integration, advanced time-series functionality and Miletus, a new
      dedicated Julia package for implementation of complex trading
      strategies

    • Julia launches on Microsoft Azure’s Data Science Virtual Machine

    2017 – Q2 & Q3

    • JuliaCon 2017 @ UC Berkeley

    • Julia 1.0 release

  3. Julia in the News

    • “Julia is poised to become one of the leading tools deployed by
      developers and programmers at banks, hedge funds, regulators and
      vendors.”
      WatersTechnology:
      “The Infancy of Julia: An Inside Look at How Traders and Economists
      are Using the Julia Programming Language”

    • “Julia is fast, … can call C directly without a wrapper, integrates
      top tier open source C and Fortran code into its Base library, and
      can easily call Python as well. Julia is built for parallel and
      cloud computing, and has particular interest from the analytics and
      scientific computing communities. According to KDnuggets’ most
      recent analytics software poll, Julia placed 8th on the list of most
      used programming languages.”
      KDnuggets:
      “Top Machine Learning Projects for Julia”

    • “Delivering Hadoop style parallelism, … [Julia] is destined to
      make a major impact.” Coding
      Dojo      :
      “7 New Programming Languages to Learn in 2016”

    • “[I]t’s notable that Julia, a high-level programming language
      built expressly for use in technical computing, has entered TIOBE’s
      list …. In industries that prize efficiency, such as finance, Julia
      has enjoyed rapid adoption by tech professionals and
      data scientists. In banking and trading, algorithmic traders and
      quants now rely on Julia because it allows them to push code as
      quickly as possible to market, without needing to rewrite.”
      Dice:
      “Julia Gains Popularity on the TIOBE Language List”

    • “Overall, Julia is a welcome addition [to the High Performance
      Computing (HPC)] community. …The future looks bright for Julia. New
      and existing HPC coders will appreciate a dirt-simple on-ramp to the
      HPC superhighway.”
      TheNextPlatform:
      “Dirt Simple HPC: Making the Case for Julia”

    • Julia should be on the radar of everyone from traders and operations
      executives to IT managers, developers and data scientists and really
      anyone who wants to expand their job options as electronic trading
      takes over and the industry as a while becomes more
      technology-centric.”
      Efinancialcareers:
      “Julia Programming Language: This Is the New Skill Hedge Funds Are
      Asking For”

  4. Julia Case Studies

    • Nobel Laureate Thomas J.
      Sargent      ,
      presenting at JuliaCon 2016, described his next-generation
      macroeconomic models as “a walking advertisement for Julia”

    • Julia for Deep
      Learning
      was showcased with IBM at SC16, using IBM’s Power8 server and NVIDIA
      GPU accelerators to increase the processing speed of medical image
      analysis 57x for diagnosing diabetic retinopathy

    • Researchers at
      Intel,
      UC
      Berkeley      ,
      Lawrence Berkeley
      Labs      , the
      National Energy Research Scientific Computing
      Center (NERSC)      ,
      JuliaLabs@MIT
      and Julia Computing developed a new
      parallel supercomputing method which increases the speed of
      astronomical image analysis 225x

    • BlackRock,
      the world’s largest asset manager, is using Julia to upgrade their
      trademark Aladdin analytics platform

    • Lincoln
      Labs       and
      the Federal Aviation
      Administration
      are using Julia to analyze 650 billion decision points for the next
      generation Aircraft Collision Avoidance System

    • The Federal Reserve Bank of New
      York       is using
      Julia to run their Dynamic Stochastic General Equilibrium model
      10-11x faster with 50% fewer lines of code

    • Voxel8 is
      using Julia for 3D printing and drone manufacture

    • UC Berkeley researchers are using Julia to guide the Berkeley
      Autonomous Race
      Car

    • An article published in
      Nature
      describes how UK cancer researchers are using Julia to model cancer
      evolution and inform interpretation of cancer genomes

    • Augmedics
      is using Julia to provide surgeons with ‘X-ray vision’ using Julia
      and augmented reality to project images of internal structures onto
      patients’ bodies in real time during surgery

    • PathBioAnalytics
      is using Julia to develop personalized precision medical treatments
      for patients

  5. Contact Julia Computing

    Please contact us at info@juliacomputing.com for any of the following reasons:

    • Julia Sales and Marketing: Can Julia Computing help you, your
      organization or industry? Are you interested in JuliaFin (for
      finance), JuliaRun (for cloud), JuliaPro Enterprise with indemnity,
      support, training or consulting? Can we make you a more effective
      advocate for Julia within your organization or industry? Please
      contact us at info@juliacomputing.com.

    • Case Studies: Are you using Julia for something interesting,
      unique, exciting or cool? Please check out the case
      studies       on our Website. If
      you have a Julia story to tell, we want to capture it. Please
      contact us at info@juliacomputing.com and we will follow up with
      you for more information.

Julia – A Fresh Approach to Numerical Computing

This post is authored by Viral B. Shah, co-creator of the Julia language and co-founder and CEO at Julia Computing, and Avik Sengupta, head of engineering at Julia Computing.

The Julia language provides a fresh new approach to numerical computing, where there is no longer a compromise between performance and productivity. A high-level language that makes writing natural mathematical code easy, with runtime speeds approaching raw C, Julia has been used to model economic systems at the Federal Reserve, drive autonomous cars at University of California Berkeley, optimize the power grid, calculate solvency requirements for large insurance firms, model the US mortgage markets and map all the stars in the sky. 

It would be no surprise then that Julia is a natural fit in many areas of machine learning. ML, and in particular deep learning, drives some of the most demanding numerical computing applications in use today. And the powers of Julia make it a perfect language to implement these algorithms.

julia

One of key promises of Julia is to eliminate the so-called “two language problem.” This is the phenomenon of writing prototypes in a high-level language for productivity, but having to dive down into C for performance-critical sections, when working on real-life data in production. This is not necessary in Julia, because there is no performance penalty for using high-level or abstract constructs.

This means both the researcher and engineer can now use the same language. One can use, for example, custom kernels written in Julia that will perform as well as kernels written in C. Further, language features such as macros and reflection can be used to create high-level APIs and DSLs that increase the productivity of both the researcher and engineer.

GPU

Modern ML is heavily dependent on running on general-purpose GPUs in order to attain acceptable performance. As a flexible, modern, high-level language, Julia is well placed to take advantage of modern hardware to the fullest.

First, Julia’s exceptional FFI capabilities make it trivial to use the GPU drivers and CUDA libraries to offload computation to the GPU without any additional overhead. This allows Julia deep learning libraries to use GPU computation with very little effort.

Beyond that, libraries such as ArrayFire allow developers to use natural-looking mathematical operations, while performing those operations on the GPU instead of the CPU. This is probably the easiest way to utilize the power of the GPU from code. Julia’s type and function abstractions make this possible with, once again, very little performance overhead.

Julia has a layered code generation and compilation infrastructure that leverages LLVM. (Incidentally, it also provides some amazing introspection facilities into this process.) Based on this, Julia has recently developed the ability to directly compile code onto the GPU. This is an unparalleled feature among high-level programming languages.

While the x86CPU with a GPU is currently the most popular hardware setup for deep learning applications, there are other hardware platforms that have very interesting performance characteristics. Among them, Julia now fully supports the Power platform, as well as the Intel KNL architecture.

Libraries

The Julia ecosystem has, over the last few years, matured sufficiently to materialize these benefits in many domains of numerical computing. Thus, there are a set of rich libraries for ML available in Julia right now. Deep learning framework with natural bindings to Julia include MXNet and TensorFlow. Those wanting to dive into the internals can use the pure Julia libraries, Mocha and Knet. In addition, there are libraries for random forests, SVMs, and Bayesian learning.

Using Julia with all these libraries is now easier than ever. Thanks to the Data Science Virtual Machine (DSVM), running Julia on Azure is just a click away. The DSVM includes a full distribution of JuliaPro, the professional Julia development environment from Julia Computing Inc, along with many popular statistical and ML packages. It also includes the IJulia system, with brings Jupyter notebooks to the Julia language. Together, it creates the perfect environment for data science, both for exploration and production.

Viral Shah
@Viral_B_Shah

New Year & New Updates to the Windows Data Science Virtual Machine

This post is authored by Gopi Kumar, Principal Program Manager in the Data Group at Microsoft.

First of all, a big thank you to all users of the Data Science Virtual Machine (DSVM) for your tremendous response to our offering in 2016. We’re looking forward to a similarly great year in 2017.

The new year also brings in some interesting new tools to our DSVM users, to help you be more productive with data science. In this post, we summarize key recent changes on the Windows Server side of our DSVM offering, below.

  1. Microsoft R Server 9.0.1 (MRS9) developer edition, a major update to the enterprise scalable R extension from Microsoft, is now available on the VM. This version brings a lot of exciting changes including several fast ML / deep learning algorithms developed by Microsoft in a new library called Microsoft ML. There’s a new architecture and interface for deploying R models and functions as web services, this follows a paradigm and interface library very similar to Azure ML operationalization. The library is called mrsdeploy. We have some R deployment samples for both notebook and R Tools for Visual Studio (RTVS) and RStudio. The olapR package in Microsoft R Server lets you run MDX queries and connect directly to OLAP cubes on SQL Server 2016 Analysis Services from your R solution. SQL Server 2016 Developer edition and the associated Microsoft R In-DB analytics is also updated to Service Pack 1.
  2. R Studio Desktop open source edition is now preinstalled into the VM, by popular demand.
  3. R Tools for Visual Studio is now updated to version 0.5, bringing in multi-window plotting and SQL tooling to run R code on SQL Server 2016.
  4. Microsoft Cognitive Toolkit (formerly called CNTK) is now on Version 2 Beta 6, and features several improvements and sample notebooks to perform fast deep learning using Python interface or the CNTK Brainscript interface.
  5. Apache Drill, a SQL based query tool that can work with various data sources and formats (e.g. JSON, CSV), was part of our previous update. We now prepackage and configure drivers to access various Azure data services such as Blobs, SQLDW/Azure SQL, HDI and Document DB. See this tutorial in our gallery for information on how to query data in various Azure data sources from within the Drill SQL query language.
  6. JuliaPro is available to DSVM users and is now pre-installed and pre-configured on the VM, thanks to Julia Computing (a company founded by the creators of Julia programming language). JuliaPro is a curated distribution of the open source Julia language along with a set of popular packages for scientific computing, data science, AI and optimization. The JuliaPro distribution comes with an Atom based IDE, Jupyter notebooks and several sample notebooks on the DSVM Jupyter instance to help you get started. Julia Computing also provides an Enterprise edition with commercial support.
  7. The Deep Learning Toolkit for the Windows DSVM is an extension to help you jump start deep learning on Azure GPU VMs, and without having to spend time installing GPU framework dependencies and drivers or configuring the various deep learning tools. This extension has been updated to include the latest versions of CNTK 2, mxNet for GPU along with new samples. It also features the Windows version of TensorFlow.

We also offer a Linux Edition of the data science virtual machine and there will be a separate post on major updates there.

Meanwhile, here are some resources to get you started with the DSVM.

Windows Edition

Linux Edition

Webinar

I’d like to end this post with a graphical summary of the DSVM, showing a [non-exhaustive] list of the various tools that are preinstalled. DSVM helps you focus more on data science and spend less time on installing, configuring and administering tools, thereby making you more productive. Give DSVM a shot today and send us feedback on how we can make it even better for your data science needs.


Gopi