Author Archives: Julia Computing, Inc.

Newsletter December 2017

By: Julia Computing, Inc.

Re-posted from: http://juliacomputing.com/blog/2017/12/05/december-newsletter.html

Happy holidays from Julia Computing and best wishes for a prosperous and productive 2018.

  1. New Julia Developments
    • Major New Release of DataFrames.jl v0.11
    • JuliaBox – Commercial Version Now Available for University and Corporate Users with Enhancements and Support
    • Improved C++ Interoperability Interface
    • JuliaPro Amazon Machine Image and Docker Image
  2. Julia Computing at SC17 and Intel HPC Developer Conference
  3. Julia Computing at Analytics Vidhya’s DataHack Summit, Sponsored by Intel
  4. Julia for Astrodynamics
  5. Julia in Linux Magazine
  6. Julia and Julia Computing in the News
  7. Julia en Français
  8. Upcoming Events Featuring Julia
  9. Recent Events Featuring Julia
  10. Contact Us


1. New Julia Developments

i. Major New Release of DataFrames.jl v0.11:

DataFrames v0.11 has been released by the Julia community with a number of important updates:

  • ‘NA’ has been replaced with ‘missing’, making it much easier to work with missing data
  • Faster joining and grouping
  • Better display of DataFrames
  • Improved documentation
  • Modeling features are now in StatsModels.jl, whereas data import/export features are now in CSV.jl
  • Many other improvements

Don’t let the version number fool you. This version has been in the works for a long time, and leverages important new features in the Julia compiler. See the release announcement and try it out!

ii. JuliaBox – Commercial Version Now Available for University and Corporate Users with Enhancements and Support:

In response to user demand, Julia Computing has introduced a new and improved JuliaBox experience with increased memory and more support. For pricing and more information about the new commercial version of JuliaBox, please contact us. The free version of JuliaBox remains available for current and new users.

iii. Improved C++ Interoperability Interface:

Cxx.jl and CxxWrap.jl allow users to wrap C++ libraries in Julia. Use Cxx.jl to write the wrapper package in Julia code or CxxWrap.jl to write it entirely in C++ and call from Julia with a single line of Julia code. It is also possible to write and call Julia code from within C++, giving Julia and C++ complete two-way interoperability.

iv. JuliaPro Amazon Machine Image and Docker Image:

JuliaPro, the fastest on-ramp for quants, data scientists and researchers, is now available as an Amazon Machine Image on AWS EC2 (Red Hat Enterprise Linux v7.4 and Ubuntu 16.04) and as a Docker image (Ubuntu 16.04 and Centos 7) for use in containerized environments such as Kubernetes. More information is available here.

2. Julia Computing at SC17 and Intel HPC Developer Conference

Denver hosted the Intel HPC Developer Conference November 11-12 and SC17 November 12-17. Julia Computing participated in both conferences and presented the Celeste case study, one of the latest and most exciting developments in high performance computing using Julia. Julia Computing’s Ranjan Anantharaman was recognized for providing the Best Tutorial at the Intel HPC Developer Conference.


Julia Computing’s Ranjan Anantharaman (left), winner of the Best Tutorial Award at the Intel HPC Developer Conference 2017

3. Julia Computing at Analytics Vidhya’s DataHack Summit, Sponsored by Intel

Julia Computing was featured as part of the Intel keynote presentation about the future of high performance computing at Analytics Vidhya’s DataHack Summit in Bangalore, India held November 9-11. Julia Computing’s Rajshekar Behar presented Julia’s work with Celeste, Intel and Intel Skylake architecture.

4. Julia for Astrodynamics

Helge Eichhorn, Software Engineer at Telespazio VEGA Deutschland, presented Astrodynamics.jl: An Open-Source Framework for Interactive High-Performance Mission Analysis at the Open Source Cubesat Workshop on Nov 23 at the European Space Operations Center (ESOC/ESA) in Darmstadt, Germany.

5. Julia in Linux Magazine

Professor Mark Vogelsberger, Theoretical Astrophysicist at MIT, published an article in Linux Magazine in Jan 2016 titled “Getting Parallel: Creating Parallel Applications with the Julia Programming Language.” According to Professor Vogelsberger: “The Julia code is … more than 100 times faster than the equivalent Python code. Multiple dispatch with function calls gives Julia extremely efficient code that is practically superior to any high-level language. Faster code in Julia can be achieved without any tricks like vectorization or outsourcing to C extensions. By contrast, such tricks are often necessary to speed up Python or R code.”

6. Julia and Julia Computing in the News

  • Tangent Works Uses Julia to Win IEEE Global Energy Forecasting Competition 2017: Tangent Works, a European machine learning company, used Julia to win the IEEE Global Energy Forecasting Competition 2017 (GEFCom2017).

  • Julia Featured in insideHPC’s “AI-HPC Is Happening Now” White Paper: insideHPC, a leading blog in the high performance computing community, featured Julia and Julia Computing in this white paper about artificial intelligence and high performance computing.

  • Julia Climbs to #35 on TIOBE Index of Most Popular Programming Languages: Julia entered the Top 50 most popular programming languages for the first time in September 2016, and has climbed to #35 since last year.

  • Julia Computing Featured Among 10 Most Innovative Startups in India That Will Rule in 2018 and Beyond: KnowStartup featured Julia Computing among the 10 Most Innovative Startups in India That Will Rule in 2018 and Beyond.

  • Julia Language Delivers Petascale HPC Performance: TheNextPlatform explains that “the Celeste team demonstrated that the Julia language can support both petascale compute and terascale big data analysis on a leadership HPC system plus scale to handle the seven petabytes of data expected to be produced by the Large Synoptic Survey Telescope (LSST) every year.”

  • Julia Computing CEO Viral Shah Featured in FactorDaily Outliers Podcast: FactorDaily’s Outliers Podcast with Pankaj Mishra featured an interview with Julia Computing CEO Viral Shah: “You can thank Viral, … along with Alan Edelman, Jeff Bezanson, Stefan Karpinski, Keno Fischer and Deepak Vinchhi … the next time you have a safe flight in US airspace.”

  • Intel Reports Faster Stock Price Estimation Using Julia: @IntelBusiness reports that Julia Computing’s stock price estimation tool runs up to 38% faster – “a big gain for a fast-moving industry.”

  • Plotting in Julia: Tom Breloff published a blog post titled “Plots: Past, Present and Future” about plotting in Julia.

  • Feigenbaum’s Alpha: Professor Stuart Brorson from Northeastern University’s Department of Mathematics published a blog post entitled “A High Precision Calculation of Feigenbaum’s Alpha in Julia”.

7. Julia en Français

Xavier Gandibleux, Professor of Operations Research and Computer Science at the Université de Nantes, is writing a book in French about using Julia and JuMP for modeling and solving linear optimization problems in Operations Research.

8. Upcoming Events Featuring Julia

Do you know of any upcoming conferences, meetups, trainings, hackathons, talks, presentations or workshops involving Julia? Would you like to organize a Julia event on your own, or in partnership with your company, university or other organization? Let us help you spread the word and support your event by sending us an email with details. Here are some upcoming events:

9. Recent Events Featuring Julia

Do you want to share photos, videos or details of your most recent conference, meetup, training, hackathon, talk, presentation or workshop involving Julia? Please send us an email with details and links.

10. Contact Us

Please contact us if you wish to:

  • Purchase or obtain license information for Julia products such as JuliaPro, JuliaPro Enterprise, JuliaRun, JuliaDB, JuliaFin or JuliaBox
  • Obtain pricing for Julia consulting projects for your enterprise
  • Schedule Julia training for your organization
  • Share information about exciting new Julia case studies or use cases
  • Spread the word about an upcoming conference, workshop, training, hackathon, meetup, talk or presentation involving Julia
  • Partner with Julia Computing to organize a Julia meetup, conference, workshop, training, hackathon, talk or presentation involving Julia

About Julia and Julia Computing

Julia is the fastest high performance open source computing language for data, analytics, algorithmic trading, machine learning, artificial intelligence, and many other domains. Julia solves the two language problem by combining the ease of use of Python and R with the speed of C++. Julia provides parallel computing capabilities out of the box and unlimited scalability with minimal effort. For example, Julia has run at petascale on 650,000 cores with 1.3 million threads to analyze over 56 terabytes of data using Cori, the world’s sixth-largest supercomputer. With more than 1.2 million downloads and +161% annual growth, Julia is one of the top programming languages developed on GitHub. Julia adoption is growing rapidly in finance, insurance, machine learning, energy, robotics, genomics, aerospace, medicine and many other fields.

Julia Computing was founded in 2015 by all the creators of Julia to develop products and provide professional services to businesses and researchers using Julia. Julia Computing offers the following products:

  • JuliaPro for data science professionals and researchers to install and run Julia with more than one hundred carefully curated popular Julia packages on a laptop or desktop computer.
  • JuliaRun for deploying Julia at scale on dozens, hundreds or thousands of nodes in the public or private cloud, including AWS and Microsoft Azure.
  • JuliaFin for financial modeling, algorithmic trading and risk analysis including Bloomberg and Excel integration, Miletus for designing and executing trading strategies and advanced time-series analytics.
  • JuliaDB for in-database in-memory analytics and advanced time-series analysis.
  • JuliaBox for students or new Julia users to experience Julia in a Jupyter notebook right from a Web browser with no download or installation required.

To learn more about how Julia users deploy these products to solve problems using Julia, please visit the Case Studies section on the Julia Computing Website.

Julia users, partners and employers hiring Julia programmers in 2017 include Amazon, Apple, BlackRock, Capital One, Citibank, Comcast, Disney, Facebook, Ford, Google, IBM, Intel, KPMG, Microsoft, NASA, Oracle, PwC, Uber, and many more.

Introduction to the packages Cxx.jl and CxxWrap.jl

By: Julia Computing, Inc.

Re-posted from: http://juliacomputing.com/blog/2017/12/01/cxx-and-cxxwrap-intro.html

Introduction

Cxx.jl is a Julia package that provides a C++ interoperability interface for Julia. It also provides an experimental C++ REPL mode for the Julia REPL. With Cxx.jl, it is possible to directly access C++ using the @cxx macro from Julia.

With Cxx.jl and CxxWrap.jl. when facing the task of wrapping a C++ library in a Julia package, authors now have 2 options:

  • Use Cxx.jl to write the wrapper package in Julia code
  • Use CxxWrap to write the wrapper completely in C++ (and one line of Julia code to load the .so)

Cxx.jl

Functionality

There are two ways to access the main functionality provided by this package. The first is using the @cxx macro, which puns on Julia syntax to provide C++ compatibility.

The macro supports two main usages:

  • A static function call @cxx mynamespace::func(args...)
  • A membercall (where m is a CppPtr, CppRef or a CppValue) @cxx m->foo(args...)

Additionally, this package provides the cxx"" and icxx"" custom string literals for inputting C++ syntax directly. The two string literals are distinguished by the C++ level scope they represent.

In summary, the two approaches to embed C++ functions in Julia discussed above would look like this :

# Using @cxx (e.g.):   
cxx""" void cppfunction(args){ . . .} """ => @cxx cppfunction(args)

# Using icxx (e.g.):
julia_function (args) icxx""" *code here*  """

The C++ REPL

This package contains an experimental C++ REPL feature. Using the package will automatically add a new pane to your REPL that is accessible by pressing the < key.

Installation

The package is installable on Julia 0.5 and newer and is available through Julia’s package manager:

Pkg.add("Cxx")

Building the C++ code requires the same system tools necessary for building Julia from source. Further, Debian/Ubuntu users should install libedit-dev and libncurses5-dev, and RedHat/CentOS users should install libedit-devel.

Using Cxx.jl with examples

Example 1: Embedding a simple C++ function in Julia

# include headers
julia> using Cxx
julia> cxx""" #include<iostream> """  

# Declare the function
julia> cxx"""  
         void mycppfunction() {   
            int z = 0;
            int y = 5;
            int x = 10;
            z = x*y + 2;
            std::cout << "The number is " << z << std::endl;
         }
      """
# Convert C++ to Julia function
julia> julia_function() = @cxx mycppfunction()
julia_function (generic function with 1 method)

# Run the function
julia> julia_function()
The number is 52

Example 2: Pass numeric arguments from Julia to C++

julia> jnum = 10
10

julia> cxx"""
           void printme(int x) {
              std::cout << x << std::endl;
           }
       """

julia> @cxx printme(jnum)
10

Example 3: Pass strings from Julia to C++

julia> cxx"""
         void printme(const char *name) {
            // const char* => std::string
            std::string sname = name;
            // print it out
            std::cout << sname << std::endl;
         }
     """

julia> @cxx printme(pointer("John"))
   John

Example 4: Pass a Julia expression to C++

julia> cxx"""
          void testJuliaPrint() {
              $:(println("\nTo end this test, press any key")::Nothing);
          }
       """

julia> @cxx testJuliaPrint()
       To end this test, press any key

Example 5: Embedding C++ code inside a Julia function

function playing()
    for i = 1:5
        icxx"""
            int tellme;
            std::cout<< "Please enter a number: " << std::endl;
            std::cin >> tellme;
            std::cout<< "\nYour number is "<< tellme << "\n" <<std::endl;
        """
    end
end
playing();

Click here for more information, examples, and documentation.

CxxWrap.jl

This package lets you write the code for the Julia wrapper in C++, and then use a one-liner on the Julia side to make the wrapped C++ library available there.

The mechanism behind this package is that functions and types are registered in C++ code that is compiled into a dynamic library. This dynamic library is then loaded into Julia, where the Julia part of this package uses the data provided through a C interface to generate functions accessible from Julia. The functions are passed to Julia either as raw function pointers (for regular C++ functions that don’t need argument or return type conversion) or std::functions (for lambda expressions and automatic conversion of arguments and return types). The Julia side of this package wraps all this into Julia methods automatically.

Installation

Like any other registered Julia package, installation completes by running the following package manager command:

Pkg.add("CxxWrap")

Features

  • Support for C++ functions, member functions and lambdas
  • Classes with single inheritance, using abstract base classes on the Julia side
  • Trivial C++ classes can be converted to a Julia isbits immutable
  • Template classes map to parametric types, for the instantiations listed in the wrapper
  • Automatic wrapping of default and copy constructor (mapped to deepcopy) if defined on the wrapped C++ class
  • Facilitate calling Julia functions from C++

A Hello, World example with CxxWrap.jl

Suppose we want to expose the following C++ function to Julia in a module called CppHello:

std::string greet()
{
   return "hello, world";
}

Using the C++ side of CxxWrap, this can be exposed as follows:

#include "jlcxx/jlcxx.hpp"

JULIA_CPP_MODULE_BEGIN(registry)
  jlcxx::Module& hello = registry.create_module("CppHello");
  hello.method("greet", &greet);
JULIA_CPP_MODULE_END

Once this code is compiled into a shared library (say libhello.so) it can be used in Julia as follows:

using CxxWrap

# Load the module and generate the functions
wrap_modules(joinpath("path/to/built/lib","libhello"))
# Call greet and show the result
@show CppHello.greet()

More such examples and documentation for the package can be found here.

This post was formatted for the Julia Computing blog by Rajshekar Behar

JuliaPro now available as an Amazon Machine Image and a Docker Image

By: Julia Computing, Inc.

Re-posted from: http://juliacomputing.com/blog/2017/11/29/juliapro-ami-and-docker-image.html

We are pleased to announce the release of JuliaPro in the form of a an AMI (Amazon Machine Image) for use on the AWS EC2 platform, as well as a Docker image for use in containerised environments, including Kubernetes.

JuliaPro is the fastest on-ramp to Julia for individual researchers, quants, traders, economists, engineers, scientists, students and others. Beginners and experts can build better software quicker while benefiting from Julia’s unparalleled high performance. It includes a Julia compiler, a profiler, and a Julia IDE (integrated development environment) bundled with over a 100 curated packages that include data visualization and plotting.

JuliaPro was always available as a single installer bundle, making it easy for desktop users to get started.
However, requiring an installation step makes devops more difficult than it should be for production workloads. We know many of our users are running Julia applications on large server clusters in production, and we wanted to make it easy to do so.

JuliaPro’s Amazon Machine Image (AMI)

We are releasing 2 variants of JuliaPro for the AMI

  • JuliaPro on Red Hat Enterprise Linux v7.4
  • JuliaPro on Ubuntu 16.04

Contents of the AMI

Both variants of JuliaPro mentioned above have the following additional softwares installed

  • [JuliaPro v0.6.1.1]((https://shop.juliacomputing.com/Products/?page_id=156134)
  • GCC
  • Python
  • CURL
  • ZMQ
  • OpenJDK
  • R
  • HDF5

JuliaPro packages such as PyCall, JavaCall, RCall, ZMQ.jl, and HDF5.jl are configured to work with pre-installed softwares, so the AMI is ready to use as soon as you boot up your instance.

Accessing the JuliaPro AMIs

JuliaPro v0.6.1.1 is installed in the following location on both AMI variants

"$HOME/JuliaPro-0.6.1.1

The JuliaPro REPL can be accessed from the following location

"$HOME/JuliaPro-0.6.1.1 /Julia/bin/julia”

Search for JuliaPro in the following regions to access our AMIs:

  • us-west-1
  • us-west-2
  • us-east-1
  • us-east-2
  • ap-south-1
  • eu-west-1
  • eu-central-1

JuliaPro’s Docker Image

The main purpose of making this image available is to enable Docker and Kubernetes users to easily work with Julia packages, and to also extend the JuliaPro infrastructure to meet their needs.

JuliaPro’s Docker Image is hosted on Dockerhub and comes with two variants of the base images:

  • Ubuntu (16.04 LTS)
  • Centos 7

The following are the available tags:

  • Ubuntu: 0.6.1.1-ubuntu,latest
  • Centos: 0.6.1.1-centos

The Docker Image can be pulled using the command

docker pull juliacomputing/juliapro:latest

The JuliaPro Installation Path in the container is

/juliapro/bin/JuliaPro-[version]/

Ways to access the JuliaPro Docker Image

  • By starting the Julia REPL with the command: docker run -it juliacomputing/juliapro:latest

  • By starting a Jupyter Notebook with the command: docker run -it -p 8888:8888 --entrypoint jupyter_notebook juliacomputing/juliapro:latest , followed by opening the displayed link in a web browser.

  • By directly running Julia Expressions: docker run -it --entrypoint julia juliacomputing/juliapro:latest -e "println(1+2)"

  • Or by running Bash: docker run -it --entrypoint bash juliacomputing/juliapro:latest