By: Blog on JuliaGPU

Re-posted from: https://juliagpu.org/2020-07-18-cuda_1.3/

Today we’re releasing CUDA.jl 1.3, with several new features. The most prominent
change is support for multiple GPUs within a single process.

Multi-GPU programming

With CUDA.jl 1.3, you can finally use multiple CUDA GPUs within a single process. To switch
devices you can call device!, query the current device with device(), or reset it using
device_reset!():

julia> collect(devices())
9-element Array{CuDevice,1}:
 CuDevice(0): Tesla V100-PCIE-32GB
 CuDevice(1): Tesla V100-PCIE-32GB
 CuDevice(2): Tesla V100-PCIE-32GB
 CuDevice(3): Tesla V100-PCIE-32GB
 CuDevice(4): Tesla V100-PCIE-16GB
 CuDevice(5): Tesla P100-PCIE-16GB
 CuDevice(6): Tesla P100-PCIE-16GB
 CuDevice(7): GeForce GTX 1080 Ti
 CuDevice(8): GeForce GTX 1080 Ti

julia> device!(5)

julia> device()
CuDevice(5): Tesla P100-PCIE-16GB

Let’s define a kernel to show this really works:

julia> function kernel()
           dev = Ref{Cint}()
           CUDA.cudaGetDevice(dev)
           @cuprintln("Running on device $(dev[])")
           return
       end

julia> @cuda kernel()
Running on device 5

julia> device!(0)

julia> device()
CuDevice(0): Tesla V100-PCIE-32GB

julia> @cuda kernel()
Running on device 0

Memory allocations, like CuArrays, are implicitly bound to the device they
were allocated on. That means you should take care to only use an array when the
owning device is active, or you will run into errors:

julia> device()
CuDevice(0): Tesla V100-PCIE-32GB

julia> a = CUDA.rand(1)
1-element CuArray{Float32,1}:
 0.6322775

julia> device!(1)

julia> a
ERROR: CUDA error: an illegal memory access was encountered

Future improvements might make the array type device-aware.

Multitasking and multithreading

Dovetailing with the support for multiple GPUs, is the ability to use these GPUs
on separate Julia tasks and threads:

julia> device!(0)

julia> @sync begin
         @async begin
           device!(1)
           println("Working with $(device()) on $(current_task())")
           yield()
           println("Back to device $(device()) on $(current_task())")
         end
         @async begin
           device!(2)
           println("Working with $(device()) on $(current_task())")
         end
       end
Working with CuDevice(1) on Task @0x00007fc9e6a48010
Working with CuDevice(2) on Task @0x00007fc9e6a484f0
Back to device CuDevice(1) on Task @0x00007fc9e6a48010

julia> device()
CuDevice(0): Tesla V100-PCIE-32GB

Each task has its own local GPU state, such as the device it was bound to,
handles to libraries like CUBLAS or CUDNN (which means that each task can
configure libraries independently), etc.

Minor features

CUDA.jl 1.3 also features some minor changes:

• Reinstated compatibility with Julia 1.3
• Support for CUDA 11.0 Update 1
• Support for CUDNN 8.0.2

Known issues

Several operations on sparse arrays have been broken since CUDA.jl 1.2, due to
the deprecations that were part of CUDA 11. The next version of CUDA.jl will
drop support for CUDA 10.0 or older, which will make it possible to use new
cuSPARSE APIs and add back missing functionality.

By: Blog by Bogumił Kamiński

Re-posted from: https://bkamins.github.io/julialang/2020/07/17/pair.html

Introduction

A recent StackOverflow question prompted me to write down a glossary in
what cases DataFrames.jl allows the use of =>. In this post I summarize it, as
indeed it has a heavy usage.

Essentially in many places we use => instead of =. The difference is that
=> is treated by the Julia parser as a two argument operator in Julia creating
a Pair object (which is much more flexible than = which has a very strict
allowed usage and it is possible to dispatch on Pair).

The functions in DataFrames.jl that have a special treatement of => are:

• DataFrame, insertcols!
• select, select!, transform, transform!, combine
• filter, filter!
• describe
• raname!, rename
• innerjoin, leftjoin, rightjoin, outerjoin, antijoin, semijoin

I have grouped these functions by the same meaning of =>, so as you can see
its meaning is context dependent. I describe these usages in sections below.

All examples were run under Julia 1.4.2 and DataFrames.jl 0.21.4. They are meant
to be executed linearly (so later examples assume that earlier examples were
run).

Column assignment: DataFrame, insertcols!

In this case the syntax is target_column_name => value and means that
target_column_name should be set to hold value, for example:

julia> using DataFrames

julia> df = DataFrame(:x => 1:4, :y => "a")
4×2 DataFrame
│ Row │ x     │ y      │
│     │ Int64 │ String │
├─────┼───────┼────────┤
│ 1   │ 1     │ a      │
│ 2   │ 2     │ a      │
│ 3   │ 3     │ a      │
│ 4   │ 4     │ a      │

julia> insertcols!(df, :a => 'a':'d', :b => exp(1))
4×4 DataFrame
│ Row │ x     │ y      │ a    │ b       │
│     │ Int64 │ String │ Char │ Float64 │
├─────┼───────┼────────┼──────┼─────────┤
│ 1   │ 1     │ a      │ 'a'  │ 2.71828 │
│ 2   │ 2     │ a      │ 'b'  │ 2.71828 │
│ 3   │ 3     │ a      │ 'c'  │ 2.71828 │
│ 4   │ 4     │ a      │ 'd'  │ 2.71828 │

Column transformation: select, select!, transform, transform!, combine

In this case there are three allowed forms:

• source_columns => transformation_function => target_column_name
• source_columns => transformation_function (target column name will be
  automatically generated)
• source_column => target_column_name (essentially a way to rename a column)

Here are some examples:

julia> transform(df, :x => maximum)
4×5 DataFrame
│ Row │ x     │ y      │ a    │ b       │ x_maximum │
│     │ Int64 │ String │ Char │ Float64 │ Int64     │
├─────┼───────┼────────┼──────┼─────────┼───────────┤
│ 1   │ 1     │ a      │ 'a'  │ 2.71828 │ 4         │
│ 2   │ 2     │ a      │ 'b'  │ 2.71828 │ 4         │
│ 3   │ 3     │ a      │ 'c'  │ 2.71828 │ 4         │
│ 4   │ 4     │ a      │ 'd'  │ 2.71828 │ 4         │

julia> combine(df, :x => maximum => :mx)
1×1 DataFrame
│ Row │ mx    │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 4     │

julia> select(df, [:x, :b] => +)
4×1 DataFrame
│ Row │ x_b_+   │
│     │ Float64 │
├─────┼─────────┤
│ 1   │ 3.71828 │
│ 2   │ 4.71828 │
│ 3   │ 5.71828 │
│ 4   │ 6.71828 │

julia> transform(df, :a => :b, :b => :a)
4×4 DataFrame
│ Row │ x     │ y      │ a       │ b    │
│     │ Int64 │ String │ Float64 │ Char │
├─────┼───────┼────────┼─────────┼──────┤
│ 1   │ 1     │ a      │ 2.71828 │ 'a'  │
│ 2   │ 2     │ a      │ 2.71828 │ 'b'  │
│ 3   │ 3     │ a      │ 2.71828 │ 'c'  │
│ 4   │ 4     │ a      │ 2.71828 │ 'd'  │

(note that what source_coulumn_names and transformation_function can be is
quite flexible — please have a look into the documentation to learn about
the available options)

Row selection: filter, filter!

In this case there allowed form is source_columns => predicate, for
example:

julia> filter(:x => >(1.5), df)
3×4 DataFrame
│ Row │ x     │ y      │ a    │ b       │
│     │ Int64 │ String │ Char │ Float64 │
├─────┼───────┼────────┼──────┼─────────┤
│ 1   │ 2     │ a      │ 'b'  │ 2.71828 │
│ 2   │ 3     │ a      │ 'c'  │ 2.71828 │
│ 3   │ 4     │ a      │ 'd'  │ 2.71828 │

Summarizing: describe

In this case there allowed form is target_column_name => aggregation_function.
Then aggregation_function is applied to every column of a data frame and the
result is stored in target_column_name column. For example:

julia> describe(df, :first => first, :last => last)
4×3 DataFrame
│ Row │ variable │ first   │ last    │
│     │ Symbol   │ Any     │ Any     │
├─────┼──────────┼─────────┼─────────┤
│ 1   │ x        │ 1       │ 4       │
│ 2   │ y        │ a       │ a       │
│ 3   │ a        │ 'a'     │ 'd'     │
│ 4   │ b        │ 2.71828 │ 2.71828 │

Names transformation: raname!, rename

The form is source_column => target_column_name to rename source_column to
target_column_name, e.g.:

julia> rename(df, "x" => "xx", "y" => "yy")
4×4 DataFrame
│ Row │ xx    │ yy     │ a    │ b       │
│     │ Int64 │ String │ Char │ Float64 │
├─────┼───────┼────────┼──────┼─────────┤
│ 1   │ 1     │ a      │ 'a'  │ 2.71828 │
│ 2   │ 2     │ a      │ 'b'  │ 2.71828 │
│ 3   │ 3     │ a      │ 'c'  │ 2.71828 │
│ 4   │ 4     │ a      │ 'd'  │ 2.71828 │

julia> rename(df, 1 => :xx, 2 => :yy)
4×4 DataFrame
│ Row │ xx    │ yy     │ a    │ b       │
│     │ Int64 │ String │ Char │ Float64 │
├─────┼───────┼────────┼──────┼─────────┤
│ 1   │ 1     │ a      │ 'a'  │ 2.71828 │
│ 2   │ 2     │ a      │ 'b'  │ 2.71828 │
│ 3   │ 3     │ a      │ 'c'  │ 2.71828 │
│ 4   │ 4     │ a      │ 'd'  │ 2.71828 │

Joining: innerjoin, leftjoin, rightjoin, outerjoin, antijoin, semijoin

Here, you can pass left_column_name => right_column_name as an on keyword
argument in the case when left and right joined data frames have different names
of columns on which the join should be performed, for instance:

julia> innerjoin(df, df2, on = :x => :x2)
4×5 DataFrame
│ Row │ x     │ y      │ a    │ b       │ a2   │
│     │ Int64 │ String │ Char │ Float64 │ Char │
├─────┼───────┼────────┼──────┼─────────┼──────┤
│ 1   │ 1     │ a      │ 'a'  │ 2.71828 │ 'a'  │
│ 2   │ 2     │ a      │ 'b'  │ 2.71828 │ 'b'  │
│ 3   │ 3     │ a      │ 'c'  │ 2.71828 │ 'c'  │
│ 4   │ 4     │ a      │ 'd'  │ 2.71828 │ 'd'  │

julia> innerjoin(df, df2, on = ["x" => "x2", "a" => "a2"])
4×4 DataFrame
│ Row │ x     │ y      │ a    │ b       │
│     │ Int64 │ String │ Char │ Float64 │
├─────┼───────┼────────┼──────┼─────────┤
│ 1   │ 1     │ a      │ 'a'  │ 2.71828 │
│ 2   │ 2     │ a      │ 'b'  │ 2.71828 │
│ 3   │ 3     │ a      │ 'c'  │ 2.71828 │
│ 4   │ 4     │ a      │ 'd'  │ 2.71828 │

(in the second example we have performed join on two pairs of columns)

I hope that you will find the examples provided above useful when working with
DataFrames.jl!

Register Now for JuliaCon 2020: JuliaCon 2020 will take place online Wednesday
July 29 through Friday July 31 from 12:30 pm – 7:30 pm GMT. Workshops are Friday
July 24 through Tuesday July 28. The schedule of events is available now.
Click here to reserve your spot

Julia Computing Virtual Booth Available During JuliaCon: Julia Computing’s
virtual booth will be available during JuliaCon. Visit JuliaCon.org during
JuliaCon to find Julia Computing’s virtual booth. Connect with a Julia
Computing expert to learn more about how to leverage Julia’s speed, power
and ease of use to improve productivity, reduce cost and speed time to market.

JuliaTeam and JuliaRun Live Demonstrations During JuliaCon: Julia Computing
will demonstrate exciting new JuliaTeam and JuliaRun product features including
VS Code integration, running Julia in your browser, single-click parallel
computing in the cloud and more. Live online demonstrations are Wednesday
July 29 (5:45 pm – 6:45 pm GMT), Thursday July 30 (11:30 am – 12:30 PM GMT)
and Friday July 31 (7:45 pm – 8:45 pm GMT). Click here to register.

Feel the Power! Julia for Energy Webinar from Julia Computing: Dr. Matt Bauman,
Julia Computing Senior Research Scientist, leads a free Julia for Energy Webinar
on Thursday July 16 from 12 noon – 1 pm Eastern (US). Space is limited
so please click here to register. Julia users
in the energy sector include AOT, EVN, Fugro Roames, Invenia, LAMPS PUC-RIO, PSR and the US Department of Energy.
Register today to reserve your spot and learn more about how Julia can help make
optimization, energy trading and modeling more efficient, faster, simpler and
less expensive.

Julia Computing Enterprise Products

• JuliaSure:
  JuliaSure from
  Julia Computing provides full service development support,
  production support and indemnification for companies using Julia.
  Subscriptions are USD $99 per month. Click here to
  subscribe.

• JuliaTeam:
  JuliaTeam from
  Julia Computing lets your entire enterprise work together
  using Julia. Collaborate, develop and manage private and public
  packages across your organization, manage open source licenses and
  benefit from continuous integration, deployment, security, indemnity
  and enterprise governance. Click here for more
  information.

• JuliaRun:
  JuliaRun from
  Julia Computing helps you scale and deploy Julia using high
  performance computing (HPC) resources, including large parallel
  simulations and analyses in the cloud: AWS, Microsoft Azure or
  Google Cloud. Click here for more
  information.

• Pumas: Pumas
  from Julia Computing is a comprehensive platform for pharmaceutical
  modeling and simulation, providing a single tool for the entire drug
  development pipeline. Click here for more
  information.

The Great ML Language (Un)Debate: Dr. Huda Nassar represented Julia in the
Great ML Language (Un)Debate
to discuss the benefits of Julia compared with alternatives
such as R, Python, Javascript and Swift. The
Great ML Language (Un)Debate is available on YouTube.

Julia for Pythonistas Colab Notebook: Aurélien Geron
published the Julia for Pythonistas Colab Notebook. As Aurélien explains, “Julia looks and feels
a lot like Python, only much faster. It’s dynamic, expressive, extensible,
with batteries included, in particular for data science.”

Julia Chinese Community 2020 Summer Conference (Julia中文社区2020年夏季会议):
The Julia community in China will hold a summer conference, tentatively
scheduled for August 18-24. More information is available here.

Julia Reads CSVs 10-20x Faster than Python and R: New benchmarks are
available demonstrating that Julia reads CSV files 10-20x faster than
Python and R. Click here for more information.

MIT JuliaLab and Microsoft Tackle Pandemic Modeling: MIT JuliaLab’s Chris
Rackauckas teamed with researchers from Microsoft, University of Oxford,
Swansea University, University of Cambridge, University of Warwick, Alan
Turing Institute and Royal Society Rapid Assistance in Modeling the Pandemic (RAMP)
to build an open platform for pandemic modeling in Julia. More information is
available here.

Julia and Spark, Better Together: Julia Computing’s Avik Sengupta published an
updated blog titled Julia and Spark, Better Together
which describes how Spark.jl enables the use of Julia programs on Spark.

Argonne National Laboratory: Julia Computing’s Mike Innes presented
Building Compilers for Numerical Programming online to the US Department of
Energy Argonne National Laboratory.

Interdisciplinary Centre for Mathematical and Computational Modeling:
Julia Computing Chief Scientist Alan Edelman presented
High Performance Computing: The Power of Language
via the University of Warsaw Interdisciplinary Centre for Mathematical and Computational Modeling’s
Virtual ICM Seminars in Computer and Computational Science as part of
Supercomputing Frontiers Europe 2020.

Julia for Climate Modeling: The Climate Modeling Alliance has published
CliMA 0.1: A First Milestone in the Next Generation of Climate Models
by Sabrina Pirzada which describes how they are using Julia to build better climate models.

Julia and Julia Computing in the News

• ZDNet: Programming Languages: Julia Touts its Speed Edge Over Python and R
• CScience: Comment Hydro-Québec Prévoit vos Besoins en Électricité
• Towards Data Science: The Great CSV Showdown: Julia vs Python vs R
• CSDN: 再见 Python，Hello Julia！
• Analytics India: Google Adds New Privacy Testing Module In TensorFlow
• Analytics India: Why Companies Still Struggle To Incorporate AI Into Existing Business Models
• EFinancialCareers: Emergent Coding Languages to Learn for Technology Jobs in Finance
• IProgrammer: SQLite Gets Jupyter Kernel

Julia Blog Posts

• Julia and Spark, Better Together: (Avik Sengupta)
• CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R: (Deepak Suresh)
• Julia vs. Python: Which Programming Language is Better?: (Devathon)
• CliMA 0.1: A First Milestone in the Next Generation of Climate Models: (Sabrina Pirzada)
• Stay Inspired while Coding with Inspiration.jl: (Zlatan Vasović)
• Analyze Mathematical Functions Interactively: (Zlatan Vasović)
• Make Your Julia Package in 10 Minutes: (Zlatan Vasović)
• Comparing dplyr vs DataFrames.jl: (Bogumił Kamiński)
• Cheap But Effective: Instituting Effective Pandemic Policies Without Knowing Who’s Infected: (Chris Rackauckas)
• Glue AD for Full Language Differentiable Programming: (Chris Rackauckas)
• How to Embed Project Environment Setup in Your Julia Script: (Bogumił Kamiński)
• Writing an AR(1) Process Simulator as a Module in Julia: (Bogumił Kamiński)
• Solving Sicherman Dice Puzzle Using DataFrames.jl: (Bogumił Kamiński)
• Learning Who Is the Author of the Current State of the Julia Language: (Bogumił Kamiński)
• Tutorials for DataFrames.jl Release 0.21. Part II: (Bogumił Kamiński)
• First Steps with Agent Based Modeling in the Julia Language: (Bogumił Kamiński)
• Julia Basics: REPL & Revise: (Ole Kröger)
• Julia Basics: Multiple Dispatch: (Ole Kröger)
• Tutorial on CubicHermiteSpline.jl: (Yi-Xin Liu)
• ConstraintSolver.jl Indicator Part 2: (Ole Kröger)
• ConstraintSolver.jl Reified Constraint: (Ole Kröger)
• Side Gig: Mentoring Julia Newcomers: (Ole Kröger)
• CUDA.jl 1.1: (Tim Besard)
• Pi vs. Tau: (Ole Kröger)
• The Meaning of Functions in Julia: (Tom Kwong)
• Naming Things Properly: (Tom Kwong)
• Programming Is Boring Art: (Tom Kwong)
• A History of Releasing Julia Packages: (Chris de Graaf)
• Franklin.jl: (Ole Kröger)
• Indicator Constraint + Benchmarks: (Ole Kröger)
• Housekeeping May 2020: (Ole Kröger)
• Data Structure Design in CSV.jl: (Jacob Quinn)
• SciML Ecosystem Update: Auto-Parallelism and Component-Based Modeling
• Forecasting the Weather with Neural ODEs: (Sebastian Callh)
• COVID-19 Epidemic Mitigation via Scientific Machine Learning (SciML): (Chris Rackauckas)

Upcoming Julia Online Events

• Webinar: Feel the Power! Julia for Energy Webinar with Matt Bauman (Julia Computing) July 16
• Virtual Conference: BuzzConf with Viral Shah (Julia Computing) July 27-31
• Online Demonstration: JuliaHub with (Julia Computing) July 29-31
• Virtual Conference: JuliaCon 2020 with (Julia Computing) July 29-31 (Workshops July 23-28)
• Virtual Booth: JuliaCon Virtual Booth with (Julia Computing) July 29-31
• Virtual Conference: Julia Chinese Community 2020 Summer Conference (Julia中文社区2020年夏季会议) August 18-24
• Online Meetup: Post-JuliaCon Gathering with Michael Mallari and (Julia5280) August 20
• Virtual Conference: Virtual AI Summit with (Julia Computing) September 2-3
• Online Meetup: Post-Labor Day / Back-to-School with Michael Mallari and (Julia5280) September 10
• Virtual Conference: ODSC Europe with (Julia Computing) September 17-19

Recent Julia Online Events

• Virtual Seminar: High Performance Computing – The Power of Language with Alan Edelman (Julia Computing) Virtual ICM Seminars in Computer and Computational Science June 4
• Virtual Conference: CogX 2020 Tools, Practice and Systems with Gwyn Jones (IQVIA) and Sebastian Vollmer (Alan Turing Institute) June 8-10
• Online Meetup: PowerDynamics.jl & Setfield.jl with Stephan Sahm and (Julia User Group Munich) June 10
• Online Training: Introduction to Julia with (Julia Computing) June 10-11
• Webinar: Bullish on Fast Code? Julia for Finance with Matt Bauman (Julia Computing) June 12
• Online Training: Introduction to Machine Learning and Artificial Intelligence in Julia with (Julia Computing) June 17-18
• Microtraining: How to Build a Model Using Flux and Productionize It with Dhairya Gandhi (Julia Computing) June 19
• Seminar: Building Compilers for Numerical Programming with Mike Innes (Julia Computing) June 24
• Online Training: Parallel Computing in Julia with (Julia Computing) June 24-25
• Webinar: Pharmacology and Pharmacometrics with Vijay Ivaturi (University of Maryland, Baltimore School of Pharmacy Center for Translational Medicine, Pumas-AI) June 30
• Online Meetup: Fall in Love with Julia: A Sparkling Hands-On Introduction to Julia 101 with Stephan Sahm and (Julia User Group Munich) July 8
• Virtual Conference: World Artificial Intelligence Conference with Viral Shah (Julia Computing) July 8-11
• Online Meetup: Post-Independence Day / Pre-JuliaCon Gathering with Michael Mallari and (Julia5280) July 8

Julia Jobs, Fellowships and Internships

Do you work at or know of an organization looking to hire Julia programmers as staff, research
fellows or interns? Would your employer be interested in hiring interns to work on open source
packages that are useful to their business? Help us connect members of our community to great
opportunities by sending us an email, and we’ll get the word out.

There are hundreds of Julia jobs currently listed on Indeed.com. Click here to apply.

Contact Us: Please contact us if you wish to:

• Purchase or obtain license information for Julia Computing products such as JuliaSure, JuliaTeam, JuliaRun or Pumas
• Obtain pricing for Julia consulting projects for your organization
• Schedule online Julia training for your organization
• Share information about exciting new Julia case studies or use cases
• Spread the word about an upcoming online event involving Julia
• Partner with Julia Computing to organize a Julia event online
• Submit a Julia internship, fellowship or job posting

About Julia and Julia Computing

Julia is the fastest high performance open source computing language for data, analytics, algorithmic trading,
machine learning, artificial intelligence, and other scientific and numeric computing applications.
Julia solves the two language problem by combining the ease of use of Python and R with the speed of C++.
Julia provides parallel computing capabilities out of the box and unlimited scalability with minimal effort.
Julia has been downloaded more than 17 million times and is used at more than 1,500 universities. Julia co-creators
are the winners of the 2019 James H. Wilkinson Prize for Numerical Software and the 2019 Sidney Fernbach Award.
Julia has run at petascale on 650,000 cores with 1.3 million threads to analyze over 56 terabytes of data using
Cori, one of the ten largest and most powerful supercomputers in the world.

Julia Computing was founded in 2015 by all the creators of Julia to develop products and provide
professional services to businesses and researchers using Julia.