By: Tim Besard

Re-posted from: https://juliagpu.org/post/2026-09-14-cuda_6.4/index.html

CUDA.jl 6.4 greatly improves support for NVIDIA Jetson boards, and further enhances caching of compiled GPU code for much faster TTFX.

NVIDIA Jetson support

One of the highlights is that NVIDIA Jetson devices are now properly supported. The CUDA.jl binary packages have been reworked to provide builds for older CUDA versions so that artifacts can be used, and many fixes have been applied to improve compatibility with Tegra hardware.

Since many Jetson devices rely on older CUDA toolkits, we have relaxed our requirement for the CUDA toolkit version: version 12 is still the minimum for full support, but we have reinstated best-effort support for CUDA 10 and 11, as used by default on older Jetson boards such as the Jetson Nano.

The following JetPack generations have been tested on:

The CUDA.jl README now clearly documents the level of support for each Jetson board and CUDA toolkit version in general.

JetPack 6+: full support on Jetson SBCs

Orin and Thor devices are fully supported. All CUDA.jl tests are expected to pass, both the core functionality that only relies on the CUDA toolkit, and any external packages that integrate with vendor libraries like cuDNN or cuTENSOR.

JetPack 5: limited support through CUDA 12

Xavier boards ship with CUDA 11.4 and, unlike a desktop GPU, you cannot simply install a newer driver: the kernel-mode driver is part of the L4T BSP. NVIDIA does publish a forward-compatibility driver for the r35 kernel driver, though, and CUDADriverjll now bundles it, enabling use of CUDA toolkit 12.5 on these devices.

With the upgraded driver and CUDA toolkit 12.5, all core CUDA.jl functionality is expected to work correctly. However, some external vendor libraries, specifically cuDNN and cuTENSOR, lack support for the hardware present on Xavier boards, or at least the versions packaged for CUDA.jl do. As a result, we do not consider Xavier boards as fully supported, however, most users should not encounter significant issues in most common use cases.

JetPack 4: best-effort support with several limitations

Previous versions of CUDA.jl used to flat-out reject the older toolkits found on JetPack 4:

ERROR: LoadError: CUDA.jl requires PTX 8.0.0, which is not supported by ptxas 10.2.89
ERROR: Failed to precompile CUDACore [bd0ed864-bdfe-4181-a5ed-ce625a5fdea2]

After fixing CUDA.jl and adding binaries for the older toolkits, the Jetson Nano can now support CUDA.jl, even without a local installation of the CUDA toolkit. The old toolkit and driver causes several issues though, which manifest as certain pieces of core functionality not being supported on these boards:

• the integrated profiler is unsupported due to crashes involving dynamic parallelism;

• certain array operations may fail due to missing APIs in the NVIDIA math libraries, without appropriate fallbacks;

• due to bugs in the vendor libraries, certain functionality (such as certain sparse matrix layouts) have been disabled.

We aim to keep CUDA.jl reasonably working on these boards, but the support is best-effort, implying that we will not degrade the experience for users on fully supported hardware or otherwise do significant development specifically to improve support for these older boards.

Caching compiled code

CUDA.jl v6.3 introduced the ability to cache inferred GPU code, greatly improving the so-called TTFX for GPU applications. With CUDA.jl v6.4 we take this further by actually caching compiled GPU code (CUBINs) as well. This relies on work in GPUCompiler.jl to make compiled code relocatable across sessions, and as such is available to any GPU back-end that opts in.

To evaluate, let's go back to the example from the previous blog post:

module Blurusing CUDA
using PrecompileToolsfunction blur_kernel!(dst, src, ::Val{R}) where R
    i = (blockIdx().x - 1) * blockDim().x + threadIdx().x
    if i <= length(dst)
        acc = zero(eltype(src))
        for k in -R:R
            @inbounds acc += src[clamp(i + k, 1, length(src))] / (1 + abs(k))
        end
        @inbounds dst[i] = sqrt(abs(acc))
    end
    return
endfunction blur(src, ::Val{R} = Val(4)) where R
    dst = similar(src)
    @cuda threads=256 blocks=cld(length(dst), 256) blur_kernel!(dst, src, Val(R))
    return dst
end@setup_workload begin
    @compile_workload begin
        blur(CUDA.zeros(Float32, 1024))
    end
endend

Timing the first call in a fresh session, on an RTX 5080 with Julia 1.13 and CUDA 13.4:

julia> using Blur, CUDAjulia> src = CUDA.rand(Float32, 1024);julia> @time Blur.blur(src);
  0.004722 seconds (255 allocations: 1.052 MiB, 80.82% compilation time)

This is a significant improvement over the 0.13s it took on CUDA.jl v6.3, and the 1.3s it took without any caching at all. It is also a major step towards supporting static compilation of CUDA.jl with JuliaC.jl, though there is still lots of work to be done in that area.

Other changes

• Support for CUDA 13.4 has been added

• The compiler has been upgraded to LLVM 23

• Logging has been unified, exposing everything through the JULIA_DEBUG env var and the enable_logging function.

The full list is in NEWS.md and the release notes. If something in here breaks for you, please file an issue. And if you are running a Jetson board we have not tested, we would like to hear about it either way.

By: n JuliaLang - The Julia programming language n

Re-posted from: https://julialang.org/blog/2026/09/julia-1.13-highlights/index.html

Highlights of the Julia 1.13 release.

By: Tim Besard

Re-posted from: https://juliagpu.org/post/2026-08-19-cutile_1.0/index.html

cuTile.jl has reached 1.0! The release adds tile windows via eachtile, masked and view-based atomics, an @atomic macro, sparse views, compiler remarks, and support for Tile IR 13.4, alongside a dedicated documentation site.

The package started as an experiment in expressing NVIDIA's tile-based programming model in Julia. Six months and three releases later, we're confident tagging an initial stable release. This comes with a dedicated documentation site.

Compared to v0.3, the 1.0 release adds the following features.

Tile windows

Previously, walking an array tile by tile meant passing the array, index, and shape to every ct.load and ct.store. The new ct.eachtile instead returns an indexable collection of fixed-shape windows. A blocked matrix multiplication shows the difference:

using CUDA, cuTile
import cuTile as ctfunction matmul!(C, A, B)
    a_tiles = ct.eachtile(A, (64, 32))
    b_tiles = ct.eachtile(B, (32, 64))
    c_tiles = ct.eachtile(C, (64, 64))    m, n = ct.bid(1), ct.bid(2)
    acc = zeros(Float32, (64, 64))
    for k in Int32(1):Int32(size(a_tiles, 2))
        acc = muladd(a_tiles[m, k], b_tiles[k, n], acc)
    end
    c_tiles[m, n] = acc
    return
endA = CUDA.rand(Float16, 256, 128)
B = CUDA.rand(Float16, 128, 256)
C = CUDA.zeros(Float32, 256, 256)
@cuda backend=cuTile blocks=(4, 4) matmul!(C, A, B)

size(a_tiles, 2) returns the number of windows along that dimension, avoiding a separate trip-count calculation. step controls the distance between window origins: a smaller value produces overlap, while a larger one leaves gaps.

adjacent = ct.eachtile(a, (8, 8))               # step defaults to the shape
overlap  = ct.eachtile(a, (8, 8); step=(4, 8))  # neighboring windows overlap

Partial edge windows are handled by the padding mode, as with a normal load. Unequal shape and step require Tile IR bytecode v13.3 or newer.

Atomics

cuTile now has three atomic-operation families, with different return values and ordering options.

The read-modify-write functions (ct.atomic_add and friends) return the old value and take a configurable memory order. In 1.0 they also accept a mask, useful for the tail block of a grid that does not divide the data evenly:

function histogram!(counts, data, n::Int32)
    pid = ct.bid(1)
    offs = (pid - Int32(1)) * Int32(128) .+ ct.arange(128)
    vals = ct.load(data; index=pid, shape=(128,))
    active = offs .<= n                    # mask off the tail
    ct.atomic_add(counts, vals, Int32(1); mask=active)
    return
end

The new ct.atomic_store_* family lowers to Tile IR's view-based atomic reductions. These reduce a tile into an array or an eachtile window and return nothing, using relaxed device-wide ordering:

function accumulate_tiles!(out, src)
    tiles = ct.eachtile(out, (128,))
    pid = ct.bid(1)
    ct.atomic_store_add(tiles, 1, ct.load(src; index=pid, shape=(128,)))
    return
end

ct.@atomic provides Base-style statement and value forms:

ct.@atomic counters[i] += update
ct.@atomic counters[i] = max(counters[i], value)
old_new = ct.@atomic counters[i] + value      # returns old => new

Statement forms default to relaxed ordering, while value forms default to acquire-release. View-based reductions and ct.@atomic require Tile IR 13.3.

Sparse views

view and @view on a TileArray now accept positive step ranges. On arrays with two or more dimensions, one dimension may instead use a 1D integer tile, creating a sparse view that ct.load and ct.store lower to a Tile IR gather/scatter view:

function pick_rows!(dst, src)
    rows = ct.arange(4; start=1, step=2)      # rows 1, 3, 5, 7
    selected = @view src[rows, 1:8]
    tile = ct.load(selected, (4, 8))
    ct.store(dst, (1, 1), tile)
    return
end

The load shape is explicit and static, while the range starts may be runtime values. Sparse loads apply the requested padding and stores clip partially out-of-bounds elements; repeated indices are fine for loads, but conflicting stores are undefined. Step ranges and sparse views require Tile IR 13.3.

Compiler remarks

tileiras can report whether it selected tensor cores, vector loads, and other optimizations. code_tiled and @device_code_tiled now print those diagnostics with remarks=true.

Compile the matmul above with Float32 inputs:

A = CUDA.rand(Float32, 256, 128)
B = CUDA.rand(Float32, 128, 256)
C = CUDA.zeros(Float32, 256, 256)
ct.@device_code_tiled remarks=true @cuda backend=cuTile blocks=(4, 4) matmul!(C, A, B)

// tileiras optimization remarks
// Name:            RemarkMemoryLoadInstructionSelected
//   - RemarkId:    3
//   - Remark:      Load instruction selected
// Name:            RemarkTensorCoreMMA
//   - RemarkId:    1
//   - Remark:      MMA operation failed to optimize to use Tensor Cores, it is using FMA instructions instead

For this kernel, the Float32 multiply uses FMA instructions. With Float16 inputs and a Float32 accumulator, the compiler instead reports:

// Name:            RemarkTensorCoreMMA
//   - Remark:      MMA operation successfully optimized to use Tensor Cores

Remarks require tileiras 13.4 or newer, which is still in early-access.

Programmatic dependent launch

Programmatic dependent launch can overlap the tail of a producer kernel with an independent preamble in the next kernel on the same stream. The producer signals when its dependents may start; the consumer is launched with dependent=true and waits before reading the producer's results:

function producer(a, producer_out)
    ct.grid_dependency_control_launch_dependents()
    tile = ct.load(a, 1, (32,))               # may overlap with the consumer
    ct.store(producer_out, 1, tile)
    return
endfunction consumer(b, producer_out, out)
    tile = ct.load(b, 1, (32,))               # independent preamble
    ct.grid_dependency_control_wait()
    ct.store(out, 1, tile + ct.load(producer_out, 1, (32,)))
    return
endstream = CUDA.stream()
@cuda backend=cuTile blocks=1 stream producer(a, producer_out)
@cuda backend=cuTile blocks=1 dependent=true stream consumer(b, producer_out, out)

The overlap is opportunistic, so correctness must never depend on the two kernels actually running concurrently. The feature requires Tile IR 13.4 and compute capability 9.0 or newer.

Other changes

• Tile IR 13.4 is supported and emitted by default when tileiras accepts it. It brings ct.insert, the inverse of ct.extract, and check_bounds=false on ct.load/ct.store, an explicit promise that the whole tile is in bounds which drops the padding and selects Tile IR's unchecked encoding.

• Explicit rounding modes on float-to-float conversions: Float32.(tile, RoundDown), with RoundNearest, RoundToZero, RoundDown, RoundUp and RoundNearestTiesAway. Supported modes and source/target pairs depend on the Tile IR version. Directed rounding generally requires 13.4; some conversions to Float8_E8M0FNU are available in 13.3.

• 64-bit indexing. TileArray gained an index-type parameter, and arrays whose sizes or strides exceed the 32-bit range automatically use Int64. Smaller arrays continue to use Int32. ct.TileArray(a; index=Int64) selects wide indexing explicitly. Wide indexing requires Tile IR 13.3.

• Array construction syntax works in kernels: [a, b, c], typed forms like Float32[a, b], bracket concatenation ([A; B], [a b; C], [A;;; B]) and cat(A, B...; dims). ct.cat was removed in favor of these.

• Multi-dimensional reductions. dims now takes an integer, an iterable of integers, or :, for both tile-level and host-level ct.Tiled reductions.

• Memory ordering on plain loads and stores, not just atomics: ct.load and ct.store accept memory_order and memory_scope.

• Configuration moved to preferences. The JULIA_CUTILE_CACHE_DIR and JULIA_CUTILE_CACHE_SIZE environment variables were replaced by the disk_cache, cache_dir, and cache_size_bytes preferences. A new compiler_timeout_seconds preference bounds each tileiras invocation.

cuTile.jl 1.0 requires CUDA.jl 6.3. See NEWS.md for the user-facing release history and the release notes for the merged pull requests. Please file an issue if you run into a problem.