Author Archives: Fabian Becker

Learn With Me: Julia – Bitwise Operators (#4)

By: Fabian Becker

Re-posted from: https://geekmonkey.org/lwm-julia-bitwise-operators/

In part 3, we learned about structs and binary I/O in Julia. We looked into how the PNG format stores metadata and how it is represented using chunks. Each chunk consisted of four components: length, type, data, and a four-byte CRC.

CRC stands for "Cyclic Redundancy Check," and these checks are commonly used in digital networks and storage devices to detect transmission or read errors in data. Sometimes one or more bits can get flipped during data transmission, and CRCs are one of the tools employed to detect this.

CRC is a so-called error detection algorithm. The simplest algorithm in this class is called the parity bit. The idea with the parity bit is that you count all the 1s in a binary string, and if the number of 1s is even, the parity bit is also even. With this technique, you can detect any odd-number bit flips but are basically out of luck when an even number of bits are flipped.

In this article, we'll look at implementing CRC32 using Julia bitwise operators. The Wikipedia page is a great starting point if you want to learn a bit more before jumping into the code.

Bitwise Operators in Julia

Like all other modern programming languages, Julia comes with bitwise operators. They apply to all primitive integer types. I don't think it's necessary for me to cover all of them here, so I'll focus on the ones we need for this post:

AND denoted in Julia by an ampersand & is true only if both operands are true.

a  b  | a & b
0  0  | 0
0  1  | 0
1  0  | 0
1  1  | 1

Bitshift is an operation that moves a sequence of bits either left or right. In Julia, the operator for this is >> or << depending on whether you want to shift the bits right or left. Shifting left by n bits has the effect of multiplying a binary number by \(2^n\). Likewise shifting right by n bits will do the inverse and divide by \(2^n\). When used, it's important to know that the first operand is always the bits operated on, and the second denotes the number of bits to shift. 2 << 5 means "shift the binary representation of 2 (10) left by 5 bits (100000).

XOR, also called exclusive or is an operator that's true if and only if the operands are different. Most languages use the ^ (caret) to symbolise XOR, Julia uses (which can be entered using LaTeX notation \xor) but also offers the xor function.

a  b  | a ⊻ b
-----------------
0  0  | 0
0  1  | 1
1  0  | 1
1  1  | 1

With the operators covered, let's now looks at implementing the CRC verification algorithm for our PNG chunks.

Bitwise CRC32

The simplest implementation of the CRC algorithm is operating bitwise. CRC is essentially an implementation of polynomial division where our data is the dividend, and the result of the division is the CRC for our data. Don't worry, we don't have to get super mathematical here – in binary polynomial division can easily be done using XOR.

For CRC32 the generator polynomial, or divisor, is $$x^{32} + x^{26} + x^{23} + x^{22} + x^{16} + x^{12} + x^{11} + x^{10} + x^{8} + x^{7} + x^{5} + x^{4} + x^{2} + x + 1$$ which can also be written in hex as 0x04C11DB7. In practice this is often reversed since it makes the algorithm easier to implement in software. The resulting constant is 0xEDB88320.

unsigned int crc32(unsigned char *message) {
    int i, j;
    unsigned int byte, crc, mask;
    
    i = 0;
    crc = 0xFFFFFFFF;
    while (message[i] != 0) {
    	byte = message[i];          // Get next byte.
        crc = crc ^ byte;
        for (j = 7; j >= 0; j--) {  // Do eight times.
            mask = -(crc & 1);
            crc = (crc >> 1) ^ (0xEDB88320 & mask);
        }
        i = i + 1;
    }
    return ~crc;
}
Basic bitwise CRC32 algorithm, taken from Hacker's Delight

The above implementation appears in one of my favorite algorithm books written by Henry S. Warren Jr., called Hacker's Delight. It features many other algorithms and techniques, and its most recent edition has a whole section on CRC32.

We can start by converting the C implementation from Hacker's Delight to Julia. Julia strings contain variable-length characters, so it is necessary to ensure they are converted to UInt8 first, since the algorithm can only operate on individual bytes.

function crc32(message::Vector{UInt8})
    crc = 0xFFFFFFFF
    for byte in message
        crc = crc ⊻ byte
        for _ in 1:8
            mask = -UInt32(crc & 1)
            crc = (crc >> 1) ⊻ (0xEDB88320 & mask)
        end
    end
    ~crc
end

We can trivially test this by picking any chunk from a PNG file and passing the type and data fields in as the message. The easiest one I found was the IEND chunk since it doesn't even have any data. This will always have the same CRC of 0xAE426082. Running the following snippet, we can see our code working:

message = "IEND"
crc = crc32(Vector{UInt8}(message))  # Returns a UInt32
print(string(crc, base=16)) # Prints: ae426082

Done! Or are we…?

Bytewise CRC32

For PNG, there's a reference implementation available in C that operates byte-wise instead of bitwise, allowing for a drastic speedup in CRC computation. We can adapt it to work in Julia as well.

The reference implementation uses a pre-built table to speed up performance. This table stores all possible combinations for a single byte resulting in a table of 256 elements. There are other implementations out there that extend this to 2 bytes or 65,536 elements.

function maketable()
    crc_table = Vector{UInt32}(undef, 256)

    for n in 0:255
        c = convert(UInt32, n)
        for _ in 1:8
           if c & 1 == 1
               c = 0xEDB88320 ⊻ (c >> 1)
           else
               c = c >> 1
           end
        end
        crc_table[n+1] = c
    end
    crc_table
end

Note that since Julia arrays are 1-indexed, we have to adapt the algorithm to take care of this.

Since the table is reusable (as long as we use the same generator polynomial), it's advisable to cache it somehow. One way of doing this is using a const variable in the global scope:

const crc_table = maketable()

The main CRC32 algorithm can now use this table to conveniently iterate over the input data byte-by-byte and save many compute cycles. Note that we had to + 1 the lookup here (as compared with the C version) to accommodate Julia's 1-indexed arrays.

function crc32(data::Vector{UInt8}, crc::UInt32)
    c = crc
    for byte in data 
        c = crc_table[((c ⊻ UInt8(byte)) & 0xff) + 1] ⊻ (c >> 8)
    end
    return c
end

With some simple multi-dispatch, we can even add a version that takes Julia strings:

crc32(data::AbstractString, crc::UInt32) = crc32(Vector{UInt8}(data), crc)

Finally, we can now apply this to our PNG chunk. The PNG spec mentions that the CRC is computed over the bytes from the type and data field and not the length field. Since we're still dealing with Vector{UInt8} we can use vcat to concatenate those two fields.

Additionally, the PNG spec demands that we initialize the CRC with 0xFFFFFFFF and XOR the result again afterward. We can implement a crc32 function for our PNGChunk type as follows:

crc32(c::PNGChunk) = crc32(vcat(c.type, c.data), 0xFFFFFFFF) ⊻ 0xFFFFFFFF

We can now validate the crc32 for each chunk against the CRC we read from the file. First, we need to fix the byte order and ensure we're comparing values of the same type. The CRC in our PNGChunk is represented as a Vector{UInt8} with the least significant byte coming first. We can reverse the order using reverse and turn the vector into a single UInt32 by using reinterpret again.

isvalid(c::PNGChunk) = crc32(c) == reinterpret(UInt32, reverse(c.crc))[1]

I've modified our Base.show function from last time to output the data a bit more clearly, and it now looks like this:

function Base.show(io::IO, c::PNGChunk)
    println(io, "Length: ", length(c))
    println(io, "Type: ",  type(c))
    println(io, "Data: ", datastr(c))
    println(io, "CRC: ", crc32(c), "\t", isvalid(c) ? "OK" : "INVALID")
    println(io, "-----")
end

That's it – we have successfully implemented a basic CRC32 algorithm in Julia.

Julia comes with a core package called CRC32c using the same algorithm but uses a different generator polynomial, so we, unfortunately, can't use it. There is, however an excellent third-party library called CRC.jl that you might want to check out.


If you like articles like this one, please consider subscribing to my free newsletter, where at least once a week, I send out my latest work covering Julia, Python, Machine Learning, and other tech.

You can also follow me on Twitter.

Learn With Me: Julia – Structs and Binary I/O (#3)

By: Fabian Becker

Re-posted from: https://geekmonkey.org/learn-with-me-julia-structs-and-binary-i-o-3/

Learn With Me: Julia - Structs and Binary I/O (#3)

Diagrams.net (formerly draw.io) is a fantastic website and tool that allows you do create rich diagrams. The service is entirely free and diagrams can be saved to you Google Drive, Dropbox or downloaded to your computer. Additionally diagrams.net allows you to export your diagrams to various formats such as SVG, JPG and PNG.

Recently it was pointed out to me that you can actually load an exported PNG diagram back into the tool and edit it again. This got me thinking – how are they doing this? Surely they aren't using image recognition techniques to identify objects in the image?

You may wonder: What does all of this have to do with the title of this post? Let's talk about the PNG format a little.

PNG File Format

PNG stands for Portable Network Graphics. The format has existed in some form since the mid-nineties. Like all binary file formats it follows a specification. From the specification we can learn a lot about how the image and its metadata is represented on disk.

File Header

A PNG file starts with an 8-byte signature. This signature tells a decoder that all bytes that follow are to be interpreted based on the PNG spec. In hexadecimal representation the header is: 89 50 4e 47 0d 0a 1a 0a

Chunks

The remainder of the PNG format follows a very simple structure. Data is represented in chunks. Each chunk starts with 4 bytes describing the length of chunk data. Then follow 4 bytes for the chunk type. This again is followed by length bytes of chunk data and finally 4 more bytes for a CRC (cyclic-redundancy check). The CRC can be computed over the chunk type and chunk data.

Length Chunk type Chunk data CRC
4 bytes 4 bytes Length bytes 4 bytes

The file specification mentions that while the length is represented using 4 bytes or 32bits the maximum length of chunk data is actually 2^31.

The chunk types are more interesting as there are plenty of them. I won't go into much detail here and instead only cover the relevant bits for this post. I encourage you to go read the specification for yourself to understand the nifty encoding techniques used here.

Since the chunk type is represented by 4 bytes, they can (mostly) be represented using 4 ASCII characters. Chunk types are split into critical and ancillary chunks – a decoder must understand all critical chunks but can safely ignore the ancillary chunks.

The critical chunks are as follows:

  • IHDR must be the first chunk in the file. It contains in specific order the image width, height, bit depth, color type, compression method, filter method and interlace method.
  • PLTE contains information about the color palette used
  • IDAT contains the actual image. There can be multiple IDAT chunks which is what allows PNG to be a streamable format in which the first smaller IDAT chunk allows a pre-render of the full image before all data is received.
  • IEND marks the end of the file

A selection of ancillary chunks:

  • tIME stores the time the image was last changed
  • tEXt stores key-value metadata. The text is encoded in ISO 8859-1. The key must be between 1 and 79 characters long and is terminated by a null character. The remainder of chunk data is the value.

Exporting a diagram from diagrams.net

Before we get started with writing some Julia code, let's first export a PNG file from diagrams.net.

This is fairly straight forward, just head over to diagrams.net, click together a diagram and hit File > Export and choose PNG. Make sure to keep the "Include a copy of my diagram" checkbox checked.

Learn With Me: Julia - Structs and Binary I/O (#3)
Sample diagram

File IO in Julia

With everything prepared we can start looking into I/O. We're not going to do anything advanced here so we'll just look at the basics.

Interacting with files, regardless of the language, always follows the same pattern:

  • Open a file for reading/writing
  • Read/write
  • Close the file descriptor

Julia is no exception to this. We can use Base.open to open a file. This will give us an IOStream instance which in turn wraps the OS file descriptor. We can either do it in a block, in which case the file will be closed automatically at the end of the block, or we call open/close separately.

open("myfile.txt", "r") do io
   # ...
end;

Furthermore there are multiple ways to read data from a file.

We'll need read and readbytes!. They both take an IOStream (the result of the open call) as the first argument. read takes a primitive type as a second argument telling it to read a single value of that type from the IO and return it. I.e. read(io, UInt32) will read the 4 bytes it takes to represent a UInt32.

readbytes! requires a vector like object to be passed as its second argument. It will read as many bytes as the vector can hold as long as there's data to read.

Reading in the PNG file

Let's put what we've just learned together. Here's the plan:

  • Open the PNG file
  • Check for the file header (remember those 8 bytes mentioned above?)
  • Read in PNG chunks by first consuming the length, the type, the data based on the length field and finally the CRC.

We can represent PNG chunks using a struct with named fields for each of the elements. The easiest way to represent a sequence of bytes is using a Vector{UInt8}. Here's the struct I came up with:

struct PNGChunk
    length::UInt32
    type::Vector{UInt8}
    data::Vector{UInt8}
    crc::Vector{UInt8}
end

It's also useful to declare a constant for holding the PNG header:

const PNG_HEADER = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]

Let's now open the PNG file and read in the first 8 bytes for the header:

io = open("Diagram.png", "r")
header = Vector{UInt8}(undef, 8)
readbytes!(io, header)

readbytes! takes an IOStream handle and a variable that it will try to fill. You can pass it an additional integer to indicate the number of bytes to read but it defaults to the length of the second argument which we've declared as a vector of UInt8s with 8 elements.

By simply comparing header with PNG_HEADER we can determine whether we're dealing with a valid PNG file:

if header ≠ PNG_HEADER
    throw(ArgumentError("File is not a PNG"))
end

Assuming our file is valid we can now attempt to read in all the chunks in the file. It's easiest to do this iteratively with a loop and consume the file until we hit EOF. Luckily Julia provides an eof function that takes an IOStream and returns whether or not we've reached the end of the file.

while !eof(io)
    length = hton(read(io, UInt32))

    type = Vector{UInt8}(undef, 4)
    readbytes!(io, type)

    data = Vector{UInt8}(undef, length)
    readbytes!(io, data)

    crc = Vector{UInt8}(undef, 4)
    readbytes!(io, crc)

    push!(chunks, PNGChunk(length, type, data, crc))
end

I'm calling hton to get the length represented properly. This is because my system (Intel-based MacBook Pro) is a little endian system (meaning the least significant byte comes first) but PNG represents all data in big endian requiring us to reorder bytes.

The loop will continue to consume bytes for the chunk type, data and the CRC and construct a PNGChunk that will then be pushed into a vector.

Note: The above code will work for a valid PNG file. There's no error checking at all so if one of the fields is corrupted or the file ends prematurely this will throw an error and fail.

Displaying chunks

Now that we're done reading the file we should take a look at its contents. For this we can add a bunch of helper functions.

We essentially want to run something like:

for chunk in chunks
    print(chunk)
end

but executing this will result in a lot of gibberish being displayed. To tell Julia how to display a PNGChunk we need to implement Base.show for our type. Base.show takes an IO object and an instance of a type. You can compare this with __repr__ in Python. An implementation that will display the length and the type of a chunk might look as follows:

function Base.show(io::IO, c::PNGChunk)
    println(io, length(c), "\t", type(c))
end

Where in other languages you declare methods on classes, in Julia you simply declare a function that operates on a type. To make the implementation of Base.show work we need to define length and type:

length(c::PNGChunk) = c.length
type(c::PNGChunk) = String(Char.(c.type))

While we could simply access chunk.length directly it's common practice to consider struct fields "private" and write functions to access them. This way you get a layer of abstraction and can easily change the layout of structs without breaking code all over the place.

To deconstruct what's going on in the second line let's start by looking at c.type. We declared the type to be a Vector{UInt8} and we consumed 4 bytes while reading the PNG file. The first thing we want to do is convert each item in the vector to its ASCII character representation. Julia provides the Char data type to represent 32-bit characters. Simply calling Char(c.type) would result in Julia attempting to consume all 4 bytes (32 bit) and won't give us the desired result.

Instead we can iterate over the items in the vector and convert each item to a Char. This could be written using a list comprehension like [Char(ch) for ch in c.type] which is rather lengthy but standard if you're coming from Python. Julia conveniently offers the dot-operator (also called broadcast) which can be applied to any function. By writing Char.(c.type) we're essentially expressing "apply each element in c.type to the Char function".

Finally we wanted to obtain the string representation of those characters and by passing a Vector{Char} to the String function we can cast it into a string.

More tenured Julia developers would probably write all of the above simply as collect(Char, c.type) |> join, but we're going to ignore this for now.

Ok, back to displaying the chunk. With Base.show and our two functions out of the way we can loop over the chunks and see what's inside our file:

13      IHDR
970     tEXt
3379    IDAT
0       IEND

So that's cool – we've got three chunks with data. IHDR contains height, width, color depth and some other metadata about the file and IDAT contains the actual image. This leaves tEXt which could contain anything.

Extracting information from IHDR

Let's see if we can make sense of the data in the IHDR chunk. First we're going to modify our Base.show implementation to also display the data field when we recognise the chunk type.

function Base.show(io::IO, c::PNGChunk)
    println(io, length(c), "\t", type(c) ,"\t", datastr(c))
end

The specification tells us that there are 13 bytes reserved for the IHDR data field and how many bytes are reserved for different properties.

The IHDR chunk must appear FIRST. It contains:

   Width:              4 bytes
   Height:             4 bytes
   Bit depth:          1 byte
   Color type:         1 byte
   Compression method: 1 byte
   Filter method:      1 byte
   Interlace method:   1 byte

The multi-byte fields will require endian conversion. Since we have already read in all data we need to reinterpret the data from our Vector{UInt8}. That's exactly the name of a Julia function that helps with reinterpreting data into another type:

hton(reinterpret(UInt32, c.data[1:4])[1])

This will take the first four bytes of chunk data and reinterpret them into a UInt32. The wrapping hton will make sure to convert from host byte order to big endian. We can repeat this for the height field and then read all the individual bytes.

function datastr(c::PNGChunk)
    if type(c) == "IHDR"
        height = hton(reinterpret(UInt32, c.data[1:4])[1])
        width = hton(reinterpret(UInt32, c.data[5:8])[1])
        depth, ct, cm, fm, im = c.data[9:13]
        return "h=$height, w=$width, d=$depth, color type=$ct, compression method=$cm, filter method=$fm, interlace method=$im"
    end
    ""
end

For my diagram I get the following output:

h=121, w=201, d=8, color type=6, compression method=0, filter method=0, interlace method=0

Obtaining the original diagram from tEXt

Finally, let's peek inside the tEXt chunk. We can first extend our datastr(c::PNGChunk) function to also have a branch to catch the tEXt type and simply print the contents of the data field:

 mxfile %3Cmxfile%20host%3D%22app.diagrams.net%22%20modified%3D%222021-05-24T09%3A22%3A42.489Z%22%20agent%3D%225.0%20
(Macintosh%3B%20Intel%20Mac%20OS%20X%2010_15_7)....

That's a bunch of gibberish. Consulting the specification tells us that the data field for tEXt consists of a key and value pair separated by a null-byte. That should be easy to parse:

key, value = split(String(Char.(c.data)), '\0')

But that's only half the equation. It looks like the value part may be URL encoded and so we need to find a way to decode it. I couldn't find this functionality in the standard library and so I ended up installing URLParser.jl which implements unescape.

(@v1.6) pkg> add URLParser

Putting everything together we can complete our datastr function by adding tEXt handling:

    elseif type(c) == "tEXt"
        key, value = split(String(Char.(c.data)), '\0')
        value = unescape(value)
        return "$key, $value"
    end

And so the final output is:

13      IHDR    h=121, w=201, d=8, color type=6, compression method=0, filter method=0, interlace method=0
970     tEXt    mxfile, <mxfile host="app.diagrams.net" modified="2021-05-24T09:22:42.489Z" agent="5.0 (Maci
ntosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" et
ag="MSmUq0enpJxQ3pDGyP_L" version="14.3.0"><diagram id="py8BCTe_me7SkJGnhe6H" name="Page-1">zZRNb4MwDEB/DcdJ
EDbaHruOdYdNm9TDdo2IC5kCRsF89dcvjFCKWKvtUGmXijw7dfxwcPxN2mw1z5MXFKAc5orG8R8cxpZLZn470FqwcHsQayl65I1gJw9g4ZBW
SgHFJJEQFcl8CiPMMohowrjWWE/T9qimVXMewwzsIq7m9F0KSiz1gtUYeAIZJ7b0ki36QMqHZNtJkXCB9QnyQ8ffaETqn9JmA6pzN3jp9z2e
iR4PpiGj32wItquqWN+tgjC+3fPXj0Ks4cbv/6XiqrQN28NSOxgAYYTYJWpKMMaMq3Ck9xrLTEBXxjWrMecZMTfQM/ATiFr7dnlJaFBCqbLR
vmZX6GxvFhVY6gguNDTMCNcx0IU8dnwDZnIBUyDdmn0aFCdZTc/B7QzFx7xRs3mwpv9g3ZtZX6f8ILN4Jn9U23mqE0mwy/m3gdrct580VqAJ
mssi543bDcy102qvqzdc1/pk+IeJTk7mPnCv5IrNXL0ppOLfmfK965kyy/E78R07+dj64Rc=</diagram></mxfile>
3379    IDAT
0       IEND

The secret to how diagrams.net embeds the diagram is solved. It's urlencoded XML embedded into a tEXt chunk inside the PNG file (now that's a fun sentence to say!).

The full code can be found at https://github.com/halfdan/geekmonkey/tree/main/julia/lwm-03

Summary

In this article we've covered a lot of different concepts in Julia. If you struggled to keep up – don't worry I'll go over all the concepts mentioned here in more detail in future posts. My approach to learning is often guided by the projects I want to do and so I often jump in at the deep end. As a result this article introduced concepts rather rapidly without spending too much time on the mechanics.

It's always fascinating when you think about how many things we take for granted in tech without thinking about the underlying mechanics. I was definitely surprised how easy it was to extract some metadata from a binary format like PNG. I've used PNG files for decades without ever thinking about their inner structure. Clearly we've only scratched the surface and haven't looked at the IDAT chunk containing all the image information, but we'll get there in time.


If you like articles like this one, please consider subscribing to my free newsletter where at least once a week I send out my latest work covering Julia, Python, Machine Learning and other tech.

You can also follow me on Twitter.

Learn With Me: Julia – Tools and Learning Resources (#2)

By: Fabian Becker

Re-posted from: https://geekmonkey.org/lwm-julia-2-tools-and-learning-resources/

Before we really dive into Julia I wanted to go over the tools and learning resources I have and will be using going forward. These resources fit my learning journey and may not directly apply to you so I encourage you to spend some time to see what's out there.

Learning Resources

I have compiled a list of learning resources that I intend to explore over the next 6 months as I progress in the Julia language. Any learning journey is prone to fail if you don't work towards an end goal. As stated in my previous post I want to use Julia to get into data science and machine learning and so it should come as no surprise that my selection of learning resources is biased toward these topics.

It was difficult to settle on a good book since the majority of books listed on the Julia website seem to be targeted towards programming beginners and seem to be lacking the depth I am looking for. Some other books have terrible ratings and so I excluded them. I ended up including only one book in the list below.

Basic

The following resources should help get a basic understanding of programming in Julia and provide a good set of problems and challenges to practice learned skills.

  • The Julia track on exercism.io where you can get your coding solutions mentored by volunteers. This provides plenty of exercises to apply some of the basics and get high quality feedback.
  • The Julia Language – A concise tutorial gitbook, as the name suggests, gives a concise overview of several important topics such as I/O, data structures, meta programming, package development and concurrency.
  • Julia Academy provides a number of interesting courses.
  • Finally, the official Julia documentation will always be my first go-to resource should I get stuck in a problem. As a shorthand for browsing the documentation I'll resort to using Julia's built-in help mode in the REPL.

Intermediate/Advanced

Eventually I'd like to dive a lot deeper into Julia performance and how to squeeze the best performance out of my code.

  • The MIT course Introduction to Computational Thinking is an incredibly well received course that uses Julia and Pluto.jl. The course teaches image analysis, machine learning, climate modelling and dynamics on networks.
  • The only book that stood out to me is Julia High Performance – Second Edition by Avik Sengupta, Alan Edelman. Just judging from its cover it seems to go into a lot more depth than most books about Julia out there.

Tools

When it comes to tools, there really isn't that much needed for Julia other than a decent computer that runs any of the three main OSes. Most of my development these days happens on Mac OS, but I'm also exploring developing on a more powerful Windows machine with WSL2.

For now I'm going to be using:

  • Visual Studio Code with the Julia for VSCode extension. It's definitely not a requirement to use this particular editor or even the extension but it's an editor I'm already very comfortable with.
  • Pluto.jl for interactive notebooks and quick data exploration

That's really all that's needed to get started. I'm sure my development workflow and tools will evolve over time but for now

If you are interested in seeing a different setup using neovim and tmux I suggest you check out Jacob Zelko's excellent post on his writing and coding workflow.

Project-driven Learning

My method of learning a new (programming) language has evolved over time. Immediately applying what you learn is essential for retention. As I read through my resources and explore Julia packages I will develop small project ideas. These projects aren't necessarily meant to evolve into production grade software, but rather produce something interesting or help me explore an area of technology.

Not all of my time will be spent on larger projects however. There's value in exploring small coding challenges from exercism.io, projecteuler and others when you're short on time or can't make any progress in your project.

As I dive deeper into Julia and get more comfortable navigating the package ecosystem I expect the focus of my projects to also shift and my projects to grow in size.

I intend to document most of these projects and invite you to follow along as I learn this exciting language and discover its secrets.


If you like articles like this one, please consider subscribing to my free newsletter where at least once a week I send out my latest work covering Julia, Python, Machine Learning and other tech.

You can also follow me on Twitter.