Keno Fischer, Julia Computing Co-Founder and Chief Technology Officer (Tools) participated in a Quora Session March 18-23. One of Keno’s responses was also featured in Forbes.

How do programming languages get created, and what goes into design decisions?

The first thing to think about in answering this question is: What is a programming language? If you ask Wikipedia that question, you will find that a programming language “is a formal language, which comprises a set of instructions that produce various kinds of output” which is of course true, but in true encyclopedia form also mostly unhelpful. It does give the right idea though. Just write down some instructions and some rules for what they do and viola you’ve created a programming language. If you write down these rules using slightly fancy language, you would call that the specification of your language and have a very good claim to have created a programming language.

Of course, in most instances, programming languages don’t start as exercises in specification writing. Instead, one starts with a program that actually does something with the programming language. Generally, this will either be a program that reads in some code written in the programming language and just does what the code says to do as it goes along (an “interpreter” – think following a recipe step by step) or one that translates the source code to the sequence of bits that the actual hardware understands (though this string of ones and zeros could also be considered a programming language that the hardware then interprets). There are a couple more exotic kinds of programs one could write to implement a programming language (e.g. type checkers, that just check that the source code is well-formed, i.e. allowed by the rules of the language, but don’t otherwise execute it) and various variations on compilers and interpreters (hybrid systems, compilers to “virtual hardware”, i.e. low level languages that are designed to be easy to map to actual hardware, compilers from one high level programming language to another, aka “transpilers”), but the key thing is that these programs “understand” the language in some way. The specification usually comes later, if ever.

Now, assuming you’ve started your own programming language, how does one decide what the language should be – what the available instructions are, what the rules and grammar of the language are, what the semantics of various things are, etc. There are a lot of things to consider when making these decisions: How does it work with the rest of the system? Is it self-consistent? Does it make sense to the user? Will the users be able to guess what’s going, just by looking at the code? Are we able to efficiently have the hardware do what the language says it should do? Is there precedent somewhere, e.g. in mathematics or in other programming languages that set users’ expectations for how thing should work? If so and we are deviating from that expectations, are there good reasons to [1]? If we are doing something different or unexpected, should we provide both or should we at least add something to make sure that users expecting the legacy behavior will easily find out what the legacy behavior is, etc? At the end, in every decision you make, you need to consider two things 1) The computer that has to run it and 2) The human that has to read it.

Both are extremely important, but there is of course a trade-off between them and languages differ where they fall on this spectrum. In Julia, we try very hard to make a program well understood by both (this was actually one of the original motivations for Julia). This isn’t easy and there are hard trade-offs to be made sometimes (e.g. it’d be nice to check overflow for all arithmetic operations, but doing this by default is too slow on current generation machines), but we try to make sure that a) We make reasonable choices by default and b) whenever we make a trade off in either directions there is ways to let the users make the opposite choice while being able to use the rest of the system without trouble. Julia’s multiple dispatch system is essential to making this work (though the details of that are a whole separate topic).

[1] E.g. we have a policy of generally spelling out names rather than using short abbreviations, so you might consider “sine” and “cosine” as more consistent names than “sin” and “cos”, but you’d be fighting against 100 years of mathematical notation. As an example on the other side, a lot of languages like to use “+” to concatenate strings. However, we considered that a serious mistake, since + is facially commutative and string concatenation is not, which is why we use “*” as our string concatenation operator.

If you were to design a new general-purpose programming language in the spirit of Julia today, what would you do differently?

Well, this one’s easy. The nice thing about having your own language is that anything you think is bad about it, you can just change without having to go through the effort of designing a whole new language from scratch. A more interesting question is what kind of language I would design if I were to do something completely different from what Julia does. Julia’s design optimizes for usability and performance. I think it would be fun to work on a language that aggressively optimizes for correctness. This probably means a language with built in proof assistant and formal verification capabilities (or something that statically enforces coverage of all corner cases). I haven’t thought too deeply about it, but I have used various proof assistants before and it’s always felt like writing assembly to me in its level of tedious. Perhaps that’s inherent to the domain, but I’d be interested in exploring what clean-room proof assistant with focus on usability would look like. Then again, there’s nothing really preventing us from bolting that kind of technology onto Julia to supplement or replace test coverage for particularly critical code. Perhaps language designers are just doomed to just keep working on the same language until legacy considerations make it impossible to change anything.

Keno Fischer, Julia Computing Co-Founder and Chief Technology Officer (Tools) participated in a Quora Session March 18-23. One of Keno’s responses was also featured in Forbes. How…

Keno Fischer, Julia Computing Co-Founder and Chief Technology Officer (Tools) participated in a Quora Session March 18-23. One of Keno’s responses was also featured in Forbes.

Why do you support open source software and open collaboration?

I consider open source software a public and social good, similar to scientific research papers and I think access to both should be available as widely as possible to spur innovation. However, I also think there’s lot of practical reasons to prefer open source, even if you couldn’t care less about the social good argument:

• Complexity. Modern software stacks are unfathomably complex and it is fairly common to have to hunt bugs through four or five layers of abstraction. If each of these layers is a separate, closed source project, you’ll spend years of your life exchanging emails with your vendor’s support engineers trying to get things debugged. I think from a practical perspective, this is probably among the top reason I refuse to work with closed source software (except in very, very, rare circumstances) that hasn’t been reverse engineered. The lost productivity and frustration is just not worth it.

• Collective Action. Say I sell you some hardware widget. It needs some software to make it work, but I have barely any budget left to get it working at all, let alone get all the niceties in there like “security”, “customizability” or even the ability to do maintenance on the software (it’s usually just a source tree on some developer’s computer, one folder per device version that got shipped – with things like CI, version control, static analysis being but a pipe dream). This is essentially the situation we have with a lot of embedded devices at the moment. One of the scariest instances of these are baseboard management controllers (BMCs). If you’re not familiar with them, small chips (usually ARM cores these days) that come with enterprise motherboards and are basically the most privileged chip in the system (more privileged than the host CPU even). Depending on the setup, they can generally read main memory, access the PCIe bus, read network traffic, flash your BIOS or just take over the machine entirely. And yet, the software that comes on these things tends to be decade old versions of Linux, hacked together with SDKs for proprietary drivers that are full of problems, Web servers that are full of security holes (did I mention that by default if you plug a network cable into the primary ethernet port the BMC will become available on the network with a default password?), and software that is so buggy you’re going to waste hours just trying to use the thing to reinstall the operating system on your server, because it keeps losing the remote mount. This is the kind of thing that keeps me up at night. It’s understandable how this kind of thing happens though. The motherboard vendors don’t have a lot of incentives to make this stuff actually work super well. They’ve sold you the board and they’ve ticket out the BMC box in the marketing materials and some of the features even kind of work. If you’re lucky you may even get a security fix every other year when somebody rediscovers the existence of these chips. Open source can help fix this problem by providing a venue for people to come together and solve this as a collective problem. People who buy these boards do care about security and usability and fixing bugs, but very few organizations have enough servers that it would make sense for them to build their entire own BMC stack (you’d have to be at the scale of Facebook or Google). But if it’s open source, you can easily imagine somebody submitting a patch for some corner case bug they have, or an organization that still has a few thousand previous generation servers lying around taking on the (relatively small) incremental maintenance burden of putting out images with updated security fixes for those machines. It’s also good for vendors, because they can take advantage of shared infrastructure. Instead of hacking together something for every new device iteration, they can just submit the new device model upstream and use the standard process to get binaries out. No more bespoke, per-chip directories. In the BMC world, this has been finally happening in the last few years with the OpenBMC project (penbmc/openbmc), but this kind of thing is a pattern that happens over and over. Open source is a way to coordinate collective action problems.

• Security. I touched upon it a little bit in the previous answer, but let me just re-iterate the point. If the code is open, anybody can look at it and if they spot a security bug, get in a patch and quickly get it fixed. Now, this isn’t perfect. Spotting security bugs is hard and not a lot of people are good at it, but at least it is possible. There is something that’s been happening in the last few years however, that strengthens this argument quite a bit. We’re seeing more and more powerful tools for automatic detection of security and other problems (fuzzing, static analysis, etc). If the source code is available, it is quite easy for third parties (e.g. academics working on new such techniques) to rapidly apply these techniques to a large corpus of important real world code.

• Community. Open source communities are a place for people with overlapping problems to meet that wouldn’t otherwise talk to each other. We’ve seen this happen a lot in the Julia community. Since everybody is talking in terms of the same concepts (Julia code and abstractions), we get very high quality discussions between people from very different fields (e.g. scientific application developers and computer scientists working close to the hardware). Magic can happen when you get these different perspectives into the same room together and it’s not an infrequent occurrence in the Julia community that entire new research directions get discovered by people chatting in the issue tracker or the discussion forum.

• Education. Reading the source code of important open source projects (compilers, operating systems, Web browsers) can be an extremely enlightening experience to understand how things really work under the hood. These code bases are testaments of the complexity of the real world and include all the details that are often skipped over in textbooks because they would be distracting from the main point (but turn out to be fairly important in practice). In many code bases (or in the commit log) you’ll often also be able to find out why something was done the way it is, which can give a lot of insight that is hard to convey any other way. Now, why should companies care that about this? Well, imagine having to hire somebody to work on software you use a lot. If the software is open source, chances are you’ll be able to find people that have at least read parts of the code base and will be able to hit the ground running immediately (disclaimer: works better for clean and well documented code).

Overall, I don’t think any of this is too controversial. The tech industry is basically built on top of open source. It is impossible to get anything meaningful done without touching open source software. There are problems that go along with it. It can be hard to get support sometimes for open source software, there isn’t always a clear vision for where the project should go, different people involved may have different goals and often the maintenance burden is not adequately carried by those that benefit most. I think there is some room for innovation here, but closed source is not the answer.