Bend - A Rust-Backed Language That Supercharges Parallel Computing Like Never Before
A programming language that feels like Python and scales like CUDA, without needing any effort to make your code parallel, is here.
Have you heard about Bend?
No?
You aren’t too late if you’re reading this story.
It was a few hours ago when I came across its GitHub repository titled ‘Bend — A massively parallel, high-level programming language’.
Bend is a high-level programming language that aims to alter how we perform parallel computation forever.
I know that there have been so many languages that have promised to change the world, but hold on because Bend might actually do so!
Before we move forward to learn about the good parts that Bend promises, let’s learn a bit about Parallel computation.
Parallel Computation— The Art Of Doing The Most In The Least Time
Computer programs by default run their operations one after the other.
This is because they rely on a single Thread, which is executed by a processor core one instruction at a time.
This straightforward approach is referred to as Sequential Computation.
This makes instruction flow in a program predictable and easy to debug.
However, this also means that every instruction has to wait for the previous one to end before it can start executing.
In this era of modern computing, where most processors consist of multiple cores, things can be made exponentially faster using another approach called Parallel Computation.
This is when many instructions in a program can be carried out at the same time using multiple threads, to reduce the program’s overall execution time.
But Parallel Computation Is Tough To Get Right
If you’re someone who has worked with Parallel computation before, you will agree that it’s a nightmare to get it right.
Multiple threads accessing shared resources at the same time might lead to Race conditions and completely unexpected outputs.
Or, if you’re having a bad day, you might be stuck with Deadlocks where two or more threads are waiting for resources captured by each other, indefinitely.
Issues like these have been solved by Python libraries such as threading and multiprocessing or by Goroutines along with synchronization primitives such as WaitGroup, Mutex, and RWMutex in Go, but their implementations are tough to master.
CUDA and Metal are some other programming models that enable parallel computation on NVIDIA and Apple GPUs, respectively. But, most working with them will tell you (if they are honest) how much of a hard work it is, to ace them.
The question remains —
Is there no other good alternative to get parallel computation right for the newbies?
Bend To The Rescue
Bend a language written to feel just as easy as Python, might be the one that could rescue us all from this misery.
Its philosophy is hilariously simple —
“Any work that can be done in parallel, will be done in parallel.”
No need to spend years learning CUDA to work with multi-core CPUs or GPUs, because Bend runs code in parallel, by default.
But How Does Bend Make It Possible?
Bend is powered by HVM2 or Higher-order Virtual Machine 2, a powerful computational framework designed for massively parallel execution.
When one compiles their programs written in high-level languages like Python to HVM2, they can simply run them directly on GPUs at blazingly fast speeds!
HVM2 is based on Interaction Combinators, a model for parallel computation devised by Yves Lafont in 1997, that uses graph-based structures.
In this model, each computation is visualised as a graph.
In these graphs, each node is a Combinator.
Combinators are the high-order functions that act as basic building blocks of this model.
Each one of them has a simple set of rules that decide how they interact with other combinators. For example —
Identity Combinator returns its argument unchanged
Constant Combinator takes two arguments and returns the first (ignoring the second one), and so on.
Each edge in the graph represents the connection between these combinators.
When these combinators interact (as the edges suggest), they transform according to their rules and return a result.
The best part of the Interaction Combinators model is that their graph structure allows different computations in a program to be processed by different cores (i.e. in parallel).
And, this is what makes the model so powerful!
HVM2 is a low-level compilation target for programs written in high-level languages and is not meant for direct human usage.
It reads something like this if one wants to implement a recursive sum.
@main = a
& @sum ~ (28 (0 a))
@sum = (?(((a a) @sum__C0) b) b)
@sum__C0 = ({c a} ({$([*2] $([+1] d)) $([*2] $([+0] b))} f))
&! @sum ~ (a (b $(:[+] $(e f))))
&! @sum ~ (c (d e))But worry not because Bend is a human-readable language that is written to interface with this cryptic HVM2 to make our lives easy (and our tasks parallel).
How To Use Bend
Bend is written using the Rust programming language, so the first step is to install Rust nightly, and then, install both HVM2 and Bend with it.
cargo +nightly install hvm
cargo +nightly install bend-langUnfortunately, this won’t work on Windows for the moment, so one (like me) has to use WSL2 as a workaround.
Next, we write our Bend code.
# sorter.bend
# Sorting Network = just rotate trees!
def sort(d, s, tree):
switch d:
case 0:
return tree
case _:
(x,y) = tree
lft = sort(d-1, 0, x)
rgt = sort(d-1, 1, y)
return rots(d, s, lft, rgt)
# Rotates sub-trees (Blue/Green Box)
def rots(d, s, tree):
switch d:
case 0:
return tree
case _:
(x,y) = tree
return down(d, s, warp(d-1, s, x, y))
# Swaps distant values (Red Box)
def warp(d, s, a, b):
switch d:
case 0:
return swap(s + (a > b), a, b)
case _:
(a.a, a.b) = a
(b.a, b.b) = b
(A.a, A.b) = warp(d-1, s, a.a, b.a)
(B.a, B.b) = warp(d-1, s, a.b, b.b)
return ((A.a,B.a),(A.b,B.b))
# Propagates downwards
def down(d,s,t):
switch d:
case 0:
return t
case _:
(t.a, t.b) = t
return (rots(d-1, s, t.a), rots(d-1, s, t.b))
# Swaps a single pair
def swap(s, a, b):
switch s:
case 0:
return (a,b)
case _:
return (b,a)
# Testing
# -------
# Generates a big tree
def gen(d, x):
switch d:
case 0:
return x
case _:
return (gen(d-1, x * 2 + 1), gen(d-1, x * 2))
# Sums a big tree
def sum(d, t):
switch d:
case 0:
return t
case _:
(t.a, t.b) = t
return sum(d-1, t.a) + sum(d-1, t.b)
# Sorts a big tree
def main:
return sum(18, sort(18, 0, gen(18, 0)))This is an example from the official repository, where we implement a Bitonic Merge Sort, a parallel sorting algorithm.
Since this algorithm uses the divide-and-conquer approach that can be run in parallel, Bend will run it in parallel (Remember its philosophy?).
Next, the program sorter.bend is run using these commands —
bend run sorter.bend # uses the Rust interpreter (sequential)
bend run-c sorter.bend # uses the C interpreter (parallel)
bend run-cu sorter.bend # uses the CUDA interpreter (massively parallel)The last command will automatically use our machine’s GPU without us ever handling the nitty-gritty details of it.
We can also compile the Bend file to standalone C/CUDA files with gen-c and gen-cu, for maximum performance but these commands are still not mature yet and can lead to errors.
Bend is still in its infancy but sounds extremely promising.
The resources in the next section will guide you in learning more and getting involved in its development. Take a look!
Happy parallelizing!
Thanks Ashish for a good article on yet another powerful programming language in making. All your articles are good and never disappoint. Keep writing !