Is Your Mental Model of Bash Pipelines Wrong?

Is Your Mental Model of Bash Pipelines Wrong? post thumbnail image

[Michael Lynch] encountered a wierd scenario. Why was compiling then running his program nearly 10x faster than just running the program by itself? [Michael] bumped into this situation whereas benchmarking a programming venture, pared it all the way down to its necessities for repeatability and evaluation, and found it highlighted an incorrect psychological mannequin of how bash pipelines labored.

Right here’s the scenario. The very first thing [Michael]’s pared-down program does is begin a timer. Then it merely reads and counts some bytes from stdin, then prints out how lengthy it took for that to occur. When operating the take a look at program within the following means, it takes about 13 microseconds.

$ echo '00010203040506070809' | xxd -r -p | zig construct run -Doptimize=ReleaseFast
bytes: 10
execution time: 13.549µs

When operating the (already-compiled) program straight, execution time swells to 162 microseconds.

$ echo '00010203040506070809' | xxd -r -p | ./zig-out/bin/count-bytes
bytes: 10
execution time: 162.195µs

Once more, the one distinction between zig construct run and ./zig-out/bin/count-bytes is that the primary compiles the code, then instantly runs it. The second merely runs the compiled program.

How can including an additional compile step lower the execution time? Seems that [Michael]’s psychological mannequin of how bash pipelines work was incorrect, and he does an ideal job of explaining how they actually work, and why that caused the strange behavior he was seeing.

In brief, instructions in a bash pipeline should not launched sequentially. They’re all launched on the similar time and execute in parallel. That meant that when run straight, [Michael]’s byte-counter program launched instantly. Then it waited round doing nothing a lot for about 150 microseconds whereas the echo '00010203040506070809' | xxd -r -p a part of the pipeline bought round to delivering its information for this system to learn. That is the place the additional execution time comes from when operating the already-compiled model.

So why is compiling it first operating sooner? Identical primary cause: when the zig construct run command kicks off, it spends a bit of time compiling this system first. Then when the compiled program is definitely launched (and begins its execution timer), the enter information from the bash pipeline is already prepared. So, the freshly-compiled program executes in much less time as a result of it doesn’t sit round ready for information from earlier within the pipeline to change into accessible.

It’s an fascinating have a look at how bash pipelines truly operate underneath the hood, and we’re delighted with the element [Micheal] places into the entire journey and clarification. In the end, particulars like this crop up and trigger some eyebrows to lift, just like the consumer who found troublesome edge cases regarding spaces in ssh commands.


Discover more from TechPros

Subscribe to get the latest posts to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Post