Throughout my career, I’ve been known as the engineer who solves a LOT of problems using shell scripts. With it, I’ve built:
- a tool that creates and destroys test services at Meta to help new engineers learn how to use the company’s planetary-scale CD system
- a parallel distributed shell that integrates with a company’s asset management system to generate the list of hosts to connect to
- plugins for an ‘Ops API’ that enables an engineer to approve and automatically apply requested infrastructure changes without the need for manual effort
Usually, when I write a shell script and submit it for review, I tend to get either of these responses from my peers:
- “Wow, this is really neat! This should save us a lot of time.”
- “Eww, gross! Why didn’t you use an actual programming language?”
Let’s talk about when shell scripts are the more compelling solution.
When CLI tools are available, and APIs aren’t
In some situations, the data sources/services you need to accomplish your task are best accessible using CLI tools. In that case, implement your solution in shell script and just run the CLI tools directly. That will be more efficient than using some kind of system library to execute them and writing all the boilerplate to supply arguments and capture stdout, stderr, and exit codes.
When prototyping
If the 80% solution can be implemented in shell script in an hour using curl and jq and the 100% solution can be implemented in Python in 8 hours using requests, do the shell script version first to validate the idea!
When authoring a CLI tool
CLI tools need to be fast! It’s so frustrating to run one where it takes over a second to just load library dependencies (Ruby/Rubygems, I’m looking at you!). The exception to this is if you’re already proficient in compiled languages (eg: Golang, Rust), which are great for building CLI tools.
When the shell script solution is significantly smaller
Consider this script that checks to see if my website is up:
curl -o /dev/null -s -w "%{http_code}" https://certomodo.io | grep -q 200
That one line of code is equivalent to this much larger Python script:
import requests
def check_url(url):
try:
response = requests.get(url)
return response.status_code == 200
except requests.RequestException:
return False
url = "https://certomodo.io"
check_url(url))
Tiny shell scripts won’t need the QA burden of larger projects such as build, lint, and unit tests (though tools like shellcheck and bats do exist)!
Conclusion
I end this article with the following quote:
“And who better understands the Unix-nature: he who writes the ten thousand lines, or he who, perceiving the emptiness of the task, gains merit by not coding?” - Master Foo
(Image produced by OpenAI DALL·E 2 with prompt “a computer screen displaying a picture of a seashell in the style of vaporwave”.)