Shell scripts are irreplaceable!

2021-04-11

Introduction

One line of shell script is worth 10,000 lines of C code.

Shell scripts are one the most important but at the same time most under appreciated aspect of Unix programming. After writing substantial about of code to imitate and replace shell scripts with other programming languages I have indeed realised that they are irreplaceable. Most programming language tries to imitate programs like cd, mkdir, ls, etc but fails to identify it's greatest strength, pipelines. The flow of data using pipelines feels much more natural and efficient.

Now, I will try to present the difficulties I faced in replacing shell scripts with a 'proper' programming language.

Note: Here we are referring to POSIX Shell scripts as they are easier to wrap your head around.

Python

Even though Python is easy to use, fast to get things done and have batteries included philosophy it still fails to be a good replacements for shell scripts.

I decided to write a python program to install Arch Linux and configure it to my liking (basically bootstrap it). At first I was really happy with python modules like os, pwd and getpass but as time went on these abstraction proved more costly than useful.

Now here is a snippet that I used to create a Unix user account and it's equivalent version in shell script.

Note: Any of mistakes done in implementing these programs are unintentional and if you find any them please contact me.

Python version: 

  # add repodir = directory to clone all repo's
  def set_repodir(home):
      repodir = home + '/.local/src'
      os.mkdir(repodir)
      os.chown(repodir, user_info.pw_uid, pw_gid)

  # add user account
  def add_user():
      try:
	  home = '/home/' + user
	  useradd = ['useradd', '-m', '-g', 'wheel',
		     '-s', '/bin/bash', user]
	  # make bash as default shell
	  subprocess.run(useradd, capture_output=True, check=True)
	  set_repodir(home)
      # if users exist, then modify user account
      except subprocess.CalledProcessError:
	  usermod = ['usermod', '-a', '-G', 'wheel', user]
	  subprocess.run(usermod, capture_output=True, check=True)
	  os.mkdir(home)
	  os.chown(home, user_info.pw_uid, pw_gid)
	  set_repodir(home)

Shell script version (taken from LARBS): 

  adduserandpass() { \
	  useradd -m -g wheel -s /bin/bash "$name" >/dev/null 2>&1 ||
	  usermod -a -G wheel "$name" && mkdir -p /home/"$name" && chown "$name":wheel /home/"$name"
	  repodir="/home/$name/.local/src"; mkdir -p "$repodir"; chown -R "$name":wheel "$(dirname "$repodir")" ;}

As we can see the python version is not only more lengthy but also more error prone compared to shell script version. What will happen when usermod command will fail?

Now there comes some command which cannot be scripted without pipelines and these require us to use a different subroutine called getstatusoutput to invoke shell, execute command and get status and output back. 

  # add user's password
  def add_pass(password):
      chpass= 'echo ' + user + ':' + password + ' | chpasswd'
      output = subprocess.getstatusoutput(chpass)
      # exit if output status output is not zero
      if output[0] != 0:
	err_msg(output.chpass)
	exit(1)

Elisp (Emacs Lisp)

At first it seems confusing to use a editor's extension language to program your environment but due to extensible nature of LISP it can be even used to do so. Also I generally use Emacs as my interface to UNIX rather than a text editor.

I recently wrote a Elisp library called emount to make use of Emacs completion to mount and umount USB drives and Android using Emacs! At its core it was just using mount and simple-mtpfs to mount USB drives and Android respectively.

Here is the function I used to mount USB devices with my own ut-common-shell-command which is just a wrapper around Emacs call-process command. 

  (defun ut-common-shell-command (command &rest args)
    "Run COMMAND with ARGS.
  Return the exit code and output in a list."
    (with-temp-buffer
      (list (apply 'call-process command nil (current-buffer) nil args)
	    (buffer-string))))

  (defun emount-mount-usb (usb-name mount-point)
    "Mount usb with USB-NAME at MOUNT-POINT."
    (interactive (list (emount--select-unmounted-usb (emount--get-drive-list emount--usb-drive))
		       (emount--select-mount-point)))
    (let* ((partition-cmd
	    (concat "lsblk -no fstype " usb-name))
	   (partition-type
	    (string-chop-newline (shell-command-to-string partition-cmd)))
	   user-group result exit-code output)
      (if (string-equal partition-type "vfat")
	  (progn
	    (setq result (ut-common-shell-command "sudo" "mount" "-t" "vfat"
						  "-o" "rw,umask=0000" usb-name mount-point))
	    (setq exit-code (nth 0 result))
	    (setq output (nth 1 result))
	    (if (= exit-code emount--success-code)
		(message "%s mounted successfully!" usb-name)
	      (user-error output)))
	(setq result (ut-common-shell-command "sudo" "mount" usb-name mount-point))
	(setq exit-code (nth 0 result))
	(setq output (nth 1 result))
	(if (not (eq exit-code emount--success-code))
	    (user-error output)
	  ;; TODO: test groups output
	  (setq user-group (nth 0 (split-string (shell-command-to-string "groups"))))
	  (setq result (ut-common-shell-command "sudo" "chowm" (user-login-name)
						":" user-group mount-point))
	  (setq exit-code (nth 0 result))
	  (setq output (nth 1 result))
	  (if (= exit-code emount--success-code)
	      (message "%s mounted successfully!" usb-name)
	    (user-error output))))))

Again you can see I am using (nth 0 result) to get exit code and printing error using (user-error output) if exit code doesn't match emount--success-code.

Corresponding shell script version (taken from voidrice): 

  mountusb() { \
	  chosen="$(echo "$usbdrives" | dmenu -i -p "Mount which drive?")" || exit 1
	  chosen="$(echo "$chosen" | awk '{print $1}')"
	  sudo -A mount "$chosen" 2>/dev/null && notify-send "USB mounting" "$chosen mounted." && exit 0
	  alreadymounted=$(lsblk -nrpo "name,type,mountpoint" | awk '$3!~/\/boot|\/home$|SWAP/&&length($3)>1{printf "-not ( -path *%s -prune ) ",$3}')
	  getmount "/mnt /media /mount /home -maxdepth 5 -type d $alreadymounted"
	  partitiontype="$(lsblk -no "fstype" "$chosen")"
	  case "$partitiontype" in
		  "vfat") sudo -A mount -t vfat "$chosen" "$mp" -o rw,umask=0000;;
		  "exfat") sudo -A mount "$chosen" "$mp" -o uid="$(id -u)",gid="$(id -g)";;
		  *) sudo -A mount "$chosen" "$mp"; user="$(whoami)"; ug="$(groups | awk '{print $1}')"; sudo -A chown "$user":"$ug" "$mp";;
	  esac
	  }

Conclusion

In the end I will let you decide which implementation suits you better but I still think shell literacy is important and it will help you to come up with a better solution. I would also recommend you check what Master Foo wants to say about shell scripting.