A data pipeline commonly has three phases: extract, transform, and load (ETL). The Unix philosophy of “do one thing well” can apply here by making each program handle only one ETL phase; for example, a program that only extracts data. This strategy works well whether orchestration is done with a shell script or with a Python-based de facto tool such as Apache Airflow. In contrast to mixing ETL within a single program, Airflow and shell scripts using GNU Parallel can handle retries, concurrency, logging, etc. Data pipeline engineering is not a rare case. Still, other kinds of applications, such as GUI, TUI, and web apps, are rarely built on shell scripts; however, shell scripts may work behind the scenes for many apps. Moreover, “do one thing well” still applies, but the unit may be a function, module or service instead.

wsl --install installs Ubuntu by default. If I had to use WSL, I’d try this first too.

I couldn’t find a book that exactly matches Clojure. Still, I prioritize Domain Modeling Made Functional: Tackle Software Complexity with Domain-Driven Design and F# by Scott Wlaschin. However, many concepts, such as algebraic data types, wrapper types, and monads, may need to be adapted or even ignored.

I haven’t applied clean architecture with Clojure, but I’m working on a Python codebase that uses dependency injection intensively. It works, but I personally feel the pain every time I create a mock.

Although the clean architecture book mentioned vertical-slice architecture and added a new chapter about packaging by technical layers versus by features, clean practitioners seem to prefer packaging by layers, which should be approached with caution. I don’t remember if the book mentions pushing I/O to the edges, or the imperative shell pattern, which would allow pure functions to cover more logic instead of having a service layer that interleaves I/O and logic.

The hard part for me was dynamic scope, which I found it was explained nicely in The Common Lisp Condition System: Beyond Exception Handling with Control Flow Mechanisms. Since then I have begun to feel that dynamic scope in Bash is a feature, not limitation.

Cachy, Endeavour, Garuda, etc should be fine.

Since you are a coder, I suppose you using CLI should be fine. However, there are many other alternatives, which is fast and even based on Arch Linux.

In the 90s, QBasic (IDE) and Edit didn’t work on Slackware. So, I tried Pico, Vim, and Emacs. Now, I still use Emacs.

With some certain distros, it is easy.

With CachyOS and Mint, it is very easy.

Remark: I disabled secure boot.