quicli - The ideas behind my small Rust CLI framework

I recently wrote a new Rust crate: quicli. As the name might suggest, it gives you a way of quickly writing CLI programs in Rust. This post is about the ideas and underlying philosophies behind this project.

Note: If you found this because you were looking for quicli itself, have a look at quicli’s Getting Started guide!

Some history

I’ve been writing CLI tools in Rust for a while now. It really surprised me how this language that works really well for systems programming quickly became my go-to solution for problems I’d previously write Shell or Ruby scripts for.

I published the post “5 Tips for Writing Small CLI Tools in Rust” at the end of August 2017 that you may have read, where I describe some of my take-aways from that in the form of short tips. I’ve gotten some great feedback from that post, and there were people telling me that they seriously started looking into Rust after having read that article. That totally made my day: Someone took a chance at learning something new and becoming part of the great community that Rust has because of something I did! How cool is that?

Ever since then, I wanted to condense this down into a small framework or library that you can use when you are just starting out with Rust as well as when you just want to write a quick tool. Sadly, like so often, I didn’t have the time or concentration to sit down and really do this. And here’s the good news: For some reason¹, after all these months, I’ve now published a very early version.

It was a Sunday evening when I published the code, and I did so without thinking much about it. But then something very interesting happened the very next day: After a link to quicli’s repo was posted to reddit, Garrett Berg (vitiral) replied saying that their stdcli project was very similar to my endeavor. This was unexpected. And not only were out projects similar, but we were both willing to join forces and try to make one great crate that takes the best of both approaches. Garrett opened a few issues on quicli’s issue tracker (thanks!), which basically concluded in #19, which was originally titled “merge stdcli and this lib”.

That sounded pretty awesome and made me really proud: Did my library, which was barely a day old at this point, look cool enough to compete with their existing effort?

But, before blindly agreeing to this, I needed to take a step back and evaluate what I want quicli to be. Because, let’s be honest, I hadn’t really thought about a ‘grand quicli vision’ before. And why should I? I just wanted a small framework-like thing that made writing CLI apps less of a pain. But how exactly do I want to to this? Thanks to Garrett’s questions and comments I now have an answer to this.

What do I want to quicli to be

(The following are basically quotes from #19.)

My goal is to provide an opinionated set of convenience functions to quickly build CLI apps.

In its implementation, you get what I’d call “a small framework around your main function”: quicli’s main! macro set up some basic things, and its prelude gives a few tools that are commonly required in CLI apps. These tools are either re-exports from existing Rust crates, or abstractions around library functions that are more ergonomic that better fit the CLI use-case (as well as my ergonomic requirements).

What I want you to end up with is code that is concise, well-structured and boilerplate-free, and production-ready. Okay, that’s nothing new; we all want that. But what does this even mean for a CLI app?

Concise

Ideally: You type one line to get one thing done. If you want to read a file and see if its text contains the word “foobar”, you can do:

let content = read_file("./input.txt")?;
let has_foobar = content.contains("foobar");

You want to read a file, I give you a read_file function. This skips allocating a String as buffer, opening File, and calling read_to_string on that file. While reading a file to a String this way may not be the most performant option, it is very concise.

Another aspect of conciseness: Instead of using unwrap to deal with errors, quicli’s main! allows you to use ? to propagate errors, and shows human-readable messages when your program exited with an error state. Similarly, instead of writing return Err(SomeType::new("message")), by using quicli you also get failure’s bail! and ensure!.

Well-structured and boilerplate-free

I just repeatedly wrote that this is a framework for small code bases. Nevertheless, it makes sense to structure your code well. Actually, structuring your code in a clever way helps you keep the code size small.

For example, this is how you parse CLI arguments (incl. type checking):

/// Get first n lines of a file
#[derive(Debug, StructOpt)]
struct Cli {
    /// How many lines to get
    #[structopt(long = "count", short = "n", default_value = "3")]
    count: usize,
    /// The file to read
    file: String,
}

See how you didn’t write any code to convert whatever followed --count to an integer? By the way: Adding doc comments to your CLI arguments structure is not just for when you write the code, structopt will also show them when calling your program with --help.

Production-ready

In my experience, the best way to ensure something ends up being used in production for years to come is to call it a prototype. I always want to get some basic stuff that will make using even the smallest tool less of a pain.

For example, CLI tools written with quicli (and structopt) always have a --help message, as well as --version flag, and good error messages. Similarly, you get logging for free, and if you add two more lines, you also get a --verbose/-v CLI flag to control the log level (pass -vvv for “debug” level logging).

What I don’t want quicli to be

I want to use quicli to quickly build CLI apps – And I want to stop there! If you are no longer writing a small CLI tool (I’d say something like 500 lines of code is no longer small), your problems are likely more complicated than what quicli gives you tools for. At that point, you should consider replacing your usage of (parts of) quicli with more sophisticated approaches.

Let’s contrast that with Garrett’s stdcli. It (re-)exports a lot of sophisticated tools! As Garrett writes:

stdcli’s goal is basically to make creating a CLI in rust more like creating one in python from a user experience and ergonomics point of view. This means: batteries [are] included, almost everything you need is already imported […]

quicli also re-exports a bunch of stuff, and I agree that these re-exports make the experience of writing CLIs quickly very smooth. There are some traits that are very useful, and deserve to re-exported! I want to be very deliberate in what I re-export though and ideally I want to provide abstractions instead of full-featured tools. Let me provide some reasoning for this.

For example: Having Read and Write in the prelude seems like a good idea, but AtomicBool? I’ve never used that one the last 4 years I’ve written Rust code. Let’s take a step back and look at it from another perspective: I’m pretty sure even Read and Write are only used in a few specific use cases. I say: We should try to identify these use cases, and provide convenience functions for those!

I didn’t want to re-export env_logger; instead I changed the main! macro (that quicli provides to set give you nice error handling and the ability to use ? in your “main” function) to initialize the logger automatically. Similarly, I added simple file read/write functions.

Writing this, the following dawned on me:

I ship leaky abstractions

I’d rather introduce some (simple and leaky) abstractions (that promise to only be useful 80% of the time) instead trying to give the user everything they need. I’m not delivering building blocks here, I deliver concrete tools. (This is hard for me – I really like abstract and generic things!)

Another example (which I haven’t implemented) that works this way: A regex_matches function that automatically uses lazy_static and only covers the simplest case (maybe even return a very simple type instead of a powerful iterator over capture results). Why? Simple: Have you seen the regex docs? They are wonderful. Sadly, for newcomers/forgetful people/drunk programmers/etc. they are also wonderfully complex.

So, instead of offering the user “everything”, I want to introduce some abstractions that are simple to use, and have simple but useful examples. To not get stuck, and to give user the ability to grow, learn, and discover new stuff, the documentation should point exactly to where to look when you want to use some of the features on a more complex level.

Indeed, instead of adding any feature flags to this crate (that enable additional components to be loaded/exposed), I want to have a clear line where a user is supposed to stop relying on quicli. I’d rather have an “eject” option to switch from quicli to “all crates imported manually” than add and re-export a whole bunch crates (which will at some point lead to extern crate kitchensink;).

And now add some good documentation

So far I’ve written what I want quicli to contain, but another important aspect to me is to prove that it is useful for writing small tools – even if you’ve only just started with Rust. This is why I initially wrote the Readme file in the form of a How To: In a few simple steps I explain how to set up a Cargo project, what you need to add to parse CLI arguments, and how to write the few lines necessary for implementing a simple head tool.

My hope was that this not only shows how easy it is to get started with quicli but to also show how concise and pretty Rust code can be. And this is a great motivator for developing quicli further: Since then, I’ve written two more guides, and for both I added features to the framework, exposed new functionality, and thought more about how I want quicli-based code to feel. (More on the aspect of hosting and testing these docs in a future post!)

Thanks so much to @jamesmunns, @mgattozzi, and @skade for reviewing this post! It came to be over the course of several train rides and without you it would be much less coherent.