Proper pattern for pipeline-type command chaining

[abey79]

Hello,

I'm trying to use bpaf for a pipeline-type CLI (think imagemagick), meaning that (1) commands are chained and (2) command maybe abstracted as "stage" that can be "executed" on some "state". I anticipate I will have a rather large number of commands, so instead of building one large enum, I'd rather have individual, self-contained command implementation. What is a good pattern for this?

To make this more concrete, here is a toy example where the state is just a single 2D point, and various commands can be chained to manipulate (translate, scale, etc) this point:

Original code and compiler error (see below for corrected code)

use bpaf::{construct, Bpaf, OptionParser, Parser};
use std::fmt::Debug;

/// the state that is manipulated by some pipeline stages
struct State {
    x: f64,
    y: f64,
}

/// when "executed", manipulate the state in some way
pub trait Stage: Debug + Clone {
    fn execute(&self, state: &mut State);
}

// ====================
// some command/stage

#[derive(Clone, Debug, Bpaf)]
struct Translate {
    #[bpaf(positional("TX"))]
    tx: f64,
    #[bpaf(positional("TY"))]
    ty: f64,
}

impl Stage for Translate {
    fn execute(&self, state: &mut State) {
        state.x += self.tx;
        state.y += self.ty;
    }
}

// ====================
// some other command/stage

#[derive(Clone, Debug, Bpaf)]
struct Scale {
    #[bpaf(positional("FACTOR"))]
    factor: f64,
}

impl Stage for Scale {
    fn execute(&self, state: &mut State) {
        state.x *= self.factor;
        state.y *= self.factor;
    }
}

// ====================

#[derive(Debug, Clone)]
struct Options {
    stages: Vec<Box<dyn Stage>>,
}

// parse a single stage
fn stage() -> impl Parser<Box<dyn Stage>> {
    // ideally, there would be some place were all commands are globally registered
    // then the registry content would be used here
    let translate = translate().map(|t| Box::new(t) as Box<dyn Stage>);
    let scale = scale().map(|s| Box::new(s) as Box<dyn Stage>);
    construct!([translate, scale])
}

// parse all stages
fn options() -> OptionParser<Options> {
    let stages = stage().many();
    construct![Options { stages }]
}

pub fn main() {
    let options = options().run();

    // execute the pipeline
    let mut state = State { x: 0.0, y: 0.0 };
    println!("Init: x: {}, y: {}", state.x, state.y);
    for stage in options.stages {
        stage.execute(&mut state);
        println!("Stage: x: {}, y: {}", state.x, state.y);
    }
}

Unfortunately, this code doesn't compile and generate some obscure (to me) error:

Compiling demo v0.1.0 (/private/tmp/demo)
error[E0391]: cycle detected when checking effective visibilities
   |
note: ...which requires computing type of `stage::{opaque#0}`...
  --> src/main.rs:52:15
   |
52 | fn stage() -> impl Parser<Box<dyn Stage>> {
   |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...which requires computing type of opaque `stage::{opaque#0}`...
  --> src/main.rs:52:15
   |
52 | fn stage() -> impl Parser<Box<dyn Stage>> {
   |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...which requires type-checking `stage`...
  --> src/main.rs:52:1
   |
52 | fn stage() -> impl Parser<Box<dyn Stage>> {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: ...which again requires checking effective visibilities, completing the cycle
note: cycle used when checking that `Options` is well-formed
  --> src/main.rs:48:1
   |
48 | struct Options {
   | ^^^^^^^^^^^^^^
   = note: see https://rustc-dev-guide.rust-lang.org/overview.html#queries and https://rustc-dev-guide.rust-lang.org/query.html for more information

For more information about this error, try `rustc --explain E0391`.
error: could not compile `demo` (bin "demo") due to previous error

Is this approach somehow viable? If so, any hint as to how to fix this code? Otherwise, is there a good alternative for a similar pattern?

Edit: here is the working, corrected code for reference, thanks to @pacak:

use bpaf::{construct, Bpaf, OptionParser, Parser};
use std::fmt::Debug;

struct State {
    x: f64,
    y: f64,
}

trait Stage {
    fn execute(&self, state: &mut State);
}

// ====================

#[derive(Clone, Debug, Bpaf)]
#[bpaf(command, adjacent)]
struct Translate {
    #[bpaf(positional("TX"))]
    tx: f64,
    #[bpaf(positional("TY"))]
    ty: f64,
}

impl Stage for Translate {
    fn execute(&self, state: &mut State) {
        state.x += self.tx;
        state.y += self.ty;
    }
}

// ====================

#[derive(Clone, Debug, Bpaf)]
#[bpaf(command, adjacent)]
struct Scale {
    #[bpaf(positional("FACTOR"))]
    factor: f64,
}

impl Stage for Scale {
    fn execute(&self, state: &mut State) {
        state.x *= self.factor;
        state.y *= self.factor;
    }
}

// ====================

struct Options {
    stages: Vec<Box<dyn Stage>>,
}

fn stage() -> impl Parser<Box<dyn Stage>> {
    let translate = translate().map(|t| Box::new(t) as Box<dyn Stage>);
    let scale = scale().map(|s| Box::new(s) as Box<dyn Stage>);
    construct!([translate, scale])
}

fn options() -> OptionParser<Options> {
    let stages = stage().many();
    construct!(Options { stages }).to_options()
}

pub fn main() {
    let options = options().run();

    let mut state = State { x: 0.0, y: 0.0 };
    println!("Init: x: {}, y: {}", state.x, state.y);
    for stage in options.stages {
        stage.execute(&mut state);
        println!("Stage: x: {}, y: {}", state.x, state.y);
    }
}

[pacak]

So you want a sequence of commands to operate on the same state giving you effectively (State, FnOnce(State) -> State)? Let me poke around a bit.

[pacak]

use bpaf::*;

trait Tweak {
    fn apply(&self, state: isize) -> isize;
}

#[derive(Debug, Clone)]
struct Inc;
impl Tweak for Inc {
    fn apply(&self, state: isize) -> isize {
        state + 1
    }
}

#[derive(Debug, Clone)]
struct Dec;
impl Tweak for Dec {
    fn apply(&self, state: isize) -> isize {
        state - 1
    }
}

fn inc() -> impl Parser<Box<dyn Tweak>> {
    pure(Inc)
        .map(|t| Box::new(t) as Box<dyn Tweak>)
        .to_options()
        .command("inc")
        .adjacent()
}

fn dec() -> impl Parser<Box<dyn Tweak>> {
    pure(Dec)
        .map(|t| Box::new(t) as Box<dyn Tweak>)
        .to_options()
        .command("dec")
        .adjacent()
}

fn pick() -> impl Parser<Vec<Box<dyn Tweak>>> {
    construct!([inc(), dec()]).many()
}

fn main() {
    let ops = pick().to_options().run();

    let mut state = 0isize;
    for op in ops {
        state = op.apply(state);
    }
    println!("Result: {state:?}");
}

This seem to work

[pacak]

Looking at your code now...

[pacak]

Removing Debug + Clone constraint from Stage makes it trait object safe, removing #[derive(Debug, Clone)] from Options deals with unsatisfied requirements and finally this construct!(Options { stages }).to_options() makes it compile. You'll need to change Translate and Scale into actual commands with #[bpaf(command)] but that's easy.

Seem to work.

[abey79]

Awesome, thanks a lot for that!