Implement lazy evaluation for bitwise operators

[ghost]

Version: mathjs-v5.2.3

I have found some inconsistency between MathJS and other programming languages, like JS or Java when using boolean operations and / or. Other programming languages only evaluate the right expression lazily (for example, see http://www.cafeaulait.org/course/week2/46.html).

Steps to reproduce (just an example):
console.error(math.eval('true or undefined < 1'));
console.error(math.eval('false and undefined < 1'));

Expected behaviour:
In the development tools, one can see the following lines printed:
true
false

Actual behaviour:
For both expressions an error is thrown:
Unexpected type of argument in function smaller (expected: Array or DenseMatrix or SparseMatrix or Matrix, actual: number, index: 1)

[josdejong]

That's an interesting point. We could indeed make the bitwise operations evaluate lazily. This has been implemented for example for the conditional operator.

I think the bitwise operators also have a different precedence compared to < than in JavaScript, see docs. I'm not sure whether your example is currently evaluated the way you expect:

math.parse('false and undefined < 1').toString({ parenthesis: 'all' })
// 'false and (undefined < 1)'

I'm always open to reconsider the operator precedence if there is a good reason for it.

[josdejong]

Shall we change the title of this issue to "Implement lazy evaluation for bitwise operators"?

[smith120bh]

@josdejong : I'm very keen for this feature, and would be happy to implement it. However, while I've prototyped making this work to prove that I could, I would like to do this in a way that is likely to be accepted before I spend too much time on it. I can see a few ways of doing this. The core barrier I observe is that, right now, OperatorNode._compile() pre-evaluates all the arguments to every operator function. We need to somehow be able to either pass raw arguments to the relevant function, or evaluate the arguments sequentially only. My thoughts on methodology here:

1. Specific exceptions in OperatorNode._compile(): This is how I prototyped a working modification. I added basically the following to OperatorNode. _compile():

     else if (evalArgs.length === 2 && this.fn === "and") {
       var evalArg0 = evalArgs[0];
       var evalArg1 = evalArgs[1];
       return function evalOperatorNode(scope, args, context) {
           return testCondition(evalArg0(scope, args, context))
               ? fn(
                     evalArg0(scope, args, context),
                     evalArg1(scope, args, context),
                 )
               : false;
       };

   Now I'd assume that this is philosophically probably not the way you'd want to go; I don't think you want conditions on specific functions in the _compile() code of nodes!

2. Add rawArgs functionality to OperatorNode: There's already rawArgs for FunctionNodes, and OperatorNode is essentially just a frontend for calling functions. We could add similar rawArgs functionality to the OperatorNode, and then modify the functions and, or, etc. to evaluate arguments within their own function code, and to do so in a lazy manner. This feels like it has the elegance of being parallel to something that's already done elsewhere in the codebase, and it allows each type of operator to implement whatever lazy logic is most appropriate to the individual function. It has the minor downside that it'd mean some somewhat redundant code duplicated across each of the relevant functions (and.js, or,js, etc) for the lazy evaluation logic.

3. Add a new LazyOperatorNode node type: This might be different enough behaviour to warrant a new node type entirely. And in this case, I'd think it'd make reasonable sense to have LazyOperatorNode._compile() itself evaluate the truthiness of the first argument, and then only even call the and function (or whatever function), if the evaluation can NOT be done lazily. While I wouldn't expect any of these options to be particularly processor-intensive, I'd imagine this'd be the most efficient. However, it also might be adding unnecessary complexity to the codebase with a whole new node type.

Whichever option, there is some additional logic to implement for when the arguments are matrices or arrays. We'd either need to (a) have a check on whether all(is_reqd_truthiness) for every element of the first argument's matrix, or (b) just only lazy evaluate if the first argument is a simple bool / number / unit.

At any rate, I'd love to hear your thoughts, and I'm definitely keen to create a PR if an approach like one of the above sounds viable!

[josdejong]

Thanks @smith120bh for picking this up!

I think your option 2 is indeed most neat: that will not "overload" OperatorNode with implementations for all bitwise functions, and instead neatly separate this, so users can customize on this. We can provide the lazy version of the bitwise functions (having .rawArgs = true set on the functions) and define them as a transform function by configuring { isTransformFunction: true }.

It will indeed involve duplicating logic, but that would be the case too in your option (1), right? I'm not sure what we should do with matrix evaluations: make that lazy on a per-cell level? That may be complex, and I'm not sure if that is worth it.

[smith120bh]

It will indeed involve duplicating logic, but that would be the case too in your option (1), right?

Yes, it would :-)

I'm not sure what we should do with matrix evaluations: make that lazy on a per-cell level? That may be complex, and I'm not sure if that is worth it.

Yeah, I think per-cell lazy evaluation also loses much of the benefit of lazy evaluation. At least in many use cases I can think of, there's a reasonable chance that we'd need to evaluate the whole matrix in a second arg even if we only want one cell's value (e.g. if there's a cross-product involved or similar), so there's not likely to be as much efficiency gain or user-perceptible functionality change. It's also something that could be added later if users are interested in it!

My personal use case for this, and reason for wanting this feature, sounds similar to @ghost's originally: I want to be able to write something of this (over-simplified) form:

calc_method == "b" ? b = 1+2 : c = 3 * 2;
final_result = (calc_method == "b" and b > 4 ? "fail" : "pass")  // currently errors if calc_method != "b"

Situations like this come up pretty frequently in structural engineering, where depending on certain conditions, you go through different calculation processes but still end up with the same final variable you need to determine. While we can use nested ternaries, those quickly become very hard to read.

[josdejong]

OK let's keep Matrix inputs non-lazy (at least for now). I think it would be a lot of work without much benefit.

Thanks for sharing your use case!