Using assertpy and mockito testing libraries

[dabla]

Hello,

As I'm starting to contribute in the Airflow codebase I was wondering if you guys would be open to add following 2 testing libraries in Airflow.

Having spent over 20 years writing Java code and transitioning to Python development in the past three years, we all know the importance of not only writing clean code but also ensuring its thoroughly tested – a principle we're all familiar with, I'm sure.

In my previous Java projects, I relied on two libraries that are also available in Python:

• assertpy
• mockito

One might question the need for introducing two additional dependencies when Airflow already provides sufficient tools for testing. However, the advantage of these libraries lies in their ability to facilitate writing mocking statements and assertions in a fluent manner, thereby enhancing readability.

For instance, if you wish to mock a DAG and its get_doc_md function based on specific parameters, you could achieve this succinctly as follows:

from airflow.exceptions import AirflowDagInconsistent
from airflow.models import DAG
from mockito import eq, when, mock, ANY

dag = mock(spec=DAG)
when(dag).get_doc_md(ANY).thenReturn("Returns generic Markdown contents for any parameter")
when(dag).get_doc_md(matches("test")).thenReturn("Returns specific Markdown contents when the parameter contains 'test'")
when(dag).validate().thenRaise(AirflowDagInconsistent)

Similarly, assertpy offers a more intuitive alternative to traditional assert statements. By leveraging its fluent API, you can chain assertions for improved clarity:

from assertpy import assert_that

assert_that(actual).is_not_none().is_equal_to("Returns specific Markdown contents when the parameter contains 'test'")
assert_that([1, 2, 3]).is_type_of(list).contains_only(1, 2, 3)

With assertpy, it's evident what you're asserting, as you first provide the actual value to be tested and then chain the desired assertions. There a are more examples shown on their website.

This proposal is intended to enhance testing capabilities within Airflow. I'm keen to hear your thoughts on this matter.

Kind regards,
David

[Taragolis]

I would recommend to open a discussion into the Dev List, see join community. Not all regular contributors/maintainers/PMC looking into the Github Discussions.

Below only my personal thoughts, and should not be consider as "no we do not need it"

• assertpy - Not maintained on the regular basis, it could easily breaks into the new version of Python, or conflicts with new version of pytest. Looks like it more for unittest projects users
• mockito - Not a big fan of Java style into the Python.

[potiuk]

Yes - you can start discussion on devlist that's the right thing to do. But I agree with @Taragolis - I also came with Java originally and only after I got into Python for a long time I started to appreciate things done in "Pythonic" way as better.

In our case we really have to make the right call: not to satisfy one's person is accustomed to, but most importantly the 2800+ contributors we have and their usual way of doing things, as well as (what Andrey wrote too) - the sustainability, freshness and "future-proofness" of any development tooling we get. Also - one of the worst thing that can happpen with such changes is that we introduce it partially, without a clear plan, goal, and very deliberate effort to apply it consistently across the whole codebase - if we end up with only 10% of the code following new style and 90% not (which is OK if that's a transition time but it should be only temporary), then it's far worse situation than having 0% of the "better style".

And when you want to introduce a new tooling the people who are "for " such a change have to take into consideration that they have 2800+ people ship to turn, so such a change is not "propose something because is nicer and better", but also taking responsibility for convicing others it is indeed nicer and better and more importantly - leading and supporting the 2800+ community in transition - and the last part is the most important. The world is full of better tools and ideas, but the big thing here is that if you are truly convinced that things are better for the community, yourself being convinced is not nearly enough. You need to:

• convince others
• figure out the transition plan (possibly with the help of those you convince)
• pass the vote on devlist with more "+1s" than "-1s"
• implment the change, support others, solve their questions and problems, and make sure that the transition does not fall-in-the-moiddle and that eventually it will lead to a complete transitiion eventually (that might take time but you need to be committed to lead that effort to a completion or convince someone to join you and take it over to lead it to completion).

We have done some of such big changes in the past (for example pytest transition from nose) - it took years to complete, but we had a very strong and convinced team of maintainers who together worked on a plan, implemented tooling for that and relentlessly followed the execution of it. This was some 4 years ago and even till today we remove some very last remnants of it (and some of our tests are still following a bit out-dated ways of setting up and teardown for example - and we are fixing them as we stumble upon them). And it took a deliberate effort of multiple maintainers who precisely - convinced others, voted and then followed with the execution.

See https://lists.apache.org/thread/zhzr6tqh54ppzh19mdh788vqgqkljgot as the initial proposal (which actually even got Airflow Improvement Proposal written).

But when you look for our devlist and ask for pytest you will see that over the next few years this decision has been refined and improved and we relentlessly worked (and a number of poeple at that) to make pytest transition successful.

https://lists.apache.org/list?dev@airflow.apache.org:lte=1M:pytest

So if your request is "I think it's better and I think you will see it and switch right after" then answer is - probably not. If your request is "I am strongly convinced this is a better way of writing tests for the whole of Airflow and I am willing to spend time and effort and convince others, build consensus and lead it to completion" - then by all means start a devlist discussion.

But be aware that:

It’s a dangerous business, going out of your door. You step into the road, and if you don’t keep your feet, there’s no knowing where you might be swept off to

[dabla]

Alright, it was just a suggestion, I don't think I need to start a discussion wasting your guys time when I see both of your arguments. I agree on the Phythonic way, but I feel like the way assertations are done for example, and that was the same back then in Java originally, was very old school and not always clear what was being tested and what was being asserted. If we would indeed want to transition it would require a lot of work, but you could for example decide that new test should be written using the new assertions and if tests have to be modified due to refactoring, then you could also impose this for example. But as I said this was just an idea andI was curious what you guys htought about it. Thank you both for you elaborate answers!

[potiuk]

One of my friends saying in Polish is (sorry for profanity) Lepiej ch****owo ale jednakowo which roughly translates to Better sh***ty but consistently. Consistency is the key.

As I mentioned above - it's FAR worse to have something a bit better but very differently in some places than to have it a bit worse but consisently everywhere. If you introduce inconsistent way of doing things only in some places without a clear goal and effort to introduce it everywhere, then the answer is simple: DON't. With our diverse and continuously changing contributors this will raise far too much confusion and questions - should I do this or that way? When? What's the criteria? Why not here? Why it's done here only in half of the code? Is there a good reason it's not done this way? etc. etc.

Right now I can easil seay to new contributors " just look how other tests are done". If we got agreement and plan to get change applied accross the code base, this chanves to "Look here, we are in a process of change, so better do it this new way and BTW. please also change it here to help to perform the overall migration".

If you change something only for new tests without a clear plan and leadership to drive the change to the whole codebase, then the answer wil start wiht "generally we do it this and that way, but there are some places yada yada yada....." and that willl be hugly complex, confusing and taking far too much time. And without a clear plan and agreement to do so, it just creates a mess and confusion.

So In short - no. introducing new ways of doing it without commiting effort and energy and having a plan and to imtroduce it everywhere is basically "no".

Introducing it only for my changes without educating others and commiting to consistency is nothing more than a convenient shortcut that cuts off the most important part of that change, where people doing it do that for their own convenience, but don't see how it impacts the whole project and convenience of others, basically prioritising "my benefits" with "project benefits".

"new test should be written using the new assertions and if tests have to be modified due to refactoring, then you could also impose this for example"

This is PRECISELY what I was telling about. This is HOW it was done in the past. That's the only way. You wont convert tens of thousands of our tests overnight (unless you could automate it of course).

But it won't happen on it's own. If you think it will happen magically, then no, not really.

Someone has to define how it should be done, others have to agree to it. If we want to impose it, someone has to write the rules for that and either educate and convince the committers/reviewers to include that in their reviews or (better) automate it with pre-commits and such. And someone will have to keep it updated, fix problems that will inevitably happen when people will start converting more and more tests.

Later someone has to track it while it is happening and encourage people, re-educate them and remind them to continue in this direction. People generally continue doing things the way they are used to, and they won't suddenly change their behaviours like that, so it will require reminding, tracking, encouraging, reinvigorating the efforts.

Lastly when there will be 10% of those most difficult cases no-one will want to touch, someone will have to do the final push, and do the - often boring and ground work to do so.

It's a LOT of effort to make this happen.

And this is precisely what I thought about when I wrote " I am willing to spend time and effort and convince others, build consensus and lead it to completion". In short - if you have an idea to improve things like that, there are two ways to do it - either you have to become that someone, or convince other(s) to become those someones to do it.

[dabla]

Thank you @potiuk for you answer. Yes indeed it cannot be done over one night and indeed, it can also bring confussion. As I said this was an open question and I wanted to know how you guys feel about it and tackle such idea's, now I know and I also fully understand the reasoning behind it, especially with a large code base and community as Airflow.