Plugin Creates Tasks Immediately Rather Than Lazily in Configuration or Execution Phases

[mattbertolini]

What version of OpenRewrite are you using?

I am using

• OpenRewrite v2.8.0
• Maven/Gradle plugin v6.10.0

How are you running OpenRewrite?

I am using the Gradle plugin, and my project is a multi module project. I am using convention plugins to distribute the plugin to all of the eligible modules in the build. I have over 500 modules (potentially) using the plugin. It is a private repository so I can't share any code.

What is the smallest, simplest way to reproduce the problem?

When I add the Openrewrite plugin to the build it creates its tasks immediately rather than relying on lazy configuration. I know this via our Gradle Enterprise build scans. This significantly slows down the configuration phase of our build which has hundreds of modules and tens of thousands of tasks. I am attaching a screenshot of Gradle Enterprise that shows this

What did you expect to see?

Lazy configuration has been around since Gradle 5.x and is the standard for defining tasks. I would expect tasks to be created during configuration or execution rather than creating them immediately. This is to make sure that modules that don't need those tasks don't have to wait for them to be created.

What did you see instead?

As the above image shows, I see the openrewrite tasks being created immediately.

What is the full stack trace of any errors you encountered?

Are you interested in contributing a fix to OpenRewrite?

I would be open to taking a stab at updating the plugin to register the tasks lazily but in order to do this support for Gradle 4.x would need to be removed. That is a decision I can't make unilaterally. Gradle 4.0 was released in June of 2017 and the last of the 4.x line was released in December of 2018. I do think it's reasonable to drop support for a version that is over five years old. This is going to be even more of a problem as Gradle evolves with version 9.x and beyond. I think it's time to bring this plugin into the modern Gradle world.
Until this plugin is more modern, I can't advise my company to use Openrewrite or Moderne. It's a nonstarter as it affects build performance too much.

[knutwannheden]

Thanks for your report! I can see how the overhead adds up in a large build like yours. While we would love to use newer Gradle APIs ourselves, we want to keep Gradle 4.10 compatibility for now. The reason being that we want to provide compatibility with projects using old Gradle versions.

But all is not lost! Based on this source Gradle added support for lazy task creation starting with version 4.9. It also seems like Grolifant could help with other places where we want to be able to use some new Gradle API while still supporting old versions. Here is a thread with some good pointers.

If lazy tasks are indeed available as of Gradle 4.9, I think we can probably register our tasks lazily.

[timtebeek]

Also note that there's some work ongoing in #227 to make the plugin compatible with configuration cache; while slightly different it's perhaps good to know here for a coordinate change to avoid conflicts.

As to your comment:

Until this plugin is more modern, I can't advise my company to use Openrewrite or Moderne. It's a nonstarter as it affects build performance too much.

For users of Moderne this is not an issue, as building the LSTs is a whole separate process and does not require any changes or additions to the underlying projects. Recipes can then be run at quickly and scale against these serialized LSTs.

On the OSS side of things for Gradle we also support a flow of running recipes without changing the build, but that might very well not scale very well to 500 modules. You're welcome to try, and we'll support you all the same. This is just to say that adding our plugin to your projects is optional, and not a requirement to running recipes.

Wanted the point out the above options and differences, just to avoid any confusion about the capabilities and how certain use cases are typically served. We'd love to help where we can, and make adjustments where necessary. Just know that there's no reason to be apprehensive about using OpenRewrite or Moderne, just because we also support folks coming from very old versions of Gradle. That's just the business we're in. :)

[mattbertolini]

I did a bit analysis of the changes it would take to make the tasks lazy loaded. @knutwannheden you are indeed correct that it looks like the provider API is in 4.10 and above. I wasn't aware of this. So it is possible to do this if 4.10 is an acceptable floor for version support. I had also not heard of Grolifant before. It could help with the changes but I don't see it on the classpath of the project. Is this something that you would be OK adding?

I took a look at #227 and its corresponding PR #228 and I do think the changes are overlapping a bit. There would be merge conflicts between the PRs. I am ok with waiting for that PR to land before making any additional changes to support lazy loading tasks. But the PR does seem to have gone a bit silent so I am concerned about that.

I might make a PR just to see what it would take and we can close it if the configuration cache PR gets moving again. Any thoughts?