Resources disappear from target/classes (but not .class)

[fmarot]

Hello,

I have a large multi-modules Maven project imported in Eclipse (100+ modules) and face a problem: sometimes, quite often but randomly, the resources disappear from target/classes. So I have to run manually mvn resources:resources (either on the command-line or in Eclipse). Then I can run my tests, or debug my app and sometimes if I re-run the tests/app then it fails because the resources have mysteriously been deleted. The .class are still compiled in each module's target/classes but resources are gone.

I faced this problem on 2023-03, upgraded m2e, upgraded to 2023-06 but the problem is still there and quite annoying.
Is it a config problem ? Any idea what in my poms or my config could cause the problem ?
I think this kind of problem has been seen and solved before but it is still there for me.
Regards

*** Features:
org.eclipse.m2e.feature (2.4.0.20230630-2007) "M2E Maven Integration for Eclipse Core"
org.eclipse.m2e.logback.feature (2.2.0.20230625-0847) "M2E Maven Integration for Eclipse Core"
org.eclipse.m2e.pde.feature (2.2.200.20230625-0847) "M2E Maven Integration for Eclipse Core"
org.eclipse.m2e.wtp.feature (1.6.0.20230118-0812) "Maven Integration for Eclipse WTP"
org.eclipse.m2e.wtp.jaxrs.feature (1.6.0.20230118-0812) "Maven JAX-RS Configurator"
org.eclipse.m2e.wtp.jpa.feature (1.6.0.20230118-0812) "Maven JPA Configurator"
org.eclipse.m2e.wtp.jsf.feature (1.6.0.20230118-0812) "Maven JSF Configurator"

[lalmeras]

Same here, with much smaller projects (5 - 15 modules). It seems to me it may be a race condition as problem is more frequent with larger projects or slower computer.

A working workaround is to launch Menu > Project > Clean project on the problematic module.

From my investigation, it is not related with manual mvn build (issue is triggered even when working exclusively in Eclipse). I don't find a way to trigger it consistently.

I think I encounter this problem with eclipse < 2023-03, but it is really frequent and annoying since 2023-03. 2023-06 is still problematic.

I don't know how to dig deeper in this issue. Glad to hear if anyone know if logs / configuration options can be tweaked to identify the problem.

[lalmeras]

Other things to share:

• I don't use eclipse without m2e ; I don't know if the issue exists with a non-maven project
• I'll give it a try do disable Automatically update projects configuration (preferences > maven) ; maybe there is a race condition between m2e project reconfiguration and eclipse building

[laeubi]

/CC @to-fr

I have now debugged the part a and the problem is with the new approach to skip updates of maven in target directories to fix possible infinite build recursion, because of this the following situation can occur:

1. There is a build and something changes (e.g. one has configured ${project.build.directory}/generated-resources as an output folder) as part of the build things are copied so everything looks fine
2. The delta change is reported but m2e ignores it because its part of the build
3. Then later on another file might change another build is performed and again changes something in ${project.build.directory}/generated-resources but this time the resources plugin thinks there is nothing to do because it only sees the delta of the single file!

This situation is quite hard to reproduce but might explain why it sometimes works and sometimes is missing some delta, that is in such cases where not a full build is performed and only generated items are changing inside an output folder!

I could reproduce this by the following pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>copy-resources</groupId>
	<artifactId>copy-resources</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<properties>
		<maven.compiler.source>8</maven.compiler.source>
		<maven.compiler.target>8</maven.compiler.target>
		<project.build.resourceEncoding>ISO-8859-1</project.build.resourceEncoding>
		<project.build.sourceEncoding>ISO-8859-1</project.build.sourceEncoding>
	</properties>

	<build>
		<plugins>
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>build-helper-maven-plugin</artifactId>
			</plugin>
		</plugins>

		<pluginManagement>
			<plugins>

				<plugin>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-clean-plugin</artifactId>
					<version>3.1.0</version>
				</plugin>

				<plugin>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-compiler-plugin</artifactId>
					<version>3.8.1</version>
					<configuration>
						<optimize>true</optimize>
					</configuration>
				</plugin>

				<plugin>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-install-plugin</artifactId>
					<version>3.0.0-M1</version>
				</plugin>

				<plugin>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-jar-plugin</artifactId>
					<version>3.2.0</version>
					<configuration>
						<archive>
							<manifestEntries>
								<groupId>${project.groupId}</groupId>
								<artifactId>${project.artifactId}</artifactId>
								<version>${project.version}</version>
							</manifestEntries>
						</archive>
					</configuration>
				</plugin>

				<plugin>
					<groupId>org.codehaus.mojo</groupId>
					<artifactId>build-helper-maven-plugin</artifactId>
					<version>3.2.0</version>
					<executions>

						<execution>
							<id>build-helper-maven-plugin-add-resource</id>
							<goals>
								<goal>add-resource</goal>
							</goals>
							<configuration>
								<resources>
									<resource>
										<directory>
											${project.build.directory}/generated-resources</directory>
									</resource>
								</resources>
							</configuration>
						</execution>

					</executions>
				</plugin>
			</plugins>
		</pluginManagement>
	</build>
</project>

Now one can place a file inside target/generated-resources and see that nothing is happen (what is expected as we simulate a generated file), now one should create a new file in the root of the project, this will trigger a new build but now nothing is copied (what is unexpected!) because the resources plugin asks for a delta in target/generated-resources but gets an empty list because only "newfile" is part of the delta.

A similar problem can arise if one runs a build outside of m2eclipse or would change such a file manually (as we done here), but at least a change in the project watched directories should trigger the build. Interestingly, changing a java source file seem to trigger a full build in this case what copies the resources again immediately!

[laeubi]

Interestingly, changing a java source file seem to trigger a full build in this case what copies the resources again immediately!

I now found that out, the resources plugin simply ask for the output directory if it has changes, and of course if a class is compiled there is always a change and then performs a full build instead of an incremental one, I think this is especially to account for deletion of the output directory but is kind of counter productive as this triggers a lot of full builds/copy operations.

From what I have tested/debugged, the previous "working" state is/was rather fragile because it relied on the following sequence of actions:

1. an incremental action happens that perform some change in a resource folder
2. the changed resource is not copied because the resource plugin did not see a delta
3. then hopefully a resource change is triggered (e.g. by manual refresh, polling or native events)
4. this triggered another incremental build, that creates the files again
5. now the resource plugin see the change and copy the resource
6. in some bad situation this triggers another build or if you are lucky the cycle ends here

[laeubi]

I have created https://issues.apache.org/jira/browse/MSHARED-1285 for now, but for sure this will take some time to become broadly adopted (if fixed!) so one probably need some more mitigations in the meantime. The problem is that these parts are quite deep into the m2e internals and involve some public API we can not easily change here without breaking things.

[fmarot]

(huge thanks for your work @laeubi. I have very limited knowledge of m2e internals so I can only appreciate you working on it. Thanks !)

[lalmeras]

A quick feedback about disabling « Automatically update projects configuration ». It is not a reliable workaround for this issue.

[reckart]

@fmarot thanks for mentioning that running resources:resources fixes the problem.

This is probably the most annoying bug in the last few months....

@laeubi I don't know if you have noticed - one of the two checks on your PR failed:

Error:  Failures: 
Error:    IncrementalResourceFilteringTest.testSimpleIncrementalFiltering:69->assertTime:158 expected:<[]time> but was:<[no]time>
[INFO] 
Error:  Tests run: 71, Failures: 1, Errors: 0, Skipped: 0

[reckart]

Hm... resources:resources doesn't help too much in a project with a bunch of generated-sources folders which are not properly registered unless the respective plugins actually run as well sigh

[reckart]

@laeubi is there a custom build one could install to fix this issue while waiting for upstream to make the necessary releases?

[laeubi]

instead of resources:resources you should run process-resources goal....

[reckart]

Thanks for the hint. Actually, I found that I even need to run process-test-resources for some of my projects...