Do not rely on (Equinox) OSGi resolver to compute classpath

[laeubi]

Currently the resolving of the classpath of an eclipse-bundle packaging involves creating a Resolver to resolve the dependent bundles that are previously collected as part of the P2 resolution. This serves then two purposes:

1. additional verification that dependencies are resolvable with the P2 collected results
2. adding appropriate acess-rules for the JDT compiler

even though this works good in most cases this has several drawbacks:

1. compile time can greatly increase for complex dependency chains (this is currently tracked in New Equinox Resolver in Tycho 2.3.x and above fails to resolve in aceptable time in certein situations #213)
2. even though most bundles in a build share lots of dependencies, there is no way to share the resolver results so every bundle requires resolving ob the same base bundles over and over again
3. effectively we resolve twice, once with P2 and once with Equinox/Felix
4. Resolution (and thus compilation) can fail due to missing runtime constraints, e.g. a bundle might require a specific system or environment property, an extender-capability and so on to run/properly work (mos prominent example would be JPA or DS), if such a requirement is exposed, one must manually enforce extra requirements that will appear on the class-path even though they are never used (for compilation)
5. at compile time there might be other cases that are (strictly) unsolvable (e.g the same package is exposed with the same content twice from different providers) but are fine for the compiler

Thus I'd like to propose an alternative (e.g. using a configuration option for the compiler-mojo) approach that is a bit less restrictive:

• We always use the P2 collected results
• If these results are unsatisfactory we should improve P2 to give better results, this also will improve user-experience at install time (e.g. P2 would automatically suggest a DS provider and add appropriate configuration options for a product currently not using/explicitly configuring DS if required)
• We add some code (or reuse it from equinox) to extract the access-restrictions from the jars itself, this would also allow to cache these result and reuse them for other reactor projects as well.

It was argued in the past that wee need this additional check to make sure the bundle is 'installable' but I think this is not necessary here. In fact determine at build-time if a bundle later can be installed/resolved is simply an impossible task (as out lined above) and as on install time we completely rely on P2 it would make more sense to make P2 more aware about possible problems so this could be checked e.g. with the check-product-mojo or when building features/update-sites. Nerveless we can keep the configuration option so people have the choice if they like to have this additional resolve step.

[mickaelistria]

even though most bundles in a build share lots of dependencies, there is no way to share the resolver results so every bundle requires resolving ob the same base bundles over and over again

We don't know in advance if bundles do share lots of dependencies; and in the same reactor some bundles can be mutually exclusive. So that's not really something that can apply. We could maybe cache the results and reuse them across modules when suitable, but we can't have 1 resolver for all bundles in the general case.

Resolution (and thus compilation) can fail due to missing runtime constraints, e.g. a bundle might require a specific system or environment property, an extender-capability and so on to run/properly work (mos prominent example would be JPA or DS), if such a requirement is exposed, one must manually enforce extra requirements that will appear on the class-path even though they are never used (for compilation)
[...]
It was argued in the past that wee need this additional check to make sure the bundle is 'installable' but I think this is not necessary here

I actually keep arguing here: the current approach, with OSGi resolver during build, ensures that for each module, the maintainer is able to declare at least 1 environment where it can be resolved at runtime. Although it's not necessary for compilation, this check is IMO vital in a good build system. A good build system is not only about compiling and packaging content (javac and zip can do that), it's also about ensuring the content deliver some quality at runtime. For an OSGi bundle, its ability to start in at least one pre-defined scenario is an important check and incapability to build an installable scenario is worth failing the build, rather than pretending success and letting developer publish artifacts that may never start.

at compile time there might be other cases that are (strictly) unsolvable (e.g the same package is exposed with the same content twice from different providers) but are fine for the compiler

The case of same package exposed with same content twice by different provide is solvable with OSGi.

We add some code (or reuse it from equinox) to extract the access-restrictions from the jars itself, this would also allow to cache these result and reuse them for other reactor projects as well.

How do you compute dependencies to resolve access-rules without using a resolver to ensure you look at the right packages? If you build a strategy that just looks at jars, then you may end up with exposing more content than necessary to the compiler and thus loose the package import restrictions. How would you also deal with things like "re-export" or "use"...?
Caching access rules may be interesting, but it's not really a big issue nowadays; at least I don't foresee it as something that would be worth the loss.

If these results are unsatisfactory we should improve P2 to give better results

That's utopic. p2 has some bugs open that are extremely complex and most likely won't be addressed (typically about its incapacity to expose OSGi use requirements) because of lack of resource. So building a strategy that assumes p2 becomes perfect is just not going to work. The less we rely on p2, the better.

[laeubi]

How do you compute dependencies to resolve access-rules

There are mainly two cases:

1. require-bundle: The access rules are defined by the exported packages of the destination
2. import-package: The access rules are the intersection between the exported packages and the imported packages

in both cases the computed access rules can be reused if different bundles import the same bundle/packages. Use or reexport clauses do not really matter much for class-path computations. There might even be some low-level code to compute these data without using the full resolver.

The less we rely on p2, the better.

given that tycho relies in large parts on P2 (resolving dependencies, building sites+products) this does not makes much sense to me. Its only the classpath-computation that uses a different technique in all other parts we use P2.

[laeubi]

Here is an example that shows a case where P2 resolves "too much" and thus the build fails even though for compile time it doesn't matter: #173