spring.profiles.included not permitted in externalized non-profile-specific properties

[philwebb]

Issue transferred from the Spring Boot issue tracker. See spring-projects/spring-boot#24876 originally raised by @konradczajka

---

This issue relates highly to Spring Cloud Config but was introduced directly by changes made in #24733 (released in Spring Boot 2.4.2) so I report it here. I'll move it to SCC if needed.

In my current setup I share some common configuration between applications using profiles.
Configuration repository contains application.yml file with properties shared among all apps. There are also additional application-xxx.yml files containing properties shared by a subset of services e.g. application-rabbitmq.yml
App-specific, but not profile-specific files (e.g. calculator.yml) declares spring.profiles.include=rabbitmq property with all profiles that should be used to retrieve given app's config. All apps are started without marking any profiles as active/included.
This approach seems to be supported by Spring Cloud Config Server as well as when I request some app's config using the default profile (<config-server-uri>/calculator/default) in response I receive "main" files: application.yml, calculator.yml, as well as files specific to the declared profile: application-rabbitmq.yml. When I change spring.profiles.include in application.yml to other value, application-rabbitmq.yml is not included anymore. This means SCCS recognizes this property and takes it into account when determining which properties files should be returned.

Now to the main problem - the setup described above works well with Spring Boot 2.4.1 but has stopped after the upgrade to 2.4.2.
Now the client app crashes at startup with:

16:43:55.562 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.springframework.boot.context.config.InvalidConfigDataPropertyException: Property 'spring.profiles.include' imported from location '[ConfigServerConfigDataResource@e84a8e1 uris = array<String>['http://localhost:8888'], optional = false, profiles = list['default']]' is invalid in a profile specific resource [origin: Config Server (snip)/calculator.yml:3:14]

InvalidConfigDataPropertyException throws an exception during processing of ConfigDataEnvironmentContributor instance responsible for handling cloud config server as a source. The contributor is marked as "profileSpecific" and therefore it's forbidden for it to contain "profiles properties".
When I move spring.profiles.include from calculator.yml to application.yml the app crashes as well.
I don't know what exactly causes the contributor to be marked as "profileSpecific". Spring Cloud Config Client makes a request to the Spring Cloud Config Server using "default" as profile name when the app was started without any profile, The response from the server looks like this:

    (...)
    "profiles": [
        "default"
    ],
    (...)
   "propertySources": [
        {
            "name": "(snip)/calculator.yml",
            "source": {
                (...)
            }
        },
        {
            "name": "(snip)/application-rabbitmq.yml",
            "source": {
                (...)
            }
        },
        {
            "name": "(snip)/application.yml",
            "source": {
                (...)
            }
        }
    ],

I created a simple repository with config server that presents the problem. It's even more simplified then the description above as configuration served by SCCS doesn't include an application-specific files, just appplication.yml and application-demo.yml:
https://github.com/konradczajka/cloud-config-profile-issue-demo

In my opinion it should be permitted to specify profiles in an external configuration as long as particular file is not profile-specific but I'm aware that there may be a good reason for forbidding that. I'd just like to know if that's a case and I'll just look for other solution to my sharing problem. And if It's not advised to include any profiles-related properties in any file served by SCCS then maybe its documentation should mention it.

Regards.

[philwebb]

I've briefly looked into this one and I think the problem is that ConfigServerConfigDataLocationResolver only implements resolveProfileSpecific and it returns an empty list from resolve.

The means that Spring Boot treats each contributor and profileSpecific which ultimately results in the InvalidConfigDataPropertyException.

[philwebb]

We're going to need a change in both Spring Boot and Spring Cloud. For Spring Boot we're going to add a ConfigData.Option to allow profile properties to be ignored (see spring-projects/spring-boot#24890). Spring Cloud will need to set this option on the Config Data that they return from their loader.

[philwebb]

It's also going to be worth setting ConfigData.Option.IGNORE_IMPORTS so that imports are only handled on the server side

[spencergibb]

This will need to wait for boot 2.4.3 and hence 2020.0.2

[workmanw]

It looks like Spring 2.4.3 was released, is this simply fixed by using that version or is there work that needs to be completed in this repo?

https://spring.io/blog/2021/02/18/spring-boot-2-4-3-is-now-available

[workmanw]

I tried upgrading our gateway again using 2.4.3 and we seem to still hit the same issue. @philwebb I'm curious if the issue is fixed for you with 2.4.3?

[spencergibb]

It's only fixed with boot 2.4.3 and Spring Cloud 2020.0.2-SNAPSHOT (release coming next week)

[workmanw]

@spencergibb Ohhh. My mistake. It didn't click when I read it originally, but now I see the commit you cited was for Spring Cloud. Thank you!

[konradczajka]

@philwebb @spencergibb Thank you for the fix. I can use spring.profiles.include in configuration repository in order to load additional, profile-specific properties files now.

Please don't treat the rest of the message as a bug report of any kind and more like an "FYI". You can ignore it if you like.

I came across some inconsistencies between how active and include profiles are processed depending on where they were defined (client app vs config repository) and I don't know if they are intentional or not:

• spring.profiles.active defined in the config repository affects the client's active profiles list but is ignored by the config server (i.e. it doesn't load the related profile-specific file). However, if I don't set the profile in the repo and start the client with --spring.profiles.active=... both active profiles list and config server are properly affected by it. I guess it just may be a Config Server's limitation.
• spring.profiles.include defined in the config repository is respected by the config server (i.e. the related profile-specific file is loaded) but it's not included in the client's active profiles list. However, if I start the client with --spring.profiles.include=... both active profiles list and config server are properly affected by it.
• When spring.profiles.active=a is defined in the config repository and I start the client with --spring.profiles.include=b both active profiles list ([b]) and list of files loaded by the config server (just application-b.yml) are wrong.
  However, if I don't specify profiles in the repository and start the client with --spring.profiles.active=a --spring.profiles.include=b both active profiles list ([a,b]) and list of files loaded by the config server (application-a.yml and application-b.yml) are correct

Below you'll find results of some tests I performed:

1. Client started with --spring.profiles.active=demo
   No profiles set in the config repo.
   Result:
   Environment::getActiveProfiles = [demo] (Correct)
   application-demo.yml is loaded by the Config Server (Correct)
2. Client started with --spring.profiles.include=demo
   No profiles set in the config repo.
   Result:
   Environment::getActiveProfiles = [demo] (Correct)
   application-demo.yml is loaded by the Config Server (Correct)
3. Client started with --spring.profiles.include=demo-i and --spring.profiles.active=demo-a
   No profiles set in the config repo.
   Result:
   Environment::getActiveProfiles = [demo-i, demo-a] (Correct)
   application-demo-i.yml is loaded by the Config Server (Correct)
   application-demo-a.yml is loaded by the Config Server (Correct)
4. Client started with no profiles
   spring.profiles.active: demo set in application.yml in the config repo
   Result:
   Environment::getActiveProfiles = [demo] (Correct)
   application-demo.yml is not loaded by the Config Server (Not correct?)
5. Client started with no profiles
   spring.profiles.include: demo set in application.yml in the config repo
   Result:
   Environment::getActiveProfiles = [] (Not correct?)
   application-demo.yml is loaded by the Config Server (Correct)
6. Client started with no profiles
   spring.profiles.include: demo-i and spring.profiles.active: demo-a set in application.yml in the config repo
   Result:
   Environment::getActiveProfiles = [demo-a] (Not correct?)
   application-demo-i.yml is loaded by the Config Server (Correct)
   application-demo-a.yml is not* loaded by the Config Server (Not correct?)
7. Client started with --spring.profiles.active=demo-ca
spring.profiles.active: demo-ra set in application.yml in the config repo
   Result:
   Environment::getActiveProfiles = [demo-ca] (Correct)
   application-demo-ca.yml is loaded by the Config Server (Correct)
   application-demo-ra.yml is not loaded by the Config Server (Correct)
8. Client started with --spring.profiles.include=demo-ci
spring.profiles.include: demo-ri set in application.yml in the config repo
   Result:
   Environment::getActiveProfiles = [demo-ci] (Not correct?)
   application-demo-ci.yml is loaded by the Config Server (Correct)
   application-demo-ri.yml is loaded by the Config Server (Correct)
9. Client started with --spring.profiles.include=demo-ci
spring.profiles.active: demo-ra set in application.yml in the config repo
   Result:
   Environment::getActiveProfiles = [demo-ci] (Not correct?)
   application-demo-ci.yml is loaded by the Config Server (Correct)
   application-demo-ra.yml is not loaded by the Config Server (Not correct?)