DefaultListableBeanFactory.getBean(Class) may throw NoSuchBeanDefinitionException when removeBeanDefinition is being called simultaneously to remove an unrelated bean

[waded]

We're seeing sporadic NoSuchBeanDefinitionException exceptions out of ApplicationContext's getBean(Class) when in another thread DefaultListableBeanFactory.removeBeanDefinition(String) is being called, for unrelated/different beans. It appears to be a synchronization issue in DefaultListableBeanFactory's implementation (across use of fields beanDefinitionMap and beanDefinitionNames.) I have included a Spring Boot application that reproduces the issue. Having more beans seems to exacerbate the issue, I presume because it increases time to copy beanDefinitionNames in removeBeanDefinition.

We're seeing this with spring-beans-5.1.4 and 5.1.3, unknown about older versions than that.

Here's an unrealistic application that reproduces the condition, which we ran with spring-boot-starter-parent 2.1.2.RELEASE (spring-beans-5.1.4):

package com.example.demo;

import org.springframework.beans.factory.NoSuchBeanDefinitionException;
import org.springframework.beans.factory.support.BeanDefinitionBuilder;
import org.springframework.beans.factory.support.BeanDefinitionRegistry;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.scheduling.annotation.SchedulingConfigurer;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
import org.springframework.scheduling.config.ScheduledTaskRegistrar;

@Configuration
@EnableScheduling
@SpringBootApplication
public class DemoApplication implements ApplicationContextAware, SchedulingConfigurer {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    private ApplicationContext context;

    public static class Volatile { }

    @Scheduled(fixedRate = 400)
    public void addAndRemove() {

        BeanDefinitionRegistry factory = (BeanDefinitionRegistry)context.getAutowireCapableBeanFactory();

        // Simulate add/remove of some beans in one background thread.
        // Using larger numbers here makes the exception increasingly easier to hit in get().
        for (int i = 0; i < 1000; i++) {
            String beanName = "volatile" + i;
            if (factory.containsBeanDefinition(beanName)) {
                factory.removeBeanDefinition(beanName);
            }
            factory.registerBeanDefinition(beanName, BeanDefinitionBuilder.genericBeanDefinition(Volatile.class).getBeanDefinition());
        }
    }

    public static class Stable { }

    @Bean
    public Stable stable()
    {
        return new Stable();
    }

    @Scheduled(fixedRate = 1)
    public void get() {
        try {
            // Here get a bean that is not the one(s) being added/removed. Expect to be able to get
            // it every time.
            context.getBean(Stable.class);

        } catch (NoSuchBeanDefinitionException e) {

            // Eventually NoSuchBeanDefinitionException occurs (the missing bean being one of the Volatile ones!)
            // In DefaultListableBeanFactory.removeBeanDefinition the map is modified, then it starts replacing
            // the list w/ new copy. Meanwhile in getBean it iterates across that list in doGetBeanNamesForType,
            // but then gets from the map. They're not in the map anymore.

            throw new RuntimeException("This is the problem", e);
        }
    }

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) {
        context = applicationContext;
    }

    @Override
    public void configureTasks(ScheduledTaskRegistrar scheduledTaskRegistrar) {
        ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();

        // It's necessary to have get() and addAndRemove running on separate threads
        threadPoolTaskScheduler.setPoolSize(2);
        threadPoolTaskScheduler.initialize();
        scheduledTaskRegistrar.setTaskScheduler(threadPoolTaskScheduler);
    }
}

[waded]

For real-world context, in our actual application (not the sample I provided here) the number of beans being added/removed is rather small compared to the total number in the application. They are being added/removed in a task scheduler thread, as here. The other threads in the actual application are HTTP request-handling threads that are using ApplicationContext.getBean for service location. There, we are seeing the NoSuchBeanDefinitionException exception occur sporadically when the HTTP request occurs during remove in the background. As in the sample, the bean(s) being located are not those that are being added/removed.

[dmitrysm2000]

Is there anything new about this one?
We are facing the same issue.
It looks like the doGetBeanNamesForType which loops on the beanDefinitionNames and the removeBeanDefinition are not sychronized well, so removing the definition from one thread may cause another to fail on get.