Skip to main content

Unable to add ECS cluster to Spinnaker (not authorized)

Issue​

When using a cross account AWS role to access an ECS cluster, the user is getting the following issue:Looks like this is an RBAC related error:You are not authorized to perform this operationThe configuration changes were successfully applied but in CloudDriver but the user is still seeing this error. clouddriver 2020-08-27 22:43:23.719 ERROR 1 --- [ main o.s.boot.SpringApplication : Application run failed clouddriver org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'basicAmazonDeployDescription': Unsatisfied dependency expressed through field 'regionScopedProviderFactory'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'regionScopedProviderFactory': Unsatisfied dependency expressed through field 'clusterProviders'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'amazonClusterProvider' defined in URL [jar:file:/opt/clouddriver/lib/clouddriver-aws-GCSFIX.jar!/com/netflix/spinnaker/clouddriver/aws/provider/view/AmazonClusterProvider.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cacheView' defined in class path resource [com/netflix/spinnaker/clouddriver/cache/CacheConfig.class]: Unsatisfied dependency expressed through method 'cacheView' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'catsModule' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'catsModule' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'netflixAmazonCredentials' defined in class path resource [com/netflix/spinnaker/clouddriver/aws/security/AmazonCredentialsInitializer.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.List: Factory method 'netflixAmazonCredentials' threw exception; nested exception is com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:643) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:130) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:893) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.4.RELEASE.jar:2.2.4.RELEASE clouddriver at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.2.4.RELEASE.jar:2.2.4.RELEASE clouddriver at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.2.4.RELEASE.jar:2.2.4.RELEASE clouddriver at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) ~[spring-boot-2.2.4.RELEASE.jar:2.2.4.RELEASE clouddriver at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140) ~[spring-boot-2.2.4.RELEASE.jar:2.2.4.RELEASE clouddriver at org.springframework.boot.builder.SpringApplicationBuilder$run$0.call(Unknown Source) ~[na:na clouddriver at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-2.5.10.jar:2.5.10 clouddriver at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) ~[groovy-2.5.10.jar:2.5.10 clouddriver at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127) ~[groovy-2.5.10.jar:2.5.10 clouddriver at com.netflix.spinnaker.clouddriver.Main.main(Main.groovy:78) ~[clouddriver-web-GCSFIX.jar:na clouddriver Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'regionScopedProviderFactory': Unsatisfied dependency expressed through field 'clusterProviders'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'amazonClusterProvider' defined in URL [jar:file:/opt/clouddriver/lib/clouddriver-aws-GCSFIX.jar!/com/netflix/spinnaker/clouddriver/aws/provider/view/AmazonClusterProvider.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cacheView' defined in class path resource [com/netflix/spinnaker/clouddriver/cache/CacheConfig.class]: Unsatisfied dependency expressed through method 'cacheView' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'catsModule' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'catsModule' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'netflixAmazonCredentials' defined in class path resource [com/netflix/spinnaker/clouddriver/aws/security/AmazonCredentialsInitializer.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.List: Factory method 'netflixAmazonCredentials' threw exception; nested exception is com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:643) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:130) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1422) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1304) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1224) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver ... 22 common frames omitted clouddriver Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'amazonClusterProvider' defined in URL [jar:file:/opt/clouddriver/lib/clouddriver-aws-GCSFIX.jar!/com/netflix/spinnaker/clouddriver/aws/provider/view/AmazonClusterProvider.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cacheView' defined in class path resource [com/netflix/spinnaker/clouddriver/cache/CacheConfig.class]: Unsatisfied dependency expressed through method 'cacheView' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'catsModule' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'catsModule' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'netflixAmazonCredentials' defined in class path resource [com/netflix/spinnaker/clouddriver/aws/security/AmazonCredentialsInitializer.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.List: Factory method 'netflixAmazonCredentials' threw exception; nested exception is com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:797) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:227) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1358) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1204) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1522) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1486) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1375) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1262) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1224) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver ... 35 common frames omitted clouddriver Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cacheView' defined in class path resource [com/netflix/spinnaker/clouddriver/cache/CacheConfig.class]: Unsatisfied dependency expressed through method 'cacheView' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'catsModule' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'catsModule' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'netflixAmazonCredentials' defined in class path resource [com/netflix/spinnaker/clouddriver/aws/security/AmazonCredentialsInitializer.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.List: Factory method 'netflixAmazonCredentials' threw exception; nested exception is com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:797) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:538) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1304) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1224) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:884) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver ... 51 common frames omitted clouddriver Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'catsModule' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'catsModule' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'netflixAmazonCredentials' defined in class path resource [com/netflix/spinnaker/clouddriver/aws/security/AmazonCredentialsInitializer.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.List: Factory method 'netflixAmazonCredentials' threw exception; nested exception is com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:797) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:538) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1304) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1224) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:884) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver ... 65 common frames omitted clouddriver Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'netflixAmazonCredentials' defined in class path resource [com/netflix/spinnaker/clouddriver/aws/security/AmazonCredentialsInitializer.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.List: Factory method 'netflixAmazonCredentials' threw exception; nested exception is com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:655) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:635) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:557) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1522) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1486) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1375) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1262) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1224) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:884) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver ... 79 common frames omitted clouddriver Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.util.List: Factory method 'netflixAmazonCredentials' threw exception; nested exception is com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:650) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver ... 98 common frames omitted clouddriver Caused by: com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: f0e6c419-049b-4759-9a21-e76d6145d35a) clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1799) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1383) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1359) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524) ~[aws-java-sdk-core-1.11.764.jar:na clouddriver at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:25045) ~[aws-java-sdk-ec2-1.11.764.jar:na clouddriver at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:25012) ~[aws-java-sdk-ec2-1.11.764.jar:na clouddriver at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:25001) ~[aws-java-sdk-ec2-1.11.764.jar:na clouddriver at com.amazonaws.services.ec2.AmazonEC2Client.executeDescribeRegions(AmazonEC2Client.java:13170) ~[aws-java-sdk-ec2-1.11.764.jar:na clouddriver at com.amazonaws.services.ec2.AmazonEC2Client.describeRegions(AmazonEC2Client.java:13141) ~[aws-java-sdk-ec2-1.11.764.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.DefaultAWSAccountInfoLookup.listRegions(DefaultAWSAccountInfoLookup.java:116) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.config.CredentialsLoader$1.get(CredentialsLoader.java:110) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.config.CredentialsLoader$1.get(CredentialsLoader.java:94) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.config.CredentialsLoader$Lazy.get(CredentialsLoader.java:324) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.config.CredentialsLoader.initRegions(CredentialsLoader.java:145) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.config.CredentialsLoader.load(CredentialsLoader.java:249) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.config.CredentialsLoader$load.call(Unknown Source) ~[na:na clouddriver at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-2.5.10.jar:2.5.10 clouddriver at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) ~[groovy-2.5.10.jar:2.5.10 clouddriver at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127) ~[groovy-2.5.10.jar:2.5.10 clouddriver at com.netflix.spinnaker.clouddriver.aws.security.DefaultAmazonAccountsSynchronizer.synchronize(DefaultAmazonAccountsSynchronizer.groovy:46) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.DefaultAmazonAccountsSynchronizer$synchronize.call(Unknown Source) ~[na:na clouddriver at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-2.5.10.jar:2.5.10 clouddriver at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) ~[groovy-2.5.10.jar:2.5.10 clouddriver at com.netflix.spinnaker.clouddriver.aws.security.AmazonCredentialsInitializer.netflixAmazonCredentials(AmazonCredentialsInitializer.groovy:71) ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.AmazonCredentialsInitializer$$EnhancerBySpringCGLIB$$a647d3fe.CGLIB$netflixAmazonCredentials$5() ~[clouddriver-aws-GCSFIX.jar:na clouddriver at com.netflix.spinnaker.clouddriver.aws.security.AmazonCredentialsInitializer$$EnhancerBySpringCGLIB$$a647d3fe$$FastClassBySpringCGLIB$$32d70a7d.invoke() ~[clouddriver-aws-GCSFIX.jar:na clouddriver at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver at com.netflix.spinnaker.clouddriver.aws.security.AmazonCredentialsInitializer$$EnhancerBySpringCGLIB$$a647d3fe.netflixAmazonCredentials() ~[clouddriver-aws-GCSFIX.jar:na clouddriver at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na clouddriver at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na clouddriver at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na clouddriver at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na clouddriver at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE clouddriver ... 99 common frames omitted clouddriver stream closed

Cause​

Looking at the following from the Stacktrace:com.netflix.spinnaker.clouddriver.aws.security.DefaultAWSAccountInfoLookupThe following issue is being tracked on Github:https://github.com/spinnaker/clouddriver/blob/master/clouddriver-aws/src/main/groovy/com/netflix/spinnaker/clouddriver/aws/security/DefaultAWSAccountInfoLookup.java#L116When looking at the listRegions call, it's being referenced very early. The documents here outline more about the issue:https://docs.armory.io/docs/spinnaker-install-admin-guides/add-aws-account-iam/#instance-role-part-3-creating-a-managing-account-iam-policy-in-your-primary-aws-accountThe above doc was used to see what roles the primary AWS account had to have.Summary:The cross account (worker node) has an 'assume role' assigned which gives the correct permissions but before it can assume the role, we saw that it tries to get the access to describe region access, and availability zones. It does not have permissions to these without the assume role.