I was recently trying to list number of files from S3
, and read those files one by one in a service. For doing this I was using spring-boot 2.0.0.RELEASE
, spring-cloud-starter-aws (Edgware.SR2)
library for auto-configuring
my credentials.
I configured my credentials by following the official documentation. You can give that a read in your free time. TLDR; all we have to do if we are using spring-boot
to auto-configure
your Amazon S3
client is to populate the following properties in your applications.properties
or application.yaml
file.
cloud.aws.region.static cloud.aws.credentials.secretKey cloud.aws.credentials.accessKey
Once you configure these properties, spring-boot
should auto-configure
aws client for you. But this did not happen for me. When I have these properties filled out, and run the project, I got the following exception.
Caused by: java.lang.NoClassDefFoundError: com/amazonaws/AmazonClientException at org.springframework.cloud.aws.context.annotation.OnAwsCloudEnvironmentCondition.matches(OnAwsCloudEnvironmentCondition.java:37) ~[spring-cloud-aws-context-1.2.2.RELEASE.jar:1.2.2.RELEASE] at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:109) ~[spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE] at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:217) ~[spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE] at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:606) ~[spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE] ... 14 common frames omitted Caused by: java.lang.ClassNotFoundException: com.amazonaws.AmazonClientException at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_152] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_152] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) ~[na:1.8.0_152] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_152] ... 18 common frames omitted
For some reason, spring-boot
latest version does not go hand in hand with spring-cloud-dependencies
. I tried downgrading spring-boot
version and use 1.5.9.RELEASE
but then didn’t end up solving the problem. I found out that the problem was that the amazon-java-aws-core
library had an upgrade and spring does not handle that well enough, which causes the application to fail miserably at runtime. All I had to do was to downgrade the amazon-java-aws-core
library.
spring-cloud-starter-aws (Edgware.SR2)
uses amazon-java-aws-core version 1.11.125
which was causing the AmazonClientException
. I had to change the following to my pom file.
<!-- Exclude aws-java-sdk-core --> | |
<dependency> | |
<groupId>org.springframework.cloud</groupId> | |
<artifactId>spring-cloud-starter-aws</artifactId> | |
<exclusions> | |
<exclusion> | |
<groupId>com.amazonaws</groupId> | |
<artifactId>aws-java-sdk-core</artifactId> | |
</exclusion> | |
</exclusions> | |
</dependency> | |
<!-- Include the right version of the dependency --> | |
<dependency> | |
<groupId>com.amazonaws</groupId> | |
<artifactId>aws-java-sdk-core</artifactId> | |
<version>1.11.106</version> | |
</dependency> |
AmazonClientException
and enables spring-cloud-starter-aws
to auto-configure the AmazonS3
client.
It is as simple as autowiring
your ResourcePatternResolver
bean and using it to list all the objects inside the bucket.
@Autowired private ResourcePatternResolver resourcePatternResolver; public List listS3Files(String bucketName) throws IOException { final Resource[] resources = resourcePatternResolver .getResources("s3://" + bucketName + "/*.xml"); return Arrays.stream(resources).map( resource -> { try { return resource.getURL().toExternalForm(); } catch (IOException e) { throw new RuntimeException(e); } } ).collect(Collectors.toList()); }
This code example as a project is pushed to my GitHub. You can find the project here.