Use maven assembly plugin instead of shade plugin

[eduramiba]

• Framework version: 1.0
• Implementations: Jersey / Spring / Spring Boot / Spark

Since using maven shade plugin to create a fat jar is known to be problematic because it can overwrite files by mixing jars (https://product.hubspot.com/blog/the-fault-in-our-jars-why-we-stopped-building-fat-jars), we currently use with success a safer way to deploy lambdas with dependencies: a single zip with all jars in a lib folder (as documented in https://docs.aws.amazon.com/en_en/lambda/latest/dg/create-deployment-pkg-zip-java.html)

Example code of how we do it in maven:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <configuration>
            <finalName>${theapp.brandingToken}-${project.version}</finalName>
            <archive>
                <manifest>
                    <addClasspath>true</addClasspath>
                    <classpathPrefix>lib/</classpathPrefix>
                    <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
                    <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
                </manifest>
            </archive>
        </configuration>
    </plugin>
    
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
            <execution>
                <id>copy-dependencies</id>
                <phase>package</phase>
                <goals>
                    <goal>copy-dependencies</goal>
                </goals>
                <configuration>
                    <outputDirectory>${project.build.directory}/lib</outputDirectory>
                    <includeScope>runtime</includeScope>
                </configuration>
            </execution>
        </executions>
    </plugin>
    
    <!-- Assembler to produce ditribution zip -->
    <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <executions>
            <execution>
                <id>zip-assembly</id>
                <phase>package</phase>
                <goals>
                    <goal>single</goal>
                </goals>
                <configuration>
                    <finalName>${theapp.brandingToken}-${project.version}</finalName>
                    <descriptors>
                        <descriptor>src/assembly/bin.xml</descriptor>
                    </descriptors>
                    <attach>false</attach>
                </configuration>
            </execution>
        </executions>
    </plugin>

And bin.xml file:

<assembly>
    <id>aws-lambda-package</id>
    <formats>
        <format>zip</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <fileSets>
        <fileSet>
            <directory>${project.build.directory}${file.separator}lib</directory>
            <outputDirectory>lib</outputDirectory>
            <useDefaultExcludes>true</useDefaultExcludes>
        </fileSet>
    </fileSets>
    <files>
        <file>
            <source>${project.build.directory}${file.separator}${theapp.brandingToken}-${project.version}.jar</source>
            <outputDirectory>lib</outputDirectory>
        </file>
    </files>
</assembly>

Thanks,
Eduardo

[sapessi]

Thanks for the feedback @eduramiba - I've been wondering whether it's time to build a Lambda-specific maven plugin to package an application and slim down the output jar as much as possible. I'll keep this open and think it through. I'll definitely look into adding the assembly plugin as an alternative to our samples and archetypes.

[humank]

Good to know this way to pack it. Thanks @eduramiba !

[Juchar]

@eduramiba Thank you! How would you exclude the embedded tomcat in this case?

[eduramiba]

@Juchar I guess you can use the goal options, https://maven.apache.org/plugins/maven-dependency-plugin/copy-dependencies-mojo.html

But if your dependency has provided scope, it will not be copied as includeScope is runtime.

[eduramiba]

Also, the jar manifest entries are not actually necessary in lambda runtime since it will load all jars in lib folder.

[Juchar]

@eduramiba Ok, thank you.

I changed you approach a little bit and it is also working (you only need to configure the assembly plugin this way):

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <executions>
    <execution>
    <id>zip-assembly</id>
    <phase>package</phase>
    <goals>
      <goal>single</goal>
    </goals>
    <configuration>
      <finalName>${project.artifactId}</finalName>
      <descriptors>
        <descriptor>src/assembly/bin.xml</descriptor>
      </descriptors>
      <archive>
          <manifest>
          <addClasspath>true</addClasspath>
          <classpathPrefix>lib/</classpathPrefix>
          <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
          <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
        </manifest>
      </archive>
      <attach>false</attach>
    </configuration>
    </execution>
  </executions>
</plugin>

And the bin.xml:

<assembly>
  <id>aws-lambda-package</id>
  <formats>
    <format>zip</format>
  </formats>
  <includeBaseDirectory>false</includeBaseDirectory>
  <fileSets>
    <fileSet>
      <directory>${project.build.directory}${file.separator}classes</directory>
      <outputDirectory/>
    </fileSet>
  </fileSets>
  <dependencySets>
    <dependencySet>
      <excludes>
        <exclude>org.springframework.boot:spring-boot-devtools</exclude>
        <exclude>org.apache.tomcat.embed:*</exclude>
      </excludes>
      <useProjectArtifact>false</useProjectArtifact>
      <outputDirectory>lib</outputDirectory>
    </dependencySet>
  </dependencySets>
</assembly>

[eugenevd]

I've tried both the suggested examples provided.
The first results in my own code (Handler) to be added to it's own jar, and then along with the other dependecy jars, is put into "lib/" directly inside the zip.
lib/myHandler.jar
lib/dep1.jar

The second example puts all the dependencies in lib/ again, but at the root of the zip the directory structure;
eg:
com/company/Handler.class
lib/dep1.jar

Either way, I get
Class not found: com.company.Handler: java.lang.ClassNotFoundException

I've tried both ways to define the handler:

com.company.Handler
com.company.Handler::handleRequest

What am I missing?

Background
Using the maven shade plugin has been the only way that I could get the deployment to work, but after adding the bouncycastle dependency, I got error:
Error creating shaded jar: Invalid signature file digest for Manifest main attributes

There are suggestions to remove the signatures:

<configuration>
    <filters>
        <filter>
            <artifact>*:*</artifact>
            <excludes>
                <exclude>META-INF/*.SF</exclude>
                <exclude>META-INF/*.DSA</exclude>
                <exclude>META-INF/*.RSA</exclude>
            </excludes>
        </filter>
    </filters>
    <!-- Additional configuration. -->
</configuration>

but surely these will be rejected.
So my conclusion was to find a way to package all the dependency jars as is, hence the suggestions by earlier posts here.

[sapessi]

Hi @eugenevd - I'm on the road right now but I'll try to replicate your issue with the Shade plugin and see if we can find a fix on Thursday. As for the assembly plugin questions, I'll let @eduramiba and @Juchar chime in since they clearly know more than I do on the topic.

[Juchar]

Hi @eugenevd - the assembly configuration I mention above will produce a zip file following these conventions.

For me it worked out of the box. Maybe you could show us the sam.yml, possibly your error is somewhere else?

This is the relevant part from mine:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: My Description
Resources:
  MyServiceFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: com.my.function.StreamLambdaHandler::handleRequest
      Runtime: java8
      CodeUri: target/my-function-aws-lambda-package.zip
      MemorySize: 768
      Policies: AWSLambdaBasicExecutionRole
      Timeout: 30
      Events:
        GetResource:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: GET

Outputs:
  MyFunctionAPI:
    Description: URL for application
    Value: !Sub 'https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/entity'
    Export:
      Name: MyFunctionAPI