AWS CDK Codepipeline build action with golang app with multiple commands targeting multiple Lambdas
Question:
So my team and I have been happily pushing code to our pipeline for a while now but today we’ve been getting errors because because the asset the pipeline builds is too large to deploy to a lambda (which happens to be 250mb)
The reason I think we’re getting this error is because in the buildspec for the build action of the pipeline (written in Python as part of a CDK app), I run
f"go build -o ./build -ldflags="-X 'main.CommitID={commit} -s -w'" ./..." which successfully builds all the binaries to the build folder but then the artefact is zipped up and encrypted before moving onto the deploy stage and deployed to each lambda. The storage here is somewhat wasteful because it deploys all binaries to all lambdas and the handler just picks the right binary to run.
However, I only want a single binary file deployed to the lambda but the deploy stage is actually a cdk synth followed by a CloudFormationCreateUpdateStackAction and in order to pass the artefact to the lambda I use Code.from_cfn_parameters and pass the params in as overrides in the final deploy stage.
My question is, given the build pipeline creates individual binaries for each cmd in the build, how do I then extract the binary from the artefact to pass as the parameter value?
I’m currently passing the whole artefact as a parameter to the synth stage and I don’t seem to be able to select a single binary from the artefact as the code property for the lambda.
[edit]
I’ve been using this source for my documentation as well as scouring the internet for others whom might have had similar issues but I haven’t found anything that works yet.
Answers:
In the end, I added an S3 deploy stage right after the build stage to push the zips to a bucket and then updated our lambda stack to use assets by their name concatenated with the commit id from this bucket. I needed to add the commit id as the Lambdas wouldn’t deploy with either versioning enabled or if I didn’t change the object name in the Lambda definition.
So my team and I have been happily pushing code to our pipeline for a while now but today we’ve been getting errors because because the asset the pipeline builds is too large to deploy to a lambda (which happens to be 250mb)
The reason I think we’re getting this error is because in the buildspec for the build action of the pipeline (written in Python as part of a CDK app), I run
f"go build -o ./build -ldflags="-X 'main.CommitID={commit} -s -w'" ./..." which successfully builds all the binaries to the build folder but then the artefact is zipped up and encrypted before moving onto the deploy stage and deployed to each lambda. The storage here is somewhat wasteful because it deploys all binaries to all lambdas and the handler just picks the right binary to run.
However, I only want a single binary file deployed to the lambda but the deploy stage is actually a cdk synth followed by a CloudFormationCreateUpdateStackAction and in order to pass the artefact to the lambda I use Code.from_cfn_parameters and pass the params in as overrides in the final deploy stage.
My question is, given the build pipeline creates individual binaries for each cmd in the build, how do I then extract the binary from the artefact to pass as the parameter value?
I’m currently passing the whole artefact as a parameter to the synth stage and I don’t seem to be able to select a single binary from the artefact as the code property for the lambda.
[edit]
I’ve been using this source for my documentation as well as scouring the internet for others whom might have had similar issues but I haven’t found anything that works yet.
In the end, I added an S3 deploy stage right after the build stage to push the zips to a bucket and then updated our lambda stack to use assets by their name concatenated with the commit id from this bucket. I needed to add the commit id as the Lambdas wouldn’t deploy with either versioning enabled or if I didn’t change the object name in the Lambda definition.