aws-lambda-python: Layer build fails #22012
Assignees
Labels
@aws-cdk/aws-lambda-python
bug
This issue is a bug.
effort/small
Small work item – less than a day of effort
good first issue
Related to contributions. See CONTRIBUTING.md
p1
Comments
varju
added
bug
|
I was actually unable to reproduce this @varju, I was able to synth a layer which contains the dependency you specified on the latest version with |
peterwoodworth
added
response-requested
|
Sorry, I shouldn't have been lazy, and should have created a full example of how to reproduce. This repo shows a minimal project that has the problem: https://github.com/varju/cdk-22012-example Full output during the build: Possibly of note: I'm on an M1 Mac. |
github-actions
bot
removed
the
response-requested
|
I've been able to reproduce this @varju, thanks for providing a repo to help test this. I've found that if you add these two lines to your construct, synth will fail in 2.41.0 whereas it will succeed in 2.40.0 @corymhall could you take a look at this? |
peterwoodworth
added
p1
effort/small
|
It looks like it is somehow related to running on ARM because I can't reproduce when using X86. But we should be able to fix it by moving to line 24. |
peterwoodworth
added
the
good first issue
Merged
3 tasks
mergify bot
pushed a commit
that referenced
this issue
Oct 12, 2022
…ilding container image (#22398) fixes #22012 There were reports that Dockerfile RUN commands are done out of order when building the container image on Graviton. I combined all the separate RUN commands into a single command and using the shell to explicitly enforce the sequence. ---- ### All Submissions: * [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) ### Adding new Unconventional Dependencies: * [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies) ### New Features * [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)? * [] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)? *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
4 tasks
mergify bot
pushed a commit
that referenced
this issue
Oct 19, 2022
…ll (#22512) This is another attempt at fixing #22012. #22398 intended to fix it, but the issue still occurs when building on arm64 machines with both `compatible_runtimes=[lambdas.Runtime.PYTHON_3_9]` and `compatible_architectures=[lambdas.Architecture.ARM_64]` The core problem is that the build leaves root-owned files under `/tmp/pip-cache`, which then cause permission problems when subsequent layers calling `pip install` are unable to create temporary directories. Example error: ``` WARNING: Building wheel for alembic-utils failed: [Errno 13] Permission denied: '/tmp/pip-cache/wheels/5c' ``` Expanding on the idea within #22398, this PR adds one more step to the chain of operations, removing any temporary files from the cache that are no longer needed (because the packages are now installed), and ensuring that subsequent layers are able to create temporary files inside the cache directories. Comparing the contents of the Docker image before and after this change... Before: ``` $ ls -aFl /tmp/*cache /tmp/pip-cache: total 16 drwxrwxrwx 4 root root 4096 Oct 15 00:57 ./ drwxrwxrwt 1 root root 4096 Oct 15 00:58 ../ drwxr-xr-x 18 root root 4096 Oct 15 00:57 http/ drwxr-xr-x 2 root root 4096 Oct 15 00:57 selfcheck/ /tmp/poetry-cache: total 8 drwxrwxrwx 2 root root 4096 Oct 15 00:57 ./ drwxrwxrwt 1 root root 4096 Oct 15 00:58 ../ ``` After: ``` $ ls -aFl /tmp/*cache /tmp/pip-cache: total 8 drwxrwxrwx 2 root root 4096 Oct 15 01:00 ./ drwxrwxrwt 1 root root 4096 Oct 15 01:00 ../ /tmp/poetry-cache: total 8 drwxrwxrwx 2 root root 4096 Oct 15 00:59 ./ drwxrwxrwt 1 root root 4096 Oct 15 01:00 ../ ``` ---- ### All Submissions: * [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) ### Adding new Unconventional Dependencies: * [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies) ### New Features * [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)? * [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)? *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
mrgrain
pushed a commit
to mrgrain/aws-cdk
that referenced
this issue
Oct 24, 2022
…ll (aws#22512) This is another attempt at fixing aws#22012. aws#22398 intended to fix it, but the issue still occurs when building on arm64 machines with both `compatible_runtimes=[lambdas.Runtime.PYTHON_3_9]` and `compatible_architectures=[lambdas.Architecture.ARM_64]` The core problem is that the build leaves root-owned files under `/tmp/pip-cache`, which then cause permission problems when subsequent layers calling `pip install` are unable to create temporary directories. Example error: ``` WARNING: Building wheel for alembic-utils failed: [Errno 13] Permission denied: '/tmp/pip-cache/wheels/5c' ``` Expanding on the idea within aws#22398, this PR adds one more step to the chain of operations, removing any temporary files from the cache that are no longer needed (because the packages are now installed), and ensuring that subsequent layers are able to create temporary files inside the cache directories. Comparing the contents of the Docker image before and after this change... Before: ``` $ ls -aFl /tmp/*cache /tmp/pip-cache: total 16 drwxrwxrwx 4 root root 4096 Oct 15 00:57 ./ drwxrwxrwt 1 root root 4096 Oct 15 00:58 ../ drwxr-xr-x 18 root root 4096 Oct 15 00:57 http/ drwxr-xr-x 2 root root 4096 Oct 15 00:57 selfcheck/ /tmp/poetry-cache: total 8 drwxrwxrwx 2 root root 4096 Oct 15 00:57 ./ drwxrwxrwt 1 root root 4096 Oct 15 00:58 ../ ``` After: ``` $ ls -aFl /tmp/*cache /tmp/pip-cache: total 8 drwxrwxrwx 2 root root 4096 Oct 15 01:00 ./ drwxrwxrwt 1 root root 4096 Oct 15 01:00 ../ /tmp/poetry-cache: total 8 drwxrwxrwx 2 root root 4096 Oct 15 00:59 ./ drwxrwxrwt 1 root root 4096 Oct 15 01:00 ../ ``` ---- ### All Submissions: * [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) ### Adding new Unconventional Dependencies: * [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies) ### New Features * [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)? * [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)? *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
Not working please open it up I'm facing this issue on latest versions: "@aws-cdk/aws-lambda-python-alpha": "^2.100.0-alpha.0",
"aws-cdk-lib": "2.100.0", |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
A Python layer containing a requirements.txt file no longer builds as of CDK 2.41.0.
Expected Behavior
Running
cdk synthshould have created a Docker image that can be pushed to AWS.Current Behavior
Sample error:
In case it's useful, here's the full Docker command I snagged from the environment while
cdk synthwas running:Reproduction Steps
Create a PythonLayerVersion containing a
requirements.txtfile with this package:Possible Solution
Reviewing the changes in #21945, I believe the problem is we're leaving a
/tmp/pip-cachedirectory with root-owned files in it. Whencdk synthinvokesdocker runto package the asset, it includes a-u <my-uid>parameter, which means it cannot write to the existing/tmp/pip-cache/wheeldirectory.Additional Information/Context
No response
CDK CLI Version
2.41.0 (build 6ad48a3)
Framework Version
No response
Node.js Version
v16.16.0
OS
macOS 12.5.1
Language
Python
Language Version
No response
Other information
No response
The text was updated successfully, but these errors were encountered: