First Commit

Author: nemborkar

.gitignore

@@ -0,0 +1 @@
+*.zip

Dockerfile

@@ -0,0 +1,7 @@
+# amazonlinux:2023 has python 3.9 + venv
+FROM amazonlinux:2023
+
+# We will need pip3 and zip
+RUN yum -y update && yum -y install \
+python3-pip \
+zip

README.md

@@ -0,0 +1,29 @@
+# lambda-layers
+To run Python functions in AWS Lambda with extra imported packages, we need to have these packages in Lambda Layers  
+However making the layers locally uses your local environment's Python Runtime, OS and CPU Architecture.  
+So the zip files we upload to the Lambda layers can be very hit or miss (mostly miss)  
+
+This repo tries to solve that problem with the following steps,
+1. Create an Amazon Linux container
+2. Install the required packages with requirements.txt and pip3
+3. Generate the zip file INSIDE the container
+4. Output the zip file to your local machine
+
+# Usage
+1. update requirements.txt with the packages you want in the layer
+- Use specific versions, if possible.  eg: ```requests==2.31.0``` instead of ```requests```  
+- [Syntax for requirements.txt](https://pip.pypa.io/en/stable/reference/requirements-file-format/)
+- [Introduction to requirements.txt](https://www.freecodecamp.org/news/python-requirementstxt-explained/)  
+
+2. run the runner.sh script and it will generate the python.zip file  
+```./runner.sh```
+
+3. upload this python.zip file to Lambda layers and use with,  
+Python Runtime : Python 3.9  
+Architecture : x86_64  
+
+# Change Python Runtime or Architecture
+- If you wish to use a different Python runtime like 3.11, 3.12 etc, you can import a newer [Amazon Linux base image](https://hub.docker.com/_/amazonlinux) in the Dockerfile.  
+This generally comes with a new stable version of Python  
+- You can also install a [specific version of Python](https://www.python.org/downloads/) by updating the ```yum install``` lines of the Dockerfile
+- arm64 has known unresolved issues on Lambda with Python as of 20240322, so I recommend against it for now

docker_install.sh

@@ -0,0 +1,6 @@
+python3 -m venv python
+source python/bin/activate
+
+pip3 install -r requirements.txt -t python/lib/python3.9/site-packages
+
+zip -r9 python.zip python

python.zip

@@ -0,0 +1 @@
+requests

requirements.txt

@@ -0,0 +1,11 @@
+container_name=layer_builder_docker
+docker_image=layer_builder_image
+
+docker build -t $docker_image .
+docker run -td --name=$container_name $docker_image
+docker cp ./requirements.txt $container_name:/
+
+docker exec -i $container_name /bin/bash < ./docker_install.sh
+docker cp $container_name:/python.zip python.zip
+docker stop $container_name
+docker rm $container_name

runner.sh

@@ -0,0 +1,11 @@
+container_name=layer_builder_podman
+podman_image=layer_builder_image
+
+podman build -t $podman_image .
+podman run -td --name=$container_name $podman_image
+podman cp ./requirements.txt $container_name:/
+
+podman exec -i $container_name /bin/bash < ./docker_install.sh
+podman cp $container_name:/python.zip python.zip
+podman stop $container_name
+podman rm $container_name