Support arbitrary shell commands at build time

[jeffsw]

We can work around a current limitation (inability to install things with e.g. --platform) by allowing arbitrary shell commands to be invoked at different stages. Some environment variables will need to be prepared for this to work, and the arbitrary commands maybe should be arbitrary multi-line shell scripts, when desired.

Environment vars needed

• LAMBDA_SRC_DIR
• LAMBDA_TMP_DIR
• LAMBDA_ZIP_FILE
• LAYER_TMP_DIR
• LAYER_ZIP_FILE

Execution stages

• PRE_ZIP files added to the LAMBDA_TMP_DIR during this stage would be subject to omit behavior
• POST_ZIP files may be added to the LAMBDA_ZIP_FILE during this stage and they will be included in hash calc
• POST_HASH files may be added to the LAMBDA_ZIP_FILE after content hash calculation

Multiline vs single-line

If a script value (string) is multiple lines, it should be written out to a file and the file executed with subprocess.run(). This way, the script could be a shell script, but it could also be Python or something else.

If it's a single line, we just run it with subprocess.run().

Example configuration ❓ 💡

[lambda_zip]
script_pre_zip = """
#!/bin/bash
pip install \
  --no-deps \
  --python-version 3.9 \
  --platform manylinux2014_x86_64 \
  --only-binary=:psycopg2:
  --target=${LAMBDA_TMP_DIR} \
  psycopg2-binary
"""