Guide for setting up serverless-offline-sqs + serverless-lift + localstack sqs

[serg06]

Guide for setting up serverless-offline-sqs + serverless-lift + localstack sqs

I just spent the last few hours figuring this out, so I'll share my solution with you all.

Disclaimer: This guide is only for sls offline. It likely needs modification before deploying.

Disclaimer: You will have to modify serverless-offline-sqs as it doesn't work with lift by default.

Set up the SQS server

# Install Python and Docker beforehand

# Install CLI tools if you don't have them
pip install awscli  # `aws`
pip install awscli-local  # `awslocal`

# Start localstack
docker run -p 4566:4566 -d --restart unless-stopped localstack/localstack

Install the Node packages:

yarn add @aws-sdk/client-sqs
yarn add -D serverless-offline-sqs serverless-lift @types/aws-lambda

Set up your serverless.yaml

plugins:
  ...
  - serverless-lift
  - serverless-offline-sqs  # MUST BE ABOVE serverless-offline OR ELSE IT WON'T WORK
  - serverless-offline
  ...

constructs:
  myQueue:
    type: queue
    # fifo: true  # if you want fifo
    worker:
      handler: src/functions/myQueue.handler

provider:
  environment:
    ...
    MY_QUEUE_URL: ${construct:myQueue.queueUrl}  # You'll need this for deployments, but not for running locally

custom:
  serverless-offline-sqs:
    autoCreate: true
    region: us-east-2
    endpoint: http://localhost:4566
    apiVersion: '2012-11-05'
    accessKeyId: foobar  # localstack auth
    secretAccessKey: foobar  # localstack auth
    queueName: myQueue  # required for lift support, not supported by default but we'll fix that

Set up your publisher

import {SQSClient} from '@aws-sdk/client-sqs';
import {SendMessageCommand} from '@aws-sdk/client-sqs';

export const sqs = new SQSClient({
    region: 'us-east-2',
    endpoint: 'http://localhost:4566'
});

// Somewhere else in the code...
console.log('Publishing event...');
await sqs.send(new SendMessageCommand({
    MessageBody: '{"hello": "there"}',
    QueueUrl: 'myQueue'
}));

Set up your consumer

import {SQSBatchResponse, SQSEvent} from 'aws-lambda/trigger/sqs';
import {writeFileSync} from 'fs';

void import('source-map-support/register');  // Optional but recommended

async function server(event: SQSEvent, context: Record<string, unknown>): Promise<void | {} | SQSBatchResponse> {
    for (const record of event.Records) {
        console.log('Consuming event...');
        console.log({event: record.body});
    }
    return {};
}

module.exports.handler = server;

Patch serverless-offline-sqs:

This plugin has had issues with lift since forever:

• serverless-offline-sqs configuration issue #5
• [serverless-offline-sqs] Missing QueueName #200
• Logging Worker getlift/lift#109

In order to fix that, you need to apply a patch:

• chore(serverless-offline-sqs): allow passing queueName from config #211

Ideally you'd apply that patch in your postinstall script, but you can do it manually too.

Test it out

sls offline --verbose --useInProcess

[luismir15]

@serg06 this guide has been very helpful, so thank you for that. Also, I wanted to ask: how would you apply the patch in a postintall script?

[serg06]

@luismir15 This is my script

# scripts/patch-serverless-offline-sqs.sh

root='.'  # Project root
lib="${root}/node_modules/serverless-offline-sqs/src/sqs.js"

if grep -q patched "$lib"; then
  echo sqs.js already patched
else
  echo sqs.js is missing the patch, applying...
  sed -i "s|const sqsEvent = new SQSEventDefinition|// patched\n    const {queueName} = this.options;\n\n    if (queueName) rawSqsEventDefinition.queueName = queueName;\n\n    const sqsEvent = new SQSEventDefinition|" "$lib"
  echo done
fi

And my package.json has this:

"scripts": {
  "postinstall": "bash scripts/patch-serverless-offline-sqs.sh"
}

[luismir15]

thanks @serg06! I wanted to point out that I was getting an error while running the script

sed: 1: "node_modules/serverless ...": extra characters at the end of n command

I'm in macos so the only work around was to add an empty string in the sed command
sed -i "" "s|const sqsEvent = new SQSEventDefinition|// patched\n const {queueName} = this.options;\n\n if (queueName) rawSqsEventDefinition.queueName = queueName;\n\n const sqsEvent = new SQSEventDefinition|" "$lib"

found the solution here: https://myshittycode.com/2014/07/24/os-x-sed-extra-characters-at-the-end-of-l-command-error/

[tim-grcia]

Thank you for this. How about if there's multiple queue? How to setup the queueName in custom serverless-offline-sqs?