AWS Lambda Manager

1 Feb

I’ve been working on a project which is based on Amazon Lambda service.
We had a few lambda functions, each of them is doing only a part of a bigger job.
The problem with lambda function is that you don’t have control over the function throttling.
It’s not just for preventing hitting the concurrent execution limits of amazon, but for preventing overloading a shared resource.

So I created a “Lambda Manager”. It’s a lambda function which is responsible for receiving “tasks” (data) and execute the lambda “worker” function if available.
If a worker is not available, it will queue the task on a redis server.

There’s also a “Lambda Agent” which is used in the worker functions for notifying the manager when a job is done.

You can have a look here:

Leave a Reply

Your email address will not be published. Required fields are marked *