Where exactly is the card in this photo? Image segmentation model inside a maxed-out lambda container
A crooked phone photo goes in, a clean straight card image comes out - no GPU, and nothing running when nobody uploads. It runs on the biggest lambda AWS sells, and the biggest is not the same as enough. Introduction This is the 3rd article in the series about the image processing pipeline. In two previous articles we went over the whole image processing pipeline, which process AWS Builder Cards…
In the latest article of the series on AWS image processing pipeline, the focus is on a vision segmentation model that runs inside a Lambda container. This model's purpose is to process the filtered image from the previous lambda function, remove the background, straighten it, and crop it to accurately locate the card in the photo.
The challenge lies in running the segmentation model on CPU, inside a Lambda container, as it needs to perform pixel-by-pixel analysis, labeling each pixel as card, background, etc. The CPU and memory requirements, along with the model weights, need to be determined to ensure the model can run efficiently in the container environment.
The chosen model for this task is BiRefNet-General-Lite, which is loaded through the rembg library. This lightweight variant of the BiRefNet model was chosen to address the memory constraints of running it in a Lambda container. The memory usage of the model was a significant concern, as previous tests with the larger, more powerful BiRefNet-General model resulted in out-of-memory errors.
To mitigate this issue, the team decided to use the BiRefNet-General-Lite model, which required significantly less memory while still achieving comparable results. Additionally, the team implemented image resizing as a preprocessing step for uploads that exceed 2048 pixels in length, further reducing the memory footprint of the image processing pipeline.
In summary, the key points of this article are the need for an image segmentation model to process AWS Builder Cards, the choice of the BiRefNet-General-Lite model due to its lightweight nature and compatibility with the Lambda container environment, and the implementation of memory optimization techniques such as reducing container memory allocation and resizing images to minimize the model's memory consumption.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — it may contain errors, so check the original before relying on it.
