Cloud-Based Computer Vision System for Grocery Shelf Management

Project Overview

Our team was engaged by a client in the food retail industry to develop a cloud-based computer vision system for automating product detection, classification, and comparison on grocery shelves. The goal was to streamline the process of checking planogram execution (a store's intended product layout) against realogram data (actual product placement on shelves), minimizing manual work and errors in inventory management. The system was designed to process photos captured via a mobile app, analyze the images, and identify out-of-shelf products.

Technologies Used

We utilized a variety of cutting-edge tools and technologies:

• PyTorch & PyTorchVideo: For developing and fine-tuning deep learning models for object segmentation and product classification.
• Amazon EC2 & S3: For scalable cloud infrastructure, enabling the storage and real-time processing of large volumes of image data.
• SageMaker Neo: To optimize the deep learning models for deployment and efficient GPU utilization in an online environment.

This technology stack allowed us to create a system capable of real-time analysis and comparison of planograms with high accuracy.

Challenges

The project involved several key challenges:

• Fine-tuning deep learning models: We needed to achieve precise object segmentation and high accuracy in classifying a wide variety of products on shelves.
• Custom classification models: These models required substantial development, including the creation of datasets for both segmentation and classification tasks.
• Comparison algorithms: Developing algorithms that could reliably compare planograms (POGs) with realograms in varying lighting conditions and camera angles was crucial.
• Efficient GPU Utilization: The system had to be designed for online use, leveraging cloud-based GPUs efficiently to ensure scalability and cost-effectiveness.

Results

Our team successfully delivered a high-accuracy, automated system that:

• Significantly improved planogram execution: The system automated the comparison of planograms with realograms, enabling the client to quickly identify discrepancies in product placement.
• Reduced manual work: The automation dramatically minimized the need for store managers to manually check shelf layouts, freeing up time for more strategic tasks.
• Improved retail operations: By increasing the accuracy of product placement analysis, the system helped optimize inventory management and improve store layout compliance.

Industry

Food Retail – with a focus on streamlining inventory and layout management through automation.

Competitor Differentiation

Our approach goes beyond just building a computer vision system; we ensure that our solutions are fully tailored to the client’s specific retail needs. By fine-tuning models and optimizing their performance, we not only deliver high-accuracy detection but also provide long-term efficiency in the client’s operations. The use of cloud-based technologies ensures the system can scale as the business grows, giving the client flexibility in managing future expansions.

Long-term Impact

The system we built continues to help the client dramatically reduce manual efforts in verifying shelf layouts and product placement. The automation of this process has allowed the client to allocate workloads more effectively across managers, resulting in enhanced operational efficiency.