Cloud-Based Computer Vision System for Grocery Shelf Management

Project Overview

Our team partnered with a client in the food retail industry to develop a cloud-based computer vision system that automates product detection, classification, and comparison on grocery shelves. This system was designed to minimize manual work, reduce operational costs, and address critical issues like out-of-shelf products (empty shelf spaces). By leveraging this solution, the client could streamline the process of checking planogram execution (a store's intended product layout) against realogram data (actual product placement on shelves). Key outcomes included faster identification of inventory issues, automatic task generation for store managers, and increased efficiency in shelf management processes.

Technologies Used

We employed cutting-edge tools to ensure accuracy, scalability, and cost-effectiveness:

• PyTorch & PyTorchVideo: For training deep learning models capable of precise object detection, segmentation, and classification.
• Amazon EC2 & S3: To enable scalable cloud infrastructure for storing and processing large image datasets in real time.
• SageMaker Neo: To optimize models for efficient deployment, ensuring smooth operation on cloud GPUs and edge devices.
• Edge Device Compatibility: We adapted the models to run on edge devices, reducing operational costs by minimizing reliance on cloud processing.

Challenges and Solutions

The project involved several key challenges:

• Adapting to Varying Devices
  • Challenge: Ensuring consistent performance across different camera devices with varying resolutions and quality.
  • Solution: Developed a training tool that integrates human-in-the-loop feedback, allowing the system to retrain itself for new data and devices seamlessly.
• Reducing Empty Shelf Time
  • Challenge: Identifying out-of-stock or misplaced products promptly to prevent lost sales opportunities.
  • Solution: Automated detection reduced shelf empty time by 20%, raising gross income for specific product categories by 1-3%.
• Model Scalability and Cost Optimization
  • Challenge: Deploying models in a cost-efficient manner while maintaining high accuracy.
  • Solution: Enabled edge device compatibility, reducing cloud infrastructure costs.
• Securing Client Data
  • Challenge: Ensuring secure data handling within a cloud-based system.
  • Solution: Implemented secure VPC architecture and robust authentication (AUTH) mechanisms to safeguard sensitive information.

System Features

Our team successfully delivered a high-accuracy, automated system that:

• Mobile Application:
  Store managers and back-office teams could access real-time data, task notifications, and visual analytics on shelf performance.
  
  
• Training Tool:
  A user-friendly interface allowing the client to train models for new product categories or adapt to seasonal planograms.
  
• Dashboards:
  Centralized analytics for monitoring planogram compliance, out-of-shelf occurrences, and performance trends.

Results

• Reduced Manual Effort: Automation decreased the reliance on manual inspections, freeing up time for strategic tasks.
• Higher Operational Efficiency: Faster issue detection and resolution led to a 20% reduction in empty shelf time.
• Revenue Growth: Increased shelf availability boosted gross income in key product categories by 1-3%.

Competitor Differentiation

Unlike competitors that only address portions of the workflow, our system provided a holistic solution for detection, classification, and planogram compliance. The integration with edge devices further distinguished the platform by reducing operational costs without compromising on performance.

Long-Term Impact

The solution continues to assist the client in automating shelf management processes, reducing operational overhead, and improving inventory accuracy. Its scalability ensures the system remains effective as the client expands its operations to new regions.