legendary-octo-goggles

Count them

AR application for counting stack of pipes
View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
   • Design Solution
   • Work Flow
   • Template development
   • Model development
   • Deployment
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgements

About The Project

Won 3rd Place in Rally to Tally Hackathon

Design Solution

Given :

1. 1.66 - 48 inch in diameter
2. 30-60 feet long
3. Stacked in 3 - 20 layers depending on Outside Diameter size (but not higher than approx. 10Feet)
4. More than 1 thousand pipes in 1 acre
5. A tag and/or cap may be attached to the end of the tube
6. In the case of large diameter pipes, the pipe stacks are often several hundred meters long and the distance between the stacks is often so narrow that it can be difficult to take a picture from the front.
7. Since each pipe weighs up to several tons, it is difficult to calculate the number of pipes by weighing them all at once.

Actions :

1. Variable diameter is taken in solution for feature detection
2. Side view will be not be the best view
3. Stacks are not too high, so view will be landscape
4. Manual collection of images is not possible
5. Extra features need to be considered in solution
6. Considered at 3.
7. Weighing them is not considered.

Workflow

1. Land vehicle bot to collect recordings of a particular stack in the field.
2. Recording to done in one direction, only front of stack will be necessary.
3. Only one stack of pipes should be taken for each recording. This ensures that the type of pipe is different for different templates.
4. Take the key frames from each part of recording.(Where the pipe front is visible, we need to cover all possible types of covering on pipes here)
5. Develop templates of each pipe: Circular region template - with tag or cap or normal.
6. Identify each count of each image in the scene by putting a number on each template.
7. For new recording if, the old templates can be used then the accuracy of model increases over time

Deployment is made on an android app. Now we can just walk around the pipe field in a given direction and it will detect the stack with the given count. The idea is that we are given with list of detections in one direction, and the app should keep a track of detections.

Final count = Number of detections * Template count

TEMPLATE DEVELOPMENT

1. Hough Circles are taken to identify pipe circular structure and templates are saved.

2. Manual crops of pipe with tags and caps should taken by human in loop.

3. Choose a template according to a part of image (very important)

AI MODEL DEVELOPMENT

1. Use the templates to train a feature generation model

2. Use the old database created from other videos, if type of pipes are same.

3. Once trained, model is tested on validation data for more images

4. If validation accuracy is good, then pass the model to deployment module

DEPLOYMENT

1. Android app will be used
2. The template used earlier will be detected multiple times in the view
3. The app will keep a count of current view, and keep adding it as the phone moves in a single direction
4. Use of the app in the land robot in future for more ease of use.

Built With

• mediapipe
• Colab

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

• mediapipe
• Colab

Installation

1. Clone the repo

   git clone https://github.com/Princep/legendary-octo-goggles.git
   

2. Develop the template

Colab offers customizable solutions for images.

OpenCV: Built-in fast Hough transform inference for processing. Modify radius for size	Result: Each circle is properly detected, Easy front view can be taken
Manual: Select a area which seems to be repeatable area	Area count = 4: Can be an area which contains tags or atleast 3 pipes

1. Save the template image with the decided count

2. Matching Your Own Template Images

• Put all template images in a single directory called templates

• To build the index file for all templates in the directory, run

   bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 \
   mediapipe/examples/desktop/template_matching:template_matching_tflite
  

   bazel-bin/mediapipe/examples/desktop/template_matching/template_matching_tflite \
   --calculator_graph_config_file=mediapipe/graphs/template_matching/index_building.pbtxt \
   --input_side_packets="file_directory=<template image directory>,file_suffix=png,output_index_filename=<output index filename>"
  

  The output index file includes the extracted KNIFT features.

1. Use mediapipe

• Replace mediapipe/models/knift_index.pb with the index file you generated, and update mediapipe/models/knift_labelmap.txt with your own template names with count for the tempate.

cd mediapipe
# Switch to OpenCV 4
sed -i -e 's:3.4.3/opencv-3.4.3:4.0.1/opencv-4.0.1:g' WORKSPACE
sed -i -e 's:libopencv_java3:libopencv_java4:g' third_party/opencv_android.BUILD

# Build and install app
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/templatematchingcpu
adb install -r bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/templatematchingcpu/templatematchingcpu.apk

• Build and run the app

Usage

• Install the apk
• Point it at the site

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Your Name - @pp_spector - prince.patel.14@gmail.com

Project Link: https://github.com/Princep/legendary-octo-goggles

Acknowledgements

• mediapipe

This site is open source. Improve this page.