By Tofeeq

2012-01-12 06:36:05 8 Comments

I found many ways to detect different shapes. But hard luck when I am going for a physical object. From what I read we should have a black border around images to make a pattern file. If I follow this concept and generate a pattern then my application detects images on printout. But in the real world a physical object not necessarily has a black border square shape around it.


Although I accept an answer, my question remains unsolved. As there is still no solution for detecting physical object.

Any further research and links are welcome!


@ARGeo 2019-04-26 06:37:39

Now you have a tool to detect objects in Android AR apps.

On 8th of May 2018 Google announced a mobile SDK – ML Kit, that's a machine learning API for Android and iOS developers that you can use along with ARCore SDK (what now has Augmented Images API and Augmented Faces API). At the moment it's still in beta stage but it's got the following working features:

enter image description here

ML Kit engineers say about it this:

ML Kit brings Google's machine learning expertise to Android and iOS apps in a powerful yet easy-to-use package. Whether you're new or experienced in machine learning, you can implement the functionality you need in just a few lines of code. There's no need to have deep knowledge of neural networks or model optimization to get started. On the other hand, if you are an experienced ML developer, ML Kit provides convenient APIs that help you use your custom TensorFlow Lite models in your mobile apps.

@naXa 2016-10-21 07:47:56

Have you considered OpenCV for detecting and tracking object? Take a look at Find Objects with a Webcam tutorial (C++ / Qt). OpenCV is available for Android - OpenCV 4 Android.

Enjoy Detection!

@dabhaid 2012-01-18 14:45:55

The bad news is, you can't use AndAR to detect physical objects. AndAR is based on a fiducial marker approach, where the marker is made of two components: a solid border and an interior pattern. The pattern encodes a value that can be used to address a particular model to render on the marker, and the border makes it easy to determine the relative orientation of the marker to the device. Clearly this is just planar image recognition.

To do object recognition on a 3D object is a more complicated problem, and I don't know of any Android libraries that provide a turn-key solution, but recognizing just one object is probably feasible on a mobile device.

One possibility might be to investigate the available Android AR toolkits (Layar, Junaio, Qualcomm AR SDK) which all now support some image recognition. It may be that by taking images of your teapot at various rotations and using those as the images you want your app to match against that you might get this solution working, but keep in mind they are only designed to do planar matching on images, not real 3D objects, so the performance might not be great. (Well, the Metaio Mobile SDK Pro does 3D recognition and tracking, but it's very expensive).

While object recognition is perhaps best done by comparing camera frames with images of the object you wish to recognize (or by comparing image features from the camera frames with pre-computed image features etc), tracking is a different matter. If you want to accurately track your 3D object in 3D space you'll really need to have or build a 3D model of it, and for each frame determine point correspondences between the camera image and the 3D object for tracking. True unassisted (i.e. no depth-camera) 3D tracking is hard.

I hope this gives you some background you can use to evaluate your next steps.

Update: Qualcomm's Vuforia SDK allows you to track "multi targets", which are objects with a set of planar tracking surfaces with a fixed spatial relationship. If you made a "cube" different photos of the 6 sides of your object (teapot) that might work somewhat.

End of 2013 Update:

I have no experience with these, but:

Metaio now offer 3D tracking of CAD models:'s LinkAR promises object matching.

I would note the use of the word "matching" - I think the use case here is you know the object you want to overlay (a toy-box, and engine etc). Differentiating between multiple 3D objects may be entirely out of scope.

@Tofeeq 2012-01-19 06:34:01

Thank you..i am very disappointed that except open source nature, there is no library available for physical object detection.Can you have sample to comparing two images using camera frame so that i can go for this.But i really appreciate your work. +1

@dabhaid 2012-01-19 11:56:16

Would you like an example of doing image matching from scratch, or how to set up e.g. Junaio Glue to do image matching?

@Tofeeq 2012-01-19 12:16:25

Yes give the example link for Matching image with live camera stream

@dabhaid 2012-01-19 12:23:41

This is a very broad question :) Perhaps the best approach for mobile would be the histogram matching method discussed here: - it's fast to computer, it's very easy to match against the frame, but it's not very robust, and you won't know exactly where in the image the object is located.

@Tofeeq 2012-01-19 12:26:57

Thanks again..Is not any other way to match image with camera preview?

@dabhaid 2012-01-19 12:37:07

There are a great many approaches to image matching. If you describe the entire use case, maybe another approach might appear better, but histogram matching would be the most straightforward for recognizing a 3D object on a mobile device in real time. If you need the position (for augmenting), you could try to follow the recognition step with template matching. There are many more solutions, but they are more sophisticated.

@Tofeeq 2012-01-19 12:49:04

Thanks for sharing lots of knowledge with me.I will wait for another response from community.Then i will decide about bounty :).But really thank you for your efforts

@Tofeeq 2012-01-19 12:50:36

Do you have time to discuss in chat?

@dabhaid 2012-01-19 13:20:42

Not any more today, and tomorrow is a busy day, but maybe over the weekend?

@Tofeeq 2012-01-19 13:21:45

ok..i will wait..Bye Good night

@Tofeeq 2012-01-25 09:18:19

i appreciate your help by awarding you bounty.I need a favor if you donot mind.can you upvote my question so that it will show in popular i need its all possible solution

@dabhaid 2012-01-25 10:55:59

Sure, visual 3D object detection is still an area of active research though, so I would be surprised if you got a "cookbook" solution from SO.

@Tofeeq 2012-01-25 11:03:18

nice to talk you..i will continue my work.What you do dabhaid?

@dabhaid 2012-01-25 11:25:44

Good luck! I'm a mixed and augmented reality researcher.

@Tofeeq 2012-01-25 11:27:43

can i have your personal email to talk you later

@Rui Marques 2013-02-15 20:11:24

The Vuforia link no longer works, maybe this was it:

@Amit Hooda 2013-11-27 04:16:34

is there any other sdk that provides tracking of real objects ? I also need this kind of functionality.

@dabhaid 2013-12-02 18:55:23

I've updated the answer a bit, the landscape is quickly moving in the direction of 3D tracking.

@Sanket Prabhu 2016-04-02 12:14:38

Hi Sir, Finally there is any solution now?

@Pedro 2012-04-26 22:17:55

I recently read about research being doing on hierarchical shape vocabularies used for object representation. Of course, there is no library available for download, but in case you are interested in the general approach here you can find some papers.

Also you might be interested in this paper. It describes an algorithm for detecting objects based on a set of contours.

@Tofeeq 2012-04-27 04:24:24

Currently i am working on different project as soon as i got time i will study your suggest link.Anyway thank you

Related Questions

Sponsored Content

26 Answered Questions

[SOLVED] How to check if a service is running on Android?

9 Answered Questions

48 Answered Questions

34 Answered Questions

6 Answered Questions

77 Answered Questions

23 Answered Questions

[SOLVED] How do I rotate the Android emulator display?

22 Answered Questions

[SOLVED] augmented reality framework

9 Answered Questions

2 Answered Questions

[SOLVED] Augmented Reality SDK with OpenCV

Sponsored Content