Determining pose of glass objects

[dj4park]

Has there been any experiment done with using DOPE to determine the pose of a glass bottle? If not, would there be any reason as to why it would not work?

Currently I am experimenting with a Soju bottle. I have trained the network using 6000 basic NDDS images for 60 epochs using a batch size of 4 and a learning rate of 0.001 but I am having difficulties in detection.

Could the problem be due to the hyperparameters, image quantity/quality? Or could it be due to the texture of the glass bottle?

[TontonTremblay]

Do you have symmetries on you object, could you show me some training examples and testing examples? I never tried transparent objects with DOPE.

[dj4park]

Following a similar strategy to @Abdul-Mukit and his video, I created 6000 simple images using NDDS and trained the model for 60 epochs on a NVIDIA Quadro P3200 Mobile setting the learning rate to 0.001 and the batch size to 4.

Here is a photo of the object that I created using Blender.

And here are some training examples that I made with NDDS.

And here is the actual object itself with which I am testing the model.

[Abdul-Mukit]

The contributors/authors will give the best suggestions, but as I am working on things similar to this I would like to share some opinion.
Your 3D model may need improvements. Firstly the 3D model is missing the exact labels as the original bottle in the main body and the neck of the bottle. You can scan the original labels then use u-v mapping to use them as textures on your 3D models - not sure if it will work but I would have tried. Second, separate materials should be used for the glass, main-label, neck-label and the cap. The metallic and roughness have to be corrected for cap and glass in your model. I feel like the cap is too reflective. Color of the cap doesn't match either. The glass material, on the other hand, doesn't seem transparent at all and lacks roughness a bit. Third, the mesh of the cap is quite different from the original.
Other than that, you may try randomizing the background with images. In my small experience, I have felt background randomization with real images tends to prevent overfitting better compared to the small number of abstract background generated by NDDS.

[TontonTremblay]

I second!

[aditya2592]

Hi @dj4park. Were you able to successfully train for your custom object?

[dj4park]

Hello @aditya2592. Due to the difficulty of creating an exact 3D model of a glass soju bottle, I have moved to create a pringles model with Blender. There were many resources available online for this object.

I will begin training the model with 15000 simple images generated with NDDS with the following hyperparameters.

Epochs: 220
Batch Size: 4
Learning Rate: 0.001 for first 150 epochs then 0.0001 for the rest

I will let you know once the training is complete.

[aditya2592]

Are you randomizing your backgrounds as well?

[dj4park]

I am randomizing the background with the default images provided with NDDS.

[sanjaysswami]

Following a similar strategy to @Abdul-Mukit and his video, I created 6000 simple images using NDDS and trained the model for 60 epochs on a NVIDIA Quadro P3200 Mobile setting the learning rate to 0.001 and the batch size to 4.

Here is a photo of the object that I created using Blender.

And here are some training examples that I made with NDDS.

And here is the actual object itself with which I am testing the model.

How did you insert that exact texture (green color and written things etc) in blender? Rather can you please explain me how did you create this 3d model? I a also trying to generate the dataset for my own object but I don't know how to add textures. If you can help me with this issue it would be really greate help to me.