New to OWL and Looking for some guidance

[UAVSkies]

Hi,

Very new to the entire OWL project so looking for some guidance on following topics:

What is the highest and best resolution camera that has ever been tested with the software and where can I find it?

Is this the latest camera available? https://www.raspberrypi.com/products/camera-module-3/

Beside Pi, what would be a more powerful processing platform people have tested the software with and it works without any tweaking to the software. And if you suggest another hardware platform, what camera would work with it?

As new to the project, I want to take advantage of the most power platform to start with from day 1.

Thanks for your help.

[geezacoleman]

Welcome to the OWL project and thanks for the interest in building one!

For cameras, our recent publication showed the Arducam AR2034 was the highest performing with a global shutter, 2.3MP camera. We haven't tried any of the latest Raspberry Pi camera modules including the one you linked, given it requires system changes to Picamera2/libcamera. If you're willing to go down that path, the Raspberry Pi Global Shutter camera is likely to perform well, with a much higher pixel area than even the Arducam.

As for processing, the only other platform officially tested is a RockPi but that is lower power than the Pi. You could also run it on a Jetson Nano or equivalent Jetson/nvidia platform but that would require a fair amount of development on your part to make sure it works.

But the benefit of an open system is being able to adapt it to your needs - we'd certainly be interested in hearing about your experience if you go down that path!

[UAVSkies]

@geezacoleman Can you please confirm this is the right camera you pointed out before?

https://www.arducam.com/product/arducam-2-3mp-ar0234-global-shutter-color-b0368/

Also do you know where I can buy a plastic camera mount?

I am about to purchase the camera for initial testing. If there is latest camera supported by the software without any software modifications, please let me know.

Lastly, I want to perform some camera lab testing. What if I point the camera to a computer screen and make and appear different weeds (Green on Green, brown etc.), will the software detect it? May be display movie on a computer screen? try different speeds?

What would be the ideal lab testing setup to first understand the accuracy of the software detection capabilities before trying it into the field.

Also, how do we add more weed types and shapes to the library? and where do I find the list of weeds this software can detect off the shelf?

I am a firm believer of lab testing to iron out glitches. Things like what would happen if a rover hits a bump? rover speed, lightning conditions, distance accuracy etc.

Thanks for your guidance.

[geezacoleman]

I'm not quite sure of the difference to the one you linked (they're both AR0234) but this is the Arducam we tested in our recent preprint: https://www.arducam.com/2-3mp-ar0234-camera-module/. A warning though that it is quite complex and difficult to set up, but the image quality is great. I just noticed It looks like I may have linked to that variant incorrectly too. Use the one I have linked here.

If there is latest camera supported by the software without any software modifications, please let me know.

If you want off-the-shelf performance with effectively plug&play capability and the current standard for the OWL, I would use a Raspberry Pi HQ Camera with the 6mm wide angle lens. This is very simple to get working and relatively low cost.

Though we haven't tested it, the Raspberry Pi Global Shutter Camera has large pixel sizes on the sensor, so we believe it will be similarly well performing to the Arducam. The issue is, it doesn't work with Picamera, only Picamera2, which is still in beta... and we currently don't support that but will be working on it. In the long run, this Raspberry Pi Global Shutter camera I think will be the way to go.

Here's a comparison of image quality from the preprint:

Hopefully this helps! Either way, keep us updated with how you go.

[geezacoleman]

Apologies I missed your other questions.

Lastly, I want to perform some camera lab testing. What if I point the camera to a computer screen and make and appear different weeds (Green on Green, brown etc.), will the software detect it? May be display movie on a computer screen? try different speeds?

It depends what you want to test, if it's purely that everything is connected and working, then pointing it at a screen may be ok, but generally I wouldn't recommend doing this - the image quality (colour/brightness etc.) from a screen is very different to the field. I would recommend bringing some real plants inside and pointing the camera at those. Best case would be using a treadmill or other device to replicate some sort of speed.

Alternatively, if you record some video outside, you can use the command line python owl.py --show-display --input VIDEO_FILE.mp4 to run the OWL software on the video recorded in realistic conditions.

Also, how do we add more weed types and shapes to the library? and where do I find the list of weeds this software can detect off the shelf?

Unfortunately there are no in-crop, off-the-shelf weeds that this detects or models provided. It is designed for off-the-shelf fallow weed detection (i.e. anything that is green). That can work for between row weed detection, but for true in-crop detection you'll need to collect and annotate your own image data and train models on that. You can check out our other project Weed-AI where some images are hosted already if it's of interest.

I am a firm believer of lab testing to iron out glitches. Things like what would happen if a rover hits a bump? rover speed, lightning conditions, distance accuracy etc.

Absolutely! It sounds really obvious (and definitely not being condescending!) but still surprises me when I take cameras from the lab to the field - our eyes are very good and the sun is very bright. Basically, indoor lighting has strong colour hues our eyes automatically correct for and indoor lighting is so much dimmer than sunlight too. Equally, different parts of the day at different times of year in different weather conditions will vary a lot. Lab testing was really good for us to test the integration, but tuning detection parameters needed to be done in the field (or with field collected videos) and unfortunately there was no substitute!

Keep the questions coming if you run into any other issues or want to discuss anything.

[UAVSkies]

Thanks, I just ordered the "[Raspberry Pi HQ Camera] with the 6mm wide angle lens.

I am trying to solve two things in my head first :)

1. First, what is an ideal rover design that can handle different terrain while weed detection software is working at its ultimate capacity??

There is one company called "ecorobotix https://ecorobotix.com/en/ " just by looking at their design, it appears they are using a clever technique where they are using large hoods, and limiting the sun exposures to plants to improve on weed detection and letting the cameras to be at their best performance.

Then there is another company called, "Carbon Robotics" in US who is doing weed detection and using lasers to burn out the weeds with 5mm accuracy.

I am trying to find a middle ground for a rover design that can work in conjunction with your software. I am planning to hire few programmers to take your solution to the next level once I have a better understanding of the basic principles behind weed detection algorithms.

2. I am little confused about your statement above.

"Unfortunately there are no in-crop, off-the-shelf weeds that this detects or models provided. It is designed for off-the-shelf fallow weed detection (i.e. anything that is green). That can work for between row weed detection, but for true in-crop detection you'll need to collect and annotate your own image data and train models on that. You can check out our other project Weed-AI where some images are hosted already if it's of interest."

I read something about green on green module supported by owl. Is that related to Weed-AI platform? I will check the link shortly.

By the way any plans on visiting US? Love to meet you some day.

[geezacoleman]

Thanks, I just ordered the "[Raspberry Pi HQ Camera] with the 6mm wide angle lens.

Exciting! It's a good way to go to get things off the ground without being bogged down in driver issues.

1. First, what is an ideal rover design that can handle different train while weed detection software is working at its ultimate capacity??

A few things to consider if you haven't already, though I am by no means a robotics expert:

1. what crops are you working in? What area do you need to cover and what time period do you have available to do it? Is a shroud for the device appropriate? Large scale agriculture with large boom sizes make shrouds a hassle. Small scale, intensive production systems mean shrouds can work. With large enough image datasets in diverse enough conditions, you can potentially get around having a shroud. Hence, companies like John Deere, Bilberry, GreenEye, Carbonbee etc etc. that work without shrouds on self-propelled systems.
2. how are you controlling the weeds? Spray application will generally limit you to 15 - 20kmh.
3. What is the payload? How often should you refill? This will help with robot size and power.
4. Is a robot necessary? Why not automate a tractor and make a three-point linkage smart sprayer?

I built something similar to the Solinftec Solix a while back and it worked ok. Weight and run time were issues though. There are a lot of robotic platforms out there now though Naio is another company working in CA, University of Nebraska Lincoln seem to have a few. Then there are retrofit companies working on tractor automation. Check out FIRA in the US - they have some great summaries of robots available.

I am trying to find a middle ground for a rover design that can work in conjunction with your software. I am planning to hire few programmers to take your solution to the next level once I have a better understanding of the basic principles behind weed detection algorithms.

Sounds great! Will be good to see what you come up with.

I am little confused about your statement above.

Green-on-green is supported in that we now have the software in place to use trained deep learning algorithms such as YOLOv8. It's not that we provide any models to use. I.e. we've built the house but you just need to furnish it! Weed-AI can provided a starting point for annotated imagery to train models, but the best approach is to collect the data on the system you will use and then train it for the conditions you expect. Hopefully that clarifies it?

By the way any plans on visiting US? Love to meet you some day.

Hopefully! Need to finish a PhD first...

[UAVSkies]

I can't seem to get the camera to display a clear image. is there something wrong with my new camera or its a software issue?

No matter what focus settings I try it will not show any clear image. its blurry and out of focus. I even tried raspberry pi camera testing (without using owl) and still blurry image. Its clearly a focus problem.

The documentations says, "NOTE: the HQ lens comes with a C-CS mount adapter which needs to be removed before fitting to the camera sensor base. The image won't focus unless the adapter is removed."

If remove the CS-Mount then threads of the camera lens are not wide enough to fit on to the camera. what am I missing? The lens does say its 6mm IR lens.

p.s. I watched some you tube videos and they are recommending this lens for Raspberry Pi Hi res. camera.

My Camera and lens setup.

DID I BOUGHT THE WRONG CAMERA "or" The wrong lens?

[UAVSkies]

Thanks. got the focus issue resolved, however this camera and lens is very poor mechanical design. I will keep experimenting with it.

[geezacoleman]

Glad to hear it worked out. What sort of issues are you still having with the camera?

[geezacoleman]

I have added additional explanation in the main hardware assembly guide here. Hopefully this makes it a bit more clear.

HQ camera C-CS mount adapter	Camera and adapter separated	Lens fitted without adapter

[UAVSkies]

Hi, I got it work. As per you suggestion, that black ring was the culprit. The software is working as expected. Its detecting Green.

Next step I am adding Green on Green so just purchased Google Coral USB edge TPU ML Accelerator, should arrive tomorrow.

After that planning to Try NVidia. if I recall correctly, to use NVIDIA the code will require modifications correct?

Also, NVIDIA seems have a number of different model's. Which Model will you recommend?

[UAVSkies]

Hi,

I followed the steps to install Green on Green.

1. Till this step everything went well:

Step 3
The final step is to test the installation.

Open up a Python terminal by running:

(owl) pi@raspberrypi:~/owl/models$ python

2. next when I type ">>> import pycoral"

I am getting the following message;

Traceback (most recent last):
File "", Line 1, in
Import Error: no module named pycoral

Notes:

1. I did not encounter any errors performing step 1.
2. I did see a message about "Install MAX Frequency" which I said YES too.
3. I did receive this warning: "You are using pip version 21.2.2; however, version 23.1.2 is available. You should consider.

Should I upgrade the Pip version?

I posting the install screen shot below for your review.

[UAVSkies]

[UAVSkies]

Hi,

I think i was able to install Pycorl as there is now module called "Pycoral"

I am little confused about how to execute these setting for indoor settings?

Your instructions says;

"NOTE If you are testing this inside, the camera settings will likely be too dark (and the image will appear entirely black) so you may also need to specify the --exp-compensation 4 and --exp-mode auto."

So, I tried this command and getting an error message. Not a python syntax expert so maybe I don't know how execute the above or in what order.

1. I tried;

(owl) pi@raspberrypi:~/owl $ python owl.py --show-display --algorithm gog --exp-compensation 4 --exp-mode auto

2. I also tried (owl) pi@raspberrypi:~/owl $ python owl.py --exp-compensation 4

to see if I can run command at a time.

Regardless, The video feed screen is not showing up.

I think I am not executing commands correctly in correct order :)

The other problem is that the minute you will execute;

(owl) pi@raspberrypi:~/owl $ python owl.py - and whatever settings.

the camera gets locked into the owl application.

Even if you try;

ps -C owl.py, its not giving me the PID # in use to Sudo kill it.

Kind of lost now...

[UAVSkies]

1. I Rebooted the Raspberry Pi.
2. Upon reboot the Camera is automatically used by owl so I used "ps -C owl.py
3. used Sudo Kill 555 to free up the camera
4. Tried

(owl) pi@raspberrypi:~/owl $ python owl.py --exp-compensation 4 --exp-mode auto --show-display --algorithm gog

Notice changed the variables positions.

The video screen sort of came up and then disappeared and program stopped.

See screen shot below.

Interestingly. if only execute command;

(owl) pi@raspberrypi:~/owl $ python owl.py --show-display

everything shows up i.e. "Detection Output", "Adjust Detection Thresholds", "Binary Threshold", "HSV Threshold on EXG" screens..

My other confusion is how one differentiates between if you are using Green on Green code vs. Green on Brown?

I am assuming both modules are there.

Sorry, soo many questions :)

[geezacoleman]

Nice work - looks like you're making progress with this!

Everything you're doing seems right - the error seems to be that local variable weed_detector referenced before assignment. This would be from L262 - 266 of owl.py.

This makes me think that GreenOnGreen() isn't being instantiated correctly. I would try a couple of things:

1. My first thought was to run it as default, and you've already done that, so we know that the camera etc all works. It must be an issue with the GoG instantiation.
2. I think it may be to do with the fact you're running it as ´python owl.py ....´ and not ´./owl.py´. Executing it the second way changes the way it runs slightly and has solved issues in the past.
3. Next, check if you have a model file in the models folder. I would use a default model file available from Google Coral official repo in the first instance.

That's all I can think of currently - I can try testing it on my own system this week to see if I can replicate it. Keep persisting with it! Eventually it will work out - and keep the questions coming too, it helps me better understand how things can be improved, so they are all really valuable and worthwhile :)

A few responses to your questions:

My other confusion is how one differentiates between if you are using Green on Green code vs. Green on Brown?

What do you mean by this? The flag will differentiate it and in the future hopefully a button in the cab of a tractor or on a controller.

(owl) pi@raspberrypi:~/owl $ python owl.py --exp-compensation 4

You should always use --show-display. That's why the video feed wasn't showing up here and then being locked in the background. The order of the command line flags (e.g. --algorithm or --show-display) isn't too important, because the flag tells it where to look. I think the issue here is you're running it as python owl.py and not ./owl.py, as mentioned above.

Even if you try;
ps -C owl.py, its not giving me the PID # in use to Sudo kill it.

This hopefully means that the program isn't running anymore - try ps -C greenonbrown.py as well if it is still locked up. and you can't find owl.py.

[UAVSkies]

Thanks, I will try your suggestions on Wednesday.

I think I am struggling to better understand the inner workings of the program.

for example first we made owl to do just green detection which worked fine. Not sure what is the correct terminology to call greenonbrown.py a “module” or a software?

Next we installed green on green, module/software.

what confusing me is that when we execute owl how does it know which module to use? :) I know silly question but clearly tells you I don’t know Python well enough.

The debate in my head is does the owl program now will do both green detection as well as green on green or just green on green or these two completely separate porgrams?

is just green detection module a separate program and green on green is separate program or at the end they are combined together?

Sorry, all these confusions are due to my limited knowledge of python and how it functions :)

[UAVSkies]

p.s. I checked the models folder, there is one file called

ssd_mobilenet_v2_coc_quant_postprocess_edgetpu.tflite

I also tried to run the program using command

~/owl/./owl.py --show-display --algorithm gog

The program gives error message, "local variable 'weed_detector' refrenced before assignment

Interestingly if you only execute command

~/owl/./owl.py --show-display

without --algrithm gog, the video window does open up.

There is also new window called "Binay Threshold" not sure what that is for.

[UAVSkies]

I tried the command >> help('modules') and I am not seeing any module called "pycoral"