-
Notifications
You must be signed in to change notification settings - Fork 340
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exposure for indoors #86
Conversation
5ed0082
to
08ced78
Compare
08ced78
to
ee88862
Compare
@jgoppert I've not setup my development environment for PX4Flow. Could you provide a binary to upload? |
@ecmnet, I've uploaded the built firmware to my google drive here: https://drive.google.com/open?id=0B3wIEs1gcVahNktlQkhCMndIeUE |
@jgoppert Thanks a lot! I'll give it a try a soon as I've mounted flow to my device again. |
@jgoppert Unfortunately, my PX4Flow does not work after upgrading (does not connect to QGC at all) |
Weird, will try uploading the image I created for you tonight on my device. |
@jgoppert I've setup my environment now and was able to compile and upload this branch. Is it correct to have some white square dots in the image now? |
Yes, the dots indicate high quality optical flow in that region of the image. |
Or more specifically, that 8x8 tile had enough difference between its pixels (f thrsh param) that it included that tile in the optical flow calculation. |
@jgoppert Thanks for clarification. I will do some testflights as soon I've mounted flow again. |
@jgoppert Flow quality seems not to be reasonable with this patch. Even, if I darken the lens, quality is reported at 220-250 in my case. |
It increases the gain and exposure when you cover the lens so you start picking up noise. Look at the image in qgc. The max gain and max exposure can be lowered to limit the effect, but if you don't cover the lens I think it gives good low and high light performance. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. I think I can give this a test the next few days
@@ -699,7 +707,7 @@ int main(void) | |||
uint16_t frame = 0; | |||
for (frame = 0; frame < image_size_send / MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN + 1; frame++) | |||
{ | |||
mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) current_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]); | |||
mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) previous_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did you change this to previous_image
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so that it gets overlayed with the dots for flow quality
@@ -79,7 +79,7 @@ void mt9v034_context_configuration(void) | |||
uint16_t new_height_context_b = FULL_IMAGE_COLUMN_SIZE * 4; | |||
|
|||
/* blanking settings */ | |||
uint16_t new_hor_blanking_context_a = 709 + MINIMUM_HORIZONTAL_BLANKING;// 709 is minimum value without distortions | |||
uint16_t new_hor_blanking_context_a = 425 + MINIMUM_HORIZONTAL_BLANKING;// 709 is minimum value without distortions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment needs to be adapted
I've tested this and it doesn't work very well at all for me. It might be my hardware of course but I get pretty heavy banding with this. |
@lericson Go into qgc and play with exposure_max and brightness. Lowering brightness down should fix it for you (try brightness 10, max_exposure 300). I've tested this a lot, so trust me, it is a huge difference for indoors. But watch out for specular reflections and shadows. I've seen the banding before as well, it is also there on the defualt firmware. It just can be more obvious when the auto gain is adjusting etc. |
@jgoppert I played around a bit with the parameters. Couldn't get very good results sadly. For your perusing, here is my merged version of this PR and #90: https://github.com/KTH-CAS-UAV/Flow/tree/master-with-exposure-pr |
Don't have time to load it myself. What does your flow quality say? It should be much higher with this patch. Also, when you evaluate the quality, don't just look at the image, enable the flag SEND_LPOS. This will give you output of a system onboard estimator of the position, assuming that the board is always aligned with north. Then I usually walk in a 2 m square. |
It said 255 all the time and the image quality was not very good with heavy banding. The flow estimation was also very noisy, which isn't so surprising. 🙂
- Ludvig
… On 16 Jun 2017, at 18:14, James Goppert ***@***.***> wrote:
@jgoppert commented on this pull request.
In src/modules/flow/main.c:
> @@ -699,7 +707,7 @@ int main(void)
uint16_t frame = 0;
for (frame = 0; frame < image_size_send / MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN + 1; frame++)
{
- mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) current_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]);
+ mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) previous_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]);
so that it gets overlayed with the dots for flow quality
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
When you set the parameters it doesn't stick after a reboot, if you set brightness to 8 and max exposure to 200 (in the firmware and rebulid), that is bascially the same as the default firmware. This firmware is basically just giving you more control over the settings. Brightness is basically the desired mean of the pixel values and it plays with auto gain to and auto exposure to get that. It also shows you the debug for the optical flow, the dots mean that the algorithm is using that cell for flow calculation. If you turn it really dark and you see no dots, there is not enough difference in intensity between the pixels in that cell for it to use it for the flow calculation. It also stabilizes the flow rate output to the rate you set instead of depending on your exposure setting. |
@@ -50,7 +50,7 @@ OBJDUMP = $(CROSSDEV)objdump | |||
|
|||
# Check if the right version of the toolchain is available | |||
# | |||
CROSSDEV_VER_SUPPORTED = 4.7.4 4.7.5 4.7.6 4.8.4 4.9.3 | |||
CROSSDEV_VER_SUPPORTED = 4.7.4 4.7.5 4.7.6 4.8.4 4.9.3 5.4.1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This supports the latest gcc, so I included it here.
@@ -40,7 +40,7 @@ | |||
#define ONBOARD_PARAM_NAME_LENGTH 15 | |||
#define BOTTOM_FLOW_IMAGE_HEIGHT 64 | |||
#define BOTTOM_FLOW_IMAGE_WIDTH 64 | |||
#define BOTTOM_FLOW_SEARCH_WINDOW_SIZE 4 | |||
#define BOTTOM_FLOW_SEARCH_WINDOW_SIZE 8 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can play with this, but this makes it possible for it to see a large max velocity, but you have to watch the update rate and make sure the cpu isn't maxing out.
PARAM_HDR, | ||
PARAM_AEC, | ||
PARAM_AGC, | ||
PARAM_BRIGHT, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here I exposed settings that were hard coded in the firmware.
dma_reconfigure(); | ||
buffer_reset(); | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is just setting the parameters that you request in QGC.
|
||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This piece of code was commented out and shows you flow quality indicators for each grid cell.
if (counter % (uint32_t)global_data.param[PARAM_BOTTOM_FLOW_SERIAL_THROTTLE_FACTOR] == 0)//throttling factor | ||
uint32_t now = get_boot_time_us(); | ||
uint32_t time_since_last_pub = now - time_last_pub; | ||
if (time_since_last_pub > (1.0e6f/global_data.param[PARAM_BOTTOM_FLOW_PUB_RATE])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the logic to publish at a fixed rate instead of based on the exposure setting.
lpos.vz = 0; | ||
lpos.vx = ground_distance*accumulated_flow_x/integration_timespan; | ||
lpos.vy = ground_distance*accumulated_flow_y/integration_timespan; | ||
lpos.vz = 0; // no direct measurement, just ignore |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I included the velocity here for debug.
strcpy(global_data.param_name[PARAM_AGC], "AGC"); | ||
global_data.param_access[PARAM_AGC] = READ_WRITE; | ||
|
||
global_data.param[PARAM_BRIGHT] = 20; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you have trouble with washout/ banding, try lowering this value. 8 is the minimum.
strcpy(global_data.param_name[PARAM_BOTTOM_FLOW_PUB_RATE], "BFLOW_RATE"); | ||
global_data.param_access[PARAM_BOTTOM_FLOW_PUB_RATE] = READ_WRITE; | ||
|
||
global_data.param[PARAM_EXPOSURE_MAX] = 500; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Increasing this value helps in low light, but it also makes the max velocity you can go lower. If you have trouble with velocity maxing out, lower this to around 200 or 100.
strcpy(global_data.param_name[PARAM_EXPOSURE_MAX], "EXPOSURE_MAX"); | ||
global_data.param_access[PARAM_EXPOSURE_MAX] = READ_WRITE; | ||
|
||
global_data.param[PARAM_GAIN_MAX] = 16; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gain can make the image pixelated. If your image has a lot of noise, try lowering this to 4 or 8.
@lericson I tried to explain my changes in this patch, hope that helps and you find some of it useful for your work. |
@jgoppert I have bench tested this PR so far. Looks very good. I increased the flow feature threshold because it reported good quality even though it was just noisy and pointing to a plain white surface. I think it's safer this way. |
@LorenzMeier @bkueng @ecmnet @mhkabir please test, this is a huge improvement for indoors and I suspect outdoors as well.
Changes