[GOBBLIN-1827] Add check that if nested field is optional and has a non-null default…

[Will-Lo]

…, then it should order the types with its default type first instead of null

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-1827

Description

• Here are some details about my PR, including screenshots (if applicable):
  Consider the following description:

 * Record R1 {
 *   fields : {[
 *      {
 *        Union : [
 *          null,
 *          Record R2 {
 *            fields : {[
 *                {
 *                  String S1
 *.                  default: String S3
 *                }, {
 *                  String S2
 *                }
 *            ]}
 *          }
 *      }
 *   ]}
 * }
 * will be flattened to:
 * Record R1 {
 *   fields : {[
 *      {
 *        Union : [ null, String S1]
 *.       default: String S3
 *      }, {
 *        Union : [ null, String S2]
 *      }
 *   ]}
 * }

Due to the addition of the AvroCompatibilityHelper, Avro flattening can run into an issue in the above scenario due to the nested non-null default S3 since the default type needs to be the first type in the field order. So R1 should have field S1 as Union [String S1, null]

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 46.98%. Comparing base (e47aef7) to head (eae0712).
Report is 321 commits behind head on master.

Files with missing lines	Patch %	Lines
...in/java/org/apache/gobblin/util/AvroFlattener.java	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3689      +/-   ##
============================================
- Coverage     46.99%   46.98%   -0.02%     
- Complexity    10788    10790       +2     
============================================
  Files          2138     2138              
  Lines         84040    84042       +2     
  Branches       9340     9341       +1     
============================================
- Hits          39497    39485      -12     
- Misses        40955    40968      +13     
- Partials       3588     3589       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZihanLi58]

+1