Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-12164] Added integration test for transaction boundaries and transaction ID ordering. Made small fixes in ordered by key integration test. #16977

Merged
merged 69 commits into from
Mar 18, 2022

Conversation

nancyxu123
Copy link
Contributor

Added integration test for transaction boundaries and transaction ID ordering. Made small fixes in ordered by key integration test.

…ordering. Made small fixes in ordered by key integration test.
@pabloem
Copy link
Member

pabloem commented Mar 1, 2022

Run Java PostCommit

1 similar comment
@pabloem
Copy link
Member

pabloem commented Mar 9, 2022

Run Java PostCommit

@nancyxu123
Copy link
Contributor Author

Seems like Java PostCommit did not run.

@pabloem
Copy link
Member

pabloem commented Mar 9, 2022

Run Java PostCommit

emilymye and others added 22 commits March 9, 2022 15:07
…ependencies to 24.3.0

* Update GCP Libraries BOM version to 24.3.0

* Update associated dependencies
…ent method to get a default example for each SDKs

* Implement method to get a default example for each SDKs

* Add error handling

* Added saving of precompiled objects catalog to cache at the server startup

* Added caching of the catalog only in case of unspecified SDK

* Update regarding comments

* Update regarding comments

* Simplified logging regarding comment

* Get defaultExamplePath from the corresponding config

* Refactoring code

* Add the `link` field to response

* Remove gjson;
Resolve conflicts;

* Refactoring code

* Getting default precompiled object from cache

* Refactoring code

* Added saving of precompiled objects catalog to cache at the server startup

* Added caching of the catalog only in case of unspecified SDK

* Update regarding comments

* Update regarding comments

* Simplified logging regarding comment

* Updates regarding comments

* Update for environment_service_test.go

* Get default example from catalog

* GetCatalogFromCacheOrStorage method

* Update licenses

* Update licenses;
Resolve conflicts;

* [BEAM-13633][Playground]
Change saving default precompiled objects to the cache

* [BEAM-13633][Playground]
Change logic of saving and receiving info about default precompiled objects

* [BEAM-13633][Playground]
Separate for each sdk

* [BEAM-13633][Playground]
regenerate proto files

* Add code of the default example to response

* Revert "Add code of the default example to response"

This reverts commit da6baa0.

* Refactoring code

* Refactoring code;
Add test;

* Edit commentaries

* Refactoring code

* Add bucket name to methods

Co-authored-by: Artur Khanin <artur.khanin@akvelon.com>
Co-authored-by: AydarZaynutdinov <aydar.zaynutdinov@akvelon.com>
Co-authored-by: Pavel Avilov <pavel.avilov>
…row and proto (apache#16875)

* Adding schema support.

* Addressing feedback.
* [BEAM-13964] Bump kotlin to 1.6.x

* [BEAM-13964] Bump kotlin to 1.6.x

* [BEAM-13964] fix warnings in Kotlin compilation
…rnings (no-name-in-module/no-value-for-parameter)

* [BEAM-13051] Pylint no-name-in-module and no-value-for-parameter warnings enabled

* [BEAM-13051] Fixed no-value-for-parameter warning for missing default values

* [BEAM-13051] Fixed parameters warnings
@codecov
Copy link

codecov bot commented Mar 10, 2022

Codecov Report

Merging #16977 (f47357a) into master (62a6610) will decrease coverage by 0.00%.
The diff coverage is 60.60%.

❗ Current head f47357a differs from pull request most recent head 9e3a121. Consider uploading reports for the commit 9e3a121 to get more accurate results

@@            Coverage Diff             @@
##           master   #16977      +/-   ##
==========================================
- Coverage   73.94%   73.94%   -0.01%     
==========================================
  Files         669      667       -2     
  Lines       88158    88071      -87     
==========================================
- Hits        65191    65125      -66     
+ Misses      21855    21835      -20     
+ Partials     1112     1111       -1     
Flag Coverage Δ
python 83.64% <70.89%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
sdks/go/pkg/beam/core/graph/window/windows.go 82.60% <ø> (ø)
sdks/go/pkg/beam/core/runtime/exec/util.go 74.28% <ø> (ø)
sdks/go/pkg/beam/core/runtime/harness/harness.go 11.26% <0.00%> (ø)
sdks/go/pkg/beam/runners/direct/buffer.go 77.46% <ø> (ø)
sdks/go/pkg/beam/util/gcsx/gcs.go 27.41% <ø> (ø)
sdks/go/pkg/beam/util/harnessopts/cache.go 100.00% <ø> (ø)
...apache_beam/runners/dataflow/internal/apiclient.py 77.39% <ø> (ø)
...s/dataflow/internal/clients/cloudbuild/__init__.py 77.77% <ø> (ø)
...ernal/clients/cloudbuild/cloudbuild_v1_messages.py 100.00% <ø> (ø)
sdks/go/pkg/beam/core/runtime/exec/plan.go 48.61% <22.22%> (ø)
... and 56 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62a6610...9e3a121. Read the comment docs.

@pabloem
Copy link
Member

pabloem commented Mar 17, 2022

Run Java PostCommit

@pabloem
Copy link
Member

pabloem commented Mar 17, 2022

LGTM

@pabloem
Copy link
Member

pabloem commented Mar 17, 2022

Run Java PreCommit

@pabloem
Copy link
Member

pabloem commented Mar 18, 2022

LGTM thanks @nancyxu123 !

@pabloem pabloem merged commit 31ed331 into apache:master Mar 18, 2022
@kennknowles
Copy link
Member

Oops - we have test suites that pull this in but it needs to be excluded: https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV1/1390/

Filed https://issues.apache.org/jira/browse/BEAM-14151 and gave to Pablo since I couldn't find your account.

@pabloem
Copy link
Member

pabloem commented Mar 22, 2022 via email

@lukecwik
Copy link
Member

Pablo, it looks like the Java PostCommit fails for some of these ITs with the direct runner, filed https://issues.apache.org/jira/browse/BEAM-14193.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.