Handling exceptions session parameters #10
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…is not ybc enabled Summary: Summary: We should only show the parallel threads based on the following criteria. # Hide "parallel thread", only if current backup is ybc enabled and target universe is ybc enabled. # hide "parallel thread", if current backup is not ybc , but target universe is ybc enabled # Block restore , if the backup is ybc enabled and target universe is not ybc # for all other cases, show "parallel threads" Test Plan: Tested manually. I created a ybc universe and took a backup(ybc enabled) | YBC On universe | YBC on Backup | Parallel threads | err msg| | enabled | enabled| hidden| hidden | |enabled | disabled | hidden | hidden| |disabled|enabled|hidden|shown| |disabled|disabled|shown|hidden| Reviewers: asathyan, jmak, rmadhavan, lsangappa Reviewed By: lsangappa Subscribers: jenkins-bot, ui, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21008
…ric shows sudden peaks. Summary: This diff addresses the issue of "cdcsdk_sent_lag_micros" showing sudden peaks. This was beacause "cdcsdk_last_sent_physicaltime" metric was dropping to zero, and as a result a large value was assigned to "cdcsdk_sent_lag_micros" metric(//cdcsdk_last_sent_physicaltime// is used to calculate the //cdcsdk_sent_lag_micros//). This gives peaks in the graphs for the lag metric. The issue comes up when CDC doesn't get any records from WAL/IntentDB, as a result //cdcsdk_last_sent_physicaltime// was being set to zero. We are fixing this by passing the existing value of the //cdcsdk_last_sent_physicaltime// metric which is returned in case CDC doesn't find any record from from WAL/IntentDB. Test Plan: Running existing C and Java tests Reviewers: sdash, rvenkatesh, vkushwaha, abharadwaj, skumar Reviewed By: abharadwaj, skumar Subscribers: ycdcxcluster Differential Revision: https://phabricator.dev.yugabyte.com/D20964
…y once during the process start Summary: It should be added during startup. Test Plan: Tested with node-agent. It was failing before some because methods did not call this (addProvider). Reviewers: amalyshev, sb-yb, cwang Reviewed By: cwang Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21030
Summary: Simplify PgLibPqTest::DBCatalogVersion unit test Test Plan: ``` ./yb_build.sh --gtest_filter PgLibPqTest.DBCatalogVersion ``` Reviewers: jason, myang Reviewed By: myang Subscribers: yql, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D21029
Summary: Added the changes to move the arm64 architecture enum to aarch64 architecture enum. Made changes to the software release, ybc release and provider region. Test Plan: Tested manually that the previous behavior of YBA not being able to list arm-based releases on arm provider does not exist anymore. Manually created an aarch-based release (local and cloud both), arm/aarch-based provider, and the arm/aarch-based universe. Reviewers: muthu Reviewed By: muthu Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D20628
…ly top level keys.
Summary: Without flattening compare the overrides and finding the unknown keys.
Test Plan:
Manully tested with following overrides.
"universeOverrides": "master:\n podLabels:\n env: test\ntserver:\n podLabels:\n env: test",
"azOverrides": {"us-west1-a":" tserver:\n podLabels:\n env: west-1a-test"},
and observed no errors in response.
Reviewers: sanketh, nsingh
Subscribers: yugaware
Differential Revision: https://phabricator.dev.yugabyte.com/D21031
Summary: * Flag introduced in reference.conf to enable or disable 'hot reload certs' feature, defaulted to 'false'. Only restarts will happen, i.e., Hot Cert Reload feature is disabled by default * Renamed the existing configs 'yb.features.cert_rotate' to 'yb.features.cert_reload' * Fixing a bug to send the boolean 'ybcInstalled' to createRestartTask() method Test Plan: Make sure only restarts happen during cert rotate Hot cert reload feature is completely nuked. It can be enabled by setting the above flag to 'true' at later point of time Reviewers: vpatibandla Reviewed By: vpatibandla Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21058
…ver certificates in docs (yugabyte#14577) * Added content * Implemented review comments
…pdate system cache Summary: When issuing direct DMLs to system tables (whether in an upgrade more or not), PG system caches are only updated for `INSERT` but not for `UPDATE`/`DELETE`. This is due to the fact that code path for `UPDATE`/`DELETE` was different for regular and system tables (latter were only called internally by DDLs). This diff changed DML `UPDATE`/`DELETE` with catalog tuple invalidation to align with DDLs. (Full code path unification was discussed but left out of the scope of this diff) Additionally: * Optimized arguments for internal YB DML pggate functions. * Renamed `yb_mt_is_single_row_update_or_delete` to `yb_fetch_target_tuple` (with inverted meaning) to avoid ambiguity with `yb_es_is_single_row_modify_txn` Other changes are minor. --- Resolves yugabyte#13500 Test Plan: ybd --java-test 'org.yb.pgsql.TestYsqlUpgrade#dmlsUpdatePgCache' Reviewers: mihnea, tverona, myang, jason, amartsinchyk Reviewed By: amartsinchyk Subscribers: amartsinchyk, yql Differential Revision: https://phabricator.dev.yugabyte.com/D18667
Summary: Replace DEFINE_ with DEFINE_UNKNOWN_ These are older flags that we need to review and determine if they are RUNTIME, or NON_RUNTIME. Until then tagging them as UNKNOWN. GH PR for easier review: yugabyte#14953 Created with https://gist.githubusercontent.com/hari90/760596c0c8cb1d9b5b6fd99d643dba01/raw/3914a3a03187813738c78587f31bd0cfeeb58038/add_define_unknown.txt And fixed some minor lint errors Fixes yugabyte#14945 Test Plan: All jenkins tests Reviewers: bogdan, mihnea, smishra Reviewed By: bogdan Subscribers: jenkins-bot, yugaware, asrivastava, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D21002
…borted retryable task Summary: JIRA Story: [[ https://yugabyte.atlassian.net/browse/PLAT-6141 | PLAT-6141 ]] This is to make the retry link easily visible such that the user does not have to go the task list and click retry. A retryable task for the universe has a boolean field set to true to indicate retryability. 1. Change the Status from the current generic “Error“ to {taskname} Failed. Reflect the more changed status name on other sections of the platform where Universe status is exposed such as but not limited to a. Universe List page 2. If the task is retryable add a Retry button in front of the task status. It should only appear if the tasks API returns retryable as true a. Retry is a transitory button. It will go away once the user clicks on the button. It only appears if the task fails 3. If the task is not retryable , add a button in front of the status which reads “View Details“. a. The View Details button only appears if the task fails Test Plan: This will have effect on 4 places: 1. YBUniverseItem -> **List view of /universes** 2. UniverseDetail -> **Overview page of Universe** 3. UniverseDisplayPanel -** Dashboard page (edited) ** 4. TableDetail -> **Individual Table Details page** Please refer to the videos: Case 1 video: Task retryable and not retryable {F33219} Case 2 video: How to display task name when its too long (value hardcoded for video purpose) {F33230} Reviewers: asathyan, kkannan, lsangappa, jmak Reviewed By: jmak Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D20978
Summary: The D21029 / 9d900b3 breaks compilation by gcc11 This diff solves the issue. Test Plan: Jenkins Reviewers: mbautin, steve.varnau, hsunder Reviewed By: hsunder Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D21066
Summary: Recent changes to enable ybc by default caused itests to fail. This diff adds the necessary install steps for moving ybc packages into the right location and specifying the path in the YBA conf. Test Plan: Run install with updated yba-installer Verify ybc packages are found/moved to right locations Verify create universe succeeds and ybc packages were moved during Configure step Reviewers: dshubin, vbansal, sanketh Reviewed By: vbansal Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21048
…ter a pitr restore. Summary: Adds a test confirming after a pitr restore all restored child tablets which were not compacted at the snapshot time are compacted after the restore is successful. This diff also does a hefty amount of refactoring between yb-backup and yb-admin-snapshot-schedule test files to share helper code. There are some similar tests between these two files and they were using copy-pasted helper functions. Test Plan: test only change. Reviewers: msun, skedia Reviewed By: skedia Subscribers: slingam, ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D20047
…e is change to Ephemeral
Summary:
Do not reset num volumes and num size unless instance type is change to Ephemeral
1. In case of Azure, I checked in dev portal that even when we select Azure provider ULTRA DISK SSD or any other SSD, the volume size and count resets to the same value as there is no concept of ephemeral.
2. In case of GCP, I checked in dev portal that even when we select GCP provider and `local scratch` , for all GCP instance types, the volume size and count resets to the same value as there is no concept of ephemeral.
There are 3 cases:
1. TO EPHEMERAL: Let's assume user is in AWS, selects 3 num volumes which is 500 GB during create step. Now in EDIT step, when they switch to ephemeral, num volumes and volume size gets reseted (non editable).
2. FROM EPHEMERAL: Let's assume user selects AWS EPHEMERAL INSTANCE, which is by default 1 num volume and size is 900 GB during create step. Now in EDIT step, when they switch to non-ephemeral, num volumes and volume size DOES NOT get reseted, they need to explicitly change that.
3. WITHIN NON-EPHEMERAL: Whatever num volumes and volume size we enter regardless of CREATE or EDIT step, we preserve it and do not reset.
Test Plan:
Please refer to the video:
{F32902}
Reviewers: cwang, jmak
Reviewed By: jmak
Subscribers: jenkins-bot, yugaware
Differential Revision: https://phabricator.dev.yugabyte.com/D20682
Summary: Two problems being addressed as suggestedby @gjalla in PLAT-6233. 1. Don't use == to compare Integer objects, use .equals() method instead. 2. Move the disk resize task to the end of edit universe workflow so that the pods have moved into the new placement by then. Test Plan: Run the failed itest testcomplexops and make sure "Shrink Universe" passes. (ongoing) Reviewers: sanketh, bgandhi, anijhawan, vkumar, gjalla Reviewed By: gjalla Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D20994
Summary: When generating a CREATE TABLE statement for a range-partitioned table, `ysql_dump` can dump a wrong statement if the CREATE TABLE statement is long enough and has a SPLIT AT clause. Before this diff, `getYbSplitClause` returns a pointer to freed memory, so it may be overwritten. It is more likely to be overwritten when the CREATE TABLE statement is long because, when that string is reallocated by `appendPQExpBuffer`, it can more easily allocate over the same address. This diff fixes this issue by calling `pg_strdup` to make a copy of the string `SPLIT AT VALUES ...`. In addition, `TestYsqlDump.java` is changed to allow multi-line queries in the input file to set up `ysql_dump` tests. Test Plan: ./yb_build.sh --java-test 'org.yb.pgsql.TestYsqlDump#ysqlDumpWithYbMetadata' Reviewers: tverona, jason Reviewed By: jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D21034
…ror filesystem Summary: Reorganize all the support bundle files collected in platform by node name, and mirror filesystem. Segregate the types of support bundle components into 2 types: 1. Node level 2. Platform level (aka YBA level, aka Global level). The structure of the support bundle before this PR: ``` yb-support-bundle ├─ application_logs │ ├─ application.log ├─ consensus_meta │ ├─ yb-admin-sahith-universe-sb-n1.tar.gz │ ├─ yb-admin-sahith-universe-sb-n2.tar.gz │ ├─ yb-admin-sahith-universe-sb-n3.tar.gz ├─ error_files │ ├─ yb-admin-sahith-universe-sb-n1.tar.gz │ ├─ yb-admin-sahith-universe-sb-n2.tar.gz │ ├─ yb-admin-sahith-universe-sb-n3.tar.gz ... ``` The support bundle structure after this PR: ``` yb-support-bundle ├── YBA │ └── application_logs │ └── application.log │ └── k8s_info ├── yb-admin-sahith-universe-sb-n1 │ ├── master │ │ ├── conf │ │ │ └── server.conf │ │ ├── consensus-meta │ │ │ └── 00000000000000000000000000000000 │ │ ├── instance │ │ ├── logs │ │ │ ├── yb-master.yb-admin-sahith-universe-sb-n1.yugabyte.log.ERROR.20221028-055825.7700 │ │ ├── master.err │ │ ├── master.out │ │ └── tablet-meta │ │ └── 00000000000000000000000000000000 │ └── tserver │ ├── conf │ │ └── server.conf │ ├── consensus-meta │ │ ├── 43a5f824e33d4602bc93e0f1c8b91bad │ ├── instance │ ├── logs │ │ ├── postgresql-2022-10-28_055840.log │ │ ├── yb-tserver.yb-admin-sahith-universe-sb-n1.yugabyte.log.INFO.20221028-055839.7841 │ └── tablet-meta │ ├── 43a5f824e33d4602bc93e0f1c8b91bad ├── yb-admin-sahith-universe-sb-n2 ... ├── yb-admin-sahith-universe-sb-n3 ``` Where the platform level components go under the YBA/ directory and the respective node level components go under each node directory, to try and preserve the directory structure on the db nodes, hence making debugging much easier. Test Plan: Manually tested creating and downloading a support bundle with all components ticked on the UI. Need to modify UTs. Integration tests already present, might need to tweak them. Reviewers: sneelakantan, vkumar Reviewed By: vkumar Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D20578
…ugabyte#14951) Summary: The <namespace> placeholder and the <SA_NAMESPACE> from the RBAC YAML were creating confusion on which one to replace while running the commands. With this change we set an environment variable and use it across all the commands, so that users can just set one variable and easily execute all the commands. Fixes yugabyte#11475 Test Plan: - Tried all the commands from this page with the variable value set and without it. Signed-off-by: Bhavin Gandhi <bhavin7392@gmail.com>
…overrides parsing. Summary: To fix 2.15.3 compilation error, changing the JsonProcessingException type to IOException. Test Plan: NA Reviewers: sanketh, nsingh Subscribers: yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21070
…Key on Master Init" Summary: Reverted because of a data race condition mentioned in yugabyte#14976 Test Plan: Checking if the TSAN test failure disappear. Reviewers: dfelsing Reviewed By: dfelsing Differential Revision: https://phabricator.dev.yugabyte.com/D21082
…ncryption is enabled Summary: Allow newly created universes with db versions at least 2.17.1.0-b14 to have https enabled on master/tserver web UI. We will piggyback node-to-node encryption for this first phase. That is, on universe creation, if universe db version is at least 2.17.1.0-b14 and node-to-node encryption is set as true, we will enable https, otherwise, we will set to http as before. To keep track of this, we have a `httpsEnabledUI` key in the universe config which has the values `false|true`, which determines whether we use http/https respectively. If `httpsEnabledUI` doesn't exist, we assume that the value is false. Already created universes with db version at least 2.17.1.0-b14 will currently not have their https unless they disable and then enable node-to-node encryption. We also have a new runtime config, called `node_ui` which will current be used to create a custom wsClient, which allows you to make api requests, is configured to currently accept any certificates and has a specified timeout. From this change: yugabyte@b737017. We currently able to just use our http endpoint when making requests. The request will be redirected to the associated `https` endpoint as required. Related to the above change, to enable https, we need the follow gflags set: 1. `webserver_redirect_http_to_https` - Allow node UI endpoints to be redirected from http -> https automatically 2. `webserver_certificate_file` - certificate file 3. `webserver_private_key_file ` - private key 4. `webserver_ca_certificate_file` - the certificate's CA For this commit, if the httpsEnabledUI bit is bi-directional. However, we currently don't support turning https on if the universe is upgraded to 2.17.1.0-b14. However, a user can turn off then turn on node-to-node encryption to enable this Test Plan: Fixed unit tests Made sure that all locations where node endpoints are used works when we set the gflags. This includes: 1. UniverseCRUDHandler (createUniverse) - check that we set the config properly 2. SessionController (proxyRequest) - When in the nodes page, we are able to click the master/tserver urls and whether or not the current node is a master leader, we should see the entire UI show the correct information. 3. TabletServerController (listTabletServers) - In the nodes page, we should be able to see the RAM used, SST size, etc. 4. prometheus.yml - Go to localhost:9090 and make sure that when we make a query on a node in the universe, we should get results back. Ex: `max_over_time(up{node_prefix="yb-admin-cwang-21710b19-universe"}[30s])`. Can also go to metrics page and check that the graphs are showing properly. 5. node_health.py (http_request) - Make sure that when going to the health tab, all checks should pass. 6. LiveQueryExecutor (call) - run yb-sample-apps, Ex: `docker run -d yugabytedb/yb-sample-apps --workload SqlInserts --nodes 10.9.119.167:5433,10.9.135.35:5433,10.9.204.146:5433 --username yugabyte --password <password>`. In the queries page, go to live queries and you should be able to see inserts into the table. 7. QueryHelper (queryUniverseNodes) 8. TlsToggle - Check that universes with version >= 2.17.1.0-b14 can have https turned on/off based on whether node-to-node encryption is turned on/off Reviewers: sanketh, sb-yb Reviewed By: sanketh, sb-yb Subscribers: bogdan, esheng, jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D19465
Summary: Handle JS exception "Cannot destructure Universe UUID"
Test Plan:
Please refer to the video
{F33358}
Reviewers: asathyan, lsangappa, kkannan, jmak
Reviewed By: jmak
Subscribers: jenkins-bot, yugaware
Differential Revision: https://phabricator.dev.yugabyte.com/D21074
* Launch and Manage tidyups * Develop tidyups * Update docs/content/preview/deploy/manual-deployment/start-tservers.md Co-authored-by: Aishwarya Chakravarthy <achakravarthy@yugabyte.com> * Update docs/content/preview/deploy/manual-deployment/verify-deployment.md Co-authored-by: Aishwarya Chakravarthy <achakravarthy@yugabyte.com> * review comment * renamed ORMs * review comments Co-authored-by: Aishwarya Chakravarthy <achakravarthy@yugabyte.com> Co-authored-by: aishwarya24 <ashchakravarthy@gmail.com>
* Intro/setup for Explore * Explore index page changes * try it a different way * Misc edits * minor edits * minor edits * add setup boilerplate to some topics * comments, edit, format * review comments, cleanup * format * style fix on tab widget * add shortcode for explore setup * test push * indexes and constraints * review comments * observability changes * tidyups * tidyups * tidyups * tidyups * links, format * tidyups * delete file * minor edit Co-authored-by: aishwarya24 <ashchakravarthy@gmail.com>
* added dapper orm content * Update docs/content/preview/develop/build-apps/csharp/ysql-dapper.md Co-authored-by: Dwight Hodge <79169168+ddhodge@users.noreply.github.com> * Update docs/content/preview/develop/build-apps/csharp/ysql-dapper.md Co-authored-by: Dwight Hodge <79169168+ddhodge@users.noreply.github.com> * changes from review * renamed ORMs Co-authored-by: Dwight Hodge <79169168+ddhodge@users.noreply.github.com>
Summary:
**Context**
The logic that makes the field 'volume size' readonly depends in part on
whether the selected instance type in the form is different than the instance type
in the cluster.
We currently set `currentInstanceType` to the instance type of the primary cluster if
it exists.
This issue arises when we are adding a read replica cluster. If we select an instance type
which matches that of the primary cluster, then the UI logic will consider this as 'no instance type change'
and thus prevent the 'volume size' field from being editable.
**Change**
This diff assigns `currentInstanceType` to the instance type of the current cluster:
- primary cluster if we're creating/editing primary cluster
- read replica if we're creating/editing read replica
This fixes the issue where 'volume size' become uneditable when adding a read replica with
the same instance type as the primary cluster.
Test Plan:
Create a universe
{F33370}
Try to add a read replica with the same instance type.
{F33373}
Verify that the volume size is editable
Reviewers: lsangappa, asathyan, rmadhavan
Reviewed By: rmadhavan
Subscribers: jenkins-bot, yugaware
Differential Revision: https://phabricator.dev.yugabyte.com/D21069
Summary: Use map binder and concrete class references instead of strings and reflection. Test Plan: Ran unit Test and itest Synced to head and tried failed unittest locally `Jenkins > com.yugabyte.yw.controllers.PitrControllerTest::Platform unit tests / API server / testListPitrConfigs` and it passed without problem. So this is unrelated test failure. Reviewers: nsingh Reviewed By: nsingh Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21073
Summary: Remove FLAG_TAG_pg and FLAG_TAG_auto so that it cannot be used in TAG_FLAG macros. DEFINE_RUNTIME_PG, and DEFINE_RUNTIME_AUTO should instead be used for these flags. Fixes yugabyte#14928 Test Plan: Jenkins builds Reviewers: rsami Reviewed By: rsami Subscribers: bogdan, asrivastava, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D20960
…ng read of catalog only data Summary: Catalog only reads like authentication, and \dt should use latest reads (tablet level consistency), as we always want to read latest data in non-user system tables. This makes sure we always respect the latest databases security policies like users, and roles. Also when the safe time for the database is not valid yet, it allows us to still start a connection. We can then set `yb_xcluster_consistency_level` to `tablet` if needed and access the partial data. Fixes yugabyte#14607 Test Plan: XClusterConsistencyTest Reviewers: slingam, myang, rahuldesirazu, dmitry Reviewed By: rahuldesirazu Subscribers: dmitry, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D21003
Summary: **Context** We made https the default protocol for replicated install. This made peer certifciates mandatory for HA config setup since we'll be using https. **Change** Make peer certificates mandatory when setting up an HA config. JIRA: https://yugabyte.atlassian.net/browse/PLAT-5631 Test Plan: Unit tests: managed/ui/src/components/ha/replication/HAReplicationForm.test.tsx Verify that HA config create button is disabled until at least one peer cert is added. {F33459}{F33460} Reviewers: sb-yb, rmadhavan Reviewed By: rmadhavan Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21292
… collector Summary: Moved the expired backup deletion logic to the backup garbage collector as now we have a separate scheduler thread specific to backups deletion. This will also allow us to increase the time interval for deleting expired backups. Test Plan: Tested Manually by deleting expired backups and moved/modified unit tests. Reviewers: vkumar, vpatibandla Reviewed By: vpatibandla Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21291
Summary: This Diff - Removes Epel Repo that we added as part for installation of `xxhsum`. As we are now copying over the pre generated binaries to the nodes. - Use Root Level Scope for configuring/starting systemd services for AmazonLinux2 based machines. - Minor Cleanup wrt to epel repo. Test Plan: - Tested Universe creation against AMI `ami-0d593311db5abb72b`. - iTest Pipeline. Reviewers: skhilar, muthu Reviewed By: muthu Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21080
… pinned object cache Summary: In yugabyte#3503 / D9279 we introduced a dedicated cache to speedup lookups for pinned objects. It assumed that pinned object set is immutable, so the cache was loaded once per session and never modified. However, YSQL upgrade changed that invariant, now we do direct `pg_depend` inserts as part of various migrations (e.g. new system functions definition). YSQL upgrade didn't update that cache properly though, leading to a system catalog content mismatch when a new system function was defined in the same session as creating a view using that (pinned) function, given that cache was initialized already. We didn't define such system views before, so we didn't encounter the described behaviour in practice. For more details, please refer to comments in `TestYsqlUpgrade#pinnedObjectsCacheIsUpdated`. As a fix, direct `pg_depend` inserts mentioned above, as well as new system table creation, now properly update the pinned objects cache. Limitation of this diff is that other backends will not be affected. Since missing an entry in this cache at worst might lead to a new dependency record created, this is not a big deal to be addressed separately. Some additional changes include a minor cleanup of pinned dependency cache, and name changes (`YB* -> Yb*`) Test Plan: ybd --java-test 'org.yb.pgsql.TestYsqlUpgrade#pinnedObjectsCacheIsUpdated' Reviewers: dmitry, myang, jason Reviewed By: jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D16734
…er's css
Summary:
Fixed:
Global overriding of bootstrap's dividers css
Test Plan:
Tested Manually.
{F33563}
Reviewers: asathyan, lsangappa
Reviewed By: lsangappa
Subscribers: jenkins-bot, ui, yugaware
Differential Revision: https://phabricator.dev.yugabyte.com/D21481
Summary: Need to increase max attempts for waiter during snapshot creation to avoid reaching max attempts limit issue that we have observed
Test Plan:
- Created cluster
- VM image upgrade
{F33551}
Reviewers: sanketh, daniel, yshchetinin, nsingh
Reviewed By: daniel, nsingh
Subscribers: jenkins-bot, yugaware
Differential Revision: https://phabricator.dev.yugabyte.com/D21453
… in metrics.yml Summary: Currently we return metric lines in wrong order sometimes - which causes most important metrics to be shown last in top K nodes/tables mode. Fixed metric lines order so that most important lines are shown first always. Test Plan: Check universe metrics dashboard in regular mode. Make sure metric lines are shown in the order, defined in metrics.yml. Switch to Top K mode. Check universe metrics dashboard in regular mode. Make sure metric lines are shown in the order, defined in metrics.yml, in Overall, Outlier nodes and Outlier tables mode. Reviewers: vbansal Reviewed By: vbansal Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21487
Summary: Fix of https://yugabyte.atlassian.net/browse/PLAT-5979 Code fix given for several string comparison issues which were earlier reference checks and now are string value checks. Test Plan: All tests passing successfully. Reviewers: svarshney, agunta, vkumar, vpatibandla Reviewed By: vpatibandla Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21436
Summary: Compaction task metrics currently only record task state metrics (`active`,`paused`,`queued`) for all compactions together rather than separating them by type. For easier debugging, this separates metrics as follows: Two state types: `active` and `nonactive` (`nonactive` is a composite of `paused` and `queued`). Three compaction types: `background` (i.e. minor), `full` (both admin-triggered and scheduler-triggered) and `postsplit` This diff also deprecates `paused`, `queued`, and `total` compaction metrics so that current UI graphs will not break, facilitating a transition to the new graphs. Also adds some compaction-related log messages for easier debugging. Test Plan: ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.ManualCompactionTaskMetrics ./yb_build.sh --cxx-test integration-tests_compaction-test --gtest_filter CompactionTest.BackgroundCompactionTaskMetrics Reviewers: rthallam, timur Reviewed By: timur Subscribers: ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D20893
Summary: The help text specifies the default value of --ui is false. This is incorrect and needs to be changed to true. Test Plan: No tests Reviewers: djiang Reviewed By: djiang Subscribers: nikhil Differential Revision: https://phabricator.dev.yugabyte.com/D21524
Summary: - Add json blob details to hold ProviderDetails - There are two types of fields in this new class. - Set of fields that are in access key and were also present in Provider.java. these were transient fields. So we added delegating setters to set these in details class. ``` String sshUser; Integer sshPort = 22; boolean airGapInstall = false; List<String> ntpServers = Collections.emptyList(); boolean setUpChrony = false; boolean showSetUpChrony = false; ``` - The fields that were in access key but did not really belong there. These are just placeholders for now ``` boolean passwordlessSudoAccess = true; String provisionInstanceScript = ""; boolean installNodeExporter = true; public Integer nodeExporterPort = 9300; public String nodeExporterUser = "prometheus"; public boolean skipProvisioning = false; ``` Only fields that will be left in access key: ``` String publicKey; String privateKey; String vaultPasswordFile; String vaultFile; boolean deleteRemote = true; ``` Test Plan: Unittests Reviewers: yshchetinin, svarshney Reviewed By: svarshney Subscribers: jenkins-bot, sanketh, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21377
Summary: Performing a reboot using the cloud provider API seems to causing an issue in case of GCP, where the networking changes seem to not be getting applied correctly. Reverting to the old behavior. Test Plan: Use an existing deployed platform where the cluster connectivity fails. Change the reboot to the old way, and verify the cluster creation goes through. Reviewers: sanketh, amindrov, daniel Reviewed By: daniel Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21532
…E_YBC_PKG=1 Summary: Download the ybc packages from S3 to the local ybc release directory. This would help us to get away from manual steps to download ybc whenever we update the ybc version on the Platform. Setup instructions: ``` Check whether AWS CLI is installed on your machine using aws --version If it is not installed, follow the below steps: Linux curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install Mac curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg" sudo installer -pkg AWSCLIV2.pkg -target / Check whether AWS CLI is installed using aws --version Configure aws CLI aws configure AWS Access Key ID: ****************C4M2 (Your AWS Access key) AWS Secret Access Key: ****************rkDM (Your AWS Secret key) Default region name: us-west-2 Default output format: None (Hit return) When we configure the AWS CLI, the access and secret keys are persisted in the file /home/<user-name>/.aws/config If these keys are added as ENV variables, then the ENV vars override the keys present in the .aws/config file. Set MOVE_YBC_PKG=1 as ENV variable on your dev machine. ``` Test Plan: Set MOVE_YBC_PKG=1 in my bashrc sbt compilePlatform is already downloading the ybc packages from S3 to /yugabyte-db/managed/src/universal/ybc Now we are copying these ybc packages to /opt/yugabyte/ybc/release Reviewers: sanketh, vbansal, sb-yb, amalyshev Reviewed By: vbansal, amalyshev Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21501
Summary: When the backups are cancelled, the underlying ybc task is not aborted and deleted. This was a regression while working on ybc backup/resume task. This has been fixed now. At the same time in the case of Platform shutdown though, we should not abort and delete the underlying backup/restore ybc tasks. The second change is to set the backup completion time in the case of backup failures. The third change is to set the enableYbc=false for DB < 2.15 before persisting it in the DB. Test Plan: # Take a regular backup. Check that it works as expected # Take a backup and see that it can be cancelled. Also, verify that the underlying ybc task has been aborted and deleted # While a backup is running, shut down the Platform and confirm that the backup is resuming on Platform restart. # Perform a restore backup operation. Check that it works as expected # Perform a restart backup operation. Abort the underlying restore task. Verify that the ybc task is deleted from YBA and the restore task fails. # While a restore backup is running, shut down the Platform and verify that the restore operation is resuming on Platform restart. # Verify that the backup completion time is set in the above backup failure cases. It is not being set currently. # Create a universe with DB version < 2.15 and verify that the universe is getting created with enableYbc=false in the universeTaskDetails # Create a universe with DB version >= 2.15 and verify that the universe is getting created with enableYbc=true in the universeTaskDetails Reviewers: nsingh, vkumar Reviewed By: vkumar Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21545
Summary: Because the SQL query that we run on a universe to get the slow query parameters contains a JOIN clause, it can take a rather long time to get the response in a multi-region configuration as several messages between the tserver and the possible far leader master are required. This diff leverages Batch Nested Loop JOINs (added at D20060) to increase the efficiency of running JOIN statements by looking up several batches of outer table tuples into the inner table at once. Also, because the BNL feature is not stable yet, a runtime config flag is added to disable it. Test Plan: Created a universe with version `2.15.3.0-b220` and a universe with version `2.17.1.0-b238` with a multi-region configuration and the same set of tables. Confirmed that the slow query feature works on both universes and it is faster on the universe with the higher version (It took 10.9 seconds for the old universe and 8.3 seconds for the universe with the higher version). Set the runtime config flag to false and confirm the slow query API request takes longer (In my environment, it took about 13.6 seconds when this flag was false and it took about 8.3 seconds when this flag was true). A sample command to set the flag to `false`: ``` curl --location --request PUT 'http://192.168.56.102:9000/api/v1/customers/f33e3c9b-75ab-4c30-80ad-cba85646ea39/runtime_config/6051b5bc-2a39-4311-8b1f-ca3267c0a313/key/yb.query_stats.slow_queries.set_batch_nested_loop' \ --header 'X-AUTH-YW-API-TOKEN: 2894987a-4fdf-47cf-95a2-be3f2039cf79' \ --header 'Content-Type: text/plain' \ --data-raw 'false' ``` Reviewers: sanketh, amalyshev Reviewed By: amalyshev Subscribers: yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21480
Summary: As a quick fix for install reporting success even if services don't start, we now will validate all services and running after an install and succeed/fail based on that. In order to enable this change, I have added some additional status utils, mainly a new status struct. This struct is what gets returned by [service].status now, and while the status command will use this struct to print to stdout, it can also be used to validate services are running. I did also add a systemd package, though I think this will be later replaced by an official systemd library that leverages the DBUS. Test Plan: Ran YBA-installer, validated the new status printing, and that we exitted successfully (with services running) Reviewers: tramer, sanketh, muthu Reviewed By: muthu Subscribers: jenkins-bot, sanketh, muthu, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21420
…ed-ui api server Summary: Use pgxpool instead of pgx for yugabyted-ui api server Test Plan: no test plan Reviewers: nikhil Reviewed By: nikhil Subscribers: djiang Differential Revision: https://phabricator.dev.yugabyte.com/D21567
Summary: Move TopK metrics FF to runtime config 1.Wait for the following diff to be merged: D20670 2.Remove enableTopK metrics FF in /features page 3.Create a runtime config property for new topk metrics view 4.runtime config should be editable at super admin level to individual universes Test Plan: Please refer to the video: [[ https://drive.google.com/drive/folders/1Ffq7x5o61PluZtQ6pi-fvexsIQvS8k2n?usp=share_link | PLAT-6114 ]] Reviewers: asathyan, lsangappa, kkannan, jmak Reviewed By: jmak Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D20717
Summary: As part of cleanup `Update Package` task was removed from the SubTaskGroupType enum, which can cause issue while loading tasks(if this task was run at some point in time) as the ebean cannot load the objects if this subtask type already exists in some task that was executed previously Depends On D21080 Test Plan: itest Pipeline Reviewers: hzare Reviewed By: hzare Subscribers: jenkins-bot, yugaware Differential Revision: https://phabricator.dev.yugabyte.com/D21564
…moteBootstrapTest Summary: Test `CreateTableITest.TableColocationRemoteBootstrapTest` consistently failed on TSAN build. It should be disabled in TSAN, as per most of our YSQL tests. Test Plan: All existing Jenkins tests Reviewers: bogdan Reviewed By: bogdan Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D21565
Co-authored-by: Dwight Hodge <79169168+ddhodge@users.noreply.github.com> resolves yugabyte#15012
…ablet splitting Summary: ``` create table demo ( id int generated always as identity, data text, primary key(id asc) ); copy demo(data) from program 'base64 -w 16384 /dev/urandom | head -c 629145600'; ``` Creates and splits a single tablet into 412MB/190MB instead of ~2*300MB. This might happen because we take split key (midpoint) of the largest SST file and it just contains continuous range of IDs. To resolve such issues partially, we can take each SSTable file's mid-point, and do a weighted mid-point of mid-points. Each mid-point's weight could be based on the size of the SSTable maybe because we are doing size based splitting. Test Plan: Added an test in table_test yb_build.sh --gtest_filter TableTest.MiddleOfMiddleKey -n 100 -- -p 1 Reviewers: arybochkin, timur Reviewed By: timur Subscribers: ybase, arybochkin Differential Revision: https://phabricator.dev.yugabyte.com/D21214
jayant07-yb
force-pushed
the
Handling_exceptions_session_parameters
branch
from
78cb174 to
26a07a4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.