Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simple cap by including sysvars and native programs #13884

Merged
merged 11 commits into from
Dec 14, 2020

Conversation

ryoqun
Copy link
Member

@ryoqun ryoqun commented Dec 1, 2020

Problem

Simplify capitalization calculation by removing special casing.

Summary of Changes

Adjust cap while adding/updating sysvar and native program accounts correctly to simplify cap calculation.

Also, this fixes a regression bug and a new bug introduced at #13403. In this sense, this pr is a follow up to #13403, #11750. (EDIT: this was done by separate pr)

@ryoqun ryoqun added the noCI Suppress CI on this Pull Request label Dec 1, 2020
@ryoqun ryoqun added CI Pull Request is ready to enter CI v1.4 and removed noCI Suppress CI on this Pull Request labels Dec 3, 2020
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Dec 3, 2020
Comment on lines 4449 to 4450
let existing_sysvar_account_count = 9;
let existing_native_program_account_count = 7;
Copy link
Member Author

@ryoqun ryoqun Dec 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I killed the cluster, this would mean I even fail the elementary math class, I guess... ;)

@@ -2028,13 +2043,16 @@ impl Bank {
// flush the Stakes cache
account.lamports = 0;
self.store_account(&program_id, &account);
None
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a test justifying this new behavior

Copy link
Member Author

@ryoqun ryoqun Dec 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improved existing: https://github.com/solana-labs/solana/pull/13884/files#r535949045
Also wrote a new test case specifically for this: (test_add_native_program_squated_while_not_replacing)

@codecov
Copy link

codecov bot commented Dec 3, 2020

Codecov Report

Merging #13884 (4582de3) into master (52c2cbd) will increase coverage by 0.0%.
The diff coverage is 97.5%.

@@           Coverage Diff            @@
##           master   #13884    +/-   ##
========================================
  Coverage    82.2%    82.2%            
========================================
  Files         384      384            
  Lines       94803    94965   +162     
========================================
+ Hits        77948    78103   +155     
- Misses      16855    16862     +7     

Copy link
Member

@mvines mvines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read through this draft, looks good

std::process::exit(1);

#[cfg(test)]
panic!("process::exit(1) is intercepted for friendly test failure...");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice idea

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! Ideally, I want to get rid of all process:exit(1)s in lib code... But, I don't have enough bandwidth; this is a tiny 10-ish lines of hacky work-around for the current situation...

@@ -197,6 +196,14 @@ pub struct Validator {
ip_echo_server: solana_net_utils::IpEchoServer,
}

fn abort() -> ! {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my til: !! (never type)

runtime/src/bank.rs Outdated Show resolved Hide resolved
@@ -8441,13 +8630,34 @@ pub(crate) mod tests {
assert!(bank.stakes.read().unwrap().vote_accounts().is_empty());
assert!(bank.stakes.read().unwrap().stake_delegations().is_empty());
assert_eq!(bank.calculate_capitalization(), bank.capitalization());
assert_eq!(
Copy link
Member Author

@ryoqun ryoqun Dec 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these additional assertions to a test written at #11750 exposes a regression introduced at #13403.

runtime/src/bank.rs Outdated Show resolved Hide resolved
})
},
|old, new| {
// creating new sysvar twice in a slot shouldn't increment capitalization twice
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(repost)

Very subtle bug in the original draft pr was caught here.

Combined with new use of inherit_specially_retained_account_balance in this (now-parameterized) test, this new code block illustrates why it was a bad idea to update bank.capitalization in inherit_specially_retained_account_balance (this was my original draft pr changes)...

@ryoqun
Copy link
Member Author

ryoqun commented Dec 10, 2020

Status update: well greatly interrupted by other things, but I've finally finished almost all work locally. This pr is just waiting to be rebased on #14042 after its merge.

@ryoqun ryoqun marked this pull request as ready for review December 12, 2020 17:07
if let Some(account) = self.get_account(&feature_set::simpler_capitalization::id()) {
if let Some(feature) = feature::from_account(&account) {
if feature.activated_at == Some(0) {
return true;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is pretty safe, not having any possibility of falsely activating this feature on testnet/mainnet-beta.

let mut existing_native_program_account_count = 4;

if self.get_account(&sysvar::rewards::id()).is_some() {
existing_sysvar_account_count += 1;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh, casual looking of coverage revealed I'm missing to test this branch... ;) b6aa451

image

@@ -1998,6 +2002,9 @@ impl Bank {
self.store_account(pubkey, account);
self.capitalization.fetch_add(account.lamports, Relaxed);
}
// updating sysvars (the fees sysvar in this case) now depends on feature activations in
// genesis_config.accounts above
self.update_fees();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was a hard to debug thing... ;p

@ryoqun ryoqun requested a review from mvines December 12, 2020 18:44
@ryoqun
Copy link
Member Author

ryoqun commented Dec 12, 2020

@mvines hooray! i needed various supllemental prs for this (see referenced links...). But finally this pr is finally ready for serious review. I think this pr is pretty solid now compared from 9 days ago #13884 (review)

mvines
mvines previously approved these changes Dec 14, 2020
Copy link
Member

@mvines mvines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I think I'd prefer simpler_capitalization be called simple_capitalization as a global search and replace.

@ryoqun ryoqun changed the title Simpler cap by including sysvars and native programs Simple cap by including sysvars and native programs Dec 14, 2020
@mergify mergify bot dismissed mvines’s stale review December 14, 2020 05:54

Pull request has been modified.

@ryoqun
Copy link
Member Author

ryoqun commented Dec 14, 2020

Looks good to me. I think I'd prefer simpler_capitalization be called simple_capitalization as a global search and replace.

Thanks for the review! Yeah, I've renamed: 31a3cd6

@ryoqun ryoqun merged commit de9ac43 into solana-labs:master Dec 14, 2020
@ryoqun
Copy link
Member Author

ryoqun commented Dec 14, 2020

After long and extensive testing, I finally merged this. :)

mergify bot pushed a commit that referenced this pull request Dec 14, 2020
* Simpler cap by including sysvars and native programs

* Fix tests

* Add comment

* revert some unrelated code

* Update test_bank_update_sysvar_account for cap.

* Test cap. for add_native_program using new helper

* Improve the cap adjustment with new tests

* Fix typo...

* Adjust test for improved code coverage

* Rename simpler_capitalization => simple_capitalization

* More rename and bonus commenting

(cherry picked from commit de9ac43)
mergify bot added a commit that referenced this pull request Dec 14, 2020
* Simpler cap by including sysvars and native programs

* Fix tests

* Add comment

* revert some unrelated code

* Update test_bank_update_sysvar_account for cap.

* Test cap. for add_native_program using new helper

* Improve the cap adjustment with new tests

* Fix typo...

* Adjust test for improved code coverage

* Rename simpler_capitalization => simple_capitalization

* More rename and bonus commenting

(cherry picked from commit de9ac43)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants