Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

protocol upgrade activation mechanism (also implements PREACTIVATE_FEATURE and ONLY_LINK_TO_EXISTING_PERMISSION protocol features) #6831

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
5799292
initial work on #6429
arhag Jan 30, 2019
1d2e24b
more work on #6429
arhag Feb 16, 2019
19e7720
some fixes for #6429
arhag Feb 20, 2019
0784a75
Merge branch 'protocol-feature-foundations' into 6429-protocol-upgrad…
arhag Feb 22, 2019
08318b7
update protocol-feature structs to use new fc::reflector_init; loggin…
arhag Feb 23, 2019
1fd25de
enum hash function needs to be explicit to work with old gcc compiler…
arhag Feb 25, 2019
9940b2c
use nth_element rather than sort in calc_dpos_last_irreversible
arhag Feb 25, 2019
cee12cc
switch protocol_feature_activation_handlers to unordered_map #6429
arhag Feb 25, 2019
a72eda9
basic support for preactivation #6429 #6431
arhag Feb 25, 2019
2d4d0d3
initial work on #6437
arhag Feb 28, 2019
feb6456
bug fixes and improvements #6437
arhag Feb 28, 2019
2aeb626
fix pending_schedule_hash
taokayan Mar 5, 2019
9d14248
Merge pull request #6865 from taokayan/6429-fix-kayan
arhag Mar 5, 2019
3e9a2c8
Merge branch 'protocol-feature-foundations' into 6429-protocol-upgrad…
arhag Mar 6, 2019
2ce825f
Merge branch 'protocol-feature-foundations' into 6429-protocol-upgrad…
arhag Mar 6, 2019
97f57de
add dependencies to hash that generates feature digest
arhag Mar 7, 2019
1459875
remove digest from protocol feature JSON filenames since they can bec…
arhag Mar 7, 2019
8c915c9
set description digest for PREACTIVATE_FEATURE
arhag Mar 7, 2019
6e3c7cb
fix bug in protocol_feature_manager::add_feature leading to undefined…
arhag Mar 7, 2019
bfead42
Merge branch '6437-intrinsics' into 6429-protocol-upgrade-activation-…
arhag Mar 9, 2019
71c4187
Merge branch 'protocol-feature-foundations' into 6429-protocol-upgrad…
arhag Mar 9, 2019
66ef224
Merge branch 'protocol-feature-foundations' into 6429-protocol-upgrad…
arhag Mar 9, 2019
1dd9a35
add preactivate_feature and is_feature_activated intrinsics; add get_…
arhag Mar 11, 2019
df1379e
fix bug in pending_block_header_state::_finish_next which created an …
arhag Mar 11, 2019
c64c963
add intrinsics on PREACTIVATE_FEATURE activation
arhag Mar 11, 2019
9deb6f7
add protocol_feature_tests/activate_preactivate_feature unit test
arhag Mar 12, 2019
9340e2c
Initial modification to tester for pfm without modifying current test…
Mar 12, 2019
2398d8c
Change default setup_policy to full and fixes the existing test
Mar 12, 2019
8fb0d29
rename schedule_all_builtin_protocol_features to preactivate_all_buil…
arhag Mar 12, 2019
6bd09b1
for now switch back to using old bios contract in smoke tests until t…
arhag Mar 12, 2019
6ec6afd
Merge pull request #6917 from EOSIO/6429-test
arhag Mar 12, 2019
5c36f61
avoid redundant validate_and_extract_header_extensions in fork_databa…
arhag Mar 12, 2019
9ad228b
check for invalid protocol feature activations earlier when producing…
arhag Mar 12, 2019
0b00d78
correct action name for preactivate_protocol_features
arhag Mar 12, 2019
625fce6
add dependency checking to controller::preactivate_feature
arhag Mar 12, 2019
3628676
bug fix in start_block: improper count to check if all preactivated f…
arhag Mar 13, 2019
a4bde90
add get_supported_protocol_features to producer_api_plugin
arhag Mar 13, 2019
caaa5c9
Fix reversed condition in get supported protocol API
Mar 14, 2019
488b8e6
fix a case if started block get aborted
taokayan Mar 14, 2019
421bc24
fix duplication check bug in controller_impl::start_block
arhag Mar 14, 2019
1b5cece
add ONLY_LINK_TO_EXISTING_PERMISSON protocol feature (enables writing…
arhag Mar 14, 2019
a7158b1
add additional unit tests for protocol activation; fix bug that lead …
arhag Mar 14, 2019
1cdf8ed
Merge pull request #6935 from taokayan/6429-producerfix-kayan
arhag Mar 14, 2019
d9fe459
Add capability to activate preactivate feature in the python testing …
Mar 13, 2019
a9d9152
decode binary RPC response in sendRpcApi before converting to JSON
arhag Mar 13, 2019
e7440bb
Use API to get feature digest for preactivate protocol feature
Mar 14, 2019
2383606
Change python cluster to use setup policy enum
Mar 14, 2019
5b77b49
remove dependency on enum.auto (not available to all platforms)
arhag Mar 14, 2019
424374d
Preactivate feature inside bios_boot.sh created by launcher
Mar 15, 2019
2211e9a
fix single producer node case for preactivation
taokayan Mar 15, 2019
355f143
Redesign protocol_feature_manager to fix bug which would lead to inco…
arhag Mar 16, 2019
f23069d
add protocol_feature_tests/activate_and_restart unit test
arhag Mar 16, 2019
a15322a
port over @taokayan's producer_plugin fixes from EOSIO/eos#6949
arhag Mar 16, 2019
ab24a3f
Merge branch '6429-protocol-upgrade-activation-mechanism' into 6429-p…
arhag Mar 16, 2019
0d564cf
fix python tests
taokayan Mar 18, 2019
a903640
fix bug in block_header_state::next regarding producer_to_last_produc…
arhag Mar 19, 2019
21ec01f
Add some helper functions for Cluster and Node
Mar 19, 2019
d85b415
Remove dependency on enum module
Mar 19, 2019
2a93b33
Remove wrong type checking
Mar 19, 2019
43d8eb9
fix python test that failed in some servers
taokayan Mar 19, 2019
a1b7b56
Merge pull request #6924 from EOSIO/6429-python-test
arhag Mar 19, 2019
e1e776f
Merge pull request #6949 from taokayan/6429-producerfix-kayan
arhag Mar 19, 2019
dfe276b
Set PFSetupPolicy to be NONE for prod preactivate test
Mar 19, 2019
53d77f4
Merge pull request #6965 from EOSIO/6429-fix-after-merge
taokayan Mar 19, 2019
ad8da6b
enforce preactivation_required in controller_impl::start_block; add u…
arhag Mar 19, 2019
568476a
protocol_feature_set::make_default_builtin_protocol_feature can be st…
arhag Mar 20, 2019
c4631a5
Add subjective restriction procotol feature test
Mar 19, 2019
a6bed12
Add test to CMakeList.txt
Mar 19, 2019
ec1e2f1
nodeos_protocol_feature_test now only tests JSON read feature
Mar 20, 2019
a54dd46
Label nodeos_protocol_feature_test as nonparallelizable_tests
Mar 20, 2019
ff870f5
prevent producer plugin from scheduling a feature that require preact…
taokayan Mar 20, 2019
ccb1a8c
Add unit test to test subject restrictions
Mar 20, 2019
765d831
Add more coverage on the test case
Mar 20, 2019
de1565e
Change permission of nodeos_protocol_feature_test to executable
Mar 20, 2019
a860f9f
Merge pull request #6972 from taokayan/6429-producerfix2-kayan
arhag Mar 20, 2019
dde3b35
pass custom_subjective_restrictions by reference into make_protocol_f…
arhag Mar 20, 2019
d89e525
some cleanup in protocol_feature_tests/subjective_restrictions_test
arhag Mar 20, 2019
57c2f53
Merge pull request #6964 from EOSIO/6429-subj-protocol-python-test
arhag Mar 20, 2019
6dfb043
allow is_feature_activated to be called by unprivileged contracts
arhag Mar 20, 2019
79c2be8
fix bios_boot script for feature_digest param
taokayan Mar 21, 2019
59bce76
Merge pull request #6977 from taokayan/6429-scriptfix-kayan
arhag Mar 21, 2019
ab4a0ef
store whitelisted_intrinsics in portable snapshot as a set of intrins…
arhag Mar 22, 2019
a8cd499
remove wlogs from get_activated_protocol_features
arhag Mar 22, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion libraries/chain/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,16 @@ add_library( eosio_chain
webassembly/wabt.cpp

# get_config.cpp
# global_property_object.cpp
#
# contracts/chain_initializer.cpp


transaction_metadata.cpp
protocol_state_object.cpp
protocol_feature_activation.cpp
protocol_feature_manager.cpp
genesis_intrinsics.cpp
whitelisted_intrinsics.cpp
${HEADERS}
)

Expand Down
39 changes: 39 additions & 0 deletions libraries/chain/block_header.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,44 @@ namespace eosio { namespace chain {
return result;
}

vector<block_header_extensions> block_header::validate_and_extract_header_extensions()const {
using block_header_extensions_t = block_header_extension_types::block_header_extensions_t;
using decompose_t = block_header_extension_types::decompose_t;

static_assert( std::is_same<block_header_extensions_t, block_header_extensions>::value,
"block_header_extensions is not setup as expected" );

vector<block_header_extensions_t> results;

uint16_t id_type_lower_bound = 0;

for( size_t i = 0; i < header_extensions.size(); ++i ) {
const auto& e = header_extensions[i];
auto id = e.first;

EOS_ASSERT( id >= id_type_lower_bound, invalid_block_header_extension,
"Block header extensions are not in the correct order (ascending id types required)"
);

results.emplace_back();

auto match = decompose_t::extract<block_header_extensions_t>( id, e.second, results.back() );
EOS_ASSERT( match, invalid_block_header_extension,
"Block header extension with id type ${id} is not supported",
("id", id)
);

if( match->enforce_unique ) {
EOS_ASSERT( i == 0 || id > id_type_lower_bound, invalid_block_header_extension,
"Block header extension with id type ${id} is not allowed to repeat",
("id", id)
);
}

id_type_lower_bound = id;
}

return results;
}

} }
141 changes: 104 additions & 37 deletions libraries/chain/block_header_state.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ namespace eosio { namespace chain {
/// 2/3 must be greater, so if I go 1/3 into the list sorted from low to high, then 2/3 are greater

if( blocknums.size() == 0 ) return 0;
/// TODO: update to nth_element
std::sort( blocknums.begin(), blocknums.end() );
return blocknums[ (blocknums.size()-1) / 3 ];

std::size_t index = (blocknums.size()-1) / 3;
std::nth_element( blocknums.begin(), blocknums.begin() + index, blocknums.end() );
return blocknums[ index ];
}

pending_block_header_state block_header_state::next( block_timestamp_type when,
Expand Down Expand Up @@ -54,6 +55,7 @@ namespace eosio { namespace chain {
result.timestamp = when;
result.confirmed = num_prev_blocks_to_confirm;
result.active_schedule_version = active_schedule.version;
result.prev_activated_protocol_features = activated_protocol_features;

result.block_signing_key = prokey.block_signing_key;
result.producer = prokey.producer_name;
Expand Down Expand Up @@ -129,6 +131,7 @@ namespace eosio { namespace chain {
}
}
}
new_producer_to_last_produced[prokey.producer_name] = result.block_num;

result.producer_to_last_produced = std::move( new_producer_to_last_produced );

Expand Down Expand Up @@ -161,9 +164,12 @@ namespace eosio { namespace chain {
return result;
}

signed_block_header pending_block_header_state::make_block_header( const checksum256_type& transaction_mroot,
const checksum256_type& action_mroot,
optional<producer_schedule_type>&& new_producers )const
signed_block_header pending_block_header_state::make_block_header(
const checksum256_type& transaction_mroot,
const checksum256_type& action_mroot,
optional<producer_schedule_type>&& new_producers,
vector<digest_type>&& new_protocol_feature_activations
)const
{
signed_block_header h;

Expand All @@ -176,36 +182,65 @@ namespace eosio { namespace chain {
h.schedule_version = active_schedule_version;
h.new_producers = std::move(new_producers);

if( new_protocol_feature_activations.size() > 0 ) {
h.header_extensions.emplace_back(
protocol_feature_activation::extension_id(),
fc::raw::pack( protocol_feature_activation{ std::move(new_protocol_feature_activations) } )
);
}

return h;
}

block_header_state pending_block_header_state::_finish_next( const signed_block_header& h )&&
block_header_state pending_block_header_state::_finish_next(
const signed_block_header& h,
const std::function<void( block_timestamp_type,
const flat_set<digest_type>&,
const vector<digest_type>& )>& validator
)&&
{
EOS_ASSERT( h.timestamp == timestamp, block_validate_exception, "timestamp mismatch" );
EOS_ASSERT( h.previous == previous, unlinkable_block_exception, "previous mismatch" );
EOS_ASSERT( h.confirmed == confirmed, block_validate_exception, "confirmed mismatch" );
EOS_ASSERT( h.producer == producer, wrong_producer, "wrong producer specified" );
EOS_ASSERT( h.schedule_version == active_schedule_version, producer_schedule_exception, "schedule_version in signed block is corrupted" );

EOS_ASSERT( h.header_extensions.size() == 0, block_validate_exception, "no supported extensions" );

if( h.new_producers ) {
EOS_ASSERT( !was_pending_promoted, producer_schedule_exception, "cannot set pending producer schedule in the same block in which pending was promoted to active" );
EOS_ASSERT( h.new_producers->version == active_schedule.version + 1, producer_schedule_exception, "wrong producer schedule version specified" );
EOS_ASSERT( prev_pending_schedule.schedule.producers.size() == 0, producer_schedule_exception,
"cannot set new pending producers until last pending is confirmed" );
}

protocol_feature_activation_set_ptr new_activated_protocol_features;

auto exts = h.validate_and_extract_header_extensions();
{
if( exts.size() > 0 ) {
const auto& new_protocol_features = exts.front().get<protocol_feature_activation>().protocol_features;
validator( timestamp, prev_activated_protocol_features->protocol_features, new_protocol_features );

new_activated_protocol_features = std::make_shared<protocol_feature_activation_set>(
*prev_activated_protocol_features,
new_protocol_features
);
} else {
new_activated_protocol_features = std::move( prev_activated_protocol_features );
}
}

auto block_number = block_num;

block_header_state result( std::move( *static_cast<detail::block_header_state_common*>(this) ) );

result.id = h.id();
result.header = h;

result.header_exts = std::move(exts);

if( h.new_producers ) {
result.pending_schedule.schedule = *h.new_producers;
result.pending_schedule.schedule_hash = digest_type::hash( result.pending_schedule );
result.pending_schedule.schedule_hash = digest_type::hash( *h.new_producers );
result.pending_schedule.schedule_lib_num = block_number;
} else {
if( was_pending_promoted ) {
Expand All @@ -217,13 +252,20 @@ namespace eosio { namespace chain {
result.pending_schedule.schedule_lib_num = prev_pending_schedule.schedule_lib_num;
}

result.activated_protocol_features = std::move( new_activated_protocol_features );

return result;
}

block_header_state pending_block_header_state::finish_next( const signed_block_header& h,
bool skip_validate_signee )&&
block_header_state pending_block_header_state::finish_next(
const signed_block_header& h,
const std::function<void( block_timestamp_type,
const flat_set<digest_type>&,
const vector<digest_type>& )>& validator,
bool skip_validate_signee
)&&
{
auto result = std::move(*this)._finish_next( h );
auto result = std::move(*this)._finish_next( h, validator );

// ASSUMPTION FROM controller_impl::apply_block = all untrusted blocks will have their signatures pre-validated here
if( !skip_validate_signee ) {
Expand All @@ -233,10 +275,15 @@ namespace eosio { namespace chain {
return result;
}

block_header_state pending_block_header_state::finish_next( signed_block_header& h,
const std::function<signature_type(const digest_type&)>& signer )&&
block_header_state pending_block_header_state::finish_next(
signed_block_header& h,
const std::function<void( block_timestamp_type,
const flat_set<digest_type>&,
const vector<digest_type>& )>& validator,
const std::function<signature_type(const digest_type&)>& signer
)&&
{
auto result = std::move(*this)._finish_next( h );
auto result = std::move(*this)._finish_next( h, validator );
result.sign( signer );
h.producer_signature = result.header.producer_signature;
return result;
Expand All @@ -250,28 +297,48 @@ namespace eosio { namespace chain {
*
* If the header specifies new_producers then apply them accordingly.
*/
block_header_state block_header_state::next( const signed_block_header& h, bool skip_validate_signee )const {
return next( h.timestamp, h.confirmed ).finish_next( h, skip_validate_signee );
block_header_state block_header_state::next(
const signed_block_header& h,
const std::function<void( block_timestamp_type,
const flat_set<digest_type>&,
const vector<digest_type>& )>& validator,
bool skip_validate_signee )const
{
return next( h.timestamp, h.confirmed ).finish_next( h, validator, skip_validate_signee );
}

digest_type block_header_state::sig_digest()const {
auto header_bmroot = digest_type::hash( std::make_pair( header.digest(), blockroot_merkle.get_root() ) );
return digest_type::hash( std::make_pair(header_bmroot, pending_schedule.schedule_hash) );
}

void block_header_state::sign( const std::function<signature_type(const digest_type&)>& signer ) {
auto d = sig_digest();
header.producer_signature = signer( d );
EOS_ASSERT( block_signing_key == fc::crypto::public_key( header.producer_signature, d ), wrong_signing_key, "block is signed with unexpected key" );
}

public_key_type block_header_state::signee()const {
return fc::crypto::public_key( header.producer_signature, sig_digest(), true );
}

void block_header_state::verify_signee( const public_key_type& signee )const {
EOS_ASSERT( block_signing_key == signee, wrong_signing_key, "block not signed by expected key",
("block_signing_key", block_signing_key)( "signee", signee ) );
}
digest_type block_header_state::sig_digest()const {
auto header_bmroot = digest_type::hash( std::make_pair( header.digest(), blockroot_merkle.get_root() ) );
return digest_type::hash( std::make_pair(header_bmroot, pending_schedule.schedule_hash) );
}

void block_header_state::sign( const std::function<signature_type(const digest_type&)>& signer ) {
auto d = sig_digest();
header.producer_signature = signer( d );
EOS_ASSERT( block_signing_key == fc::crypto::public_key( header.producer_signature, d ),
wrong_signing_key, "block is signed with unexpected key" );
}

public_key_type block_header_state::signee()const {
return fc::crypto::public_key( header.producer_signature, sig_digest(), true );
}

void block_header_state::verify_signee( const public_key_type& signee )const {
EOS_ASSERT( block_signing_key == signee, wrong_signing_key,
"block not signed by expected key",
("block_signing_key", block_signing_key)( "signee", signee ) );
}

/**
* Reference cannot outlive *this. Assumes header_exts is not mutated after instatiation.
*/
const vector<digest_type>& block_header_state::get_new_protocol_feature_activations()const {
static const vector<digest_type> no_activations{};

if( header_exts.size() == 0 || !header_exts.front().contains<protocol_feature_activation>() )
return no_activations;

return header_exts.front().get<protocol_feature_activation>().protocol_features;
}

} } /// namespace eosio::chain
15 changes: 12 additions & 3 deletions libraries/chain/block_state.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,24 @@ namespace eosio { namespace chain {

block_state::block_state( const block_header_state& prev,
signed_block_ptr b,
const std::function<void( block_timestamp_type,
const flat_set<digest_type>&,
const vector<digest_type>& )>& validator,
bool skip_validate_signee
)
:block_header_state( prev.next( *b, skip_validate_signee ) )
:block_header_state( prev.next( *b, validator, skip_validate_signee ) )
,block( std::move(b) )
{}

block_state::block_state( pending_block_header_state&& cur,
signed_block_ptr&& b,
vector<transaction_metadata_ptr>&& trx_metas,
const std::function<void( block_timestamp_type,
const flat_set<digest_type>&,
const vector<digest_type>& )>& validator,
const std::function<signature_type(const digest_type&)>& signer
)
:block_header_state( std::move(cur).finish_next( *b, signer ) )
:block_header_state( std::move(cur).finish_next( *b, validator, signer ) )
,block( std::move(b) )
,trxs( std::move(trx_metas) )
{}
Expand All @@ -25,9 +31,12 @@ namespace eosio { namespace chain {
block_state::block_state( pending_block_header_state&& cur,
const signed_block_ptr& b,
vector<transaction_metadata_ptr>&& trx_metas,
const std::function<void( block_timestamp_type,
const flat_set<digest_type>&,
const vector<digest_type>& )>& validator,
bool skip_validate_signee
)
:block_header_state( std::move(cur).finish_next( *b, skip_validate_signee ) )
:block_header_state( std::move(cur).finish_next( *b, validator, skip_validate_signee ) )
,block( b )
,trxs( std::move(trx_metas) )
{}
Expand Down
Loading