Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ip address validation #2

Closed
massa-bot opened this issue Nov 8, 2021 · 28 comments
Closed

Ip address validation #2

massa-bot opened this issue Nov 8, 2021 · 28 comments
Labels

Comments

@massa-bot
Copy link

In GitLab by @flipchan

Verify that there is no invalid ip addresses in the file, so that someone can't "fool" the program to submit bad/invalid ips

@massa-bot
Copy link
Author

In GitLab by @flipchan

support for ipv6?

@massa-bot
Copy link
Author

In GitLab by @flipchan

rfc 1918?

@massa-bot
Copy link
Author

In GitLab by @flipchan

https://tools.ietf.org/html/rfc1918

@massa-bot
Copy link
Author

In GitLab by @flipchan

blacklisted ips:

0.0.0.0
127.0.0.1
0:0:0:0:0:0:0:1
::1
255.255.255.255

@massa-bot
Copy link
Author

In GitLab by @flipchan

ip validation regex would be good to have

@massa-bot
Copy link
Author

In GitLab by @damip

@flipchan let's avoid regexes and manual blacklists. How about this stuff ?

Can you check if this is what we want and that there are no corner cases ?

@massa-bot
Copy link
Author

In GitLab by @damip

assigned to @flipchan

@massa-bot
Copy link
Author

In GitLab by @damip

@flipchan can you also check if it would consider docker network IPs as global by default ? And if not, what should we do in our docker compose to emulate a global IP range ?

@massa-bot
Copy link
Author

In GitLab by @flipchan

@damip

Sample

use std::net::IpAddr;

fn validate_ip(ip: IpAddr) -> bool {
    !ip.is_loopback() && !ip.is_unspecified() && !ip.is_multicast()
}

fn main() {
let testipv6: IpAddr = "0000:0000:0000:0000:0000:0000:0000:0001".parse().unwrap();
let testipv4: IpAddr = "127.0.0.1".parse().unwrap();
 validate_ip(testipv6);//false
validate_ip(testipv4);//false
}

https://doc.rust-lang.org/std/net/enum.IpAddr.html#method.is_multicast

@massa-bot
Copy link
Author

@massa-bot
Copy link
Author

@massa-bot
Copy link
Author

In GitLab by @damip

@flipchan IpAddr already validates the syntax when it is created. After that, IpAddr.is_global() checks if the IP is globally routable.

@massa-bot
Copy link
Author

In GitLab by @flipchan

@damip is_global is in nightly

@massa-bot
Copy link
Author

In GitLab by @flipchan

    #[rustc_const_unstable(feature = "const_ip", issue = "76205")]
    pub const fn is_global(&self) -> bool {
        match self {
            IpAddr::V4(ip) => ip.is_global(),
            IpAddr::V6(ip) => ip.is_global(),
        }
    }
    #[rustc_const_unstable(feature = "const_ipv4", issue = "76205")]
    pub const fn is_global(&self) -> bool {
        // check if this address is 192.0.0.9 or 192.0.0.10. These addresses are the only two
        // globally routable addresses in the 192.0.0.0/24 range.
        if u32::from_be_bytes(self.octets()) == 0xc0000009
            || u32::from_be_bytes(self.octets()) == 0xc000000a
        {
            return true;
        }
        !self.is_private()
            && !self.is_loopback()
            && !self.is_link_local()
            && !self.is_broadcast()
            && !self.is_documentation()
            && !self.is_shared()
            && !self.is_ietf_protocol_assignment()
            && !self.is_reserved()
            && !self.is_benchmarking()
            // Make sure the address is not in 0.0.0.0/8
            && self.octets()[0] != 0
    }

@massa-bot
Copy link
Author

In GitLab by @flipchan

is_global is still under dev rust-lang/rust#76205

@massa-bot
Copy link
Author

In GitLab by @damip

@flipchan we are using nightly :) just need to activate the ip flag:

#![feature(ip)]

@massa-bot
Copy link
Author

In GitLab by @flipchan

the ci system is building the stable release, https://gitlab.com/massalabs/massa-network/-/blob/master/.gitlab-ci.yml

@massa-bot
Copy link
Author

In GitLab by @damip

@flipchan Does it compile with the current CI ? If the ip feature is absent from rust:latest which we are currently using, we can switch to this: https://hub.docker.com/r/rustlang/rust/

@massa-bot
Copy link
Author

In GitLab by @flipchan

hmm.. maybe? It should because of the release flag(--release), doe removing --release might compile it as nightly, not sure

@massa-bot
Copy link
Author

In GitLab by @flipchan

@damip is_global calls is_private, so if there is a node on the same local network it will not work :/

@massa-bot
Copy link
Author

In GitLab by @damip

@flipchan that was the point of the question I asked you 4 days ago (see above):

"can you also check if it would consider docker network IPs as global by default ? And if not, what should we do in our docker compose to emulate a global IP range ?"

@massa-bot
Copy link
Author

In GitLab by @flipchan

what is an invalid ip address?

  • non routable
  • multicast

@massa-bot
Copy link
Author

@massa-bot
Copy link
Author

In GitLab by @flipchan

created merge request !10 to address this issue

@massa-bot
Copy link
Author

In GitLab by @flipchan

mentioned in merge request !10

@massa-bot
Copy link
Author

In GitLab by @flipchan

best recommended action seems to be: try to connect, if it fails, it fails

But still should validate that its not a multicast/broadcast address, if its a public ip addr and not routable by the gateway/system/firewall, socket will fail to connect and it will raise a failure in PeerInfo

@massa-bot
Copy link
Author

In GitLab by @damip

is_global is enough, just need to activate the "ip" feature at crate level.

@massa-bot
Copy link
Author

In GitLab by @flipchan

@damip if someone wants to connect to nodes listening on 127.0.0.1 it wont work when using is_global

yvan-sraka pushed a commit that referenced this issue Nov 14, 2021
Resolve "Client: clean output of wallet_info"

Closes #2

See merge request massalabs/massa!3
AurelienFT added a commit that referenced this issue May 1, 2023
* Delete old network and protocol

* Use the new massa-protocol

* Change import of ProtocolCommandSender to ProtocolController

* Fix all compile errors.

* Fix all error in tests and use the keypair file.

* Use limiter from settings

* Use routable ip

* Update peernet

* Add all variables to config to launch the node

* Fix compilation of API with new protocol

* Fix compilation of the node

* Remove network exports from bootstrap and replace interaction with protocol ones

* Fix(tests) : test_peer_connected && test_list_peers (#3882)

* Remove network and fix tests compilation

* Fix all tests

* Use initial peers from bootstrap and change initial peers file

* Remove old network category in config.

* Change print stop to info

* Don't test peers if we are already connected to

* Don't test our local ips.

* Increase maximum limitation socket.

* Update peernet

* Add debug prints

* Update peernet

* Update peernet

* Fix endorsement merge deletion

* Try to disable endorsement pool to test

* Fix peer not registered when received from remote.

* Add debug print

* Add debug prints

* Add debug print

* Fix endorsement note endorsements

* Add debug print

* Add more debug print

* Update peernet

* Update peernet

* Update peernet

* Remove all debug prints

* Display all len of all vectors to spot memory leak

* Remove debug memory print

* Add print when new peer connected

* Add setup nasm in CI

---------

Co-authored-by: Modship <yeskinokay@gmail.com>
AurelienFT added a commit that referenced this issue May 3, 2023
* Add skeleton new massa protocol.

* Add architecture handler

* Add basic peer management to massa

* Setup a simple test.

* Get best peer from peer manager

* Fix connections to not retry if unnecessary

* Add operations handler

* Add handler skeleton endorsement

* Add block handler skeleton

* Improve announcement serialization

* Add endorsement default behavior

* Add pool controller

* Add channels to communicate from handlers to outside world

* Fix tests.

* Add test endorsements to complete

* Add todo ip

* Rework network deserialize messages (#3750)

* Use new branch peernet

* Change handlers messages deserialization architecture.

* Remove un-necessary serialization of message ID in messages

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* Also deletes for the other type of message

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* Remove unused lines

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* Refactor serializing of message (#3786)

* Add new way of serializing messages.

* Use peernet default branch

* Format and add comment

* Rework id management

* Add comment messages

* Clean basic test

* add command channel in PeerManagementHandler (#3809)

* improves the peer management and communication logic

* Add structure of operation handler (#3817)

* Add structure of operation handler

* Update cache to use a simplified Lru and share it. Use the same channel for all communications to the propagator

* Add send of op and remove peers from cache when needed

* Update lock

* Add management of messages received from retrieval.

* Change timers management and add pruning

* Update caches and add storage to keep ops to be propagated

* Remove useless line and add a comment

* Remove useless write locks.

* ban peer if note_operations_from_peer is Err (#3826)

---------

Co-authored-by: Modship <yeskinokay@gmail.com>

* Add basic endorsement handler code (#3827)

* Add structure of operation handler

* Update cache to use a simplified Lru and share it. Use the same channel for all communications to the propagator

* Add send of op and remove peers from cache when needed

* Update lock

* Add management of messages received from retrieval.

* Change timers management and add pruning

* Update caches and add storage to keep ops to be propagated

* Remove useless line and add a comment

* Remove useless write locks.

* Add basic code for endorsement handler

* Fix runtime error in tests (#3836)

* Network refactoring tester (#3841)

* add 'thread_tester_count' in config.toml

* refactor Tester::new

* move thread_tester_count from NetworkConfig to ProtocolConfig

* create shared channel for thread tester  // add stop command to PeerManagementHandler

* Tester : test the listener on recv msg

* remove unwrap and print error

* Add basic skeleton for block handler and fix endorsement one (#3840)

* tmp

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* drop manager

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* Add two different thread for block handler

* fixup

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* Add cache of block

* Comment

* Fix handler handshake in massa protocol.

* Add basic block handler

* Add skeleton reception message of block infos

* Add all the code of the block handler

* Finish code of block retrieval thread

* Add the commands of propagation handling and fix compilation error

* Add debug print and fix the peer that didn't connected after handshake

* Fix listeners created too late.

* Fix review notes.

* Change ban node to warn in case it fails

---------

Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Co-authored-by: Litchi Pi <litchi.pi@proton.me>

* Add stop command to all thread network refactoring (#3858)

* tmp

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* drop manager

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* Add two different thread for block handler

* fixup

Signed-off-by: Litchi Pi <litchi.pi@proton.me>

* Add cache of block

* Comment

* Fix handler handshake in massa protocol.

* Add basic block handler

* Add skeleton reception message of block infos

* Add all the code of the block handler

* Finish code of block retrieval thread

* Add the commands of propagation handling and fix compilation error

* Add debug print and fix the peer that didn't connected after handshake

* Fix listeners created too late.

* Fix review notes.

* Change ban node to warn in case it fails

* Add channels for exterior world on all handlers thread and add stop command

* Update ports test

* Add stop command on handler so that even if there is clone of the sender they are killed

---------

Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Co-authored-by: Litchi Pi <litchi.pi@proton.me>

* send 100 peers to connected peers (#3864)

* get_peers_to_send

* update fn

* send 100 random peers to all peer connected each 10 sec

* use tuple for sender/receiver in PeerManagementHandler for readability

* Wrap interaction with peernet to mock them (#3865)

* Add basic mocks for tests

* Wrap all interactions to peernet with a controller to mock it

* Avoid overriding cache value

* Add mock in test and port the first test

* Update massa-protocol-worker-2/src/tests/ask_block_scenarios.rs

Co-authored-by: Modship <yeskinokay@gmail.com>

* Update massa-protocol-worker-2/src/tests/ask_block_scenarios.rs

Co-authored-by: Modship <yeskinokay@gmail.com>

* Change a todo

---------

Co-authored-by: Modship <yeskinokay@gmail.com>

* Fix cargo lock

* Fix compil error after merge

* Port tests network refactoring. (#3869)

* Port tests block workflow.

* Add more test on block handler and fix some bugs found.

* Port tests in block operations and add propagation of operations from block handler to operation one

* Add scenarios endorsements

* Port operation tests

* Port test ban_node and add disconnect of a banned node.

* Add cache scenario

* Fix compilation tests.

* Fix last endorsement test

* Change comment ban scenarios

* Fix tests protocol

* Update config on new protocol network and clean-up (#3875)

* Add name to threads and more config values.

* Bound all channels remove unwraps

* Fix clippy warnings

* Update config tests

* network_refactoring - send peers in handshake (#3870)

* send peers in handshake

* remove peer_db.read() and use previous lock

* remove useless scope

* restore scope and send peers when perform_handshake start

* fix : use saturating_sub

* update massa network with new peernet trait

* add serializer for peer management message in MassaHandshake to avoid many new serializer call

* Add Clone derive to struct for serializer

* send peers on failback_function + remove unwrap

* add panic hook in test

* remove revision tag for peernet in Cargo.toml and update it.

* update OutConnectionConfig::Tcp with default value (can set rate limit)

* remove dbg

* Use new protocol (#3879)

* Delete old network and protocol

* Use the new massa-protocol

* Change import of ProtocolCommandSender to ProtocolController

* Fix all compile errors.

* Fix all error in tests and use the keypair file.

* Use limiter from settings

* Use routable ip

* Update peernet

* Add all variables to config to launch the node

* Fix compilation of API with new protocol

* Fix compilation of the node

* Remove network exports from bootstrap and replace interaction with protocol ones

* Fix(tests) : test_peer_connected && test_list_peers (#3882)

* Remove network and fix tests compilation

* Fix all tests

* Use initial peers from bootstrap and change initial peers file

* Remove old network category in config.

* Change print stop to info

* Don't test peers if we are already connected to

* Don't test our local ips.

* Increase maximum limitation socket.

* Update peernet

* Add debug prints

* Update peernet

* Update peernet

* Fix endorsement merge deletion

* Try to disable endorsement pool to test

* Fix peer not registered when received from remote.

---------

Co-authored-by: Modship <yeskinokay@gmail.com>

* Use new protocol #2 (#3886)

* Delete old network and protocol

* Use the new massa-protocol

* Change import of ProtocolCommandSender to ProtocolController

* Fix all compile errors.

* Fix all error in tests and use the keypair file.

* Use limiter from settings

* Use routable ip

* Update peernet

* Add all variables to config to launch the node

* Fix compilation of API with new protocol

* Fix compilation of the node

* Remove network exports from bootstrap and replace interaction with protocol ones

* Fix(tests) : test_peer_connected && test_list_peers (#3882)

* Remove network and fix tests compilation

* Fix all tests

* Use initial peers from bootstrap and change initial peers file

* Remove old network category in config.

* Change print stop to info

* Don't test peers if we are already connected to

* Don't test our local ips.

* Increase maximum limitation socket.

* Update peernet

* Add debug prints

* Update peernet

* Update peernet

* Fix endorsement merge deletion

* Try to disable endorsement pool to test

* Fix peer not registered when received from remote.

* Add debug print

* Add debug prints

* Add debug print

* Fix endorsement note endorsements

* Add debug print

* Add more debug print

* Update peernet

* Update peernet

* Update peernet

* Remove all debug prints

* Display all len of all vectors to spot memory leak

* Remove debug memory print

* Add print when new peer connected

* Add setup nasm in CI

---------

Co-authored-by: Modship <yeskinokay@gmail.com>

* Update port tests

* Add todo on api

* Add new hack comment

* Remove useless scope

* Use type PeerMessageTuple in messages handler

* Added todo

* Fix get rand peers and debug print.

* Send myself in bootstrap peers & don't send local addresses.

* Fix add of myself in bootstrap peers.

* Change add myself

* Add debug print and reduce lock length

* Fix received fallback data in handshake.

* Fix error offset in tester handshake

* Update peernet

* Fix tests and remove debug print

* Update peernet.

* Add debug print

* Update peernet

* Add more print deadlock

* Update peernet

* Remove debug print

* Update announce in peerdb when handshake worked.

* Readd some debug print.

* Fix do not test our ip.

---------

Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Co-authored-by: Litchi Pi <litchi.pi@proton.me>
Co-authored-by: Modship <yeskinokay@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant