Moving away from IRC as a messaging channel

[chris-belcher]

IRC was only ever intended as a starting point, eventually another messaging channel will be found because ultimately IRC will always been inferior. There are a few ideas for which p2p network to use, the most promising being https://github.com/cpacia/Subspace still in development which is explicitly designed for coinjoin. That will surely allow Tor too.

The JoinMarket codebase allows for easy separation of messaging channel code from the rest. To extend it to use subspace or something else, only a new subclass of MessagingChannel has to be written.

[ande2101]

I may be interested in implementing this one.

How would a one-to-many broadcast work in Subspace? Two connections, a standard one for private messages and a shared key for the group channel?

Also what about flooding? Centralized systems solve flooding by having privileged users, Bitmessage solves it by making people do proof of work before they can broadcast.

Finally there's traversing routers, be it home or Tor.

[chris-belcher]

I barely know anything about subspace. Can't help you much.

[chris-belcher]

Now it's quite difficult to find IRC networks that accept tor without too many other conditions. There are networks which require you to register with NickServ, or to have had an account for a number of days.

What would be the effects of going on IRC networks with no tor support, as well as going on tor-only networks? It would mean the liquidity would be split up, and market makers who wanted the most coverage would have to run tor all the time.

It wouldn't be good to slowly exclude tor users from the fast and big clearnet-only liquidity, and it wouldn't be good to only have tor users joining with each other. We want the widest variety of users with all kind of motivations and circumstances to use JoinMarket. Hopefully simple incentives would solve this, that yield generators want to sell to as many bots as possible so they'll have to get on the tor networks too.

[justusranvier]

I really wish Tor was developed primarily as a library, with the actual application being a thin wrapper around the core library.

That would make it easier for developers to to build applications that connect to Tor without requiring their users to install and maintain additional pieces of software.

That would make "running tor all the time" no more or less complicated than running the maker bot all the time.

[lacksfish]

Why don't we use PeX or DHT just like bittorrent clients do?

PeX: https://en.wikipedia.org/wiki/Peer_exchange
DHT: http://www.bittorrent.org/beps/bep_0005.html

[chris-belcher]

So thats a p2p network just like bitcoin itself.
That could work yes. I'm wondering how it would work against censorship. If you're a yield generator and you see other people's offers being announced, will you relay those messages?

[chris-belcher]

Other ideas are telegram.

Also SIP looks like another widely deployed decentralized communication protocol, if only one can send text messages over it

[tailsjoin]

I don't know if you saw this or not: https://github.com/equalitie/np1sec. TorMessenger will use it for multiparty comms in the future: https://blog.torproject.org/blog/tor-messenger-beta-chat-over-tor-easily.

[koinster]

Could something like http://matrix.org/ help?

[riclas]

i actually overlooked matrix.org when thinking about this issue... it seems quite straightforward to change from IRC to matrix. end-to-end encryption is still in the works, but we could probably start using it without that.

[chris-belcher]

@riclas has just been telling me about this on IRC and from the sounds of it it could be good.

[AdamISZ]

I took a quick look, it seems at least worthy of further investigation (matrix.org). One could argue whether it would have a meaningful benefit over private IRC servers (possibly easier to set up, with multiple federated, not sure yet)

[riclas]

If you were thinking about going for a multi-server irc setup as the next step, this should be much easier to setup as such.

[chris-belcher]

From what I've read, matrix is one single network that's connected together a little bit like SMTP. So it's already a maximally federated network of servers, unlike IRC which is more of a protocol that has many of its own little islands of IRC instances.

The maximally federated network is a good model for being resistant to censorship I think. Better than a pure p2p model where the market makers and relay peers are the same entity, because market makers have a direct incentive not to relay messages from other makers who compete with them.

[AdamISZ]

Better than a pure p2p model where the market makers and relay peers are the same entity, because market makers have a direct incentive not to relay messages from other makers who compete with them.

Yes, but otoh there's a distinction between the broadcast element of our protocol (basically, the orderbook) and the p2p section; note that we already have a "private" part (in the sense of not using the messagechannel) - the bitcoin transaction broadcast; it would be strictly superior to make the transaction negotiation p2p after peer discovery via the orderbook (one minor example, it would remove snooping concerns like encrypted message length leakage). More importantly I guess is the fact that making that part p2p would remove a scalability bottleneck that we have at the moment (the N-way broadcast of the encoded transaction).

Not to overcomplicate things, but I think it's worth considering that distinction.

[chris-belcher]

Yes so a hybrid option. Matrix or another federated server network for broadcasting offers, p2p for exchanging bitcoin transaction information.

OTOH the p2p method involves users having to open ports on their routers. I remember that being difficult and annoying to teach people (although I last wrote software like that more than 5 years ago so maybe things changed) Perhaps only the maker could have open ports (although noobs should be able to user makers too, e.g. patientsendpayment.py)

[chris-belcher]

A conversation I had with someone from #matrix on freenode. Matrix looks like a good solution with a few small caveats that need to be ironed out.

The caveats are that the client might need to trust a server not to censor (depending on whether the client can download the hash-linked-list of messages), otherwise the client can run their own server and be trust-free but that requires the server to be reachable from the outside internet (so cant be behind a NAT)

how hard is it to write a bot for matrix? like an irc bot
there are APIs i assume?
Pretty easy, its just some HTTP and JSON :)
ty :)
i understand matrix is like a federation of servers, how easy would it be for one or two of them to censor messages ? so if there was a channel and a bot was announcing things inside, how easy or hard would it be to stop that
so tor has fairly good censorship-resistant properties because everything is encrypted so the nodes cant see what they're relaying, obviously there'd be no point encrypting in matrix because everyone has to read it
it depends if a user on that server has admin rights in that room
so rooms live on a single server?
or is it like irc where a channel somehow exists on many servers
Rooms are distributed, whether or not a message is "allowed" depends on the state of the room
ah so you mean if the room owner/moderator/operator allows it? okay lets suppose she did
so a concret example, theres a room that has an irc bot that announces magnet links for torrenting, how hard would it be for one or two servers to shut it down ?
err obviously, not an irc bot
A room is a directed acyclic graph of events, new messages reference older messages using hashes (in a similar vein as git). Thus servers can see if messages are missing or not, and can decide who is allowed to do what based solely on that graph
ah interesting, so a server could know if messages have been missed out
and if one server doesnt relay it messages in an attempt to censor, our server would know about it and request them from somewhere else
If a user/server has admin rights, they can just ban everyone including the bot if they wanted to
but if you had a room where no one had admin rights (which is possible), then no server would really be able to censor anything (other than to clients that are connected to that server)
okay, so an admin doesnt have control over the entire network, only over their own room and/or server
Yup
okay :)
so if a client wants to be more sure they're not missing out information, can they connect to two or three servers at once?
so all three need to conspire to make the censorship work, and even then the client would notice the graph of hashes is broken
The easiest way would be to just run your own server somewhere, really. Accounts are tied to particular servers atm (for better or for worse) so you would need multiple accounts to connect to multiple servers
ah okay
And generally the events sent down to clients via the Client-Sever API don't contain all the hashes and things, since generally clients don't care
although there may be an option to get the full events, im not sure
but right now we're happy enough to go with "if you don't trust your server, run your own"
okay :)
that may very well change, but we have so much to do that it is relatively low down the list of things to look at :)
i understand
so someone could run a server on their own machine, and with their client connect to localhost, if they cant port forward then their server makes outbound connects to the rest of the matrix network
atm the server needs to be reachable from the internet
since the federation protocol is currently very "push" orientated
alright
ill have a look at both ways, asking the server for the hashes to check nothing has been missed out, and running your own server
thank you M-erikj1
no problem :)
thanks for dropping by

[AdamISZ]

Result of a quick look through setup today:

synapse (seems to be a stable implementation using Python):

Install as per instructions here seems to work fine here on ubuntu 15.04. The initial config requires a domain name; if you don't have DNS you can just set your IP address (and port forward as required for remote). On IRC they mentioned this is not the intended way, but works for testing. This (setting up a server) is presumably something we won't need to do much, albeit we're obviously interested in running our own homeservers.

The install automatically creates an ssl cert and keys.

They note there that you can enable registration with a setting in the config yaml file. I did that.

Client: this repo seems to fit the bill quite well. Here's a quick run of the snippet client code that seemed to work OK from a remote client:

from matrix_client.client import MatrixClient
client = MatrixClient("https://<ipaddr>:8448", valid_cert_check=False)
#there is also a client.register_with_password that works from remote 
#(presumably this is specifically due to the server config mentioned above). 
# Still need to look more into this.
token = client.login_with_password(username="adam5", password="stuff")
#has to be done once ofc
#room = client.create_room("my_public_room", is_public=True)
room = client.join_room("#my_public_room:<ipaddr>")
room.send_text("Hello!")

Once this is setup I can see the messages on localhost at port 8448 (there's a default web client that comes with synapse, a simple chat interface). The client.py in the SDK code has listening threads and callbacks etc. Will be reading it more.

valid_cert_check=False - obviously this is no good for non-testing but we'll have to look into how to work with the ssl cert issue if we want our own homeservers.

[chris-belcher]

Another conversation. I think matrix looks pretty good.

Soon I'll write a short post about matrix and it's properties and post it around on bitcointalk and reddit to get ideas and comments from more people interested in the project.

would banning users be the only way to censor their messages in a public room? and would any such event be publically viewable by all room members?
admins can redact any messages I think?
*room admins
i see. and would that event be visible, or could they hide it?
redaction events are visible to clients
M-Tor: OK, thanks, that's probably enough censorship resistance to be getting on with :)
I wonder why bridge nicks are network_nick and not nick_network or nick[network]
most significant info should be first
and weechat default nick color hash ignores the part after _
could a server somewhere just not relay messages ? which could be a kind of censorship
i don't think they could maintain consistency in federation. they could be isolated right.
second para here seems relevant: http://matrix.org/docs/spec/r0.0.1/intro.html#id3
what words does that paragraph start with ?
belcher: sorry was out. "Clients typically"
yeah link doesn't work, sorry.
so this event graph looks like it could be useful for anti-censorship
clients could check it to get an idea of whether events have been missed out ?
but it also looks like the event graph is only for server-server communcation
well a malicious server could spout any bs it wants to its clients
but that would just be adding events right? not taking them away?
belcher (IRC): If you're connected to a malicious server, your HS can censor all it wants to its client
M-roblabla could the client download the event graph somehow? would that help?
Pretty sure it can take them away for its clients.
Well the client could use the Server-to-Server APIs and act like its own HomeServer. I believe there are plans to explore this
so for anticensorship, could the user run their own server ?
Yeah sure.
In a federated environment, it gets harder to censor things.
i read running a homeserver requires ports being open, so NAT-traversal, in theory could there be a way without that ?
eh, I don't know. The current federation model is based on the idea that the servers can "talk" to eachother by doing HTTP requests in both directions.
ok ty
It's certainly possible on the technical level, but I believe it'd require lots of changes to the protocol.
is it (easily) possible for a client to connect to many servers at once? so a single server finds it hard to censor
belcher (IRC): It's easy for a Server to federate with many Servers
Most clients today only connect to one server, though I guess that's mostly a UI/UX problem
M-roblabla do you know much about the api/libraries for writing matrix bots? can they be used to connect to more than one ?
You can have multiple bridges to matrix.
ok, thats probably a good-enough method for our requirements
one last question: after connecting to one server can you request the ip addresses of other servers known to it ?
I'm fairly sure you can find them out through the userids (@username:server) but I'm not 100% sure
ok ty
Right, belcher (IRC) you can figure them out by doing http://matrix.org/docs/spec/r0.0.1/server_server.html#resolving-server-names
if you already know the DNS yes
i was thinking a client can connect to a few servers from a hardcoded list, and then download more server addresses from them for next time connecting
So for instance, my server is cmc.im, you should do a SRV lookup for cmc.im _matrix._tcp , which will give you an IP and a port to connect to. If that fails, you do a normal lookup for cmc.im and use port 8448.
You do, it's part of the user's UserID
Yeah ? Well it can find them by looking at what other servers users used to connect
ah okay
My understanding of the spec is limited, there might be a better way, but that seems good enough
so what about which other servers the server itself has connected to, im sure matrix.org knows about many other servers
Thing is, unless a user connects to matrix.org's rooms, it won't communicate with it
And if it does connect to matrix.org, it will show up in the event graph with his userid :P
ah
right, i keep thinking its like irc where servers are permanently connected to each other
so does that mean the users need to be in the same room as you ? for you to see their userid
Hmm, probably.
Since the syncs are per-room IIRC
so my software could connect to a big channel with many users, get the all the userids from there
seems each event has an event_id field containing the server name.
belcher (IRC): yup. Also look for federated rooms.
my concern is, if you connect to multiple servers, the data flow you get from each may be inconsistent w.r.t ordering (which may or may not matter)
waxwing shouldnt matter for our case where we just broadcast offers
belcher: for offers, yes (probably), for privmsg different, but again it may not be an issue
isn't it supposed to be eventually consistent though ?
do private messages exist in matrix ?
causality would prevent out of order in that case
:)
belcher: we already did it right?
oh yes, hah
waxwing so everything checks out with matrix?
i think having to get a data feed from multiple servers for the same room sounds a bit off-script from how matrix is designed.
but that's only an impression.
i realise "having to" is a bit strong
you'd get duplicates i guess, one from each connection
arguably it's not worse than IRC even with one, in terms of censorship resistance?
one server, i mean
yes i think so
how would private messages work when you have multiple connections? just send it down each one i guess
and deal with duplicates
i wouldn't think you'd need to use multiple servers for that
messages can have a sequence number in the protocol and we just drop duplicates
okay then, so just choose one server and send the whole thing through that ?
ah i forgot about multi-part tx messages
which raises the other question: what kind of rate limits are used (by default) in matrix? limits on msg size?
also whats the performance like? hopefully there wont be too many lags
but all these questions can be answered

[sneurlax]

I use Matrix daily, FWIW, and may be able to contribute development experience, if not a homeserver itself

[AdamISZ]

@sneurlax thanks for the offer, please see #541 and #650

[chris-belcher]

The maximally federated network is a good model for being resistant to censorship I think. Better than a pure p2p model where the market makers and relay peers are the same entity, because market makers have a direct incentive not to relay messages from other makers who compete with them.

I think I may have come up with a part of a solution to this problem.

An issue is that a relaying p2p node may be a maker and can see the message they're relaying. So if they see an offer that is more attractive than theirs, they simply don't relay it.

The solution may be to use encryption. When the taker requests the orderbook, they also include an encryption pubkey. Each maker encrypts their offer and only that taker can decrypt it. And maker p2p nodes can't read what the offer is, or even that it's an offer at all. This encrypted packet would be indistinguishable from an encrypted packet between takers and makers when creating a coinjoin, so if a maker p2p node blocks all packets then they might just stop coinjoins happening.

[chris-belcher]

<waxwing> This could be interesting, although I haven't read it in detail yet.
<waxwing> https://github.com/HelloZeroNet/ZeroNet

https://github.com/HelloZeroNet/ZeroNet

[chris-belcher]

Also how about this: each maker runs a tor HS, and maybe 5-10 volunteers run HS http servers (which we can call seeder servers) which makers can connect to and register their own .onion address on, and those same http servers will hand out the list of .onions to any taker who asks. Takers then connect to the maker's onions over Tor to download their offers and communicate the joinmarket protocol. The onion addresses of the seeder nodes are included by default into JoinMarket (perhaps in the cfg file, or a separate file)

This idea requires fidelity bonds for makers, because otherwise the volunteer seeder servers can be easily flooded with spam onions.

The seeder servers shouldnt require much bandwidth, the system can tolerate temporary downtimes so they don't need to be too reliable either. So it's easy to imagine that volunteers would run them, perhaps from an always-on raspberry pi in their own home.

[adlai]

If anyone in that scenario needs a significant fidelity bond, wouldn't that be the operators of directory servers?

[chris-belcher]

Fidelity bonds are a solution to sybil attacks. Sybil attacks can happen when its cheap to anonymously join a certain group. Directory servers / seeder servers are a very hard group to join, because that would require editing the config file and most users just leave the default unchanged.

By contrast makers are an easy group to join (which is desired, as its good for privacy) and so it's possible to sybil attack by running many makers. Fidelity bonds therefore make sense as a way to raise the cost of sybil attacking.

Also, each additional maker in the seeder server costs that server extra disk space and bandwidth, so fidelity bonds are useful to rate-limit the adding of new makers. That way it's not easy to exhaust the resources of the seeder servers.