Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposed Direction for Fork Development #1

Closed
tyjvazum opened this issue Feb 15, 2023 · 39 comments
Closed

Proposed Direction for Fork Development #1

tyjvazum opened this issue Feb 15, 2023 · 39 comments

Comments

@tyjvazum
Copy link

I think the pub feature, being able to make generic inscriptions without them being tied to the ordinal system, would be great for the ecosystem overall.

There are many potential use cases for inscriptions that don't fit into the ord protocol design as is. This includes protocols that would benefit from additional metadata, but there's no accessible way to add that when using the ord protocol. As an example, the pub protocol might benefit from having a field to specify an external public key and signature, so it can be (optionally) connected with other systems.

In this direction, there are parts of the original codebase that should be removed, such as the server code that produces the website. That's primarily the subcommand/server.rs and subcommand/preview.rs files, along with the associated points of interaction, like the parts of subcommand.rs that use Server and Preview. Other areas to remove include subcommand/traits.rs, decimal.rs, degree.rs, epoch.rs, media.rs, rarity.rs, and templates.rs.

There's more stuff that isn't needed for the raw inscription functionality, but that's a large chunk to get started cutting out and should help provide more understanding of the codebase.

As an additional note, I think a protocol should have a different identifier when it's desirable to be able to parse that particular group of inscriptions out of the blockchain and act on them in some way specific to the protocol, as well as when it doesn't make sense for the inscriptions to be connected to an ordinal, have a concept of transferring the inscription, or for their inscribed data to automatically show up in default explorers.

I've been working on a (currently unpublished) fork to implement arbitrary protocols and some related features. I think the goals are very complementary, but I do plan on keeping optional support for indexing. @jotapea, if you're interested, I'm willing to help, and perhaps we can combine our efforts more directly as things progress.

@jotapea
Copy link
Owner

jotapea commented Feb 15, 2023

I have been thinking about how to proceed, because if this non-indexing variation of the ord software still produces valid ordinals, then it could continue using the same protocol_id. But at the same time... maybe it shouldn't?

Expanding on what you wrote in https://github.com/casey/ord/issues/1548, I have been thinking that for this eternal publishing concept we are discussing, an additional field that makes a lot of sense to me is the content hash.

This can auto-validate the protocol, as in the publications that don't match can be ignored (or treated different). And this could have its own focused content hash to transaction index, to avoid writing duplicate data (but can be flexible, in like only focusing on certain amount of blocks).

so it can be (optionally) connected with other systems

Yes it can have multiple optional features.

@tyjvazum
Copy link
Author

tyjvazum commented Feb 15, 2023

I have been thinking about how to proceed, because if this non-indexing variation of the ord software still produces valid ordinals, then it could continue using the same protocol_id. But at the same time... maybe it shouldn't?

Perhaps it could, but I think it'd require modification to enable that, so I'd prefer to generalize inscriptions to achieve the same result. In my view, inscriptions aren't being used to their full potential without arbitrary protocol support.

Ordinals is primarily a set of two great innovations, of course the ordinal system, but also the inscription system. I consider the ord protocol to be one of many protocols that can be implemented on top of inscriptions, with another being your pub protocol.

Expanding on what you wrote in casey#1548, I have been thinking that for this eternal publishing concept we are discussing, an additional field that makes a lot of sense to me is the content hash.

Yes, that's a great point. I think a content hash field is something that makes sense at the top layer, for all protocols to have, since it's general purpose. A hash would be part of the overall metadata, then individual protocols can specify any additional metadata specific to their use case as key-value pairs in a header object.

This would allow the system to be used to easily define arbitrary protocols, working more like a developer making an app than needing to get into the really low-level stuff needed to make an inscription protocol as it currently stands. I think that should enable quite a rich ecosystem to be built on top of Bitcoin. Sometimes it can make sense to use the ordinal system, but for other use cases it can be better to use these generic inscriptions.

This can auto-validate the protocol, as in the publications that don't match can be ignored (or treated different). And this could have its own focused content hash to transaction index, to avoid writing duplicate data (but can be flexible, in like only focusing on certain amount of blocks).

Yes, exactly, that's another great point. That functionality can be used to deduplicate data that's being written, and it can be a feature that a given protocol can use or not use (although perhaps on by default to be space, and fee, efficient). These are the types of features that can be implemented in a straightforward way when arbitrary protocols are supported.

so it can be (optionally) connected with other systems

Yes it can have multiple optional features.

Exactly. I provided the example of tying into an external public key infrastructure just as an example. It may or may not make sense for pub specifically, but as you've expressed in your proposals, it can be essentially any feature once the right interface is exposed for protocol developers to use. Features that have potential to be used across multiple protocols make sense to define at the top layer, and then specific protocols can implement additional features as necessary.

@jotapea
Copy link
Owner

jotapea commented Feb 16, 2023

Ordinals is primarily a set of two great innovations, of course the ordinal system, but also the inscription system.

Agree. Ordinals introduced this data envelope standard which can now serve as a foundation for everyone to build on.

Maybe this repo doesn't need to focus on a single protocol id. While it can make sense to have a basic file publishing protocol id set as default ('pub' or another name), this could also serve as a bootstrap for any protocol id that wants to be based on inscriptions. So it can continue supporting the 'ord' protocol id in some capacity, at least in read/hash/index capabilities.

@tyjvazum
Copy link
Author

I'm working on preparing an initial release with the improvements I've described, so I don't think it makes sense to duplicate efforts in terms of indexing or similar. However, I think your original idea for a minimal tool to build and parse inscriptions has a lot of value in terms of allowing other programs to easily integrate inscription functionality. That can support arbitrary protocols as well, perhaps named something like sip (Simple Inscription Program) to reflect its focus.

@stet
Copy link

stet commented Feb 20, 2023

@jotapea @tyjvazum I would like to participate in this effort as well.
maybe more can be discussed in another channel?
@sull

@tyjvazum
Copy link
Author

tyjvazum commented Feb 21, 2023

I've set up a discussion room on Gitter. It's easy to use, with your GitHub account as the sign-in method. Here's the link to join: https://app.gitter.im/#/room/#arb-proto:gitter.im

@jotapea
Copy link
Owner

jotapea commented Feb 21, 2023

Maybe at this point is still better to continue the conversations in GitHub. @tyjvazum when can I expect to learn more about your efforts? As you said we should avoid duplicate work. @stet @sull welcome and glad you discovered this. Read you comment in https://github.com/casey/ord/issues/1501#issuecomment-1436436306 and it seems you are well aligned with the vision / potential we see for inscription-first protocols.

@tyjvazum
Copy link
Author

Maybe at this point is still better to continue the conversations in GitHub. @tyjvazum when can I expect to learn more about your efforts? As you said we should avoid duplicate work. @stet @sull welcome and glad you discovered this. Read you comment in casey#1501 (comment) and it seems you are well aligned with the vision / potential we see for inscription-first protocols.

We're already having some discussion on Gitter, but sure, I'm happy to converse on GitHub as well. I'm working toward an initial release as quickly as I can. I think it'll be relatively soon.

@0attack
Copy link

0attack commented Mar 6, 2023

i noticed that the gitter is encrypted so i can't see any previous messages. maybe that should be changed? i like the direction being discussed.

@tyjvazum
Copy link
Author

tyjvazum commented Mar 6, 2023

i noticed that the gitter is encrypted so i can't see any previous messages. maybe that should be changed? i like the direction being discussed.

Oh, that's a major issue that wasn't clear in the settings, thank you for letting me know.

Unfortunately, encryption couldn't be turned off after being on so I had to create the room again with the same link.

I'm glad you're interested. I'm working on finishing code for an initial release. I can't provide a solid date since it's novel work, but I'm aiming to have it out in the next week.

@jotapea
Copy link
Owner

jotapea commented Mar 6, 2023

@0attack glad to have you here!

@tyjvazum @stet can we just have all communications in GitHub? Even if @tyjvazum decides to make his code in another repo, we will still be able to reference each other.

I tried joining gitter and it asks to login and permissions and I just feel these add unnecessary friction to anyone that wants to participate in this effort. And its not only the friction, is also the dilution of the communication / information shared. And even more if the gitter is going to be public anyway... I just don't see clear benefits but see these drawbacks.

Just my opinion.

@stet
Copy link

stet commented Mar 7, 2023

yeah whatever is easiest 👍

@tyjvazum
Copy link
Author

tyjvazum commented Mar 7, 2023

@jotapea, I'm fine communicating wherever people choose to, GitHub, Gitter, or even somewhere else. Gitter is being used to provide a chat room for Arbitrary Protocols. It simply provides another channel, with different features and a real-time style of communication. A lot of projects use Discord, which has much more friction, requiring an email address and phone number, and is not open source like Gitter, so I think it's a nice option.

@jotapea
Copy link
Owner

jotapea commented Mar 7, 2023

@tyjvazum Any particular topic of interest that was discussed in the original room? Assuming you will make a separate repo for your project, will it be forked from casey/ord or will it be completely new?

@stet
Copy link

stet commented Mar 7, 2023

@tyjvazum Any particular topic of interest that was discussed in the original room? Assuming you will make a separate repo for your project, will it be forked from casey/ord or will it be completely new?

could post screenshots, not too much in there i don’t think.

@tyjvazum
Copy link
Author

tyjvazum commented Mar 7, 2023

@tyjvazum Any particular topic of interest that was discussed in the original room? Assuming you will make a separate repo for your project, will it be forked from casey/ord or will it be completely new?

It's based on casey/ord, with changes to add functionality, but the same overall structure.

@stet
Copy link

stet commented Mar 7, 2023

Not much but this is the meat of a brief chat.

Screen Shot 2023-03-07 at 10 20 12 AM

@jotapea
Copy link
Owner

jotapea commented Mar 7, 2023

Thanks for the screenshot @stet, all valuable comments. We all seem to be aligned so I'm looking forward to @tyjvazum work for next steps...

@0attack
Copy link

0attack commented Mar 7, 2023

i found this after reviewing issues. awesome concept. i had a similar idea for a fork, even similar name arb but no time to code it yet. also don't know much rust. could you explain anything more?

@tyjvazum
Copy link
Author

tyjvazum commented Mar 8, 2023

Sure, it's based on casey/ord, but extended to support arbitrary protocols on top of Bitcoin. Developers can create a simple JSON specification file to declaratively express their protocol, which the software can then load sort of like an extension. I'm implementing a Bitcoin Identifier/Username protocol, which is helping me to implement the generalized system, and I also have some more protocols planned for the future.

@jotapea
Copy link
Owner

jotapea commented Mar 8, 2023

Developers can create a simple JSON specification file to declaratively express their protocol

🚀🚀🚀

@stet
Copy link

stet commented Mar 14, 2023

Sure, it's based on casey/ord, but extended to support arbitrary protocols on top of Bitcoin. Developers can create a simple JSON specification file to declaratively express their protocol, which the software can then load sort of like an extension. I'm implementing a Bitcoin Identifier/Username protocol, which is helping me to implement the generalized system, and I also have some more protocols planned for the future.

are you proceeding and any other discussion being done with ord team?

@tyjvazum
Copy link
Author

Yes, I am.

I just released the code that generates the list of domains for the sunrise period of the username protocol. I'll be releasing some specifications in the middle of the week, and then the client code later in the week. After that, it'll be an ongoing development effort.

There's no discussion currently, although I'm quite open about everything. However, there was some more chat in the Gitter if you haven't seen that.

@stet
Copy link

stet commented Mar 14, 2023

alright sounds good, guess will wait for your preliminary release before contributing more.

@tyjvazum
Copy link
Author

I'm always open to feedback if you'd like to discuss anything sooner.

At the moment I'm working on the generalized system that provides cryptographic keys for the username protocol, and any other protocols, to use. It uses the Bitcoin Core wallet as a basis to generate Ed25519 keys from, and can be extended more in the future. Once that's done, most of the major features should be in place for the initial release.

@0attack
Copy link

0attack commented Mar 15, 2023

I just released the code that generates the list of domains for the sunrise period of the username protocol. I'll be releasing some specifications in the middle of the week, and then the client code later in the week. After that, it'll be an ongoing development effort.

by specifications you mean the specification file for protocols like you mentioned before?

@tyjvazum
Copy link
Author

I've released an example of the JSON specification file for the username protocol here. It's taking me a bit longer than anticipated to get everything ready, but I'm making solid progress. I'll continue releasing pieces as it comes together, with a full release ideally this month.

I also provided some information about an identicon system, that's a part of the username protocol, here. I came up with the concept after @RagnarLifthrasir started discussion in the Gitter. I think it's an interesting way to integrate the username protocol (and potentially others) with Bitcoin NFTs, as well as something that provides utility independently.

As I said in earlier messages, the overall goal of the project is to provide a surface that enables application developers to be able to create and deploy arbitrary protocols on top of Bitcoin, and I think that's advancing very nicely.

Some aspects of the designs will be much easier to adjust earlier rather than later, so feedback is encouraged and always welcome.

@jotapea
Copy link
Owner

jotapea commented Mar 20, 2023

@tyjvazum Very cool identicons sample!

@jotapea
Copy link
Owner

jotapea commented Mar 20, 2023

About the whole usernames discussion, I am not sure if they will work as a "unique namespace everyone will adopt". In the web, each domain has the freedom of creating their own usernames. The newest Bitcoin sub-protocol Ordinals have their own names. The old Counterparty sub-protocol has their own names.

I could see some potential for a DNS system, but unfortunately, there are also plenty of projects trying it. Could this be the one that "wins" and gets adopted by everyone, maybe. But I still believe the main effort should be focused on the features provided by these arbitrary sub-protocols.

The ordinal's inscription envelope PROTOCOL_ID is already serving the purpose of an open (and simple) top-level-domain.

I might be wrong and please don't get discouraged, but I just wanted to provide my sincere feedback. And having said all this, I still see a high potential in @tyjvazum's design approach.

@tyjvazum
Copy link
Author

About the whole usernames discussion, I am not sure if they will work as a "unique namespace everyone will adopt". In the web, each domain has the freedom of creating their own usernames. The newest Bitcoin sub-protocol Ordinals have their own names. The old Counterparty sub-protocol has their own names.

I agree. It's, of course, not intended to be something that everyone has to adopt, and I'm sure there will continue to be others that people choose to use. It's just a novel option.

However, while it's not a goal for the project, there certainly is demand for universal usernames, as evidenced by the proliferation of OAuth via Big Tech services.

With regard to Ordinal names, that's a very different thing that has some significant tradeoffs (at least for the current proposal), which I went over in this comment.

I could see some potential for a DNS system, but unfortunately, there are also plenty of projects trying it. Could this be the one that "wins" and gets adopted by everyone, maybe. But I still believe the main effort should be focused on the features provided by these arbitrary sub-protocols.

Yes, exactly. It's mostly meant to help with direction for the underlying implementation, while also being a solid, decentralized naming system (for which there arguably isn't a good existing example of). It's created with features that are provided for any arbitrary protocol to use. I plan on using it in the future as a basis for related applications that need usernames.

The ordinal's inscription envelope PROTOCOL_ID is already serving the purpose of an open (and simple) top-level-domain.

Yes, that's what I'm building out infrastructure to make use of, but that only defines a protocol. It's not suitable for an individual identifier like a username since they need to be united within a common specification. Enabling use cases like that in an easy, developer-friendly way is the primary objective of the project.

I might be wrong and please don't get discouraged, but I just wanted to provide my sincere feedback. And having said all this, I still see a high potential in @tyjvazum's design approach.

I'm truly happy to have the feedback. Honest discussion is an incredibly useful tool for refining ideas. Thank you for the ongoing participation and encouragement 👍

@jotapea
Copy link
Owner

jotapea commented Mar 21, 2023

Referencing some interesting approaches to add metadata to inscriptions:


EDITS: Other threads / comments with multiple relevant links:

@tyjvazum
Copy link
Author

tyjvazum commented Apr 6, 2023

I've set up the repo for my fork arb here.

It doesn't include new features at the moment, but it has more information about the next steps I'll be taking, it gets the basic structure in place (which was a fair bit of work in itself), and it allows for feedback or questions in the issues.

I was planning on including Arbitrary Protocols, Content Compression, Hash-addressed Content, Inscription Metadata (JSON), and Off-chain Content (BitTorrent) today, but I want to do some more testing first and am currently waiting for reindexing to complete after the recent reorg (which I'd really like to implement a solution for). I'm aiming to release those features next week, maybe sooner if the final testing goes smoothly.

I look forward to any feedback and it'd be particularly helpful if someone can verify my signature on the v0.1.0 release to make sure everything was done correctly.

@0attack
Copy link

0attack commented Apr 8, 2023

I've set up the repo for my fork arb here.

i noticed you did the fork where it has no "Forked from casey/ord" link under the name, which might be bad for people finding it... it also isn't added to the list of forks. mind if i reference it in my forked repo to make it a little easier to get to? probably couldn't hurt but i wanted to ask first.

It doesn't include new features at the moment, but it has more information about the next steps I'll be taking, it gets the basic structure in place (which was a fair bit of work in itself), and it allows for feedback or questions in the issues.

I was planning on including Arbitrary Protocols, Content Compression, Hash-addressed Content, Inscription Metadata (JSON), and Off-chain Content (BitTorrent) today, but I want to do some more testing first and am currently waiting for reindexing to complete after the recent reorg (which I'd really like to implement a solution for). I'm aiming to release those features next week, maybe sooner if the final testing goes smoothly.

can't wait to see it! i actually made a post a few days ago about my small attempt at helping with the reorg bug, https://github.com/casey/ord/issues/1945#issuecomment-1496191064

I look forward to any feedback and it'd be particularly helpful if someone can verify my signature on the v0.1.0 release to make sure everything was done correctly.

what are the steps to verify it? maybe it'd be good to put instructions.

@tyjvazum
Copy link
Author

@0attack, I've updated the releases page with verification instructions and of course I don't mind if you reference it in your repo 👍

@RagnarLifthrasir
Copy link

Thoughts on this proposed Bitcoin Ordinals DID Method Specification?

@tyjvazum
Copy link
Author

I've released an updated version of arb. It took more work and time than I'd initially hoped it would, as usual, and I'm sure there are bugs or oversights but I think it's a big step forward. It'd be great if anyone can help with testing, refactoring code, or writing documentation. I'll be continuing to work on the next batch of features.


Thoughts on this proposed Bitcoin Ordinals DID Method Specification?

@RagnarLifthrasir, I took my time thinking about it. It's somewhat compatible with what I'm working on, but I don't think treating an identity as connected to a satoshi is the best approach since I don't think identities are NFTs. I view the DID concept as primarily an interoperability mechanism and that's a useful thing, so I'll keep it in mind for my work.


@jotapea, I used your pub term as the protocol identifier for non-tracked inscriptions. I think it pairs well with the ord protocol naming. I'm still willing to help if you want to move forward with your idea for a minimal inscription reading/writing tool. If so, perhaps we should start a new issue for it since this one has grown a bit long.

@stet
Copy link

stet commented May 16, 2023

I'll take it for a spin

@jotapea
Copy link
Owner

jotapea commented May 18, 2023

@tyjvazum Thanks for your offer. I'm busy on other work at the moment, so I'm not sure when (or if) I'm coming back to this.

And it seems people are already using the ord envelope for arbitrary data as evidenced by the brc20 delusion lol. I'm pretty sure these people don't care about their assigned ordinal, is all about the inscribed data. Maybe they are already using something similar to this repo but just keeping the ord protocol_id.

I glanced at your work and it looks very elaborate! What do you have in mind in terms of first proof of concept / prototype / use case to focus on? Are you working on a product that will use this?

@tyjvazum
Copy link
Author

@tyjvazum Thanks for your offer. I'm busy on other work at the moment, so I'm not sure when (or if) I'm coming back to this.

And it seems people are already using the ord envelope for arbitrary data as evidenced by the brc20 delusion lol. I'm pretty sure these people don't care about their assigned ordinal, is all about the inscribed data. Maybe they are already using something similar to this repo but just keeping the ord protocol_id.

I glanced at your work and it looks very elaborate! What do you have in mind in terms of first proof of concept / prototype / use case to focus on? Are you working on a product that will use this?

There's been interest around BRC-20 support so that will likely end up being the first novel use case for arb, then my plan is to release the bid, bnw, and bfs protocols, probably in that order. All of this functionality can be implemented using a shared set of primitives, which is the arb design direction. Prototype code exists for these, but it needs to be polished into a cohesive whole as I put it out incrementally.

I have no plans for a commercial product, I just want to build useful things on top of Bitcoin, hopefully expanding and improving the overall ecosystem. I saw this potential sitting underutilized in ord and also saw it as an opportunity for me to learn Rust, which I've wanted to do but haven't had something useful to work on until starting this project.

Perhaps larger things will emerge going forward, or perhaps someone will want to use arb as the basis for a profit-seeking venture, but I'm having fun building and learning.

I'll close this issue since it seems to have served its purpose. If anyone wants to get involved just jump into the issues or discussions. As always, the more the merrier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants