Add Aleo and Leo language support

[ghost]

Proposed changes and reference

Add support for Leo and Aleo instructions. Leo is a functional, statically typed programming language, built for writing private applications. Leo is a high-level programming language that compiles down to low-level Aleo Instructions.

References

• Leo website
  • Leo playground (live-IDE environment w/examples)
• Aleo website
  • Background on Aleo as an open source layer-1 blockchain project plus other information for one to grok.
• Developer documentation
  • Leo specific docs
  • Aleo instructions specific docs
• Leo and Aleo grammars
• Aleo applications deployed to our blockchain are viewable via an explorer like this one.
  • Leo is compiled down to Aleo Instructions when deployed.
  • A developer can write Aleo Instructions independent of Leo code.

Description

• This PR adds Leo and Aleo Instructions to the languages.yml file. This includes samples of Aleo and Leo programs, a unique ID for each, and their grammars.
• Previously, Leo and Aleo additions were attempted prematurely in PR 6021 and PR 6023 respectively and were unsuccessful because there was a lack of usage.

Checklist:

• I am adding a new language.
  • The extension of the new language is used in hundreds of repositories on github.com.
  • Search results for each extension:
    • Leo
    • Aleo
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • Leo
        • fibonacci.leo
        • groups.leo
        • twoadicity.leo
      • Aleo
        • sign_verify.aleo
        • mapping_operations.aleo
        • arrays_in_finalize.aleo
    • Sample license(s):
      • Aleo Instructions
        • 
      • Leo
        • 
  • I have included a syntax highlighting grammar: Leo and Aleo Instructions Grammars
    • 
  • I have added a color
    • Aleo
      • Hex value: #154bf9
      • Rationale: Brand colors
    • Leo
      • Hex value: #6814ec
      • Rationale: Brand colors

[ghost]

Hey there @lildude, just a friendly ping here to let you know this is ready for review. If you have any questions for us regarding Aleo or Leo, please don't hesitate to reach out. Best wishes for a wonderful start to your week and happy Monday!

[lildude]

Several issues here:

1. your grammars are not Textmate compatible so won't work. This would have been pointed out by a message along the lines of "no grammars found" when you ran the script/add-grammar script as detailed in the CONTRIBUTING.md file
2. your grammar repo doesn't have a license. We can not accept grammars not licensed by one of the documented licenses
3. One is your samples is too big - if it's suppressed in the diff, it's too big
4. Several of your samples don't appear to be examples of real-world usage.

Beyond the things in your direct control, neither extension is popular enough for inclusion when we exclude forks and the AleoHQ user (this user has an disproportionate influence on the current usage, hence I excluded it)

[lildude]

Oh yes, and we need to know the licence of all samples (they need to be covered by an open source licence)… the repo you referenced has no licence so we can't accept those samples.

[ghost]

Hey @lildude,

Thank you for the prompt response and your feedback. I believe we have sorted out all your previous comments, I've addressed them below.

Points 1-4 and 5:

1. You can find the repository containing both newly updated grammars here. As a minor point, I did run the script/add-grammar command/script as detailed in the CONTRIBUTING.md file on the ABNF grammars but incorrectly assumed they were TextMate-compatible, my mistake 🙂 I have updated this PR with the correct grammars.
2. Here are the licenses for the grammars:
3. I have removed all the previous samples and replaced them with more relevant examples, including the basic bank sample you mentioned was too large.
4. I have updated the samples to provide more real-world use cases for both Leo and Aleo Instructions based on our test cases. There are also a number of examples available via our docs that demonstrate usage for operators, commands, and provide developer resources.
5. We have added licenses covering the samples provided for Aleo Instructions - Apache 2.0 and for Leo - GPLv3.
6. I have run both the bundle exec rake test and bundle exec script/cross-validation --test tests with no errors reported for either Leo or Aleo Instructions.

On your point regarding popularity:

• When removing AleoHQ there are still 1.7k files, and without forks, there are 152 files. However, due to the nascency of the zero-knowledge-specific blockchain space (specifically within a layer-1 context), these numbers are actually quite large. Leo and Aleo Instructions are both abstraction layers for zero-knowledge circuits, which require a special set of skills and years of experience to grasp in most cases. I understand that usage needs to be present to add an extension. However, given the specialized nature of our technology and the fact it's difficult to develop early on, I wonder if there's an exception that can be made. Our community continually asks us for the support of the Linguist extension.
• In case it's helpful, we've created a comparison chart using GitHub data (via your API but with a custom indexer to provide greater accuracy). You can view a version of that dashboard here for your reference. It's a comparison between two other zk-specific DSLs. Of note is the % of unique files vs. those that are duplicated (delineated by SHAs) and the fact both the other languages are EVM-based, and so inherit the Ethereum ecosystem's users, whereas we are starting from scratch.
• Another point to mention here is that we are currently in the process of launching our network, and so are continually shipping new language features, engaging with our community via hackathons, events, workshops, and online content, which produces more contributors and, thereby, more files for Leo and Aleo Instructions but that also means support on GitHub is more important than ever because it makes it easier to develop and browse existing Leo and Aleo Instructions code with extensions and syntax highlighting. For additional context, our Discord community has over 3,000 online users at any given time and over 240,000 verified users, we have over 170,000 followers on Twitter and a growing YouTube channel. We hope that these factors offer a mitigation in terms of approval. We are happy to work with you to prove that both Leo and Aleo have real-world usage on GitHub and on the web.

Thank you for your consideration 🙏

[lildude]

We can not accept this grammar as it is licensed under the GPL-3 license. We only accept grammar with these licenses as pointed out in the CONTRIBUTING.md file.

The popularity requirement and method used to determine it still stands and is applied consistently across all contributions.

[ghost]

Hello again @lildude,

No problem. I have updated our grammar license to Apache 2.0.

Regarding popularity, that's unfortunate to hear, but we understand. I do have a few questions on my end, if that's okay:

1. Can you outline here if both Aleo Instructions and Leo need to meet the 2000 file requirement or if Aleo Instructions only needs 200 files? Aleo Instructions files are rendered at compile time and only appear once unless a developer explicitly writes using it. Referencing your guidelines outline here.
2. Would it be possible to confirm the below search queries are what you will use to determine if each language's popularity meets requirements?

Criterion:

• Leo
  • Extension: .leo
  • Required number of files: 2000
  • Search query: https://github.com/search?q=path%3A*.leo+aleo+NOT+owner%3AAleoHQ+NOT+is%3Afork&type=code
• Aleo Instructions
  • Extension: .aleo
  • Required number of files: 200
  • Search query: https://github.com/search?q=path%3A*.aleo+aleo+NOT+owner%3AAleoHQ+NOT+is%3Afork&type=code

Once I've heard back from you, I will work with our teams internally to come up with a strategy to get those numbers up so we can get this merged. In any case, best wishes for a great rest of your evening and an even better Friday + weekend 🙂

[lildude]

Please re-add the grammar using script/add-grammar --replace <submodule path> <repo_url> and add the cached license file that will be downloaded and added to the repo.

[lildude]

1. Can you outline here if both Aleo Instructions and Leo need to meet the 2000 file requirement or if Aleo Instructions only needs 200 files? Aleo Instructions files are rendered at compile time and only appear once unless a developer explicitly writes using it. Referencing your guidelines outline here.

From your description, it sounds like the Aleo files will meet the criteria for only 200 unique :user/:repos. Note, this doesn't mean 200 files as as you've pointed out, users can write these themselves and there is evidence that a few repos have multiple Aleo files. There's currently no easy way of determining total unique :user/:repos from GitHub's search so I'll use my discretion which will definitely require more than 200 files... probably closer to 230-250.

2. Would it be possible to confirm the below search queries are what you will use to determine if each language's popularity meets requirements?

Yes, that's where I'll start. I then use discretion to determine if there is a user having a disproportionate influence on the results, like the AleoHQ user, or if there are signs of gaming the system–it's quite obvious when this is happening 😉. I also tend to ignore obvious "hello world" files as they're not real usage.

Once I've heard back from you, I will work with our teams internally to come up with a strategy to get those numbers up so we can get this merged.

Sounds good, but please don't try and game the system, we want organic real usage.

Another thing to note, I won't conduct a thorough popularity search until I'm close to making the next release which is when I merge most PRs. See this note for more details.

[ghost]

Happy Wednesday/Thursday @lildude! I hope things have been well for you since our last chat.

We have been working hard to grow both Leo and Aleo instructions numbers over the last few months. From our side, we can see the numbers have grown to 375 Aleo specific files and 1.1k Leo files when using GitHub search given the criteria we discussed before. Could it be that GItHub's indexer is a bit off when using search? I've refreshed the search for the last several weeks and seen the same numbers despite there being steady growth aligned with our marketing and developer relations efforts using our internal dashboard. I have created a version you can view here. On it, you can see there are more than 2k files (we generate these numbers using a custom indexer of GitHub API data).

Any thoughts as to why these numbers (GitHub search vs. GitHub API) might be off and when we can expect GitHub search to update? We are hoping to get both Leo and Aleo instructions merged before the New Year.

Looking forward to continuing the conversation. Until then, best wishes for a great rest of your morning/afternoon/evening, wherever you are 😄

[lildude]

Any thoughts as to why these numbers (GitHub search vs. GitHub API) might be off and when we can expect GitHub search to update?

Yes, GitHub search does not index everything. See the search restrictions which is likely to explain the difference.

[ghost]

Okay, thank you for the clarification @lildude.

Following up, if I submit a PR specifically for Aleo Instructions to be merged since those numbers are at 380, would that be approved?

Right now, Aleo Instructions are more valuable from a syntax highlighting perspective since many of the larger programs that folks look at on GitHub are written in it. For example, credits.aleo.

If that makes sense to you, I'll create a second PR for just Aleo Instructions and leave this PR as is, waiting until Leo numbers reach 2k.

[d0cd]

@lildude This PR is outdated, please feel free to close at your convenience.
For reference, the correct PRs are here:

• Add Leo #6970
• Add Aleo instructions #6969