[WIP] Add Cargo buildsystem #564

chriskilding · 2023-09-20T19:11:39Z

Adds a Flatpak buildsystem for Cargo.

Implements #15

Note: I am opening this PR at an early stage to gather initial feedback and requirements for what needs to be done. I'm not familiar with the internals of Flatpak, so will gratefully accept any help.

src/builder-module.c

chriskilding · 2023-10-17T21:36:47Z

I'm also a bit stuck on how to add the new test cases to the script / entry point that runs the test suite - could someone advise where this is done, so that they can be run in the build?

TingPing · 2023-10-28T03:25:03Z

The entry point is https://github.com/flatpak/flatpak-builder/blob/main/tests/test-builder.sh

They are just simple shell scripts.

TingPing · 2023-10-28T03:27:49Z

So we need to make this have value beyond what buidlsystem: simple just calling cargo does. That means that we have something that actually expands Cargo sources or similar.

chriskilding · 2023-10-31T18:09:48Z

@TingPing agreed - that was always the intention. I just wanted to get the test harness set up around it first to TDD my changes.

Would you still recommend tomlc99, or are there any better (perhaps Cargo-specific) parser options? Ideally we'd want a C TOML library which is readily installable from the OS package manager.

TingPing · 2023-10-31T18:33:02Z

I have no opinion at all of TOML libraries.

On Fedora these are packaged:

~~libtoml (C) (DEAD)~~
toml11 (C++ Header)
tomlplusplus (C++)
~~cpptoml (C++) (DEAD)~~

TingPing · 2023-10-31T18:35:58Z

Not packaged but maybe interesting:

tomlc99 (C)
~~tomlc (C) (DEAD)~~

TingPing · 2023-10-31T18:37:55Z

I think I'd lean towards tomlc99 as a Meson subproject.

chriskilding · 2023-11-04T16:10:48Z

Another thought about parsing the Cargo / Cargo.lock files....

If we take the Toml parser route with tomlc99 or similar, effectively we would have to reimplement all of Cargo's parse+fetch logic in C. Furthermore the Cargo project occasionally changes the Cargo.lock file format (as indicated by its version number), so each time this changes we'd have to update the flatpak-builder reimplementation. This will likely be non-trivial.

I am therefore thinking that either of the following strategies would be better than reimplementing the Cargo dependency fetch logic ourselves:

Use the cargo crate and some kind of Rust-C FFI bindings to call the fetch Rust function from C directly. (This presumes that this function exists and is publicly available within Cargo's internal library. I imagine the FFI bindings would make this option more difficult to implement.)
Do a shell call within flatpak-builder to effectively exec cargo fetch. (I think this will be simpler to implement, and also make it easier to accommodate newer versions of Cargo and the Cargo.lock file format. We would however have to take care to show any errors from the delegated cargo fetch call to the user.)

Thoughts?

hfiguiere · 2023-11-09T03:47:53Z

src/builder-module.c

@@ -1585,6 +1587,8 @@ builder_module_build_helper (BuilderModule  *self,
    simple = TRUE;
  else if (!strcmp (self->buildsystem, "qmake"))
    qmake = TRUE;
+  else if (!strcmp (self->buildsystem, "cargo"))
+    cargo = TRUE;


this will need to be documented. See data/flatpak-manifest.schema.json and doc/flatpak-manifest.xml

hfiguiere · 2023-11-09T03:49:05Z

src/builder-module.c

@@ -1741,6 +1761,10 @@ builder_module_build_helper (BuilderModule  *self,
              configure_cmd = "meson";
              configure_final_arg = g_strdup ("..");
            }
+          else if (cargo)
+            {
+              // nothing to do - Cargo does not have a configure step


so how do you select features?
or building a specific binary.

Good point. Just to be more specific about this, do you mean switches like:

Profile: cargo build --release / --debug

Target: cargo build --target <architecture>

Cargo features: cargo build --features <feature>,<feature>,...

hfiguiere · 2023-11-09T03:55:35Z

Also one key thing here is that cargo isn't part of the base sdk. So the failure mode might not be ideal.

chriskilding · 2023-11-21T20:11:05Z

@hfiguiere yes, also a good point. I suppose this prompts the question of how flatpak-builder should provide buildsystems for any language/toolchain that's not in the base SDK - not just Rust/Cargo.

chriskilding · 2023-11-21T20:16:59Z

One possibility is that the buildsystem API could provide a feature where you can declare the sdk-extensions that the buildsystem depends on.

There would then be a couple of options for how to resolve those sdk-extensions when needed. A 'soft' resolution option would be to use the extension if it's installed, but if it's not, flatpak-builder prompts the user to add them to the Flatpak manifest with a message like:

"The buildsystem depends on the following sdk-extensions [ foo, bar, baz ], but these were not present. Please declare them in your Flatpak manifest."

Meanwhile a more automated option would be for flatpak-builder to just go ahead and resolve all the buildsystem's sdk-extension dependencies by itself. This would, however, mean that those sdk-extensions are not declared in the Flatpak manifest. There might be unintended consequences to that, but I don't have a view yet on what those could be.

alufers · 2024-01-08T11:17:36Z

"The buildsystem depends on the following sdk-extensions [ foo, bar, baz ], but these were not present. Please declare them in your Flatpak manifest."

Meanwhile a more automated option would be for flatpak-builder to just go ahead and resolve all the buildsystem's sdk-extension dependencies by itself. This would, however, mean that those sdk-extensions are not declared in the Flatpak manifest. There might be unintended consequences to that, but I don't have a view yet on what those could be.

The problem I see with this is that you can choose multiple different rust SDK extensions (stable, nightly etc.):

https://github.com/flathub/org.freedesktop.Sdk.Extension.rust-stable
https://github.com/flathub/org.freedesktop.Sdk.Extension.rust-nightly

To add insult to injury, those SDK extensions put cargo in different paths, and they have to be appended to PATH manually in the manifest.

Add Cargo buildsystem

9da587b

chriskilding mentioned this pull request Sep 20, 2023

Discuss how to package Rust/Cargo applications (and other non-standard build systems) #15

Open

(WIP) Add Cargo test cases, add cargo check step

048fc7a

nanonyme reviewed Oct 17, 2023

View reviewed changes

src/builder-module.c Show resolved Hide resolved

hfiguiere reviewed Nov 9, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Add Cargo buildsystem #564

[WIP] Add Cargo buildsystem #564

chriskilding commented Sep 20, 2023

chriskilding commented Oct 17, 2023

TingPing commented Oct 28, 2023

TingPing commented Oct 28, 2023

chriskilding commented Oct 31, 2023 •

edited

Loading

TingPing commented Oct 31, 2023 •

edited

Loading

TingPing commented Oct 31, 2023

TingPing commented Oct 31, 2023

chriskilding commented Nov 4, 2023

hfiguiere Nov 9, 2023

hfiguiere Nov 9, 2023

chriskilding Nov 21, 2023

hfiguiere commented Nov 9, 2023

chriskilding commented Nov 21, 2023

chriskilding commented Nov 21, 2023 •

edited

Loading

alufers commented Jan 8, 2024

[WIP] Add Cargo buildsystem #564

Are you sure you want to change the base?

[WIP] Add Cargo buildsystem #564

Conversation

chriskilding commented Sep 20, 2023

chriskilding commented Oct 17, 2023

TingPing commented Oct 28, 2023

TingPing commented Oct 28, 2023

chriskilding commented Oct 31, 2023 • edited Loading

TingPing commented Oct 31, 2023 • edited Loading

TingPing commented Oct 31, 2023

TingPing commented Oct 31, 2023

chriskilding commented Nov 4, 2023

hfiguiere Nov 9, 2023

Choose a reason for hiding this comment

hfiguiere Nov 9, 2023

Choose a reason for hiding this comment

chriskilding Nov 21, 2023

Choose a reason for hiding this comment

hfiguiere commented Nov 9, 2023

chriskilding commented Nov 21, 2023

chriskilding commented Nov 21, 2023 • edited Loading

alufers commented Jan 8, 2024

chriskilding commented Oct 31, 2023 •

edited

Loading

TingPing commented Oct 31, 2023 •

edited

Loading

chriskilding commented Nov 21, 2023 •

edited

Loading