Linux Support

[Joannis]

I've tried getting Sourcery working on Linux as a test. SourceKit and SourceKitten can be compiled for Linux, I've already managed that, but the FileWatcher used in Sourcery seems to be the only library that doesn't support Linux. If Sourcery moves to SPM you could fetch all dependencies except the FileWatcher using SPM and compile it. --watch wouldn't work, but Sourcery would be working on Linux which is good for Server Side Swift development. Anywhere a feature isn't supported/supportable on Linux you can place a #if !os(Linux) or preferably #if os(macOS) os(iOS) os(watchOS).

Just as an added note: I'm not suggesting this as a must for 1.0. But I'd definitely like this to be supported in the future.

[krzysztofzablocki]

FileWatcher pod is no longer used for Sourcery as I needed more powerful daemon, we'll just remove it completely (I thought I already did)

[Joannis]

I'm not sure if you've removed it. I might still be running too old code in my tests.

[damirstuhec]

@krzysztofzablocki Any updates on this? 🙇

[felix91gr]

We’re using this watcher to add support for Linux in SourceKittenDaemon. It’s a wrapper of the inotify C library present in the Linux kernel + a basic daemon.

[felix91gr]

Ponyboy47, the guy behind the original wrapper, has been developing a new FS Watcher here. The guy really knows what he’s doing, so maybe that’s a better option. We’ll see.

[mxcl]

Hi, I need this for various reasons, would you accept the patch that removes --watch on Linux as for my use (and I think most other people too) we need Sourcery in our CI environments but we don’t need it as a workflow tool (on Linux at least).

[ilyapuchka]

@mxcl I guess this is acceptable for now, but why not just commit generated code instead of running Sourcery on CI?

[mxcl]

but why not just commit generated code instead of running Sourcery on CI?

Two reasons:

• Humans make mistakes, if I have to generate it manually, I'll forget.
• I don't like committing generated files (especially generated files that are only used for CI on two platforms out of six)

I guess this is acceptable for now

Well, if you aren't cool with it then I'd rather not. Since maintaining something that core don't want is not fun.

[ilyapuchka]

I think it is fine as I suppose watcher does not work on Linux any way, but I didn't try it. @krzysztofzablocki you probably know better if thats the case?

[felix91gr]

I agree with @mxcl

Committing generated code seems like a very bad idea. The whole point of Sourcery is that the canonical code are your macros. If you’re committing generated code, you’re not committing canonical code. I might be wrong, but that seems like a very bad idea.

[krzysztofzablocki]

@mxcl we'd be happy to accept a PR that enables Linux support, even without the watcher especially since its nice to have and not a critical feature 👍

I totally agree with Humans make mistakes, if I have to generate it manually, I'll forget., the whole idea behind Sourcery is to limit human mistakes as much as possible, its the driving principle for this project.

@felix91gr I do disagree on not committing generated files in most projects, but there are few reasons behind this:

• I believe that you should always be able to fetch your repository and run it instantly without having to run pod install and things like that.
• Sourcery generates the exact code you ask it to generate, its not opinionated and I think we should still review that code in our PR's, this is your production code after all, by commiting it you will see Diffs in the PR's
• If for some reason the tool would ever breaks on new version of Xcode / OSX, your code is still working and you can just move to maintaining it by hand like you did before using Sourcery

I definitely think that limiting human mistakes by having the codegen run with each build is more important than whether you commit the code or not, although I've been burned by not commiting it in the past so my recommendation is to do it for the above reasons :)

[ilyapuchka]

@krzysztofzablocki I could not explain it better, completely agree with all the points (the only thing is probably LinuxMain is really not that necessary to commit, but if so, I would consider it as exception). I think we should put it in README as general recommendation.

[mxcl]

K great, I'll spin up my docker.

[felix91gr]

@mxcl how did it go? I'm willing to help. I'm in need of generating LinuxMain files and already two people have pointed out to me that this is trivial with Sourcery 😅

Maybe we can make a PR to start enabling Linux support :)

[felix91gr]

Also: @krzysztofzablocki @ilyapuchka, what's left in general to enable Linux support?

[mxcl]

I made some progress but it was going to be a lot more work than I anticipated, mainly because Sourcery doesn't handle #if os() and I would need to make that work too.

[felix91gr]

Oh no :(

mainly because Sourcery doesn't handle #if os()

How so?

[Tomaz-Vieira]

Hello, everyone
Do we have any progress with this? I'd be supremely interested in using Sourcery on Linux, and I'd also be willing to help if someone could give me some pointers.

[Nikoloutsos]

Any updates on linux compatibility?

[art-divin]

👋 Hi all

I have installed ubuntu VM through tart and updated to 22.04 according to this guide.

I had to run the following commands prior to being able to run bundle install in Sourcery:

1. sudo apt install libffi-dev
2. sudo apt install build-essential
3. sudo apt install libsqlite3-dev

Then I was able to successfully run bundle install.

Unfortunately SourceryRuntime contains import of CommonCrypto which are binding codebase to Apple platforms.
I wonder if there's a way to cut these references out ❔

I'll continue investigation 🕐

[krzysztofzablocki]

We could drop CommonCrypto and use some replacement, I think this is only used for cache hash

[art-divin]

We could drop CommonCrypto and use some replacement, I think this is only used for cache hash

Indeed, there's some replacement available in swift-tools-support-core, albeit being deprecated - SHA256

[Joannis]

Or SwittCrypto

[art-divin]

👋🏻 Hey, a little update:

as it appears to be, there are a lot of issues compiling code under AST and other parts of code, mainly related to @objcMembers throwing an error on Linux.

Wrapping it into

#if os(macOS)
@objcMembers 
#endif

helps, so I am continuing this journey.

[art-divin]

👋🏻 Hey all, a little update:

I was able to reduce number of errors down to a hundred or less, but now there are plenty in Sourcery.swift and mostly related to JavaScriptCore framework.

Other changes I needed to make is to disable almost all SourceryRuntime/Generated extensions due to inheritance from NSObject and use of @objc annotations. This would take a while to automate I suspect 🤔

My current plan is to make it buildable, although some features will be broken.

[art-divin]

👋🏻 Hey all, some news:

I was able to resolve all of the build errors in Sourcery, though some of the fixes I applied I'd need to refactor. At the moment I face these issues coming from swift-llbuild:

error: link command failed with exit code 1 (use -v to see invocation)
/usr/bin/ld.gold: error: cannot find -lncurses
.build/checkouts/swift-llbuild/lib/llvm/Support/Unix/Process.inc:340: error: undefined reference to 'setupterm'
.build/checkouts/swift-llbuild/lib/llvm/Support/Unix/Process.inc:358: error: undefined reference to 'tigetnum'
.build/checkouts/swift-llbuild/lib/llvm/Support/Unix/Process.inc:362: error: undefined reference to 'set_curterm'
.build/checkouts/swift-llbuild/lib/llvm/Support/Unix/Process.inc:363: error: undefined reference to 'del_curterm'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

My setup:

:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.2 LTS
Release: 22.04
Codename: jammy

:~$ swift -version
Swift version 5.8.1 (swift-5.8.1-RELEASE)
Target: aarch64-unknown-linux-gnu

[art-divin]

I figured it out, I needed to install sudo apt-get install libncurses5-dev and ... build is successful! Sourcery was compiled under Ubuntu! 🔥

[art-divin]

Hi all 👋🏻

Linux Support #1188 is ready for review. What a ride! It works 🔥 🎆