Code portability

[hbiyik]

SCOPE:

This is neither a requirement or a feature request, but an open discussion about Tribler initiated from an enthusiast. So feel free to bring your beer along 😊.

MOTIVE, WHY PORTABILITY?: TLDR: No-one uses only desktop anymore

Currently no software fits to a lifecycle anymore if it only targets Desktop Environment. Even basic system tools like winzip has mobile ports.

WHAT ARE TARGET PLATFORMS?: TLDR: Windows, Linux, Android

Desktop: Windoz, Linux, OSX
Mobile: Android, IOS (I preferably ignore anything related to apple or anyone who uses apple products.)
IOT – HTPC: Custom Linux or Android Variants that people use for their media centers. They have fair amount of CPU and RAM size but have limited storage size
IOT – SMART TVS: Custom Linux / Android Variants, low storage, low cpu, low memory.
IOT – NAS: Custom Linux Distros that people use for their home file sharing service. Limited memory/cpu capacity but high storage size.
IOT – CONTROLLERS: Low CPU/RAM/STROAGE but fair amount of networking capabilities

WHAT ARE TARGET CPUS: TLDR: arm6/7/8(64), x86, amd64

Arm : ARM7 can execute arm6 code but crypto operation will be lower speed, ARM8 can both execute arm6 and arm7 code. But if the Kernel is compiled in upper version of arm platform, then the library interact with SYSCALL will fail if in lower version arm code. IE: If kernel is ARMv7 code, but netifaces is ARMv6 code, then it will fail. Same dependency is valid if the library is not static compiled and has a dynamic linkage (libc dependency of Android for example)
Mips: Mips currently looks like an abandoned platform in embedded market and dying each ad everyday but some manufacturers still prefer this cpu architecture due to low cost of architecture licensing. Mips has a subset for hard floating operations this is much way more preferred and called mipsel.
x86, amd64: These are the ones for desktop and very small portion of mobile devices.

WHAT IS THE PROBLEM WITH TRIBLER SOURCE CODE FROM PORTABILITY POV: TLDR: it is not portable

To my personal POV the biggest architectural problem is that, gui and the core is not totally abstract with each other. There are tons of import in gui code that direct imports from the core. This basically means that the gui functions which is provided with direct import instead of REST api, can not be ported to other platform. Because core should be portable but gui can be re-implemented with a native language (native, java, javascript, html etc..) depending on the platform capabilities.

Second problem is the of course the C libraries that the core use, and the variance of import mechanisms of each related components. There are several ways to import a C written library to python interface. Ctypes, CFFI, Cython/SWIG, and setup tools (ptython.h wrapper). You can find each every of those in the existing code which makes the portability of the code even crazier.

SUGGESTIONS / SOLUTIONS:Feel free to comment.

[devos50]

Thank you for your write-up! 👍

To my personal POV the biggest architectural problem is that, gui and the core is not totally abstract with each other. There are tons of import in gui code that direct imports from the core.

True, the GUI imports some code from the core indeed. Before 2016, the GUI and Core were even part of the same package and were running in the same process. We changed that and made a clearer abstraction between the two with a RESTful API. The reason for these imports is mostly to avoid code duplication, however, it decreases seperation between the two modules. Addressing this should not be very hard. We could even think about moving the GUI to a separate repo (which is actually what many applications nowadays are doing).

Second problem is the of course the C libraries that the core use, and the variance of import mechanisms of each related components.

This problem is not so easy to address since most of the C-based libraries we are using, are necessary, e.g., libtorrent and most of the crypto libraries. We already experience difficulty when compiling some of these libraries for all supported platforms. I also notice that Docker containers are getting more and more common. This might a solution for developers. Not sure about releases to end-users though.

[arvidn]

as for libtorrent. it's portable to at least windows, linux, mac, iOS and android. its CI builds and tests it for x86, amd64 and ARM. I'm more than happy to accommodate new platforms if they are reported to me.

[hbiyik]

@devos50 yeah great improvement on rest api and its abstraction as well, kudos to team, also async now. I really enjoy project moving step by step further each year even though visible gui stuff looks same, under the hoods stuff is moving further for sure.

+1 on docker, but they practically cover linux OSes on Desktop and Embedded as well (ie, corelec, libreelec, armbian etc..) in production. I think whats left is only Android. (ignoring mac apple things)

[Solomon1732]

We could even think about moving the GUI to a separate repo (which is actually what many applications nowadays are doing).

If this were to happen some of the code could be potentially reviewed (for example older code from py2 days, performance improvements, etc.). This is a lot of work, however. I don't expect this to happen, certaintly not overnight. And like @hbiyik said, kudos for the team - you're putting a lot of work into the project.

[qstokkink]

This update is a few years late, but we now have a fully fledged web-GUI coming in Tribler 8.0. That is done.

I guess the bare metal installation of random dependencies is unsolvable. However, in the past years we also got ourselves a Docker image.

With that, I consider this issue solved.