Add ntopng #3856

hgy59 · 2020-01-18T18:11:48Z

Motivation: Revival of the oldest PR at this time
Linked issues: closes #1683

Since I did not succeed to rebase the existing PR #1683, I created a new branch and merged the basic work of @cytec.

Checklist

Build rule all-supported completed successfully (except ppc853x due DSM<6)
(n/a) Package upgrade completed successfully
New installation of package completed successfully

Updates related to #1683

Update ntopng to version 3.8.1 (of December 2019)
implement DSM 6 service setup (indeed DSM <= 5.2 is not supported)
move data dir to {pkg}/var/data
move ntopng.conf to {pkg}/etc and document most of the options
add new dependencies (lua, libxml2, curl, glib)
add nDPI module (from ntop.org)
update rrdtool (add new modules: pango, cairo, pixman; groff, urw-base35-fonts, uchardet)
add libmaxminddb
add libmysqlclient (mysql-connector-c)
add native modules libmysqlclient and ghostscript
update libpcap
update libsodium
update geoip
remove unused module cross/termcap from repo
update ntopng icon
add initial GeoLite2 content files (further updates are not available anymore without API key)
add patches (by sed) to set synology specific infos into generated code
add (former empty) module cross/libcap (incl. patches for older kernel, workaroud for cross compile)

Remarks

It was some work to get it all cross compiled. Especially the cross/libcap
cross/libcap is required to add network discovery capabilities and for the special code that changes the owner of the pid file during startup.
This service must be started as root (see src/conf/privilege) and on startup it changes the owner of the pid- and other files (to sc-ntopng) before starting in the background. Therefor the generic start-stop-status does not work and I had to create a custom one (that I derived from the ntop init.d sample for debian/centos).
Just while developing this module, the download of geoip database (by MaxMind) got restricted to registered users with an API key. I integrated the latests public version to have (at least) outdated geo information available (for some hints on how to update see ntop issue #3182 and Important Geolocation Changes in ntop Products)
As I successfully run this package on DS218+ (x64) and DS115j (armada370) I have enough courage to do remove the BETA flag.
This is the Community Edition of ntopng and (AFAIK) not capable to support Professional and Enterprise Licenses - available only for binary packages from ntop.org.

ymartin59 · 2020-02-03T21:54:39Z

spk/ntopng/src/conf/privilege

+	"defaults":{
+		"run-as": "package"
+	},
+	"username": "sc-ntopng",


I would set defaults to "run-as : root" and probably discard "username" if there is no interest to have such a package specific user either for service execution or package file ownership.

Only the service start needs to be done as root. So as root the ownership of some files (pid, ...) is set to sc-ntopng and the service itself is run as this package specific user.
So we definitifly need the package user.
And the package needs libcap (that makes this possible as far as I understand)
[Libcap implements the user-space interfaces to the POSIX 1003.1e capabilities available in Linux kernels. These capabilities are a partitioning of the all powerful root privilege into a set of distinct privileges. ]

spk/ntopng/src/start-stop-status.sh

ymartin59

A large PR with many dependencies for such a "simple" service.
Some minor improvements requests (as usual)
Great job.

cross/json-c/patches/remove-variable-size.patch

ymartin59 · 2020-03-10T20:05:17Z

cross/libcap/Makefile

+.PHONY: libcap_install_target
+libcap_install_target:


I prefer without _target suffix, it is obviously a make target.

And I was used to take the original with pkg_name as prefix, since there is not such an example in the wiki page (except for POST_STRIP -> extra_install).
I will use here your preference....

cross/libmysqlclient/Makefile

ymartin59 · 2020-03-10T20:07:50Z

cross/json-c/Makefile

+LICENSE  =
+
+GNU_CONFIGURE = 1
+PRE_CONFIGURE_TARGET = myPreConf


Proposal: jsonc_pre_configure

taken untouched from cytex...

cross/ndpi/Makefile

cross/urw-base35-fonts/Makefile

ymartin59 · 2020-03-10T20:22:38Z

native/libmysqlclient/Makefile

+COMMENT  = MySQL C API (libmysqlclient) 
+LICENSE  = GNU GPLv2
+
+CONFIGURE_TARGET = mysql-connector-c_configure_target


Without _target

native/libmysqlclient/Makefile

spk/ntopng/Makefile

ymartin59 · 2020-03-12T19:33:09Z

cross/json-c/Makefile

+PKG_NAME = json-c
+PKG_VERS = 0.12-20140410


Notice a more recent version is available and expected for introduction from another PR #3873
https://github.com/SynoCommunity/spksrc/blob/364de55b69714ed8c806caa38953064b5e7efe55/cross/json-c/Makefile

ymartin59

May you please update cross/json-c to align version with PR for ffpmeg

ymartin59 · 2020-03-14T10:59:52Z

spk/ntopng/Makefile

+REQUIRED_DSM = 6.1
+
+MAINTAINER = hgy59
+DESCRIPTION = ntopng is the next generation version of the original ntop, a network traffic probe that monitors network usage. ntopng is based on libpcap and it has been written in a portable way in order to virtually run on every Unix platform, MacOSX and on Windows as well.


Is it worth to mention it is "Community Edition", like in your PR description?

The description on ntop.org says:

Available Versions
ntopng comes in three versions, Community, Professional (Small Business Edition) and Enterprise. The Community version is free to use and opensource (code can be found on Github). The Professional and Enterprise offer some extra features that are particularly useful for SMEs or larger organizations. Features are highlighted in the following table.

So IMO it is not needed to explicitly declare this package as "Community Edition", since only this is opensource.

When I was developping the package, the german magazin c't had a series about ntopng, and for raspberry pi there are Pro-Versions for Embedded ARM available (payable).

ymartin59 · 2020-03-14T11:00:26Z

@hgy59 Do you consider this as tested enough to be published (even after json-c update)?

hgy59 · 2020-03-14T11:35:17Z

@ymartin59 yes, I tested it with DS218+ (x64) after json-c update.

ymartin59 · 2020-03-14T20:38:31Z

cross/libmysqlclient/Makefile

+PKG_NAME = libmysqlclient
+PKG_REAL_NAME = mysql-connector-c
+PKG_VERS = 6.1.6


Is there a reason why cross/libmysqlclient and native/libmysqlclient does not compile same version? If none, I expect here 6.1.11

Thanks, this is a mistake of mine.

@hgy59 May you please fix version so that I may publish

hgy59 · 2020-04-05T22:19:22Z

@ymartin59 sorry, forgot to commit my changes made three weeks ago.

- update ntopng to version 3.8.1 - implement DSM 6 service setup (indeed DSM <= 5.2 is not supported anymore) - move data dir to {pkg}/var/data - move ntopng.conf to {pkg}/etc - add new dependencies (lua, libxml2, curl, glib) - add nDPI module (from ntop.org too) - update rrdtool (add new modules: pango, cairo, pixman; groff, urw-base35-fonts, uchardet) - add libmaxminddb - add libmysqlclient (mysql-connector-c) - add native modules libmysqlclient and ghostscript - update libpcap - update libsodium - update geoip - remove unused module cross/termcap from repo - remove empty module cross/libcap (maybe we need it later) - update ntopng icon - add initial GeoLite2 content files (further updates are not available anymore without API key) - add patches (by sed) to set synology specific infos into generated code

- ntopng needs libcap for some network discovery functions - without linking to libcap the pid file is not created on the initial start - service start would work only after manually killing the initial process

- use custom start-stop-status.sh script with SSS_SCRIPT - generic service-setup does not work for background-service changing pid-file owner on start

ymartin59 · 2020-04-07T18:48:03Z

@hgy59 I see in application features "Lua scripting". We have already Lua here, but I wonder it is available in "community edition"...

ymartin59 · 2020-04-07T21:38:08Z

Package published

djmax9999 · 2020-05-12T14:57:41Z

Hi, is there any chance to get NTOPNG updated to Version 4.0 ? It would be great, because there are several nice features added to Version 4.0x.
Or is there any documentation how to get it cross compiled by myself ?
I am using a DS216j.
Thanks so far....

hgy59 · 2020-05-12T18:21:01Z

@djmax9999 there is already issue #3984 that requests such an update.
Unfortunately it is not just update and deploy. My first try failed already for building the updated cross/ndpi...

ymartin59 reviewed Feb 3, 2020

View reviewed changes

spk/ntopng/src/start-stop-status.sh Show resolved Hide resolved

This was referenced Feb 11, 2020

[Package Request] Bitwarden #3712

Open

ImageMagick not converting SVG to Bitmap #3881

Closed

hgy59 added the status/ready-to-merge label Feb 29, 2020

ymartin59 requested changes Mar 10, 2020

View reviewed changes

ymartin59 mentioned this pull request Mar 12, 2020

FFMPEG 4.2.2: Update, fixes and cleanup #3873

Merged

3 tasks

ymartin59 reviewed Mar 12, 2020

View reviewed changes

ymartin59 approved these changes Mar 12, 2020

View reviewed changes

ymartin59 approved these changes Mar 13, 2020

View reviewed changes

ymartin59 reviewed Mar 14, 2020

View reviewed changes

ymartin59 requested changes Mar 14, 2020

View reviewed changes

ymartin59 assigned hgy59 Apr 5, 2020

ymartin59 removed the status/ready-to-merge label Apr 5, 2020

cytec and others added 9 commits April 7, 2020 20:43

added ntop

9721e7f

add libcap

65779a3

- ntopng needs libcap for some network discovery functions - without linking to libcap the pid file is not created on the initial start - service start would work only after manually killing the initial process

libcap: add cross compile and fix build for old kernels

69f5d02

ntopng: fix start-stop-status

2c7fffb

- use custom start-stop-status.sh script with SSS_SCRIPT - generic service-setup does not work for background-service changing pid-file owner on start

cleanup

606957b

incorporate code review

3e0a7d1

update json-c

b77d473

update libmysqlclient

fbf3b48

ymartin59 force-pushed the cytec-ntop branch from 80e2894 to fbf3b48 Compare April 7, 2020 20:59

ymartin59 approved these changes Apr 7, 2020

View reviewed changes

ymartin59 merged commit dc27c24 into SynoCommunity:master Apr 7, 2020

hgy59 deleted the cytec-ntop branch April 9, 2020 23:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ntopng #3856

Add ntopng #3856

hgy59 commented Jan 18, 2020 •

edited

Loading

ymartin59 Feb 3, 2020

hgy59 Feb 3, 2020

ymartin59 left a comment

ymartin59 Mar 10, 2020

hgy59 Mar 10, 2020

ymartin59 Mar 10, 2020

hgy59 Mar 10, 2020

ymartin59 Mar 10, 2020

ymartin59 Mar 12, 2020

ymartin59 left a comment

ymartin59 Mar 14, 2020

hgy59 Mar 14, 2020

ymartin59 commented Mar 14, 2020

hgy59 commented Mar 14, 2020

ymartin59 Mar 14, 2020

hgy59 Mar 14, 2020

ymartin59 Apr 5, 2020

hgy59 commented Apr 5, 2020

ymartin59 commented Apr 7, 2020

ymartin59 commented Apr 7, 2020

djmax9999 commented May 12, 2020

hgy59 commented May 12, 2020

		.PHONY: libcap_install_target
		libcap_install_target:

		PKG_NAME = json-c
		PKG_VERS = 0.12-20140410

Add ntopng #3856

Add ntopng #3856

Conversation

hgy59 commented Jan 18, 2020 • edited Loading

Checklist

Updates related to #1683

Remarks

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ymartin59 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ymartin59 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ymartin59 commented Mar 14, 2020

hgy59 commented Mar 14, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hgy59 commented Apr 5, 2020

ymartin59 commented Apr 7, 2020

ymartin59 commented Apr 7, 2020

djmax9999 commented May 12, 2020

hgy59 commented May 12, 2020

hgy59 commented Jan 18, 2020 •

edited

Loading