Remove sudo in build

[madars]

Currently, ./scripts/configure.sh requires root privileges to build OpenCBDC. These privileges are used for three purposes:
a) installing build environment (e.g. clang, gmake on macOS, etc);
b) installing certain packaged dependencies (e.g. GoogleTest/libgtest-dev); and
c) after fetching and compiling sources from the internet, installing certain libraries (e.g. LevelDB, NuRaft, Lua, etc) into OS default install path (usually, /usr/lib).

This is suboptimal in multiple ways. First, invocation of sudo is spread all the way through the script, making this dependency harder to audit. Second, installing user-compiled dependencies in /usr/lib can interfere with OS package management (e.g., when user has already installed Lua) and makes it harder to develop two OpenCBDC instances side by side, if one of them, for example, wants to bump up its dependencies (e.g. have a never version of evmc).

The two commits below try to amend the above as follows:

• We split out installing build environment (a) and OS packages (b) into a separate script, ./scripts/install-build-tools.sh. This script requires sudo.
• We also adjust ./scripts/configure.sh to install dependencies in a separate, user-owned prefix directory (currently, this is called prefix and lives alongside build directory). This makes installation of these dependencies (c) not rely on root privileges and separates them between different instances of OpenCBDC (and system itself).
• We adjust ./scripts/build.sh and CMakeLists.txt to include this prefix directory in dependency search.

If this looks good, I can also produce appropriate changes to the documentation (README.md).

[HalosGhost]

strong concept ACK. It looks like the commits are missing a DCO, and it does look like a large portion of the CI workflows need to be updated.

[HalosGhost]

This looks great, but I believe there might be a small issue to resolve: I get linker errors from the static libcurl.a put in prefix/. Looks like it's expecting to find nghttp2 and openldap; are these expected to be compiled in statically, or installed from the new build-dependencies script?

/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `http2_disconnect':
http2.c:(.text+0xbc): undefined reference to `nghttp2_session_del'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `http2_getsock':
http2.c:(.text+0x114): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: http2.c:(.text+0x124): undefined reference to `nghttp2_session_get_remote_window_size'
/usr/bin/ld: http2.c:(.text+0x148): undefined reference to `nghttp2_session_get_stream_remote_window_size'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `on_begin_headers':
http2.c:(.text+0x178): undefined reference to `nghttp2_session_get_stream_user_data'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `on_header':
http2.c:(.text+0x1d5): undefined reference to `nghttp2_session_get_stream_user_data'
/usr/bin/ld: http2.c:(.text+0x4f0): undefined reference to `nghttp2_submit_rst_stream'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `on_stream_close':
http2.c:(.text+0x5da): undefined reference to `nghttp2_session_get_stream_user_data'
/usr/bin/ld: http2.c:(.text+0x62c): undefined reference to `nghttp2_session_set_stream_user_data'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `on_data_chunk_recv':
http2.c:(.text+0x6af): undefined reference to `nghttp2_session_get_stream_user_data'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `data_source_read_callback':
http2.c:(.text+0x7ea): undefined reference to `nghttp2_session_get_stream_user_data'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `h2_session_send':
http2.c:(.text+0x9b0): undefined reference to `nghttp2_priority_spec_init'
/usr/bin/ld: http2.c:(.text+0xa02): undefined reference to `nghttp2_submit_priority'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `h2_process_pending_input':
http2.c:(.text+0xaa4): undefined reference to `nghttp2_session_mem_recv'
/usr/bin/ld: http2.c:(.text+0xae3): undefined reference to `nghttp2_session_check_request_allowed'
/usr/bin/ld: http2.c:(.text+0xb25): undefined reference to `nghttp2_session_want_read'
/usr/bin/ld: http2.c:(.text+0xb32): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: http2.c:(.text+0xba2): undefined reference to `nghttp2_strerror'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `http2_conncheck':
http2.c:(.text+0xd1d): undefined reference to `nghttp2_submit_ping'
/usr/bin/ld: http2.c:(.text+0xd45): undefined reference to `nghttp2_session_send'
/usr/bin/ld: http2.c:(.text+0xd56): undefined reference to `nghttp2_strerror'
/usr/bin/ld: http2.c:(.text+0xd7b): undefined reference to `nghttp2_strerror'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `http2_handle_stream_close':
http2.c:(.text+0xf34): undefined reference to `nghttp2_http2_strerror'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `http2_recv':
http2.c:(.text+0x1111): undefined reference to `nghttp2_session_want_read'
/usr/bin/ld: http2.c:(.text+0x1126): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `http2_send':
http2.c:(.text+0x1453): undefined reference to `nghttp2_session_resume_data'
/usr/bin/ld: http2.c:(.text+0x145a): undefined reference to `nghttp2_is_fatal'
/usr/bin/ld: http2.c:(.text+0x1474): undefined reference to `nghttp2_is_fatal'
/usr/bin/ld: http2.c:(.text+0x15ce): undefined reference to `nghttp2_priority_spec_init'
/usr/bin/ld: http2.c:(.text+0x1669): undefined reference to `nghttp2_submit_request'
/usr/bin/ld: http2.c:(.text+0x16cb): undefined reference to `nghttp2_session_want_read'
/usr/bin/ld: http2.c:(.text+0x16ec): undefined reference to `nghttp2_session_resume_data'
/usr/bin/ld: http2.c:(.text+0x1738): undefined reference to `nghttp2_session_want_read'
/usr/bin/ld: http2.c:(.text+0x174c): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: http2.c:(.text+0x1788): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `Curl_http2_ver':
http2.c:(.text+0x186f): undefined reference to `nghttp2_version'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `on_frame_recv':
http2.c:(.text+0x1a24): undefined reference to `nghttp2_session_get_stream_user_data'
/usr/bin/ld: http2.c:(.text+0x1a6d): undefined reference to `nghttp2_submit_rst_stream'
/usr/bin/ld: http2.c:(.text+0x1a74): undefined reference to `nghttp2_is_fatal'
/usr/bin/ld: http2.c:(.text+0x1a9d): undefined reference to `nghttp2_session_get_remote_settings'
/usr/bin/ld: http2.c:(.text+0x1ab0): undefined reference to `nghttp2_session_get_remote_settings'
/usr/bin/ld: http2.c:(.text+0x1f0c): undefined reference to `nghttp2_submit_rst_stream'
/usr/bin/ld: http2.c:(.text+0x1f13): undefined reference to `nghttp2_is_fatal'
/usr/bin/ld: http2.c:(.text+0x1fbb): undefined reference to `nghttp2_session_set_stream_user_data'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `Curl_http2_done':
http2.c:(.text+0x2171): undefined reference to `nghttp2_submit_rst_stream'
/usr/bin/ld: http2.c:(.text+0x21ab): undefined reference to `nghttp2_session_set_stream_user_data'
/usr/bin/ld: http2.c:(.text+0x2200): undefined reference to `nghttp2_session_send'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `Curl_http2_request_upgrade':
http2.c:(.text+0x22ca): undefined reference to `nghttp2_pack_settings_payload'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `Curl_http2_done_sending':
http2.c:(.text+0x241c): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: http2.c:(.text+0x244b): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: http2.c:(.text+0x2475): undefined reference to `nghttp2_session_resume_data'
/usr/bin/ld: http2.c:(.text+0x2491): undefined reference to `nghttp2_session_want_write'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `Curl_http2_setup':
http2.c:(.text+0x2612): undefined reference to `nghttp2_session_callbacks_new'
/usr/bin/ld: http2.c:(.text+0x262a): undefined reference to `nghttp2_session_callbacks_set_send_callback'
/usr/bin/ld: http2.c:(.text+0x263a): undefined reference to `nghttp2_session_callbacks_set_on_frame_recv_callback'
/usr/bin/ld: http2.c:(.text+0x264a): undefined reference to `nghttp2_session_callbacks_set_on_data_chunk_recv_callback'
/usr/bin/ld: http2.c:(.text+0x265a): undefined reference to `nghttp2_session_callbacks_set_on_stream_close_callback'
/usr/bin/ld: http2.c:(.text+0x266a): undefined reference to `nghttp2_session_callbacks_set_on_begin_headers_callback'
/usr/bin/ld: http2.c:(.text+0x267a): undefined reference to `nghttp2_session_callbacks_set_on_header_callback'
/usr/bin/ld: http2.c:(.text+0x268a): undefined reference to `nghttp2_session_callbacks_set_error_callback'
/usr/bin/ld: http2.c:(.text+0x269d): undefined reference to `nghttp2_session_client_new'
/usr/bin/ld: http2.c:(.text+0x26a8): undefined reference to `nghttp2_session_callbacks_del'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `Curl_http2_switched':
http2.c:(.text+0x2807): undefined reference to `nghttp2_submit_settings'
/usr/bin/ld: http2.c:(.text+0x2827): undefined reference to `nghttp2_session_set_local_window_size'
/usr/bin/ld: http2.c:(.text+0x28dc): undefined reference to `nghttp2_session_upgrade2'
/usr/bin/ld: http2.c:(.text+0x28fa): undefined reference to `nghttp2_session_set_stream_user_data'
/usr/bin/ld: http2.c:(.text+0x296b): undefined reference to `nghttp2_strerror'
/usr/bin/ld: http2.c:(.text+0x2993): undefined reference to `nghttp2_strerror'
/usr/bin/ld: http2.c:(.text+0x29b7): undefined reference to `nghttp2_strerror'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `Curl_http2_stream_pause':
http2.c:(.text+0x2a1f): undefined reference to `nghttp2_session_set_local_window_size'
/usr/bin/ld: http2.c:(.text+0x2a53): undefined reference to `nghttp2_strerror'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-http2.o): in function `h2_session_send':
http2.c:(.text+0xa28): undefined reference to `nghttp2_session_send'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_disconnect':
openldap.c:(.text+0x23): undefined reference to `ldap_unbind_ext'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_done':
openldap.c:(.text+0xb4): undefined reference to `ldap_abandon_ext'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_recv':
openldap.c:(.text+0x204): undefined reference to `ldap_result'
/usr/bin/ld: openldap.c:(.text+0x229): undefined reference to `ldap_msgtype'                                                                                                                                                           [9/1820]
/usr/bin/ld: openldap.c:(.text+0x243): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x2a6): undefined reference to `ldap_parse_result'
/usr/bin/ld: openldap.c:(.text+0x303): undefined reference to `ldap_memfree'
/usr/bin/ld: openldap.c:(.text+0x30d): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x33d): undefined reference to `ldap_get_dn_ber'
/usr/bin/ld: openldap.c:(.text+0x34b): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x369): undefined reference to `ldap_err2string'
/usr/bin/ld: openldap.c:(.text+0x3d4): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x3eb): undefined reference to `ldap_err2string'
/usr/bin/ld: openldap.c:(.text+0x409): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x434): undefined reference to `ldap_err2string'
/usr/bin/ld: openldap.c:(.text+0x476): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x4b0): undefined reference to `ldap_get_attribute_ber'
/usr/bin/ld: openldap.c:(.text+0x665): undefined reference to `ber_memfree'
/usr/bin/ld: openldap.c:(.text+0x680): undefined reference to `ber_free'
/usr/bin/ld: openldap.c:(.text+0x68a): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x741): undefined reference to `ber_memfree'
/usr/bin/ld: openldap.c:(.text+0x79b): undefined reference to `ldap_get_attribute_ber'
/usr/bin/ld: openldap.c:(.text+0x7af): undefined reference to `ber_free'
/usr/bin/ld: openldap.c:(.text+0x7e3): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x850): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0x868): undefined reference to `ber_memfree'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_perform_bind':
openldap.c:(.text+0x8fe): undefined reference to `ldap_sasl_bind'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_cancel_auth':
openldap.c:(.text+0x9ca): undefined reference to `ldap_sasl_bind'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_connecting':
openldap.c:(.text+0xa8c): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0xac0): undefined reference to `ldap_abandon_ext'
/usr/bin/ld: openldap.c:(.text+0xae7): undefined reference to `ldap_result'
/usr/bin/ld: openldap.c:(.text+0xb58): undefined reference to `ldap_parse_result'
/usr/bin/ld: openldap.c:(.text+0xbb6): undefined reference to `ldap_parse_sasl_bind_result'
/usr/bin/ld: openldap.c:(.text+0xbcd): undefined reference to `ldap_err2string'
/usr/bin/ld: openldap.c:(.text+0xc1d): undefined reference to `ber_bvfree'
/usr/bin/ld: openldap.c:(.text+0xc2a): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0xcb0): undefined reference to `ldap_parse_sasl_bind_result'
/usr/bin/ld: openldap.c:(.text+0xd19): undefined reference to `ldap_msgtype'
/usr/bin/ld: openldap.c:(.text+0xe5d): undefined reference to `ldap_set_option'
/usr/bin/ld: openldap.c:(.text+0xe67): undefined reference to `ldap_msgfree'
/usr/bin/ld: openldap.c:(.text+0xf4a): undefined reference to `ldap_get_dn_ber'
/usr/bin/ld: openldap.c:(.text+0xf75): undefined reference to `ldap_get_attribute_ber'
/usr/bin/ld: openldap.c:(.text+0xfa7): undefined reference to `ldap_get_attribute_ber'
/usr/bin/ld: openldap.c:(.text+0x1017): undefined reference to `ber_memfree'
/usr/bin/ld: openldap.c:(.text+0x1023): undefined reference to `ldap_err2string'
/usr/bin/ld: openldap.c:(.text+0x105b): undefined reference to `ldap_err2string'
/usr/bin/ld: openldap.c:(.text+0x10b8): undefined reference to `ber_free'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_connect':
openldap.c:(.text+0x11af): undefined reference to `ldap_init_fd'
/usr/bin/ld: openldap.c:(.text+0x11d8): undefined reference to `ldap_set_option'
/usr/bin/ld: openldap.c:(.text+0x11e8): undefined reference to `ldap_set_option'
/usr/bin/ld: openldap.c:(.text+0x1248): undefined reference to `ldap_search_ext'
/usr/bin/ld: openldap.c:(.text+0x12a3): undefined reference to `ldap_err2string'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_url_parse':
openldap.c:(.text+0x133e): undefined reference to `ldap_url_parse'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_do':
openldap.c:(.text+0x1436): undefined reference to `ldap_search_ext'
/usr/bin/ld: openldap.c:(.text+0x1447): undefined reference to `ldap_free_urldesc'
/usr/bin/ld: openldap.c:(.text+0x14a9): undefined reference to `ldap_err2string'
/usr/bin/ld: openldap.c:(.text+0x14e2): undefined reference to `ldap_abandon_ext'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_perform_auth':
openldap.c:(.text+0x1598): undefined reference to `ldap_sasl_bind'
/usr/bin/ld: /home/halosghost/prj/desudoization/prefix/lib/libcurl.a(libcurl_la-openldap.o): in function `oldap_setup_connection':
openldap.c:(.text+0x1652): undefined reference to `ldap_free_urldesc'
/usr/bin/ld: openldap.c:(.text+0x1686): undefined reference to `ldap_pvt_url_scheme2proto'
collect2: error: ld returned 1 exit status
make[2]: *** [tests/integration/CMakeFiles/run_integration_tests.dir/build.make:330: tests/integration/run_integration_tests] Error 1
make[1]: *** [CMakeFiles/Makefile2:2290: tests/integration/CMakeFiles/run_integration_tests.dir/all] Error 2
[100%] Built target run_unit_tests
make: *** [Makefile:91: all] Error 2

[HalosGhost]

needs rebase, and potentially a fix regarding another dependency (see comment above).

[maurermi]

Just going to make a comment for future reference: this removes sudo in configure, our build script does not require sudo

[anders94]

Not to pile on too much here but I'd suggest renaming ./scripts/configure.sh to something like ./scripts/install-dependencies.sh while we're at it.

[HalosGhost]

merged via #249