T2.3 decission support tool #179

JBenda · 2023-03-14T17:47:46Z

Adding decision support to JULEA.

New:

trace level access which logs server backend operations.
tool access-replay which takes a JULEA_TRACE=access trace and replays the backend operations of it
scripts/decission-support.py runs access-replay multiple times for different configurations described in a config file
scripts/decission-support.R converts the output of a scripts/decission-support.py to plots and tables, which can be viewed as text or webpage
scripts/style.css and scripts/interactive.js for interactive and nice HTML page, where to put this file properly?

Further documentation can be found in doc/decission-support.md

Additional analytics possible on this data, but not yet implemented:

Cardinality of objects access (regarding different processes)
read/write ratio for different object categories
analytics regarding access type clusters and frequency

Added build option julea_trace to enable tracing also for release builds

JBenda · 2023-03-14T17:49:32Z

backend/db/jbson.c

@@ -317,15 +317,22 @@ j_bson_iter_value(bson_iter_t* iter, JDBType type, JDBTypeValue* value, GError**

 			break;
 		case J_DB_TYPE_SINT64:
-			if (G_UNLIKELY(!BSON_ITER_HOLDS_INT64(iter)))
+			if (G_UNLIKELY(!BSON_ITER_HOLDS_INT64(iter) && !BSON_ITER_HOLDS_INT32(iter)))


bson containing int32 should also be able to be parsed

JBenda · 2023-03-14T17:51:57Z

lib/core/jconnection-pool.c

+			if (client_program_name == NULL)
+			{
+				client_program_name = "<unkown>";
+			}


Different ways to get a program name, if there is a better way, please let me know.

lib/core/jtrace.c

JBenda · 2023-03-14T17:56:47Z

lib/core/jtrace.c

+}
+
+static guint32
+count_keys_recursive(const bson_t* bson)


Number of keys in query bson ↔ complexity measure

lib/core/jtrace.c

JBenda · 2023-03-14T17:58:13Z

lib/core/jtrace.c

+				}
+				else
+				{
+					/// \TODO handle unknown backends


warning? panic?

warning imo

JBenda · 2023-03-14T18:03:06Z

tools/access_replay.c

+		ret = j_backend_db_schema_get(db_backend, batch, parts[NAME], schema, &error);
+		g_array_append_val(bsons, schema);
+		// schema do not exists
+		if (error && error->code == 8)


Normal behavior is to check if i schema exists before creating a new one, therefore a schema does not exist is no error to panic

t-erxleben · 2023-03-19T18:42:25Z

lib/core/jtrace.c

+		else if (strncmp(name, "backend_", 8) == 0)
+		{
+			int type = 0;
+			if (trace_thread->access.db.type == NULL) /// \TODO more precise checking?


What do you mean by precise checking?

This part is for loading the type and config_path of each backend, which cannot be done in the init since the trace will be initialized before the backends.

Instead of checking db.type, it could be checked more of null, but this would mean an invalid state .... so think the test is enough, and will remove the todo

t-erxleben · 2023-03-19T18:44:41Z

lib/core/jtrace.c

+			trace_thread->client.uid = va_arg(args, guint32);
+			va_end(args);
+		}
+		else if (strncmp(name, "backend_", 8) == 0)


maybe add a comment here to clarify that this line filters for trace calls with backend_ prefix.

t-erxleben · 2023-03-19T18:46:22Z

lib/core/jtrace.c

+			name += 8;
+			if (strncmp(name, "kv_", 3) == 0)
+			{
+				type = 1;


those 'magic' numbers should probably be an enum for readability.

t-erxleben · 2023-03-19T18:48:05Z

lib/core/jtrace.c

+
+				j_configuration_unref(config);
+			}
+			name += 8;


Hard to catch should the prefix change later on.
Maybe add defines for the prefix and prefix length? This would still avoid pattern matching overhead.

also used later on: is it feasible to split the string (even before the match against 'backend_') on '_' and compare parts instead?

We cannot split at every _ because sometimes it appears in the name, like get_all. And doing it before the check for backend_ seems like a lot of potential string comparing and allocating because the name is constant. Therefore, we cannot simply add \0

lib/core/jtrace.c

t-erxleben · 2023-03-20T12:14:22Z

tools/access_replay.c

+		goto end;
+	}
+
+	// j_trace_init("julea-replay");


Should this be out-commented? Otherwise it is probably better to delete

t-erxleben · 2023-03-20T12:15:55Z

tools/access_replay.c

+};
+
+static gboolean
+split(char* line, char* parts[ROW_LEN])


There is also g_strsplit but I think I like your version better :)

t-erxleben · 2023-03-20T12:19:59Z

tools/access_replay.c

+		char* line = NULL;
+		size_t len = 0;
+		ssize_t read = 0;
+		memset(memory_chunk, 0, memory_chunck_size);


Should memory_chunk be initialized with random values instead of only 0?

t-erxleben · 2023-03-20T12:21:22Z

tools/access_replay.c

+		g_warning("unkown operation: '%s'", op);
+	}
+	return ret;
+}


Please add some empty lines for readability :)

tools/access_replay.c

t-erxleben · 2023-03-20T13:53:49Z

Does it make sense to enfore JULEA_TRACE=access in julea-access-replay so the user does not have to care to set it manually? It seems like this would be the default use case for the tool.
Also: are echo and access compatible? (e.g., what happens when both are specified, since they both output to stderr).

t-erxleben · 2023-03-20T13:58:07Z

I rewrote the doc file here.
Feel free to cherry-pick and/or argue about changes I made :)

JBenda · 2023-03-20T14:47:21Z

Does it make sense to enfore JULEA_TRACE=access in julea-access-replay so the user does not have to care to set it manually? It seems like this would be the default use case for the tool. Also: are echo and access compatible? (e.g., what happens when both are specified, since they both output to stderr).

You could also use replays to test a configuration for its functionality. So, you do not need to run an expensive program or made up integration test, just re-run the data access. Also reduces dependencies for such cases.

Since each echo line starts with [timestamp] it would be easy to filter out these lines. For debugging reasons (e.g. check if the access records are correct) it would make sens to see both in chronologically order.

t-erxleben · 2023-03-20T16:18:14Z

Does it make sense to enfore JULEA_TRACE=access in julea-access-replay so the user does not have to care to set it manually? It seems like this would be the default use case for the tool. Also: are echo and access compatible? (e.g., what happens when both are specified, since they both output to stderr).

You could also use replays to test a configuration for its functionality. So, you do not need to run an expensive program or made up integration test, just re-run the data access. Also reduces dependencies for such cases.

Since each echo line starts with [timestamp] it would be easy to filter out these lines. For debugging reasons (e.g. check if the access records are correct) it would make sens to see both in chronologically order.

Alright :)

t-erxleben · 2023-03-20T16:35:39Z

I get warnings during the build:

➜ ninja -C bld
ninja: Entering directory `bld'
[24/176] Compiling C object libjulea.so.p/lib_core_jtrace.c.o
../lib/core/jtrace.c: In function ‘parse_backend_operation’:
../lib/core/jtrace.c:645:1: warning: function returns an aggregate [-Waggregate-return]
  645 | parse_backend_operation(const gchar* backend_operation)
      | ^~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_enter’:
../lib/core/jtrace.c:753:32: warning: function call has aggregate value [-Waggregate-return]
  753 |                         type = parse_backend_operation(backend_operation);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_leave’:
../lib/core/jtrace.c:1075:61: warning: function call has aggregate value [-Waggregate-return]
 1075 |                                 BackendTypeOperation type = parse_backend_operation(backend_operation);
      |                                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[176/176] Linking target julea-fuse

t-erxleben · 2023-03-21T13:06:38Z

I worked through the examples given in the documentation and everything worked fine.

As an addition it could be nice to mention the R package dependencies in the docs (probably as a command that install them?). I ran through a few iterations of trying to execute the R script and installing missing packages.

JBenda · 2023-03-22T08:47:42Z

I get warnings during the build:

➜ ninja -C bld
ninja: Entering directory `bld'
[24/176] Compiling C object libjulea.so.p/lib_core_jtrace.c.o
../lib/core/jtrace.c: In function ‘parse_backend_operation’:
../lib/core/jtrace.c:645:1: warning: function returns an aggregate [-Waggregate-return]
  645 | parse_backend_operation(const gchar* backend_operation)
      | ^~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_enter’:
../lib/core/jtrace.c:753:32: warning: function call has aggregate value [-Waggregate-return]
  753 |                         type = parse_backend_operation(backend_operation);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_leave’:
../lib/core/jtrace.c:1075:61: warning: function call has aggregate value [-Waggregate-return]
 1075 |                                 BackendTypeOperation type = parse_backend_operation(backend_operation);
      |                                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[176/176] Linking target julea-fuse

This is fixed now.

JBenda commented Mar 14, 2023

View reviewed changes

t-erxleben reviewed Mar 19, 2023

View reviewed changes

lib/core/jtrace.c Outdated Show resolved Hide resolved

t-erxleben reviewed Mar 19, 2023

View reviewed changes

lib/core/jtrace.c Outdated Show resolved Hide resolved

t-erxleben reviewed Mar 19, 2023

View reviewed changes

lib/core/jtrace.c Outdated Show resolved Hide resolved

t-erxleben reviewed Mar 19, 2023

View reviewed changes

lib/core/jtrace.c Outdated Show resolved Hide resolved

t-erxleben reviewed Mar 20, 2023

View reviewed changes

tools/access_replay.c Outdated Show resolved Hide resolved

JBenda force-pushed the feature/t23-decission-support-tool branch from 534bee5 to 7839ad2 Compare March 26, 2023 12:07

JBenda added 8 commits May 23, 2023 13:20

data backend access tracing

674d1a4

Add traces for db and kv

4644c07

Fix for unknown program name

e811c4c

sceletcton of replay tool

b7ba030

Working replay, but with segfault

f7d2274

first reply example

e59a3ba

Add complexity guess for db selectors

3a9d3fe

Formatting

c1e7b02

JBenda added 28 commits May 23, 2023 13:20

Add plotting evaluation for decission support summary

10c30a8

start adding tables

effe650

Adding plotting and table code in R script

4436e35

Report design changes

cddc274

Changed decission support execution script to python

e63cc75

Update docu for <tmp> path part

8a788bb

nicer pälots

0f61910

even nicer plotting

863d35a

better interaction

7feb441

More interactive plot

1f56386

start cleaning plotting script

93f592d

Code formatting

77df281

Format

72ab9c7

Reduce warnings

3ef8ddb

use GString for frequently changing strings

2bd7ed4

Reduce magic numbers and repeating string constants

6fbc3e5

formating

5f7230b

Init replay_access memory chunk with random values

2e9b446

format

cea756c

Fix g_string usage

3db27e3

fix docs

bf2c36f

Check if all R packages are installed

f8d8edb

enable traces in relase builds with option

dfaf42d

auto format

8939000

add option file

c8ddc50

add semantics to access trace

eed749d

adapt csv headers and prints

41c26c6

auto format

6921602

JBenda force-pushed the feature/t23-decission-support-tool branch from ab1f9c6 to 6921602 Compare May 23, 2023 11:23

fixe meson.build

ca04017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

T2.3 decission support tool #179

T2.3 decission support tool #179

JBenda commented Mar 14, 2023 •

edited

Loading

JBenda Mar 14, 2023

JBenda Mar 14, 2023

JBenda Mar 14, 2023

JBenda Mar 14, 2023

t-erxleben Mar 19, 2023

JBenda Mar 14, 2023

t-erxleben Mar 19, 2023

JBenda Mar 20, 2023

t-erxleben Mar 19, 2023

t-erxleben Mar 19, 2023

t-erxleben Mar 19, 2023

t-erxleben Mar 19, 2023

JBenda Mar 20, 2023

t-erxleben Mar 20, 2023

t-erxleben Mar 20, 2023

t-erxleben Mar 20, 2023

t-erxleben Mar 20, 2023

t-erxleben commented Mar 20, 2023

t-erxleben commented Mar 20, 2023

JBenda commented Mar 20, 2023 •

edited

Loading

t-erxleben commented Mar 20, 2023

t-erxleben commented Mar 20, 2023

t-erxleben commented Mar 21, 2023

JBenda commented Mar 22, 2023

T2.3 decission support tool #179

Are you sure you want to change the base?

T2.3 decission support tool #179

Conversation

JBenda commented Mar 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

t-erxleben commented Mar 20, 2023

t-erxleben commented Mar 20, 2023

JBenda commented Mar 20, 2023 • edited Loading

t-erxleben commented Mar 20, 2023

t-erxleben commented Mar 20, 2023

t-erxleben commented Mar 21, 2023

JBenda commented Mar 22, 2023

JBenda commented Mar 14, 2023 •

edited

Loading

JBenda commented Mar 20, 2023 •

edited

Loading