How can i use my custom dataframe in it

[trinikoko]

Hello!

While I looked on your launching example, i still dont get how to use my "custom" dataframe(okay, dataframe.value, i mean np array) in it?
I dont need to use part with "generate_tabular_synth_data" at all in this case?
But where should i get c_train/c_test/ground_truth_concept_masks then?
Or i just can assign them as None?
And look, i tried to split my initial df into x_train/x_test and so for y, then i ".value"-ed all of these df-s, then, i've gotten this:

Error itself:

InvalidArgumentError Traceback (most recent call last)
Cell In[352], line 9
7 pretrain_epochs = 50
8 batch_size = 1024
----> 9 pretrain_hist = end_to_end_model.fit(
10 x=x_train,
11 y=y_train,
12 epochs=pretrain_epochs,
13 batch_size=batch_size,
14 validation_split=validation_size,
15 verbose=1,
16 )

File D:\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py:122, in filter_traceback..error_handler(*args, **kwargs)
119 filtered_tb = _process_traceback_frames(e.traceback)
120 # To get the full stack trace, call:
121 # keras.config.disable_traceback_filtering()
--> 122 raise e.with_traceback(filtered_tb) from None
123 finally:
124 del filtered_tb

File D:\anaconda3\Lib\site-packages\tensorflow\python\eager\execute.py:53, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
51 try:
52 ctx.ensure_initialized()
---> 53 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
54 inputs, attrs, num_outputs)
55 except core._NotOkStatusException as e:
56 if name is not None:

InvalidArgumentError: Graph execution error:

Detected at node compile_loss/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits defined at (most recent call last):
File "", line 198, in _run_module_as_main

File "", line 88, in _run_code

File "D:\anaconda3\Lib\site-packages\ipykernel_launcher.py", line 17, in

File "D:\anaconda3\Lib\site-packages\traitlets\config\application.py", line 992, in launch_instance

File "D:\anaconda3\Lib\site-packages\ipykernel\kernelapp.py", line 701, in start

File "D:\anaconda3\Lib\site-packages\tornado\platform\asyncio.py", line 195, in start

File "D:\anaconda3\Lib\asyncio\windows_events.py", line 321, in run_forever

File "D:\anaconda3\Lib\asyncio\base_events.py", line 608, in run_forever

File "D:\anaconda3\Lib\asyncio\base_events.py", line 1936, in _run_once

File "D:\anaconda3\Lib\asyncio\events.py", line 84, in _run

File "D:\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 534, in dispatch_queue

File "D:\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 523, in process_one

File "D:\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 429, in dispatch_shell

File "D:\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 767, in execute_request

File "D:\anaconda3\Lib\site-packages\ipykernel\ipkernel.py", line 429, in do_execute

File "D:\anaconda3\Lib\site-packages\ipykernel\zmqshell.py", line 549, in run_cell

File "D:\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3051, in run_cell

File "D:\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3106, in _run_cell

File "D:\anaconda3\Lib\site-packages\IPython\core\async_helpers.py", line 129, in _pseudo_sync_runner

File "D:\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3311, in run_cell_async

File "D:\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3493, in run_ast_nodes

File "D:\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3553, in run_code

File "C:\Users\Timur.c\AppData\Local\Temp\ipykernel_10576\2716014408.py", line 9, in

File "D:\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py", line 117, in error_handler

File "D:\anaconda3\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 314, in fit

File "D:\anaconda3\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 117, in one_step_on_iterator

File "D:\anaconda3\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 104, in one_step_on_data

File "D:\anaconda3\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 54, in train_step

File "D:\anaconda3\Lib\site-packages\keras\src\trainers\trainer.py", line 316, in compute_loss

File "D:\anaconda3\Lib\site-packages\keras\src\trainers\compile_utils.py", line 609, in call

File "D:\anaconda3\Lib\site-packages\keras\src\trainers\compile_utils.py", line 645, in call

File "D:\anaconda3\Lib\site-packages\keras\src\losses\loss.py", line 43, in call

File "D:\anaconda3\Lib\site-packages\keras\src\losses\losses.py", line 22, in call

File "D:\anaconda3\Lib\site-packages\keras\src\losses\losses.py", line 1722, in sparse_categorical_crossentropy

File "D:\anaconda3\Lib\site-packages\keras\src\ops\nn.py", line 1567, in sparse_categorical_crossentropy

File "D:\anaconda3\Lib\site-packages\keras\src\backend\tensorflow\nn.py", line 638, in sparse_categorical_crossentropy

Received a label value of 200000000 which is outside the valid range of [0, 540). Label values: 18000000 16000000 37000000 11300000 <...> 27000000 48000000 [[{{node compile_loss/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits}}]] [Op:__inference_one_step_on_iterator_68526]

Ah, and my splitted arrays (at least what i tried to get as "splitted") have such props:

x_train has shape (13024, 12) and type float64
y_train has shape (13024,) and type int64
x_test has shape (3257, 12) and type float64
y_test has shape (3257,) and type int64

You know, before this, i tried to give into "generate_tabular_synth_data" function my df, as variable named "latent" there. It worked in that block, but also, i was getting very strange, about -10E6-order losses (namely minus) and zero accuracy.
And even after this "half-done" block, next block was giving to me such error:
"ValueError: Only one class present in y_true. ROC AUC score is not defined in that case."
And i could understand this error in the way of absence in "generate_tabular_synth_data" function any specifying of y variable itself.

So, i think it's just i didnt get something very important, so I'd be happy if you help me to understand that all)