Setting up the software

[hentsd]

Hey. Since I'm quite new to programming then it is a bit confusing for me. I've set up the hardware and installed the software, but like, what now? What would the next steps be to get the program up and running?

[danielsk78]

Hi @hentsd,
Thanks for your interest in the sandbox project.
In which step are you stuck in the installation? Is it in the hardware configuration or a problem with the code itself? Please describe the exact problem you have, next to your hardware specifications, to give you a detailed solution.
Also, be sure to follow all the steps provided in the Readme.

[hentsd]

Hey @danielsk78 , thanks for the quick response.
Right now I'm at a point where I've completed all actions in the Readme file. The hardware is setup as well thanks to earlier AR Sandbox configuration.
Right now I have problems starting the program. Tried to start the sandbox_server with "panel serve sandbox_server.py --show" but it gave me only blank white pages. I'm not sure if I've done something wrong or is there anything else I should to with the code.
Hardware: MSI GL63E laptop (i7-8750H with RTX2070), BenQ MX631ST projector.

[danielsk78]

Hi @hentsd,
I just checked what is wrong, and yes, "panel serve sandbox_server.py --show" is not working properly. This is because a major refactoring in the widgets for all modules was made, and the API needs to be updated. I am working on it.

For the moment, use the Jupyyter notebooks. You can find them under "notebooks/tutorials". Try running the calibration first, and then you can start looking into the different available modules. We are also working on expanding the notebooks' documentation, so if any questions, don't hesitate to open new issues.

[hentsd]

Hey @danielsk78,
I tried running calibration using Jupyter Notebook but kept getting ValueErrors. Once again, since it is my first time operating with Python I might be making some stupid errors.
When running "proj = Projector(use_panel=True)"

---

ValueError Traceback (most recent call last)
in
1 #use_panel = True will open automatically a panel server
----> 2 proj = Projector(use_panel=True)

~\open_AR_Sandbox\sandbox\projector\projector.py in init(self, calibprojector, use_panel, p_width, p_height, **kwargs)
71 self.profile = None
72 self.sidebar = None
---> 73 self.create_panel()
74 if use_panel is True:
75 self.start_server()

~\open_AR_Sandbox\sandbox\projector\projector.py in create_panel(self)
104 tight=False,
105 dpi=self.dpi,
--> 106 css_classes=['frame']
107 )
108 plt.close(self.figure) # close figure to prevent inline display

~\Anaconda3\envs\sandbox-env\lib\site-packages\panel\pane\base.py in init(self, object, **params)
105
106 def init(self, object=None, **params):
--> 107 applies = self.applies(object, **(params if self._applies_kw else {}))
108 if (isinstance(applies, bool) and not applies) and object is not None :
109 self._type_error(object)

~\Anaconda3\envs\sandbox-env\lib\site-packages\panel\pane\plot.py in applies(cls, obj)
139 is_fig = isinstance(obj, Figure)
140 if is_fig and obj.canvas is None:
--> 141 raise ValueError('Matplotlib figure has no canvas and '
142 'cannot be rendered.')
143 return is_fig

ValueError: Matplotlib figure has no canvas and cannot be rendered.

From there on I can't open the widget as well.

I tried to figure it out using Google but got nowhere..

[danielsk78]

I could not recreate the error. So I just corrected what seems to be the error.

• Please update your master branch by doing a git pull and try again to run the notebook

pn.extension() is repeated in several parts of the main code and modules. This may cause conflict with the css_classes that define the color in the Projector module. I made a partial change that hopefully will solve the problem. But I will look more in detail at how I am using this extension over the whole code.

Which version of panel and matplotlib are you using?
I am running with panel== 0.10.2 and matplotlib >= 3.2.1.

[hentsd]

Big thanks for your time and for the help @danielsk78. I'm running panel 0.9.7 and matplotlib 3.3.2.
The first calibration now worked (calib_projector). Right after I keep having problems with calib_sensor.
"calib_proj = _calibration_dir + 'my_projector_calibration.json'
module = CalibSensor(calibprojector = calib_proj, name = 'kinect_v2')" successfully launches the server, but gives me:
AttributeError Traceback (most recent call last)
in
1 calib_proj = _calibration_dir + 'my_projector_calibration.json'
----> 2 module = CalibSensor(calibprojector = calib_proj, name = 'kinect_v2')

~\open_AR_Sandbox\sandbox\sensor\calibration_sensor.py in init(self, calibprojector, name, **kwargs)
37 plt.close() # close figure to prevent inline display
38
---> 39 pn.state.add_periodic_callback(self.update_panel_frame, 5)
40 #self.projector.panel.add_periodic_callback(self.update_panel_frame, 5)
41

AttributeError: '_state' object has no attribute 'add_periodic_callback'

And after that the widget box won't run as well. "widget = module.calibrate_sensor()
widget" gives me:
NameError Traceback (most recent call last)
in
----> 1 widget = module.calibrate_sensor()
2 widget

NameError: name 'module' is not defined

[danielsk78]

Hi @hentsd, found the problem. Thank you.

please upgrade panel to version 0.10.2. Use pip install --upgrade panel. This should solve the compatibility problem.

• Please update your master branch by doing a git pull and try again to run the notebook

I modified the requirements.txt and setup.py file with the correct version of panel, plus a small deprecation warning of matplotlib in the calibration_sensor.py script.

[hentsd]

Hey @danielsk78, getting further every time.
When initializing aruco detection under main_thread
(from sandbox.markers import MarkerDetection
aruco = MarkerDetection(sensor=sensor) gives me nameerror:

NameError Traceback (most recent call last)
in
1 #Initialize the aruco detection
2 from sandbox.markers import MarkerDetection
----> 3 aruco = MarkerDetection(sensor=sensor)

~\open_AR_Sandbox\sandbox\markers\markers_plotting.py in init(self, sensor, **kwargs)
8 def init(self, sensor, **kwargs):
9 self.sensor = sensor
---> 10 self.Aruco = ArucoMarkers(sensor=sensor, **kwargs)
11 self.df_aruco_position = None
12 self.lines = None

~\open_AR_Sandbox\sandbox\markers\aruco.py in init(self, sensor, aruco_dict, **kwargs)
26 def init(self, sensor=None, aruco_dict=None, **kwargs):
27 if not aruco_dict:
---> 28 self.aruco_dict = aruco.DICT_4X4_50 # set the default dictionary here
29 else:
30 self.aruco_dict = aruco_dict

NameError: name 'aruco' is not defined

[danielsk78]

Hi @hentsd
It seems you didn’t install the correct opencv package properly.

Please do:

pip uninstall opencv-python

pip install opencv-contrib-python

[hentsd]

@danielsk78, thanks, that was the issue in that case.
sensor.Sensor._stop() in main_thread gives:
AttributeError Traceback (most recent call last)
in
----> 1 sensor.Sensor._stop()

~\open_AR_Sandbox\sandbox\sensor\kinectV2.py in _stop(self)
85 Stop the thread when _platform is linux
86 """
---> 87 if self._thread_status is not 'stopped':
88 self._thread_status = 'stopped' # set flag to end thread loop
89 self._thread.join() # wait for the thread to finish

AttributeError: 'KinectV2' object has no attribute '_thread_status'

and later in markerdetection_calibration i once again get:
ModuleNotFoundError Traceback (most recent call last)
in
1 #save the paths to the calibration files
----> 2 from sandbox import _calibration_dir
3 _calibprojector = _calibration_dir + "my_projector_calibration.json"
4 _calibsensor = _calibration_dir + "my_sensor_calibration.json"

ModuleNotFoundError: No module named 'sandbox'

after:
from sandbox import _calibration_dir
_calibprojector = _calibration_dir + "my_projector_calibration.json"
_calibsensor = _calibration_dir + "my_sensor_calibration.json"

[danielsk78]

2 comments:

1. Are you using Linux to run the sandbox?
2. If you don't have a local installation of the sandbox, be sure to have a relative import to the sandbox main code. In your specific case, the notebook have an incorrect relative import. I will fix this when the notebook documentation for the markerdetection_calibration is ready. For the moment, just replace the first lines of codes in that notebook with:

import os,sys
sys.path.append('../../../')

[hentsd]

Got it.
I'm using Windows 10 at the moment.

[hentsd]

When trying to play around with the BuFaTa model, I once again get the same "ModuleNotFoundError: No module named 'sandbox'" error despite of changing the first lines to:
import os,sys
sys.path.append('../../../') which fixed the same error last time.

[danielsk78]

This is a relative import. Is when you don't have the pip version installed.

For the BuFaTa model use:
import os,sys
sys.path.append('../../../../../') #Sandbox

If you want an easier solution, so you don't worry anymore about the relative import, use the setup.py provided for a local installation. (pip install -e .)

[hentsd]

Alrighty, huge thanks @danielsk78, seems like I've almost got it set up.
However theres a problem with the topomodel widget as it's giving me:

AttributeError Traceback (most recent call last)
in
1 #Topography Widgets
----> 2 TopoModule.widgets()

AttributeError: 'TopoModule' object has no attribute 'widgets'

[danielsk78]

The structure of the sandbox was recently changed to have consistency between modules. Ideally every module have or will have a function in the form of [ModuleName].show_widgets().

For your case just change the name of widgets() to show_widgets().

These "errors" are part of the documentation that we are currently working on.

[hentsd]

Hey @danielsk78, last time i added gempy module everything worked fine, today i'm getting
AttributeError Traceback (most recent call last)
in
----> 1 widget = gpsb.widget_model_selector()
2 widget

AttributeError: module 'gempy' has no attribute 'widget_model_selector'

and

AttributeError Traceback (most recent call last)
in
----> 1 gpsb.show_widgets().show()

AttributeError: module 'gempy' has no attribute 'show_widgets'

[danielsk78]

When exactly was that "last time"? You need to correctly initialize the GempyModule class to be able to show the widgets.

[hentsd]

The last time was 2 days ago, but seems like reinstalling gempy fixed the problem.

[hentsd]

Hey @danielsk78, thank you again for the thorough cooperation for getting the sandbox running. I'm just curious, what is your recommended PC hardware or what systems are you using yourself? Asking it because it's loading/updating time with i7-8750H (at 2.2GHz), RTX2060 (6GB) and 16GB of RAM is quite horrendous.

[danielsk78]

Hi @hentsd, your system is more than enough!
If the problem is from the computation time in the GemPy module, then this is because, for every new frame of the sandbox, we need to recalculate the whole GemPy model and depending on how big the model is, this will take some time. The solution to this should be addressed directly as a GemPy issue and not sandbox (but I'm trying to find a partial solution to this... like only compute the model when the frame change).
If the problem is outside the GemPy module, then the error may be related to issue #3. We are still looking at what could trigger this problem and how to solve it, but it seems to be a conflict with matplotlib and threading. Any help would greatly appreciate it.

[hentsd]

Hey @danielsk78,got it.
Okay, that's exactly what the problem is with GemPy.
Also, when initializing the main thread, the topography doesn't seem to update live at all (outside GemPy). Only once you have stopped the thread.
I'll look issue #3 and see if i can get it solved.

[danielsk78]

Hey @hentsd,
So if the thread doesn't seem to update is because, for every new frame, the error in issue #3 appears in the form of a warning and skip the process. You can try to turn off the aurco detection and the live frame update (update only if the frame changes) by doing:
main.check_change = False; main.ARUCO_ACTIVE = False;
or using the widgets.
If you see any other error/ warning appearing, please open a new issue with the problem.

[danielsk78]

Discussion is now part of issue #3. Follow this conversation there.