LEVBDIM: A Light EVent builder based on DIM
2016-08-22
Abstract. We developed a light data acquisition system, based on DIM and mongoose-cpp frameworks. Providing binary data collection, events building, web accessible finite state machine and process control, it is well suit to manage distribute data source of laboratory or beam test. It supports only simple event building (one unique process)
DIM [1] is an HEP acquisition framework developed in the DELPHI experiment. It embeds binary packet exchange in messages published by server and subscribed by client. It is light and TCP/IP based and can be installed on nearly all currently used OS nowdays. Mongoose-cpp is a tiny web server that is used to bind CGI commands to the DAQ application, either in the finite state machine of the process or in standalone mode.
LEVBDIM is a simple acquisition framework where DIM is used to exchange structured binary buffer. It provides several functionalities often used in modern data acquisition:
- Event Building
- Predefine structure of buffer
- Publication of data
- Collection of data from numerous publisher
- data writing
- Run control
- Generic Finite State Machine
- Web access
- Process control
- Dedicated FSM used to create process on remote computers with a JSON[5] description of the environment.
All those functionalities are mainly independent and can be used on their own. The last section of this documentation details a full example using all the capabilities of LEVBDIM.
2.1. Additional packages {#sec-additional-packages .h2 data-heading-depth="2" style="display:block"}
Few standard packages are needed and can be found on any Linux distributions: boost, jsoncpp, curl, git and scons
sudo apt-get -y install libboost-dev libboost-system-dev
libboost-filesystem-dev libboost-thread-dev libjsoncpp-dev
libcurl4-gnutls-dev git scons
Two network libraries should be installed. The first one is DIM that can be download and compile from https://dim.web.cern.ch/dim/. The second one is used to give web access to the application, it's Mongoose-cpp[2] based on Mongoose [3]. One version is distributed with levbdim and can be compiled before the installation.
2.2. LEVBDIM installation {#sec-levbdim-installation .h2 data-heading-depth="2" style="display:block"}
2.2.1. Getting the levbdim software {#sec-getting-the-levbdim-software .h3 data-heading-depth="3" style="display:block"}
The software is distributed on GitHub. The master version is download with
git clone http://github.com/mirabitl/levbdim.git
2.2.2. Installing the software {#sec-installing-the-software .h3 data-heading-depth="3" style="display:block"}
A snapshot of mongoose is copied in the web directory
cd levbdim
source web/mongoose.install
The default installation is in /opt/dhcal/levbdim directory.
The compilation is using Scons
cd levbdim
scons
The library liblevbdim.so is installed in the lib subdirectory.
3. Library functionalities {#sec-library-functionalities .h1 data-heading-depth="1" style="display:block"}
Event building consists of merging various data source that collect event fragment at the same time. Each data source should consequently have a localization tag and a time tag for each data fragment it provides. This fragment are published by a DimService and is centrally collected to build an event, i.e a collection of data fragment with an identical time tag.
The event builder proposed is not itself distributed, it is unique and so no geographical or time dependent partitioning is available. All data sources are collected by a unique process. It subscribes to all available data source and writes them on receipt in a shared memory. A separate thread scans the memory and build the event, i.e a collection of data buffer from each registered data source with the same time tag. A last thread process completed event and call registered event processors that can write event to disk in user defined format. The next sections detail the software tools provided to achieve those tasks.
3.1.1. The buffer structure {#sec-the-buffer-structure .h3 data-heading-depth="3" style="display:block"}
The levbdim::buffer class is a simple data structure that provides space to store the read data (the payload) and four tags:
-
The Detector ID: A four bytes id used to tag different geographical partition (ex. tracker barrel, ECAL end cap…)
-
The source ID: A four bytes id characterizing the data source (ex. ADC or TDC number)
-
The event number: A four bytes number of readouts
-
The bunch crossing number: an eight bytes number characterizing the time of the event in clocks count. The clock is the typical clock of the experiment. It may allows a finer events building if needed. It si currently optional
-
The payload: a bytes array containing detector data for the given event number readout.
-
The payload size: the size of the payload array.
The payload can be compressed and inflate on the fly using the gzip[4] library calls (compress,uncompress methods).
Buffer do not need to be allocated by data producers since they are embedded in the levbdim::datasource class described bellow.
3.1.2. Server side: The datasource class {#sec-server-side--the-datasource-class .h3 data-heading-depth="3" style="display:block"}
A levbdim::datasource object instantiates a levbdim::buffer with a maximal size specified. It creates also the associated DimService with the name /FSM/LEVBDIM/DS-X-Y/DATA where X is the detector id and Y the data source one. The buffer has method to access the payload in order to allow the user process to fill it.
The server process can instantiate any number of levbdim::datasource fill the associated buffers and publish them every time a new event is collected. The only requirement is that a DimServer is started at the initialization phase. The following pseudo code gives an example of usage.
// Header
std::vector<levbdim::datasource*> _sources;
// Initialisation
// Starting the DIM server
std::stringstream s0;
s0.str(std::string());
s0<<"dummyServer-"<<name;
DimServer::start(s0.str().c_str());
// Configuration
// Loop on data sources
std::cout <<"Creating datasource Detector="<<det<<" SourceId="<<sid<<std::endl;
// Maximum buffer size is set to 128 kBytes
levbdim::datasource* ds= new levbdim::datasource(det,sid,0x20000);
_sources.push_back(ds);
This is an example with one thread per data source but all data source can be filled and publish sequentially if needed.
// Create a readout thread per data source
// calling the readdata method
void dummyServer::start()
{
_running=true;
for (std::vector<levbdim::datasource*>::iterator ids=_sources.begin();ids!=_sources.end();ids++)
{
_gthr.create_thread(boost::bind(&dummyServer::readdata, this,(*ids)));
::usleep(500000);
}
}
// Loop on Events and publish data
void dummyServer::readdata(levbdim::datasource *ds)
{
// Filling the buffer with random data
// evt and bx are global data
std::srand(std::time(0));
while (_running)
{
::usleep(10000);
if (!_running) break;
if (evt%100==0)
std::cout<<"Thread of "<<ds->buffer()->dataSourceId()<<" is running "<<evt<<" "<<_running<<std::endl;
// Just fun , ds is publishing a buffer containing sourceid X int of value sourceid
uint32_t psi=ds->buffer()->dataSourceId();
uint32_t* pld=(uint32_t*) ds->payload();
for (int i=0;i<psi;i++) pld[i]= std::rand();
pld[0]=evt;
pld[psi-1]=evt;
// publishing data of current event number evt and bunch crossing number bx
ds->publish(evt,bx,psi*sizeof(uint32_t));
}
// Running is set false by the stop method , thread exits
std::cout<<"Thread of "<<ds->buffer()->dataSourceId()<<" is exiting"<<std::endl;
}
As one can see the levbdim::datasource provides an easy way to store event data and publish them on the fly.
3.1.3. Client side: The datasocket class {#sec-client-side--the-datasocket-class .h3 data-heading-depth="3" style="display:block"}
On client side the process should subscribe to the datasource published it wants to collect. It is achieved by using levbdim::datasocket class. It instantiates a DimInfo object that subscribe to the DimService named /FSM/LEVBDIM/DS-X-Y/DATA published by the levbdim::datasource. It's a DimClient and it has an infoHandler method that will collect the publish data. The data are then accessible in the levbdim::buffer. This method can also be overwritten to implement user treatment but the preferred usage is to save received data to /dev/shm so it can be used by the event builder class. This is achieved at initialization in an analog way to data sources
std::cout <<"Creating datasocket "<<det<<" "<<sid<<std::endl;
// data socket storing data in buffer of same size
levbdim::datasocket* ds= new levbdim::datasocket(det,sid,0x20000);
// Save data in share memory directory
ds->save2disk("/dev/shm/levbdim/");
_sources.push_back(ds);
The needed directory /dev/shm/levbdim and /dev/shm/levbim/closed needed by the method are not created by the code and MUST be created by the user. Real disks directory can obviously be used but performances may be deeply affected since /dev/shm is memory disk and can provide much faster access.
In this example, each datasocket object will write on reception a binary file named
Event_#det_#source_#event_#bx
in /dev/shm/levbdim containing the buffer. Once this one-event,one-source data are written, it creates an empty file with the same name in the /dev/shm/levbdim/closed directory. By scanning this later directory, the event builder will be able to read back the completed data in the memory files.
This mechanism decoupled the data reception from the event building.
3.1.4. Event building: The shmdriver class {#sec-event-building--the-shmdriver-class .h3 data-heading-depth="3" style="display:block"}
Finally the event collection is done asynchronously by the levbdim::shmdriver class. The class registers data source identified by their detectorId and their dataSourceId at initialization. It also registers levbdim::shmprocessors that will treat completed events. Two threads are then created:
-
The first one is doing the following task:
- listing the files in /dev/shm/levbdim/closed directory
- reading the corresponding files in /dev/shm/levbdim/ directory. The data are stored in a map with the key being the event number, an levbdim::buffer is pushed in an associated vector for each data source read.
- erasing the read files in /dev/shm/levbdim and closed directory
-
The second one is looping on the map entries, If one entry collected the exact number of data sources registered, the entry is processed and then removed from the map.
The shmprocessor class is a pure virtual class implementing only three methods:
- void start(uint32_t run) called at each new run start.
- void stop() called at end of run
- void processEvent(uint32_t key,std::vector dss) called for each completed pair of event number and buffer list.
An example of processor is given in levbdim::basicwriter class which is writing data in a binary format. Many processors can be registered and will be called sequentially (data writing, monitoring, visualization…).
The main limitation of LEVBDIM is that this event building is unique and the computing not distributed. It is nevertheless possible to duplicate the event building process on a different computer (for cpu consuming task) but data from data sources are then sent twice and the bandwidth might be affected if the event sizes are large.
The following pseudo code implements an example of event builder:
-
configure
// Delete existing datasockets for (std::vector<levbdim::datasocket*>::iterator it=_sources.begin();it!=_sources.end();it++) delete (*it); _sources.clear(); // Now create the builder _evb= new levbdim::shmdriver("/dev/shm/levbdim"); _evb->createDirectories(); _evb->cleanShm(); // register a processor _writer= new levbdim::basicwriter("/tmp"); _evb->registerProcessor(_writer); // register datasockets // Loop on data source ids // ... std::cout <<"Creating datasocket "<<det<<" "<<sid<<std::endl; levbdim::datasocket* ds= new levbdim::datasocket(det,sid,0x20000); ds->save2disk("/dev/shm/levbdim/"); _sources.push_back(ds); _evb->registerDataSource(det,sid); -
start
// Starts the two shmdriver threads and pass the run number to the processors _evb->start(run); -
stop
// stop the threads _evb->stop();
3.2.1. Structure of the FSM mechanism {#sec-structure-of-the-fsm-mechanism .h3 data-heading-depth="3" style="display:block"}
All data acquisition processes ( hardware readout, event builder, DB interfaces, Slow control…) may have different states corresponding to different phases of the read out process (OFF, READY, CONFIGURED, RUNNING,STOPPED…). It is therefore compulsory to have a software mechanism able to trigger the transition between the different states. A DIM-based implementation is done with the levbdim::fsm class that contains:
- A vector of states names
- A map of command names associated to a vector of levbdim::fsmTransition
- A DimRpc object (levbdim::rpcFsmMessage class) handling the command reception and its processing
An fsmTransition is a class associating an initial state, a final state and a command handler. The command handler is a boost functor with a levbdim::fsmmessage parameter. As an example one can add a transition to the fsm in a wummyServer class with the following code
fsm->addTransition("START","CONFIGURED","RUNNING",boost::bind(&wummyServer::start,
this,_1));
where
- START is the command
- CONFIGURED and RUNNING are the initial and final state
- start(levbdim::fsmmessage* m) is a method of the wummyServer class
The levbdim::fsmmessage is a class handling a Json::Value with two attributes:
- command : the command name
- content : A possibly empty set of parameters
On return of the fsm handler one may add to the content set a Json::Value named answer with the setAnswer() method.
On the client side an levbdim::fsmClient class is provided allowing the controlling program to call the execute(levbdim::fsmmessage* m) method with the message containing the required command and parameters.
This structure is performing but required that the run control is written using DIM Rpc implementation. In order to permit much lighter clients we implements a web based version of the state machine.
The levbdim::fsmweb class is an overloading of the fsm class with a mongoose-cpp [2] webserver on a port XXXX with three services:
-
htpp://mypc:XXXX/ returns the list of possible Finite State Machine commands, the list of standalone commands and the PREFIX used, in a JSON file format.
-
htpp://mypc:XXXX/PREFIX/FSM?command=NAME&content={…} calls the FSM transition NAME with the content parameters set. The answered levbdim::fsmmessage is published in JSON format
-
htpp://mypc:XXXX/PREFIX/CMD?name=NAME&toto={…} calls the boost callback registered in the fsmweb for the command named NAME.
In the last case additional commands to the FSM ones can be registered with addCommand method:
_fsm->addCommand("DOWNLOAD",boost::bind(&wummyServer::download, this,_1,_2));will register the command named DOWNLOAD and link it to the boost functor. The method download(Mongoose::Request &request, Mongoose::JsonResponse &response) is called. The request parameter is an object embedding the CGI parameters, the response is a pure Json::value object that will be published as a JSON file on return.
This approach allows to add commands not linked to state transition (monitoring,debug,..)
4. Full example of an FSM based process {#sec-full-example-of-an-fsm-based-process .h1 data-heading-depth="1" style="display:block"}
4.1. The wummyServer class {#sec-the-wummyserver-class .h2 data-heading-depth="2" style="display:block"}
The class wummyServer is a dummy data server that will publish periodically random data. Its prototype is the following
class wummyServer {
public:
wummyServer(std::string name,uint32_t port);
void configure(levbdim::fsmmessage* m);
void start(levbdim::fsmmessage* m);
void stop(levbdim::fsmmessage* m);
void halt(levbdim::fsmmessage* m);
void readdata(levbdim::datasource *ds);
void download(Mongoose::Request &request, Mongoose::JsonResponse &response);
void list(Mongoose::Request &request, Mongoose::JsonResponse &response);
void setEvent(uint32_t e) {_event=e;}
void setBx(uint64_t b) {_bx=b;}
uint32_t event(){return _event;}
uint64_t bx(){return _bx;}
private:
fsmweb* _fsm;
std::vector<levbdim::datasource*> _sources;
bool _running,_readout;
boost::thread_group _gthr;
uint32_t _event;
uint64_t _bx;
}
The constructor instantiates the state machine, registers the states and start the Web and DIM services.
wummyServer::wummyServer(std::string name,uint32_t port) : _running(false)
{
_fsm=new fsmweb(name);
// Register state
_fsm->addState("CREATED");
_fsm->addState("CONFIGURED");
_fsm->addState("RUNNING");
// Register allowed transition
_fsm->addTransition("CONFIGURE","CREATED","CONFIGURED",
boost::bind(&wummyServer::configure, this,_1));
_fsm->addTransition("CONFIGURE","CONFIGURED","CONFIGURED",
boost::bind(&wummyServer::configure, this,_1));
_fsm->addTransition("START","CONFIGURED","RUNNING",
boost::bind(&wummyServer::start, this,_1));
_fsm->addTransition("STOP","RUNNING","CONFIGURED",
boost::bind(&wummyServer::stop, this,_1));
_fsm->addTransition("HALT","RUNNING","CREATED",
boost::bind(&wummyServer::halt, this,_1));
_fsm->addTransition("HALT","CONFIGURED","CREATED",
boost::bind(&wummyServer::halt, this,_1));
// Register additional commands
_fsm->addCommand("DOWNLOAD",boost::bind(&wummyServer::download, this,_1,_2));
_fsm->addCommand("LIST",boost::bind(&wummyServer::list, this,_1,_2));
//Start Dim server
std::stringstream s0;
s0.str(std::string());
s0<<"wummyServer-"<<name;
DimServer::start(s0.str().c_str());
// Start web services
_fsm->start(port);
}
There is then handlers declared for each FSM or standalone commands
In this configure transition there is only the registration of the data sources. In real world it will also include the hardware configuration.
void wummyServer::configure(levbdim::fsmmessage* m)
{
std::cout<<"Received command "<<m->command()<<
" with parameters "<<m->value()<<std::endl;
// Delete existing datasources if any
for (std::vector<levbdim::datasource*>::iterator it=_sources.begin();it!=_sources.end();it++)
delete (*it);
_sources.clear();
// Add the data source
// Parse the json message, the format used in the example is
// {"command": "CONFIGURE", "content": {"detid": 100, "sourceid": [23, 24, 26]}}
Json::Value jc=m->content();
int32_t det=jc["detid"].asInt();
const Json::Value& dss = jc["sourceid"];
// Book a Json array to return the list of source booked
Json::Value array_keys;
// Loop on sources
for (Json::ValueConstIterator it = dss.begin(); it != dss.end(); ++it)
{
const Json::Value& book = *it;
int32_t sid=(*it).asInt();
std::cout <<"Creating datasource "<<det<<" "<<sid<<std::endl;
array_keys.append((det<<16)|sid);
levbdim::datasource* ds= new levbdim::datasource(det,sid,0x20000);
_sources.push_back(ds);
}
// Overwrite msg
//Prepare complex answer
m->setAnswer(array_keys);
}
The handler is just starting a boost thread of data “reading” for each data source
void wummyServer::start(levbdim::fsmmessage* m)
{
std::cout<<"Received "<<m->command()<<std::endl;
_running=true;
for (std::vector<levbdim::datasource*>::iterator ids=_sources.begin();ids!=_sources.end();ids++)
{
_gthr.create_thread(boost::bind(&wummyServer::readdata, this,(*ids)));
::usleep(500000);
}
}
The readdata method is defined with the datasource argument. It's typically the method that is interfacing the hardware readout of the event data.
void wummyServer::readdata(levbdim::datasource *ds)
{
uint32_t last_evt=0;
std::srand(std::time(0));
while (_running)
{
::usleep(10000);
// ::sleep(1);
if (!_running) break;
// Update data only when _event number is modified
if (_event == last_evt) continue;
if (evt%100==0)
std::cout<<"Thread of "<<ds->buffer()->dataSourceId()<<" is running "<<
last_evt<<" events, running status : "<<_running<<std::endl;
// Just fun , ds is publishing a buffer containing sourceid X int of value sourceid
uint32_t psi=ds->buffer()->dataSourceId();
// update the payload
uint32_t* pld=(uint32_t*) ds->payload();
for (int i=0;i<psi;i++) pld[i]= std::rand();
pld[0]=evt;
pld[psi-1]=evt;
// publish data
ds->publish(_event,_bx,psi*sizeof(uint32_t));
last_evt= _event;
}
std::cout<<"Thread of "<<ds->buffer()->dataSourceId()<<" is exiting"<<std::endl;
}
The stop transition is just stopping the reading threads without reconfiguring the process.
void wummyServer::stop(levbdim::fsmmessage* m)
{
std::cout<<"Received "<<m->command()<<std::endl;
// Stop running
_running=false;
::sleep(1);
std::cout<<"joining the threads"<<std::endl;
_gthr.join_all();
}
In the halt transition, the datasources are removed and a new configuration is needed
void wummyServer::halt(levbdim::fsmmessage* m)
{
std::cout<<"Received "<<m->command()<<std::endl;
if (_running)
this->stop(m);
std::cout<<"Destroying data sources"<<std::endl;
for (std::vector<levbdim::datasource*>::iterator it=_sources.begin();it!=_sources.end();it++)
delete (*it);
_sources.clear();
}
They are dummy commands:
- The DOWNLOAD can implement the download of database data for future configuration
void wummyServer::download(Mongoose::Request &request, Mongoose::JsonResponse &response)
{
std::cout<<"download "<<request.getUrl()<<" "<<request.getMethod()
<<" "<<request.getData()<<std::endl;
// Getting the data base name
std::string state= request.get("DBSTATE","NONE")
// if (state.compare("NONE")!=0) do the download
std::stringstream os;
os<<state<<" has been download"
response["answer"]=os.str();
}
- The LIST command can return the list of data sources
void wummyServer::list(Mongoose::Request &request, Mongoose::JsonResponse &response)
{
std::cout<<"list "<<request.getUrl()<<" "<<request.getMethod()
<<" "<<request.getData()<<std::endl;
Json::Value array_keys;
for (std::vector<levbdim::datasource*>::iterator ids=_sources.begin();ids!=_sources.end();ids++)
array_keys.append(((*ids)->detectorId()<<16)|(*ids)->dataSourceId());
response["answer"]=array_keys;
}
The main program just instantiate the wummyServer object. In order to emulate new event arrival the event number is updated every second
int main()
{
wummyServer s("myfirsttry",45000);
uint32_t evt=0;
// Just update the event number to trigger data publication
while (1)
{
::sleep(1);
s.setEvent(evt++);
}
}
5. The Run and process Control {#sec-the-run-and-process-control .h1 data-heading-depth="1" style="display:block"}
With the web implementation of the finite state machine, the run control is really eased and can be written with very light tools like the curl[7] library or python. One can ofcourse use more user friendly interfaces like Qt [8] or web based graphical interfaces like Wt[9] or javascript pages, since all these frameworks have http request capabilities. We will give two examples in curl and python of the control of the wummyServer process.
For example using curl, one can configure the wummyServer process with:
curl http://mypc:45000/unessai/FSM?command=CONFIGURE\&content=%7B\"detid\":100,\"sourceid\":%5
B12,13,14%5D%7D
and start a run with
curl http://mypc:45000/unessai/FSM?command=START\&content=%7B%7D
Access to the download standalone command is also easy
curl http://mypc:45000/unessai/CMD?name=DOWNLOAD\&state=MyDbStateForMyDetetector
The configuration and the control of all daq processes is then a simple bash script that can be run on any computer where curl is installed.
We used urllib and urllib2 and socket package (SOCKS support) and define two methods:
- FSM access
def executeFSM(host,port,prefix,cmd,params):
if (params!=None):
# build the URL
myurl = "http://"+host+ ":%d" % (port)
# parameter list
lq={}
# content take values from the params python list
lq["content"]=json.dumps(params,sort_keys=True)
lq["command"]=cmd
# encode it
lqs=urllib.urlencode(lq)
# Build the final url
saction = '/%s/FSM?%s' % (prefix,lqs)
myurl=myurl+saction
# send the command
req=urllib2.Request(myurl)
r1=urllib2.urlopen(req)
return r1.read()
- Command access
def executeCMD(host,port,prefix,cmd,params):
if (params!=None and cmd!=None):
myurl = "http://"+host+ ":%d" % (port)
# CGI parameter list
lq={}
lq["name"]=cmd
for x,y in params.iteritems():
lq[x]=y
# build the list
lqs=urllib.urlencode(lq)
saction = '/%s/CMD?%s' % (prefix,lqs)
myurl=myurl+saction
req=urllib2.Request(myurl)
# Check the server is alived
try:
r1=urllib2.urlopen(req)
except URLError, e:
p_rep={}
p_rep["STATE"]="DEAD"
return json.dumps(p_rep,sort_keys=True)
else:
return r1.read()
else:
# no additional parameters build the query
myurl = "http://"+host+ ":%d/%s/" % (port,prefix)
req=urllib2.Request(myurl)
# Check the server is alived
try:
r1=urllib2.urlopen(req)
except URLError, e:
p_rep={}
p_rep["STATE"]="DEAD"
return json.dumps(p_rep,sort_keys=True)
else:
return r1.read()
Using those two methods one can build any request to a fsmweb based process. The usage of python allows the developers to build a much more complex code architecture and benefit from a rich library resources (XML,JSON configuration for example).
5.2. Large architecture approach {#sec-large-architecture-approach .h2 data-heading-depth="2" style="display:block"}
The LEVBDIM package was developed to read the Semi Digital HCAL of the Calice collaboration[6]. It handles a large number of processes for readout (14), database access (1), trigger control (2), low voltage control (1) and event building(1). The configuration sequence is complex and can be coded in python but the control is then bind to a single python process and a single user. We preferred to have a hierarchical approach and created an upper levbdim::fsmweb process (WDaqServer) with a relatively simple finite state machine (INITIALISE,CONFIGURE,START,STOP,DESTROY). This process implements hierarchical (or parallel for readout) lower level FSM transitions via web or DIM message calls. It's a intrinsically a web process and can be accessed remotely by various client.
The most user-friendly client is a Wt based web service that is accessing the WDaq server. Several clients can be connected and the processing is driven by the WDaq FSM.
5.3. The process management {#sec-the-process-management .h2 data-heading-depth="2" style="display:block"}
One last requirement of a distributed acquisition system is the management of the different processes (start,kill,restart). Heavily inspired by the XDAQ [10] framework jobcontrol, we developed one application levbdim::fsmjob that is started on every computer used in the Daq. On request to this application a list of processes are forked and the specified programs are created (or killed/ restarted). The child processes are writing in PID identified log that can be requested to the application. The management of the fsmjob itself can be done using Linux daemon mechanism that will ensure that it is started during the computer boot sequence.
5.3.1. Process description {#sec-process-description .h3 data-heading-depth="3" style="display:block"}
The process are described with the following structure:
{
"NAME":"WRITER",
"ARGS" : ["-d /data/NAS/oyonax"],
"ENV" : [
"DIM_DNS_NODE=lyosdhcal9","LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:
/opt/dhcal/levbdim/lib:/opt/dhcal/lib:/opt/dhcal/root/lib:
/opt/dhcal/dim/linux:/opt/dhcal/lcio/v02-00/lib/:$LD_LIBRARY_PATH"
],
"PROGRAM" : "/opt/dhcal/bin/levbdim_build"
}
where
- NAME is a user given name of the process. Since sevral jobs can be started one the same computer the name has to be unique on agiven computer
- PROGRAM is the executable
- ARGS is an array of argument to the executable
- ENV is an array of environment variables to be set when the program runs.
The state machine of the fsmjob is described in figure 1. All jobs on a givern PC must be registered. The first possibility is to send an INITIALISE command with a local file name or an url name in the content argument. The file is read or load and parsed. The jobs corresponding to the computer name are registered. An example of the file structure is given here:
{
"HOSTS" :
{
"lyosdhcal9":[
{
"NAME":"WRITER",
"ARGS" : ["-d /data/NAS/oyonax"],
"ENV" : [
"DIM_DNS_NODE=lyosdhcal9",
"LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:
/opt/dhcal/levbdim/lib:/opt/dhcal/lib:
/opt/dhcal/root/lib:$LD_LIBRARY_PATH"
],
"PROGRAM" : "/opt/dhcal/bin/levbdim_build"
},
{
"NAME":"WIENER",...
},
{
"NAME":"DBSERVER",
"ARGS" : [],...
}
],
"lyoilcrpi17":[
{ "NAME":"MDCSERVER",
"ARGS" : [],...
}
]
}
It's a map of HOSTS (computer's names) containing a list of processes with the previously described process's structure. In order to use this initialisation, the file should be copied locally on each computer or put in an accessible web page.
Another possibility is to use the REGISTERJOB transition and to register each job independantly:
-
Send a REGISTRATION command. It clears previous job list and prepare a map of jobs
-
Send any number of REGISTERJOB commands with parameters of content being JSON strings:
- processname for NAME tag
- processargs for ARGS tag
- processenv for ENV tag
- processbin for PROGRAM tag
-
Send a ENDREGISTRATION to terminate the initialisation process
Once the process is INITIALISED one can send a START command to start all process on the PC and a KILL command to kill them. A DESTROY command put the daemon back in CREATED state.
Additionnal standalone commands are available:
Command
Parameters
Action
STATUS
No
Returns the list and status of all processes
with per process the HOST,NAME,PID and STATUS
(Running/DEAD)
KILLJOB
pid, processname,signal(9)
kill the specified process
RESTART
pid, processname,signal(9)
restart the specified process
JOBLOG
pid, processname,lines(100)
return the last lines of the log of the specified job
Figure 1. Finite State machine of the levbdim::fsmjob
5.3.2. Process control client {#sec-process-control-client .h3 data-heading-depth="3" style="display:block"}
Again it is free to the developper to write its own client using any web library. We personally used python since the JSON format is easily interfaced to the language structure and the network library simple. C++ (with jsoncpp and curl) is also used. A python example is given in the example directory.
[1] C Gaspar, DIM, 🔎
[4]GZIP ref 🔎
[5]JSON ref 🔎
[6]Calice 🔎
[7]curl 🔎
[8]QT 🔎
[9]Witty 🔎
[10]XDAQ 🔎