Welcome to DARC

For installation instructions, see INSTALL.
For documentation see doc/man.pdf (which you may need to 
cd doc && make ubuntu1204 && make man.pdf
or appropriate for your OS - to install required packages - namely
doxygen and latex).


Overview:
The Durham RTC contains the following components.
An RTC that does the number crunching (reads the cameras, controls the
mirrors): darcmain (though you should never have to start this manually)
A control program that controls the RTC and optionally reads telemetry
data from it: darccontrol
A GUI for interacting with the control program and the telemetry
server: darcgui
A command line tool: darcmagic
A python library: controlCorba.py
Telemetry tools: darcplot etc.


To run:
darccontrol path/to/configfile.py -o
To check the status
darcmagic status

Other notes:

GIT development (version control):
For git header (version) updates (as reported by darcmagic get -version):
make version
make

darc3 upcoming changes:
multirate
rationalisation of names - refSlopes, addActuators, closeLoop, v0, actuators, etc.
udp telemetry.
Improvements to parameter update - reduced jitter.
Numa awareness - ability to put parameters in different numa memory
regions.
Parameter update - improvements to reduce jitter.  Ability to warm the
cache.
16-bit float operation.

TODO:

See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=884776
Condition variables are no longer process shared.  This affects
telemetry and should be fixed!  Affects libc6 version 2.25 and
onwards. (so 2.24 is okay).  apt show libc6 or yum infl glibc shows
the version. Or ldd --version

buffer.py (Circular) acts nicely when shm removed - reopens?  Or detects it.
give plots option of what they want to subscribe too parameter wise.
e.g. a list in the initcode section of plot file, and then a darcParam
dictionary passed to the mangle area.
plot change prefix - and multiple prefixes???
GUI to automatically define subaps
Investigate rejection transfer function saturation at low frequencies
Complete modules for alternative to subapLocation - in rtcslope and rtccalibrate
Figure sensor input for refslopes too?
multicast of telemetry?
udp camera interface?
Test under VX works.
prefetching intrinsics

Recently completed features:
NUMA awareness
A widget area for plotter - so user can add custom widgets.
zeropadding during correlation
change the way calibrate and slope modules are called - allow to do
all subaps at once?
vastly improved plotting capability
ability to use cog and correlation at same time (on different subaps).
A job scheduler for pygtk.
An averaged telemetry circular buffer... - rolling average -
external to darccore.
Averager:  repeat or single average.  Rolling or collated.  shm name.
If single average, no shm - write to stdout.
added pixel reordering to sl240Int32cam.c
WPU centroider module
Update config files and libraries.
Async reconstructor module, and a socketMirror module
Adaptive windowing different thresholding types...
Objects to write to shm on each node (rather than transmitting data
twice).  If a client wants a stream kept on, it needs to register with
the object somehow - maybe keeping a socket open?  Then when all
sockets are closed, the server can turn off the stream.
The way to do this is in the shm, have an entry which each reader
increments before calling pthread_cond_wait (i.e. before blocking to
wait for more data).  The server then checks this value and sets to
zero before broadcasting.  If it is nonzero, the time is saved
(timeDataLastRequested), and if it is zero (no one has waited for
data), then timeDataLastRequested is compared with current time, and
if difference is say greater than 1 minute, the server stops receiving
data and sets decimation to zero.
PUT A GPL HEADER ON EVERYTHING.
Test logging to FITS file fully (thread safe?).
Ability to log directly to FITS file.
Getting equivalent of $Id$ in GIT:
Check that saver.WriteRaw still works with the circbuf hdr...
socket input and output interfaces
Option to use one of the subap processing threads to do the dm
stuff... (that way, it could all run in 1 thread if necessary).
Update camera libraries - waitPixel() may no longer have a mutex
locked upon entry...
Then, check subapAllocation works okay.
Way of specifying which subap processed by which thread - which
should allow getting rid of some mutexs
Look at separating out the RTC - so can be used in several instances on different computers.
Sort out the darcmain/darccore thing (ie do proper includes etc)
Ability to subscribe to an individual variable (e.g. plot subscribing
to subapLocation).
Async buffer switching - double buffered - i.e. a separate thread that
handles the buffer switch, which could take several iterations to complete
(e.g. uploading rmx to GPU).  Actually - I won't do this - but
instead, eg in the gpu case, you could have a parameter rmxToUpload
for example, which would then get uploaded into gpu, and then a
parameter useUploadedRMX or something, so swap to using it... it would
be dependent on the recon interface library...
An interface for parameter stream - ie the ability to change a
parameter every iteration...
Option for sender to send from head of circular buffer.
Have a param change interface allowing parameters to be changed each iteration.
Ability to have sequence of refCentroids (as with actuators).
Sorted out buffer switching
calibration module
centroiding modules.
plot.py - remove depreciation warnings.
Get rid of camerasFraming.  Get rid of dmCommand pointers in rtcrecon.h
On darc exit - close the libraries properly.
Adaptive windowing - being able to specify maxadapoffset on a
per-subap basis.  (so that not all windows move).
Think about rtcpxlbuf - does it still need partialadd? - Done
Think about position of sendactuators - should it be moved earlier? - Done
use agbcblas.h - Done
tidy up reconstruction module (shouldn't be passed threadInfo).
Rationalise use of copyToInactive() - done, I think.  Should now only
ever be called after a buffer switch has completed.
Rationalise thisiter - should be darc generated always, and then
camera frame numbers should be put in rtcStatusBuf???  Which should
telemetry use?  Probably darc generated one.  Also, have a
writeError() when the camera generated one does not increment by 1.
Get a log of every change made to darc...


Figure sensor usage:
Needs to accept required actuators, a_r.
The figure sensor wavefront sensor measurements are reconstructed to
give measured actuators, a_m.  
The currently applied actuators are a_c.
So, the new values to be sent to the DM are 
a_new = a_c + a_r - a_m.
This can be achieved by running the figure sensor with
reconMode=="truth".
The gain should be set to +1 (try it - not sure if this will cause it
to blow up...).  The + bit is so that using -a_m rather than +a_m.
"addActuators" should be set and a_r should be placed into "actuators"
every time new values are available.

Centroid linearity calibration:
Need:
calbounds[2,nsubaps,2],   (x/y, subap, low/hi)
calData[2,nsubaps,nsteps] (x/y, subap, step)
calsteps[2,nsubaps,nsteps] (x/y, subap, step)
About 5MB for phase C if use 1000 steps.  5kB per step.
If the centroid measurement is lower then
calsteps[i,j,calbounds[i,j,0]]
 or higher than
calsteps[i,j,calbounds[i,j,1]]
then the centroid measurement itself should be used.  This means that
caldata should be scaled appropriately (i.e. equivalent at the boundaries)...

Testing functionality for CANARY:
yorick -i canary.i 
test_all(replay=1)

NUMA awareness options:
1. Each numa node has full copy of parameter buffers.
	- requires more memory
	- lots of copying
2. Main parameter buffer plus smaller buffers on each numa node
	- potentially more complex code
	- pointed to from the main param buffer (so that all modules
	  have access to it)
	- a buffer swap would also swap the numa buffers.
	- gives user more control: can copy slowly to the numa
	  buffers.
	- lets go for this option!
	- and a customised reconmvm module.
3. Main buffer, and modules copy relevant parts to their own memory.
	- potentially more complex code
	- more jitter during buffer swap while copying is done.