PiCO QL is a C/C++ plug-in library that sets up an SQL SELECT web-accessible online interactive interface over C/C++ data structures. SQL queries access the application's data structures in place.
This branch contains releases of the generic PiCO_QL plugin for C/C++ applications. It is not guaranteed to work in the kernel or Valgrind contexts.
For C++ applications, PiCO QL supports querying of containers compatible with the C++ SGI forward container and forward iterator concept. The container classes of the STL consist a prime example of such. PiCO QL provides concept checks to verify that containers follow the above concepts. It also supports querying of custom containers (using the USING LOOP directive) and mere objects. See the PiCO QL for C/C++ applications tutorial.
For C applications, PiCO QL supports querying of data structures like arrays and linked lists (see examples/CApp) but also any other data structure for which users will provide a loop variant (using the USING LOOP directive). It also supports querying of mere structs.
PiCO QL is also configurable as a loadable Linux kernel module. It provides both a /proc and a high-level web interface to be able to issue SQL queries against kernel data structures. Currently PiCO QL supports data structures modelling processes (such as the linked list of processes) , files, virtual memory, file system, network, namespaces, devices, busses etc. Support is extensible to include more data structures. See Fast track roll your own probes tutorial in the PiCO QL LKM wiki.
PiCO QL is also configurable as an extension to Valgrind tools, that is MemCheck, CacheGrind, and CallGrind. It provides an online interactive web interface to the data structures employed by Valgrind tools that gather memory operations metadata from an application that Valgrind instruments at the time. In other words, while Valgrind tools gather metadata about how an application is using memory, at the same time users can query the metadata using SQL queries. Synchronization issues are skipped using a simple stop-the-clock implementation to poll for a query waiting to be executed.
Userspace (built-in examples):
Tested under Mac OS X 10.6.7-8, 10.8.5, 10.10.5
Linux 2.6
MSVS 2010 on Win64
Windows 10 via Cygwin (under way)
Linux kernel module:
Tested on:
Architectures: x86_64, amd64
Distributions - kernels: Fedora 18 - 3.6.9,
Debian 7.1 - 3.2.0,
Debian 8 - 3.16.7,
Ubuntu server 10.04 - 2.6.32
Valgrind extension:
Tested under Mac OS X 10.6.7-8, 10.8.5
- Userspace: Commits are verified against:
- test cases by executing a script.
- Valgrind (installed in Debian GNU/Linux 6.0.7 - kernel 2.6.32).
- Kernel: Commits are verified against an array of SQL queries by executing a script.
- Ruby Part of the library's source code (the parser/code generator) is written in Ruby.
- SQLite3 While PiCO QL does not use a database or other store to copy and/or store data, it uses the virtual table (VT) mechanism of Sqlite3 to provide a relational representation of the underlying data model. The PiCO QL source code includes SQLite, so it does not have to be installed in your system.
- SWILL Once plugged in an application, PiCO QL uses the same or a separate thread and utilises the SWILL library to present a local server with a web-like, user friendly query interface. In case of the kernel configuration this interface wraps the /proc interface.
- Boost For C++ the library accepts any container iterable in some fashion and provides concept checks (compile-time), implemented by Boost, for containers modelled after the ForwardContainer and ForwardIterator concepts
autoreconf --install (only the first time) ./configure [--enable-run-tests=yes] [--prefix=...] make make install make test # if --enable-run-tests has been used
To see the PiCO QL library in action:
- do not use --enable-run-tests
- head to the installation directory
- run one of the example executables: bankapp, chess, capp, vrp, polymoprhism
make CFLAGS='-Dcygwin_conv_to_full_win32_path\(path,fullpath\)=cygwin_conv_path\(CCP_WIN_A_TO_POSIX, path, fullpath, PATH_MAX\)'
int init(const char** pragmas, int npragmas, int port_number, pthread_t *t); void register_data(const void *collection, const char * col_name); int exec_query(const char *query, stringstream &s, int (*callback)(sqlite3 *, sqlite3_stmt *, stringstream &)); int step_mute(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s); int step_text(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s); int step_swill_html(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s); int step_swill_json(sqlite3 *db, sqlite3_stmt *stmt, stringstream &s); int progress(int n, int (*callback)(void *p), void *p); int interrupt(); int shutdown(); int create_function(const char *name, int argc, int text_rep, void *p, void (*xFunc)(sqlite3_context*,int,sqlite3_value**), void (*xStep)(sqlite3_context*,int,sqlite3_value**), void (*xFinal)(sqlite3_context*));
See src/pico_ql.h for the full C/C++ API
- At the examples/ directory, users can browse:
Behind the scenes a compiler written in Ruby parses a relational specification and creates the source code of the relational interface that is later used to execute queries on the application's data structures. Here is how the compiler is invoked and the arguments it accepts:
ruby generator.rb pico_ql_dsl.sql [debug] [kernel] [C] [no_mem_mgt] [concept_check]
There are also different configuration alternatives available at the make step configuration options.
- At the doc/ directory there is the BNF of the DSL that grounds the user description syntax and the wiki pages.
- At src/ directory, the library's source code is found.