-
Notifications
You must be signed in to change notification settings - Fork 273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
moves to centralized and uniform path handling, removes dead code #1278
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ivg
force-pushed
the
straighten-folders
branch
from
February 26, 2021 14:44
35baf64
to
0205fba
Compare
Introduction ============ This PR changes how and where we store configuration, read-only data, and caches. Instead of the previous approach, where each plugin was receiving its own location via the configure script, we now rely on the central location from `Extension.Configuration` module of the bap-main library. In addition, we introduce user-specific locations for configuration files and read-only data that. The user-specific files, unlike the system-wide, are not overwritten during bap installation. Which is especially useful for developing lifters in Primus Lisp (it is a very bad UX when your carefully written lifter is overwritten or removed on `opam update`) and for configuring bap for your own taste and feel (e.g., if you're getting annoyed with at&t syntax and want the intel's one and don't want to update the configuration files on each update). Configuration Files and Locations ================================= The configuration files are stored in two folders, the system-wide, which is `$prefix/etc/bap/` hosts the site and package specific configuration. The user-specific location follows the XDG Base Directory Specification and is either `$XDG_CONFIG_HOME/bap` or, if `$XDG_CONFIG_HOME` is not set, to $HOME/.config/bap. Both locactions are folders that hold an arbitrary number of configuration files. The files are read and applied in alphabetical order with the first read file having precedence in case if they define values for the same parameter. The syntax is the common ini-style syntax, with comments starting with `#`. Each line shall have form `<parameter> = <value>`, where a parameter matches with the name of a BAP command-line parameter with leading `--` removed, e.g., suppose we want to switch to the `intel` syntax, instead of `att` for x86 disassembler. The corresponding command-line parameter is `--llvm-x86-syntax`. We create a file in the `~/.config/bap` folder with the following contents, ``` llvm-x86-syntax = intel ``` The filename could be anything and you can create a separate file per each feature, or just have one big file. The configuration system is also integrated with the cache subsystem, so changes to configuration will be automatically detected and reflected in the cache digests. Read-only Data ============== The read-only data includes various specifications and signatures, like lisp semantics definitions, stubs, policy specifications, api files, byteweight and other signatures. We also store them in two locations, the system-wide, `$prefix/share/bap/`, where the data shipped with bap is installed. And user-specific `$XDG_DATA_HOME` (defaults to `$HOME/.local/share/bap`), which is shared by all switches and is never touched during bap installation. Locations of the Lisp Files --------------------------- The Primus Lisp library is now stored in three places: - `$prefix/share/bap/primus/lisp` - `$prefix/share/bap/primus/site-lisp` - `$XDG_DATA_HOME/bap/primus/lisp` (where `$XDG_DATA_HOME` defaults to `$HOME/.local/share/`). The `$prefix/share/bap/primus/lisp` stores the standard library and language-specific definitions of Primus Lisp. It shall not be touched by users. The `$prefix/share/bap/primus/site-lisp` folder is the folder where libraries should be installed, both provided by bap and by other packages. This approach follows conventions in Lisp and Python (and other script languages) commmunities. Finally, the location for local, user-specific, lisp libraries is `$XDG_DATA_HOME/bap/primus/lisp`. This is where you can store your own lisp codes and common libraries and expect that they will never be touched by bap installation scripts. And do not forget that the current folder is also in the search path, as well as that you can add more paths using `--primus-lisp-add` to add more folders to the search path. Together with the configurations files, it is now easy to handle those paths, e.g., by adding a `lisp-paths.conf` file to `~/.config/bap` with the following contents. ```conf primus-lisp-add = /home/user/share/primus-lisp ``` The Cache Location ================== The cache location wasn't really changed but is made more accessible for those who would like to store their own bap-specific non-essential runtime files. Just use `Extension.Configuration.cachedir` for that or, even better, use the Cache facility of the `regular` library. Breaking Changes ================ No interfaces were broken, but some files changed their location. Everything is now stored in the bap folder, and is more organized. For example, bap-api is now stored in bap/api folder, and byteweight and stubs signatures are stored in `bap/signatures`. Also, byteweight signatures are renamed to `byteweight.zip` from the old `sigs.zip`. The update should pass seamless as it is all handled by the configuation and installation script. Which will be updated separately.
ivg
force-pushed
the
straighten-folders
branch
from
February 26, 2021 17:17
0205fba
to
254ce7e
Compare
ivg
added a commit
to BinaryAnalysisPlatform/opam-repository
that referenced
this pull request
Feb 26, 2021
Syncronizes with BinaryAnalysisPlatform/bap#1278 In fact, the only necessary change was the bap-signatures update, everything else is more or less unnecessary as opam should correctly remove installed data files. We should probably remove the `remove` field altogether.
This was referenced Apr 2, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduction
This PR changes how and where we store configuration, read-only data,
and caches. Instead of the previous approach, where each plugin was
receiving its own location via the configure script, we now rely on
the central location from
Extension.Configuration
module of thebap-main library.
In addition, we introduce user-specific locations for configuration
files and read-only data that. The user-specific files, unlike the
system-wide, are not overwritten during bap installation. Which is
especially useful for developing lifters in Primus Lisp (it is a very
bad UX when your carefully written lifter is overwritten or removed on
opam update
) and for configuring bap for your own taste andfeel (e.g., if you're getting annoyed with at&t syntax and want the
intel's one and don't want to update the configuration files on each
update).
Configuration Files and Locations
The configuration files are stored in two folders, the system-wide,
which is
$prefix/etc/bap/
hosts the site and package specificconfiguration. The user-specific location follows the XDG Base
Directory Specification and is either
$XDG_CONFIG_HOME/bap
or,if
$XDG_CONFIG_HOME
is not set, to $HOME/.config/bap.Both locactions are folders that hold an arbitrary number of
configuration files. The files are read and applied in alphabetical
order with the first read file having precedence in case if they
define values for the same parameter. The syntax is the common
ini-style syntax, with comments starting with
#
. Each line shallhave form
<parameter> = <value>
, where a parameter matches with thename of a BAP command-line parameter with leading
--
removed, e.g.,suppose we want to switch to the
intel
syntax, instead ofatt
forx86 disassembler. The corresponding command-line parameter is
--llvm-x86-syntax
. We create a file in the~/.config/bap
folderwith the following contents,
The filename could be anything and you can create a separate file per
each feature, or just have one big file. The configuration system is
also integrated with the cache subsystem, so changes to configuration
will be automatically detected and reflected in the cache digests.
Read-only Data
The read-only data includes various specifications and signatures,
like lisp semantics definitions, stubs, policy specifications, api
files, byteweight and other signatures.
We also store them in two locations, the system-wide,
$prefix/share/bap/
, where the data shipped with bap isinstalled. And user-specific
$XDG_DATA_HOME
(defaults to$HOME/.local/share/bap
), which is shared by all switches and isnever touched during bap installation.
Locations of the Lisp Files
The Primus Lisp library is now stored in three places:
$prefix/share/bap/primus/lisp
$prefix/share/bap/primus/site-lisp
$XDG_DATA_HOME/bap/primus/lisp
(where
$XDG_DATA_HOME
defaults to$HOME/.local/share/
).The
$prefix/share/bap/primus/lisp
stores the standard library andlanguage-specific definitions of Primus Lisp. It shall not be touched
by users. The
$prefix/share/bap/primus/site-lisp
folder is thefolder where libraries should be installed, both provided by bap and
by other packages. This approach follows conventions in Lisp and
Python (and other script languages) commmunities.
Finally, the location for local, user-specific, lisp libraries is
$XDG_DATA_HOME/bap/primus/lisp
. This is where you can store your ownlisp codes and common libraries and expect that they will never be
touched by bap installation scripts. And do not forget that the
current folder is also in the search path, as well as that you can add
more paths using
--primus-lisp-add
to add more folders to the searchpath. Together with the configurations files, it is now easy to handle
those paths, e.g., by adding a
lisp-paths.conf
file to~/.config/bap
with the following contents.
The Cache Location
The cache location wasn't really changed but is made more accessible
for those who would like to store their own bap-specific non-essential
runtime files. Just use
Extension.Configuration.cachedir
for thator, even better, use the Cache facility of the
regular
library.Breaking Changes
No interfaces were broken, but some files changed their
location. Everything is now stored in the bap folder, and is more
organized. For example, bap-api is now stored in bap/api folder, and
byteweight and stubs signatures are stored in
bap/signatures
. Also,byteweight signatures are renamed to
byteweight.zip
from the oldsigs.zip
. The update should pass seamless as it is all handled bythe configuation and installation script. Which will be updated
separately.