forked from Feh/nocache
-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
161 lines (122 loc) · 6.25 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
nocache - minimize filesystem caching effects
---------------------------------------------
The `nocache` tool tries to minimize the effect an application has on
the Linux file system cache. This is done by intercepting the `open`
and `close` system calls and calling `posix_fadvise` with the
`POSIX_FADV_DONTNEED` parameter. Because the library remembers which
pages (ie., 4K-blocks of the file) were already in file system cache
when the file was opened, these will not be marked as "don't need",
because other applications might need that, although they are not
actively used (think: hot standby).
Use case: backup processes that should not interfere with the present
state of the cache.
Installation and Usage
----------------------
Just type `make`. Then, prepend `./nocache` to your command:
./nocache cp -a ~/ /mnt/backup/home-$(hostname)
You can select (filter, include, exclude) files which you want to cache
based on their pathnames. The cache is disabled for a file iff (`I` and not `E`)
where,
* if the environment variable `NOCACHE_PATTERN_INCLUDE` is defined,
`I` iff the pathname of the file satisfies
the POSIX Extended Regular Expression in `NOCACHE_PATTERN_INCLUDE`,
* if `NOCACHE_PATTERN_INCLUDE` is undefined, `I` is `true`,
* if the environment variable `NOCACHE_PATTERN_EXCLUDE` is defined,
`E` iff the pathname of the file satisfies
the POSIX Extended Regular Expression in `NOCACHE_PATTERN_EXCLUDE`,
* if `NOCACHE_PATTERN_EXCLUDE` is undefined, `E` is `false`.
The command `make install` will install the shared library, man
pages and the `nocache`, `cachestats` and `cachedel` commands
under `/usr/local`. You can specify an alternate prefix by using
`make install PREFIX=/usr`.
Debian packages are available, see http://packages.qa.debian.org/n/nocache.html.
Please note that `nocache` will only build on a system that has
support for the `posix_fadvise` syscall and exposes it, too. This
should be the case on most modern Unices, but kfreebsd notably has no
support for this as of now.
Testing
-------
For testing purposes, I included two small tools:
* `cachedel` calls `posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED)` on
the file argument. Thus, if the file is not accessed by any other
application, the pages will be eradicated from the fs cache.
Specifying -n <number> will repeat the syscall the given number of
times which can be useful in some circumstances (see below).
* `cachestats` has three modes: In quiet mode (`-q`), the exit status
is 0 (success) if the file is fully cached. In normal mode,
the number of cached vs. not-cached pages is printed. In verbose
mode (`-v`), an actual map is printed out, where each page that is
present in the cache is marked with `x`.
It looks like this:
$ cachestats -v ~/somefile.mp3
pages in cache: 85/114 (74.6%) [filesize=453.5K, pagesize=4K]
cache map:
0: |x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|
32: |x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|
64: |x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x|x| | | | | | | | | | | | |
96: | | | | | | | | | | | | | | | | | |x|
Also, you can use `vmstat 1` to view cache statistics.
For debugging purposes, you can specify a filename that `nocache` should log
debugging messages to via the `-D` command line switch, e.g. use `nocache
-D /tmp/nocache.log …`. Note that for simple testing the file `/dev/stderr`
might be a good choice.
Example run
-----------
Without `nocache`, the file will be fully cached when you copy it
somewhere:
$ ./cachestats ~/file.mp3
pages in cache: 154/1945 (7.9%) [filesize=7776.2K, pagesize=4K]
$ cp ~/file.mp3 /tmp
$ ./cachestats ~/file.mp3
pages in cache: 1945/1945 (100.0%) [filesize=7776.2K, pagesize=4K]
With `nocache`, the original caching state will be preserved.
$ ./cachestats ~/file.mp3
pages in cache: 154/1945 (7.9%) [filesize=7776.2K, pagesize=4K]
$ ./nocache cp ~/file.mp3 /tmp
$ ./cachestats ~/file.mp3
pages in cache: 154/1945 (7.9%) [filesize=7776.2K, pagesize=4K]
Limitations
-----------
The pre-loaded library tries really hard to catch all system calls
that open or close a file. This happens by "hijacking" the libc
functions that wrap the actual system calls. In some cases, this may
fail, for example because the application does some clever wrapping.
(That is the reason why `__openat_2` is defined: GNU `tar` uses this
instead of a regular `openat`.)
However, since the actual `fadvise` calls are performed right before
the file descriptor is closed, this may not happen if they are left
open when the application exits, although the destructor tries to do
that.
There are timing issues to consider, as well. If you consider `nocache
cat <file>`, in most (all?) cases the cache will not be restored. For
discussion and possible solutions see <http://lwn.net/Articles/480930/>.
My experience showed that in many cases you could "fix" this by doing
the `posix_fadvise` call *twice*. For both tools `nocache` and
`cachedel` you can specify the number using `-n`, like so:
$ nocache -n 2 cat ~/file.mp3
This actually only sets the environment variable `NOCACHE_NR_FADVISE`
to the specified value, and the shared library reads out this value.
If test number 3 in `t/basic.t` fails, then try increasing this number
until it works, e.g.:
$ env NOCACHE_NR_FADVISE=2 make test
Alternate Approaches
--------------------
If you have a fairly recent Linux kernel, you can also try to simply
cage a cache-intensive application in a memory-limited cgroup. For
example, I use the following command in the startup part of my backup
script:
sudo env ppid=$$ sh -c '
cgcreate -g memory:backup ;
cgcreate -g blkio:backup ;
echo 256M > /sys/fs/cgroup/memory/backup/memory.limit_in_bytes ;
echo 100 > /sys/fs/cgroup/blkio/backup/blkio.weight ;
echo $ppid > /sys/fs/cgroup/memory/backup/tasks ;
echo $ppid > /sys/fs/cgroup/blkio/backup/tasks ;
'
More info: https://www.kernel.org/doc/Documentation/cgroups/memory.txt
Acknowledgements
----------------
Most of the application logic is from Tobias Oetiker's patch for
`rsync` <http://insights.oetiker.ch/linux/fadvise.html>. Note however,
that `rsync` uses sockets, so if you try a `nocache rsync`, only
the local process will be intercepted.