-
Notifications
You must be signed in to change notification settings - Fork 43
/
README.file
366 lines (268 loc) · 13.4 KB
/
README.file
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
File services
================================================================================
Hui Cao
##########################################
# THIS CODE IS STILL EXPERIMENTAL!
# DO NOT USE IN PRODUCTION ENVIRONMENTS.
# Please send any issues to the Snort team
##########################################
Overview
================================================================================
With the volume of malwares transferred through network increasing, network file
inspection becomes more and more important. This feature will provide file type
identification, file signature creation, and file capture capabilities to help
user deal with those challenges.
There are two parts of file services: file APIs and file inspect preprocessor.
File APIs provides all the file inspection functionalities, such as file type
identification, file signature calculation, and file capture.
File inspect preprocessor provides users ability to control file services, such
as enable/disable/configure file type identification, file signature, or file
capture.
Supported protocols: HTTP, SMTP, IMAP, POP3, FTP, and SMB.
Supported file signature calculation: SHA256
Sections:
Dependency Requirements
File Services Configuration
File Inspect Preprocessor Configuration
File Type Identification
File Signature
File Capture
Dependency Requirements
================================================================================
For proper functioning of the file processing:
Stream session tracking must be enabled, i.e. stream5. TCP must be
enabled in stream5. The preprocessor requires a session tracker to keep
its data.
IP defragmentation should be enabled, i.e. the frag3 preprocessor should be
enabled and configured.
At lease one of preprocessors supported protocols must be enabled.
File Services Configuration
================================================================================
File services are configured through snort "conf" keyword.
The configuration name is "file":
config file: <options>
Option Argument Required Default
file_type_depth <depth> No 1460 bytes
file_signature_depth <depth> No 10485760 bytes
file_block_timeout <timeout> No 1 day
file_capture_memcap <memcap> No 100 megabytes
file_capture_max <max> No 1048576 bytes
file_capture_min <min> No 0 bytes
file_capture_block_size <size> No 32768 bytes
depth = 0 - 4G bytes
timeout = 0 - 4G seconds
memcap = 1 - 4G megabytes
max = 0 - 4G bytes
min = 0 - 4G bytes
size = 8 - 4G bytes
config file:\
< file_type_depth depth >,\
< file_signature_depth depth >, \
< file_block_timeout timeout >, \
< file_capture_memcap memcap >, \
< file_capture_max max >, \
< file_capture_min min >, \
< file_capture_block_size size >
Options:
< file_type_depth depth >:
Maximum file depth to identify file type. If file type is not identified
beyond this value, file type will be set "unknown". 0 means unlimited
< file_signature_depth depth >:
Maximum file depth to calculate file signature. If file size is larger
than this value, file signature will not be calculated. 0 means unlimited
< file_block_timeout timeout >:
When a file transferred through HTTP is blocked , browser might resume
file transfer with partial content in HTTP protocol. Snort block file
even if it is transferred through resume. This timeout is to set how long
it will keep blocking that file.
< file_capture_memcap memcap >:
This sets the memory limit for file buffers, in megabytes.
< file_capture_max max >:
Maximum file size we can capture. If file size is greater than this value
file will not be captured.
< file_capture_min min >:
Minimum file size we can capture. If file size is smaller than this value
file will not be captured.
< file_capture_block_size size >:
This sets the memory block size for file buffer. Smaller value might save
memory but may hurt performance. Default is 32k bytes.
Example:
config file: file_type_depth 16384, file_signature_depth 10485760, \
file_block_timeout 3600, file_capture_memcap 200, \
file_capture_max 1048576, file_capture_min 200, \
file_capture_block_size 65536,
This set the file type depth 16K bytes, file signature depth 10M bytes,
file block timeout 1 hour, file capture memcap 200M bytes, maximum file size
to capture is 1M file, minimum file size to capture is 200 bytes, file block
size is 64k.
File Inspect Configuration
================================================================================
File inspect dynamic preprocessor can be enabled during build time. The
following options must be included in ./configure:
--enable-file-inspect
The configuration name is "file_inspect":
Option Argument Required Default
type_id NONE No DISABLED
signature NONE No DISABLED
capture_memory NONE No DISABLED
capture_disk <dir> <disk size> No DISABLED
capture_network <hostname portno> No DISABLED
capture_queue_size <size> No 3000
blacklist <file> No NULL
greylist <file> No NULL
portno = 0 - 65535
size = 0 - 4G files
disk size = 0 - 4G megabytes
preprocessor file_inspect:\
< type_id >,\
< signature >, \
< capture_memory >, \
< capture_disk dir <disk size> >, \
< capture_network hostname portno >, \
< capture_queue_size size >, \
< blacklist file>, \
< greylist file>
Options:
< type_id >:
Enable file type identification
< signature >:
Enable file signature calculation
< capture_memory >:
Capture file to memory (file buffers). File analysis can be done on file
buffers directly without the overhead of storing to disk.
< capture_disk dir <disk size> >:
Store files to the directory specified in dir, and capture no more than
disk size (in megabytes). If this limit is reached, no more file capture.
< capture_network hostname portno >:
Send files to other host, specified by hostname (or IP address) and
port number.
< capture_queue_size size >:
Set the maximum number of files can be queued to process (saved to disk
or sent to network).
< blacklist file >:
Specify a SHA256 list file. Files match SHA256 will be blocked.
< greylist file >:
Specify a SHA256 list file. Files match SHA256 will be alerted.
SHA list file format
one SHA per line.
Use # for comments.
#start of block.txt
BDC68D4C856558E618BB5C992A2A6B0276D4809F46A8C44E4D4993DDC31E91B2
69DABBDB754B358AC4FE4B22DE04C0E4C93076816F14BB0730CAA9FD223996FC
#end of block.txt
Example:
preprocessor file_inspect: type_id, signature, \
capture_disk /home/file_capture/tmp/ 300, \
capture_network localhost 8000, \
capture_queue_size 5000, \
blacklist block.txt
File Type Identification
================================================================================
File magic rule:
file type: <name>; id: <number>; ver: <version>; category: <name>; \
group: <group>[,<group>]; msg: <message>; \
content: |<hex values>|; offset:<number>; rev: <number>
type: defines the name of file type. Name should be limited to any
alphanumeric string including periods, dashes, and underscores.
id: used to uniquely identify different file types, similarly to rule ID.
ver: used to indicate different versions for the same file type.
category: defines the categories of file type. Name should be limited to
any alphanumeric string including periods, dashes, and underscores.
Categories can be Executables, PDF files, FLASH files, Office Documents,
Archive, Graphics, Multimedia etc.
group: logical grouping of file rules, used for IPS rule keyword `file_group`.
See README.file_ips for more information on 'file_group'.
msg: the message to print when file type information needs to be described.
content: defines the file magic using binary data. The binary data is
enclosed within the pipe (|) character and represented as bytecode,
i.e hexadecimal numbers.
offset: defines the location of file magic relative to the start of file.
rev: used to uniquely identify revisions of file type definition.
Similar to snort rule, it can be used through include directives. Similar file
types can be put into the same file. Multiple file type configurations can be
included in snort configuration.
Pre-packaged file magic rules:
A set of file magic rules is packaged with Snort. They can be located at
"etc/file_magic.conf". To use this feature, it is recommended that the
these pre-packaged rules are used; doing so requires that you include
the file in your Snort configuration as such:
include etc/filemagic.conf
Example:
file type:GIF; ver:87; category:graphics; msg:"GIF87a"; id: 1;
content: |47 49 46 38 37 61|; offset:0; rev:1;
file type:GIF; ver:89; category:graphics; msg:"GIF89a"; id: 1;
content: |47 49 46 38 39 61|; offset:0; rev:1;
The previous two rules define GIF format, because file magics are different.
Full set of file type magic rules are not included. Future release will include
file type magic and rule options to use file type information.
Enable file type
File type events are generated when file type preprocessor rules are enabled.
Each file magic rule requires one file type preprocessor rule. File type use
146 as generator ID.
Example:
alert ( msg: "GIF87a file "; sid: 87; gid: 146; rev: 1;
metadata: rule-type preproc; )
alert ( msg: "GIF89a file "; sid: 89; gid: 146; rev: 1;
metadata: rule-type preproc; )
preprocessor file_inspect: type_id
File Signature Calculation
================================================================================
File signature events are generated when file signature preprocessor rules are
enabled. File signature use 147 as generator ID.
Example:
alert ( msg: "File signature "; sid: 1; gid: 147; rev: 1;
metadata: rule-type preproc; )
Enable Openssl support
File services provide its own SHA calculations, but users can optionally enable
openssl implementation. Openssl might provide performance optimization based on
the platform snort runs. To enable it, the following options should be included
in the ./configure command.
--with-openssl-includes= <path to openssl include>
--with-openssl-libraries= <path to openssl lib>
Example:
--with-openssl-includes=/usr/local/ssl/include
--with-openssl-libraries=/usr/local/ssl/lib
File Capture
================================================================================
File capture relies on file type and file signature enabled. If file type is
unknown, that file will not be captured. File signature is used as file ID to
avoid downloading duplicated files.
Example:
preprocessor file_inspect: type_id, signature, \
capture_disk /home/file_capture/tmp/, \
capture_queue_size 5000
All files captured will be saved to directory "tmp". File capture queue is
set to 5000 files.
Sending captured files to another host
File inspect preprocessor provides a simple way to sending files to another
host for further analysis. In tools/ directory, file_server is provided to
receive and save all the files to disk.
Example:
preprocessor file_inspect: type_id, signature, \
capture_network 192.168.1.1 8000, \
capture_queue_size 5000
All files captured will be sent to host 192.168.1.1, to port number 8000
On the host 192.168.1.1, you can start file_server before starting snort:
file_server 8000
All the files will be saved to the directory where the command is running.
File Events
================================================================================
File inspect preprocessor also works as a dynamic output plugins for file
events. It logs basic information about file.
Format
output filelog: [<filename> [<limit>]]
<limit> ::= <number>[(’G’|’M’|K’)]
filename: the name of the log file. The default name is <logdir>/file.
You may specify ”stdout” for terminal output. The name may include an
absolute or relative path.
limit: an optional limit on file size which defaults to 128 MB.
The minimum is 1 KB.
Example:
dynamicoutput file / path to /libsf_file_preproc.so
output filelog:file
All file events will be logged in "file" in the log directory.
File event example:
07/02-16:42:50.253157 [**] [File: /wp-content/themes/images/logo.jpg,
size: 4633 bytes] [signature:
8D68FA32E67624D409247FEF10D32A413D55C125970AFA533B9CA5E7DDF07A5E] [**]
[147:1:1] [**] {TCP} 108.161.189.192:80 -> 10.4.33.54:40736