forked from zeek/zeekctl
-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathCHANGES
2593 lines (1589 loc) · 87.3 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
2.5.0-19 | 2023-03-22 13:32:10 -0700
* Make private address space locality configurable (Christian Kreibich, Corelight)
2.5.0-17 | 2023-03-20 10:02:20 -0700
* Update baseline for command.peerstatus test (Tim Wojtulewicz)
* Update test baseline for changes to Site::local_nets (Tim Wojtulewicz)
This required adding a random seed and setting the environment
variable for Zeek during testing. Otherwise the set prints out
in a different order every time and breaks the determinism
of the test.
2.5.0-14 | 2023-03-01 10:17:36 +0100
* build-zeek: Recognize ZEEK_CI_CPUS (Arne Welzel, Corelight)
nproc on Cirrus CI gives 32 even if we only allocated 4 CPUs and
building Spicy with -j32, instant OOM.
Also, ditch a bit more Travis references.
* GH-45: testing: Remove pf_ring round-robin usage in tests (Arne Welzel, Corelight)
In #45, the round-robin load balancing method for pf_ring was
removed, but there was a test using it. Fix it up.
* GH-309: Update peerstatus baselines (Arne Welzel, Corelight)
This has been reported as a regression via zeek/broker#309, but for
the time being updated it to the new world order.
* build-zeek: Recognize Cirrus CI, not Travis (Arne Welzel, Corelight)
2.5.0-9 | 2023-02-24 18:59:07 +0100
* Add a new ZeekPortWarning plugin (Arne Welzel, Corelight)
This was discussed on Slack:
1) The issue is pressing enough to actively warn users about it when
starting zeekctl.
2) We should prepare users for the change in default coming with
Zeek 5.2. We're a bit late here, but still reasonable for 5.0
to 6.0 upgrades.
This change should be included into a Zeek 5.0.x maintenance release.
2.5.0-5 | 2023-02-24 09:23:35 +0100
* Support lb_method af_packet (Arne Welzel, Corelight)
* GH-2792: plugins: Import af_packet.py (Arne Welzel, Corelight)
This is an import of af_packet.py from zeek/zeek-af_packet-plugin at
revision b8c17c898bedfe020056027036f5a7eabc815c92. However, tabs have
been replaced with spaces.
Further, we're importing this as zzz_af_packet.py to have it be
loaded and initialized after lb_custom.py on which it depends.
Related to zeek/zeek#2792.
* pluginreg: Sort py files before import (Arne Welzel, Corelight)
Make import order of plugins predictable so prefixing of filenames
with zzz or aaa can be used for basic ordering.
2.5.0 | 2023-02-01 15:47:52 -0700
* Release 2.5.0 (Tim Wojtulewicz, Corelight)
2.4.1-15 | 2023-01-23 09:13:34 +0000
* Remove the broctl symlink. (Johanna Amann, Corelight)
2.4.1-12 | 2022-12-02 18:05:32 -0800
* lb_pf_ring: Drop round-robin, make error message say "not supported" (Arne Welzel, Corelight)
* style: replace simple quotes with double quotes for consistency (V)
* pf_ring: add new 'inner' load balancing strategies for better balancing of tunneled sessions (V)
* Add CodeQL workflow (sylwia-budzynska)
2.4.1-4 | 2022-10-07 09:16:09 -0700
* Move ZeekPort out of Linux's ephemeral port range (47760 to 27760) (Arne Welzel, Corelight)
WARNING: This breaks users that have setup strict firewalls between Zeek
nodes, but at the same time fixes spurious worker failures.
On Linux, port 47760 and the following ports selected by zeekctl fall square
into the ephemeral port range. This has resulted in multiple users reporting
Zeek workers spuriously failing to start with messages as follows:
error in /usr/local/zeek-5.0.0/share/zeek/base/frameworks/cluster/./setup-connections.zeek, lines 94-96: Failed to listen on INADDR_ANY:47764 (Broker::listen(Broker::default_listen_address, Cluster::self$p, Broker::default_listen_retry))
fatal error: errors occurred while initializing
This can happen when another process on the system are using a port that
a Zeek process is supposed to listen on. They are free to do so, these
ports are in the default ephemeral port range. Even the outgoing connection
from the same or another worker to the manager or logger has been
observed to cause this.
FreeBSD users have not seen this previously, as its ephemeral port range
is above 47760 (49152), but on Linux it starts as low as 32768.
* Keep make dist from deleting all paths containing 'build' [skip ci] (Tim Wojtulewicz, Corelight)
2.4.1 | 2022-06-01 09:30:19 -0700
* Release 2.4.1
2.4.0-5 | 2022-04-08 11:26:28 -0700
* Update cmake submodule to pull in InstallSymlink fix (Christian Kreibich, Corelight)
2.4.0-3 | 2022-01-27 14:53:12 -0700
* Have `make dist` cleanup a few more wayward files before tarring (Tim Wojtulewicz, Corelight)
* Update cmake submodule to latest master (Tim Wojtulewicz, Corelight)
2.3.0-5 | 2021-06-15 11:33:52 -0700
* GH-32: Add builtin-plugins to ZEEKPATH in set-zeek-path (Tim Wojtulewicz, Corelight)
2.3.0 | 2020-12-14 21:02:33 -0800
* Release 2.3.0
2.2.0-27 | 2020-12-12 20:20:43 -0800
* Install zeekctl into Zeek's common Python library subdirectory (Christian Kreibich, Corelight)
- This removes the unused --python-install-dir option from the
configure help output, and adds --python-home and --python-prefix to
allow specifying custom Python installation folders, as done in the
Broker package. Bundled installation with Zeek inherits
PY_MOD_INSTALL_DIR, as the rest of the tree.
- The testsuite no longer hardwires assumptions about the location of
the Python module folder, and instead relies on "zeek-config
--python_dir" to obtain it. This required some rewiring of the
per-test string substitution logic. Cross fingers.
- Switches cmake's deprecated "remove_directory" command to "rm".
- Bumps trace-summary submodule to make it find Python modules in the
Zeek distribution's installation directory.
2.2.0-25 | 2020-12-10 14:11:47 -0800
* Update Broker Python binding usages to new API (Jon Siwek, Corelight)
Without properly use of context-management or explicit reset() calls,
the destruction order of subscriber objects can cause
heap-use-after-free crashes.
2.2.0-24 | 2020-12-10 15:45:03 +0000
* Baseline refresh to reflect btest 0.64 (Christian Kreibich, Corelight)
2.2.0-21 | 2020-12-07 15:06:31 -0800
* Update CMake logic to prefer Python 3 over Python 2 (Jon Siwek, Corelight)
2.2.0-17 | 2020-12-02 11:10:51 -0800
* Update minimum required CMake to 3.5 (Jon Siwek, Corelight)
2.2.0-15 | 2020-11-26 18:06:12 +0000
* Remove an empty/useless 'btest' file (Jon Siwek, Corelight)
* Remove CI testing of older Python versions and add newer versions (Jon Siwek, Corelight)
* Remove Python 2 compatibility logic from all Python scripts (Jon Siwek, Corelight)
* Update Python invocations to use explicit `python3` (Jon Siwek, Corelight)
* Update CMake logic to enforce Python >= 3.5 (Jon Siwek, Corelight)
* Update docs to reflect new Python 3.5 minimum requirement (Jon Siwek, Corelight)
* Update submodules for changes related to Python 2 EOL (Jon Siwek, Corelight)
2.2.0-6 | 2020-11-24 15:16:37 -0800
* Rely on GNUInstallDirs for definition of libdir and adopt it for installation (Christian Kreibich, Corelight)
2.2.0 | 2020-07-27 11:14:20 -0700
* Release 2.2.0
2.1.0-25 | 2020-07-21 12:55:47 -0700
* Update a test baseline for new Broker::table_store_db_directory (Jon Siwek, Corelight)
2.1.0-24 | 2020-07-21 14:45:35 +0000
* Add new "BrokerDBDir" configuration option, which sets the
location in which Zeek tables that are backed by Broker stores are
persisted. (Johanna Amann, Corelight)
2.1.0-20 | 2020-06-30 11:31:03 -0700
* Fix .travis.yml to use auxil/ instead of aux/ (Jon Siwek, Corelight)
2.1.0-18 | 2020-06-08 11:14:14 -0700
* Rename aux/ to auxil/ (Jon Siwek, Corelight)
Since "aux" is not an allowed file/dir name on Windows.
2.1.0-11 | 2020-03-26 13:33:50 -0700
* Update generated docs (Jon Zeolla)
* Clarify docs and example for multi-logger cluster (Jon Zeolla)
2.1.0 | 2020-02-08 12:32:49 -0800
* Release 2.1.0
2.0.0-39 | 2020-02-04 12:07:18 -0800
* Don't check for sqlite3 python module when cross-compiling (Fabrice Fontaine)
Don't check for sqlite3 python module support by calling
"${PYTHON_EXECUTABLE}" -c "import sqlite3" when cross-compiling as this
will check sqlite3 support on the host python interpreter and not the
target python interpreter.
2.0.0-36 | 2020-01-30 19:11:25 -0800
* No longer need to look for BROCTL_DISABLE_LISTEN. (Robin Sommer, Corelight)
* Error out when old Bro options are used. (Robin Sommer, Corelight)
* Error out when old Bro plugin API used. (Robin Sommer, Corelight)
* Remove 'bro' command from ps plugin. (Robin Sommer, Corelight)
* Abort if there's a broctl.cfg but no zeekctl.cfg. (Robin Sommer, Corelight)
* Abort when using old BroControl plugin API. (Robin Sommer, Corelight)
2.0.0-25 | 2019-11-25 10:21:18 -0800
* Fix "scripts" command in standalone mode (Jon Siwek, Corelight)
Addresses https://github.com/zeek/zeek/issues/697
2.0.0-24 | 2019-11-25 09:21:27 -0800
* Change install.py to use a relative path for the zeekctl-config.sh symlink (Craig Leres)
2.0.0-18 | 2019-10-28 20:14:23 -0700
* Remove Python 3.4 from Travis CI matrix (Jon Siwek, Corelight)
It's end-of-life and not available in Travis "dist: bionic".
2.0.0-17 | 2019-10-28 18:27:37 -0700
* Use Ubuntu 18.04 (Bionic) in Travis CI (Jon Siwek, Corelight)
To satisfy Zeek C++17 requirement
2.0.0-16 | 2019-10-28 18:25:20 -0700
* Move CMake project() after cmake_minimum_required() (Jon Siwek, Corelight)
2.0.0-12 | 2019-10-17 16:30:37 -0700
* Change gzip compression level from 9 to default #614 (JC Connell)
2.0.0-6 | 2019-08-23 06:31:33 -0400
* archive-log: Print a usage string if the number of arguments is incorrect. (Vlad Grigorescu)
2.0.0-3 | 2019-08-13 13:43:34 -0700
* Add CompressLogsInFlight option to compress logs while writing instead of upon rotation (Tim Wojtulewicz, Corelight)
2.0.0 | 2019-08-08 10:51:01 -0700
* Release 2.0.0
1.9-60 | 2019-08-06 11:48:09 -0700
* Simplify check-pid script
Still keeps support for Alpine/BusyBox version of `ps`, which lacks
the -p option, but removes the use of `kill -0`, which transiently
fails for unknown reason: see https://github.com/zeek/zeek/issues/518 (Jon Siwek, Corelight)
1.9-56 | 2019-06-21 09:55:14 -0700
* Fix alpine ps => PID issue (Jeff Barber)
1.9-52 | 2019-06-12 15:08:09 -0700
* Rename directories from bro to zeek (Daniel Thayer)
1.9-49 | 2019-05-23 19:33:47 -0700
* Rename the BROPATH environment variable (Daniel Thayer)
1.9-47 | 2019-05-20 19:37:27 -0700
* More changes for Bro to Zeek renaming (Daniel Thayer)
1.9-45 | 2019-05-15 15:00:39 -0700
* Adjust parallelism of build-zeek script (Jon Siwek, Corelight)
* Update broker.bro module usage to broker.zeek (Jon Siwek, Corelight)
1.9-43 | 2019-05-14 19:29:56 -0700
* Fix plugin.ps test (Jon Siwek, Corelight)
1.9-42 | 2019-05-14 18:19:43 -0700
* Remove the "update" command (Jon Siwek, Corelight)
1.9-41 | 2019-05-14 17:27:44 -0700
* Update Travis config to use zeek/zeekctl (Jon Siwek, Corelight)
* Update README.rst symlink (Jon Siwek, Corelight)
1.9-39 | 2019-05-14 13:12:15 -0700
* Fix legacy plugin API and add new tests (Daniel Thayer)
* Added new test cases and improved a few tests. (Daniel Thayer)
* Fixed one line in the help output to fit within an 80 character display. (Daniel Thayer)
* Some fixes for bro-to-zeek renaming and docs (Daniel Thayer)
* Updating documentation. (Robin Sommer, Corelight)
* Renamed broctl to zeekctl (Robin Sommer, Corelight)
I ended up doing the rename pretty comprehensively across all the
scripts, as it was hard to change some places but not others. So most
uses of Bro are replaced with Zeek now. I tried to maintain backwards
compatibility with the old names where user visible, including names
for options and IDs inside plugins.
Changes to maintain backwards compabibility:
- We now also puts links in place for backwards compability:
bin/broctl -> bin/zeek-wrapper (which then forwards to zeekctl)
lib/broctl -> lib/zeekctl
- If an etc/broctl.cfg exists from a previous install, we symlink
etc/zeekctl.cfg to it to keep any customizations that were made.
- We create a Python wrapper module BroControl that forwards (with a
warning) to the renamed ZeekControl, so that old plugins continue to
work.
- Old option name containing "Bro" are accepted in place of the new
Zeek variants.
- "ps.bro" is an alias for "ps.zeek".
- BROCTL_DISABLE_LISTEN is an alias for ZEEKCTL_DISABLE_LISTEN
1.9-32 | 2019-05-10 19:13:32 -0700
* Add LibDir64 option (Jon Siwek, Corelight)
And make it and LibDir optional dirs for syncing to remote nodes as
well as for use with the 'df' command.
On some platforms, certain libraries in the Zeek-ecosystem now install
into the lib64/ directory by default (per that platform's convention).
* Allow option names that have numbers in them (Jon Siwek, Corelight)
* Silence test failures due to rotate_file_by_name deprecation (Jon Siwek, Corelight)
1.9-28 | 2019-04-19 11:11:53 -0700
* Replace bro_init/bro_done usages with zeek_init/zeek_done (Seth Hall, Corelight)
1.9-24 | 2019-04-16 11:53:06 -0700
* Update some tests and baselines due to new file extension (Daniel Thayer)
* Install script files with new file extension (Daniel Thayer)
* Change file extension of all script files to ".zeek" (Daniel Thayer)
* Fix the update command
This broke due to https://github.com/zeek/zeek/pull/261, in which
errors in initialization are now fatal, but there happened to be
benign/unnoticed errors with the way `broctl update` was working.
Namely, it was incorrectly treating the bro process that it spawned
for using the control framework as a cluster node, by setting the
CLUSTER_NODE environment variable, and that causes an attempt to listen
on a port which, when a cluster is up and running as it should be,
is already listened upon, thus generating an error and completely
failing now due to change in initialization behavior. (Jon Siwek, Corelight)
1.9-11 | 2019-01-04 13:10:54 -0600
* Adding support for log rotation/expiration for distributed loggers. (Stefan Maerz)
1.9-8 | 2018-12-10 15:07:31 -0600
* Change Travis btest commnad to help isolate hung tests (Jon Siwek, Corelight)
1.9-7 | 2018-12-10 13:09:29 -0600
* Update Travis URL for cloning Zeek (Jon Siwek, Corelight)
* GH-11: Improve check-pid helper script for Alpine support (Jon Siwek, Corelight)
1.9-4 | 2018-12-07 16:31:33 -0600
* Update github/download link (Jon Siwek, Corelight)
* Update submodules to use github.com/zeek (Jon Siwek, Corelight)
1.9-2 | 2018-09-26 10:31:47 -0500
* Update broctl.rst (by running "make doc") (Daniel Thayer)
1.9 | 2018-09-18 16:47:56 -0500
* Release 1.9.
1.8-1 | 2018-09-18 16:46:20 -0500
* Fix commands that use broker python bindings (Jon Siwek, Corelight)
1.8 | 2018-09-18 14:26:09 -0500
* Release v1.8.
1.7-126 | 2018-09-07 09:56:19 -0500
* Update the broctl top command to not show the "Proc" column (Daniel Thayer)
* Update the stats-to-csv script for broker and loggers (Daniel Thayer)
* Improve `make dist` (Jon Siwek, Corelight)
1.7-122 | 2018-08-29 23:44:14 +0000
* Include Broker node ID in "control" event topics. This helps break
a message routing loop due to all "control" nodes being subscribed
to a common topic. (Jon Siwek, Corelight)
1.7-119 | 2018-08-21 13:20:35 -0500
* Change default snaplen to 9216 bytes to better accommodate
jumbo frames (Justin Azoff)
1.7-117 | 2018-08-20 14:57:41 -0500
* Change broctl to warn about unrecognized broctl options (Daniel Thayer)
* Remove deprecated node-specific SitePolicy* options (Daniel Thayer)
1.7-107 | 2018-07-16 10:57:58 -0500
* Update docs about the "update" command being deprecated (Daniel Thayer)
* Various unit test additions/improvements/cleanups (Daniel Thayer)
* Update diff-top-output script based on a recent change (Daniel Thayer)
* Reduce the number of node.cfg files used by the tests (Daniel Thayer)
* Reduce the number of broctl.cfg files used by the tests (Daniel Thayer)
* Set BRO_DEFAULT_LISTEN_ADDRESS when running tests (Daniel Thayer)
* The check and scripts tests no longer need to be serialized (Daniel Thayer)
1.7-93 | 2018-06-21 11:54:09 -0500
* Make 'check' and 'scripts' commands skip connection setup (Corelight)
1.7-92 | 2018-06-08 09:55:24 -0500
* Broker-related updates and fixes for the docs (Daniel Thayer)
* Reduce the size of the Bro build and build more quickly when running
broctl tests (Daniel Thayer)
* Travis CI improvements (Daniel Thayer)
* Fix the testing Makefile "cleanup" target to remove all test tmp files.
(Daniel Thayer)
* Fix a few commands to report error when bro is not running (Daniel Thayer)
* Improve error message for import broker failure (Daniel Thayer)
* Remove redundant output from broctl capstats command (Daniel Thayer)
* Output error messages to stderr instead of stdout (Daniel Thayer)
* Improved capstats error message when capstatspath option is not set.
(Daniel Thayer)
* Fixed exit status of capstats command when it doesn't produce any
results. (Daniel Thayer)
* Updated some test baselines. (Daniel Thayer)
1.7-81 | 2018-06-06 14:01:23 -0500
* Listen in standalone mode, but not when processing a trace (Corelight)
1.7-77 | 2018-05-21 17:46:06 +0000
* Port BroControl to use Broker. (Corelight) This includes:
- Add deprecation warning to "update" command. Bro's new
configuration framework supersedes it.
- Make Broker's control topic a configuration option.
- Add 'DefaultStoreDir' option that controls location of
persistent stores
- Remove 'IPv6Comm' and 'ZoneID'. For the former, Broker
should be able to handle IPv6 automatically. The latter is
not supported anymore for now.
* Don't open debug.log when not configured to do logging, and
catch when broctl can't open the debug log file. (Daniel Thayer)
1.7-61 | 2018-03-15 14:57:05 -0700
* Configure Travis CI email recipients and build branches. (Daniel
Thayer)
1.7-57 | 2018-02-05 15:04:41 -0800
* Add a .travis.yml file (Daniel Thayer)
* Fix a race condition in the bro__test script. (Daniel Thayer)
* Fix the build-bro script when running on Travis CI. (Daniel Thayer)
1.7-53 | 2018-01-18 13:18:38 -0600
* Allow capstats to work with af_packet (Mike Reeves)
* Fix race conditions in the "update" command test and the "start-slowstart"
command test. Cleaned up the bro__test script. (Daniel Thayer)
* Sort the list of filesystems for each node in the "df" command output.
This fixes the "df" command test on Python 3. (Daniel Thayer)
* Enable easier changing of the node type sort order in broctl command
output. A list of node types in the preferred sort order is now used
instead of using alphabetical order. (Daniel Thayer)
* Add a logger to the node.cfg for some tests to verify correct ordering
of node names in the output of various broctl commands. (Daniel Thayer)
* Update test baselines for recent change of PFRINGClusterID default value.
Also, it is no longer necessary to set a value for the PFRINGClusterID
option in broctl.cfg for the PF_RING tests. (Daniel Thayer)
1.7-44 | 2017-12-28 10:33:48 -0500
* Fix bug in broctl df command where it could skip checking the filesystem
of the "logs" directory if none of the other Bro directories were on that
filesystem, but only when a cluster config with a logger node was being
used. Also fixed the check for NFS mounted volumes to prevent broctl from
skipping non-NFS filesystems that have a colon in the name.
Addresses BIT-1880 (Daniel Thayer)
* Added "df" command tests for a standalone configuration, and for Bro
directories on different partitions. (Daniel Thayer)
1.7-39 | 2017-12-28 10:29:22 -0500
* Changed the default value of PFRINGClusterID to be 21 (instead of 0)
when PF_RING is not installed. Also changed the default value of
SendMail to /usr/sbin/sendmail (instead of SENDMAIL-NOTFOUND) when
sendmail is not installed. (Daniel Thayer)
1.7-34 | 2017-12-13 11:30:50 -0600
* Simplify broctl "start" and "stop" output to show the node type
instead of listing each node name. (Daniel Thayer)
* Code cleanup: reduce number of hard-coded node type names in the code.
(Daniel Thayer)
* Add new tests of the BroControl plugin API (Daniel Thayer)
* Reorganize, rename, and simplify numerous test scripts. (Daniel Thayer)
* Cleanup the broctl test build and setup scripts. (Daniel Thayer)
* Add test for "broctl --version" (Daniel Thayer)
* Added a test case for multiple logger nodes in the "install" command test.
(Daniel Thayer)
1.7-16 | 2017-09-26 09:16:47 -0400
* Allow broctl plugin command names that are an empty string to be run
by typing just the plugin prefix name (no dot needed). (Daniel Thayer)
* Add tests to verify that bugs fixed in the following commits
are actually fixed: 6bf5bb0f, 9f387354, and f472a05f. (Daniel Thayer)
1.7-12 | 2017-09-20 17:18:40 -0400
* Fix archive-log to correctly handle logs that are already compressed.
(Daniel Thayer)
1.7-10 | 2017-09-19 17:06:27 -0500
* Fix "install" command to preserve symlinks in "site" directory.
Addresses BIT-1846. (Jon Siwek)
* Fix broctl "print" command to not truncate output. (Daniel Thayer)
1.7-7 | 2017-07-27 14:38:10 -0500
* lb_pf_ring update: support for ZC and the new bro::pf_ring plugin (cardigliano)
1.7-5 | 2017-07-11 08:45:32 -0500
* Use SHA-1 instead of MD5 to compute config hash values. Addresses BIT-1817.
(Daniel Thayer)
1.7 | 2017-06-26 15:55:09 -0700
* Release 1.7.
1.6-3 | 2017-06-26 10:52:27 -0400
* Set a value for the global_hash_seed constant. Addresses BIT-1819.
(Daniel Thayer)
1.6 | 2017-06-06 17:43:14 -0500
* Release 1.6
* Pruning CHANGES a bit (Daniel Thayer)
1.5-49 | 2017-04-30 12:53:44 -0400
* Allow more than one logger to be defined.
This adds initial support for running a Bro cluster with multiple logger
processes. This is primarily useful for installations that use something
like Kafka or Logstash to aggregate logs. (Daniel Thayer)
* Add a "--version" option to show broctl version (Daniel Thayer)
* Added a new option MailReceivingPackets to allow users to disable
broctl cron mail that no packets were seen on an interface. (Daniel Thayer)
* A large number of unused code removal and code cleanups (Daniel Thayer)
* Fix some failing tests when using python 3 (Daniel Thayer)
* The "start" helper script now reports error if PID string is empty (Daniel
Thayer)
* Fixed the sorting of node names in command output (e.g. "worker-10"
should be output after "worker-2"). Now the order of names is based on
the "count" node attribute instead of the name. (Daniel Thayer)
* Fixed some bugs in stats-to-csv script (proxies were being handled like
workers, and it was assuming that the manager is named "manager").
Also added more error checking. (Daniel Thayer)
* Fix potential cases of unhandled IndexError and ValueError. (Daniel Thayer)
* Fixed a few cases where the ps plugin didn't return non-zero when an
error occurred. (Daniel Thayer)
* Fix shell scripts to no longer depend on bash (Daniel Thayer)
* Improve the run_cmds() and run_localcmd() functions by returning output
as a string (instead of list of strings) and check and handle output
string correctly in all cases. (Daniel Thayer)
1.5-21 | 2017-03-17 13:18:58 -0400
* Fix some tests to make sure the test tmp dir is removed (Daniel Thayer)
* Update crash-diag script due to recent change where "bro -v" now outputs
the version to stdout. Also fixed crash-diag to not show stderr output
from running "bro -N". (Daniel Thayer)
* Add a new broctl option to expire crash directories
Added functionality to broctl cron to remove crash directories older than
the number of days specified in the new option CrashExpireInterval (the
default value is 0, which means crash directories never expire). (Daniel
Thayer)
* Add a test for expiration of crash directories (Daniel Thayer)
* Reduce disk usage of post-terminate and crash-diag
Changed post-terminate and crash-diag so that the bro binary is not
copied when there is no core file. Also, the crash report is now
saved to disk only when crash-diag is run from post-terminate (i.e.,
the "diag" command will no longer create any files). (Daniel Thayer)
* Change archive-log to use "mv" instead of "cp"
Changed archive-log to "mv" (rather than "cp") logs when not using gzip
for better efficiency. This means we will not have the logs in the tmp
directory when Bro crashes, so the scripts have now been simplified to
never attempt to keep a copy of archived logs in the tmp dir (previously,
logs >100MB were always being deleted anyway). (Daniel Thayer)
1.5-12 | 2017-03-13 13:43:43 -0400
* Prevent the broctl check and scripts commands from hanging
Changed the check-config script to run bro with the "-a" option
when running "broctl check" in the hope that this will prevent broctl
from hanging for any reason. The "-a" option prevents bro
from running any bro script statements (previously, "check" would
cause bro to exit after handling the bro_init event) but should still
be able to identify the same bro scripting errors as before.
Also, to prevent "broctl scripts" from hanging, set the value
of "exit_only_after_terminate" to False (in broctl/check.bro) in case
another script sets the value of that constant to True. Since "bro -a"
prevents bro from creating the loaded_scripts.log file, that option
cannot be used with "broctl scripts". (Daniel Thayer)
1.5-9 | 2017-01-26 16:38:17 -0500
* Fix some failing tests
Added a new broctl option, called StopWait, to force the stop command
to wait for the post-terminate script to finish. This is needed
because some tests were failing due to background log-archive processes
creating logs after "broctl stop" finished, which was preventing the
test directory from being deleted. (Daniel Thayer)
* Fix post-terminate to not generate invalid timestamps
Fixed the code that tries to extract the base name and timestamp
from a log filename, because it wasn't extracting them correctly
when the base name contained a period (this doesn't happen for any
of the standard Bro logs) or if the timestamp in the filename wasn't in
the format YYYY-MM-DD-HH-MM-SS (this could happen if Bro terminates
but for some reason doesn't execute the code in the writers/ascii.bro
script that renames the log, or if someone uses a different forma
by redefining Log::default_rotation_date_format). The fix involves
first removing the log suffix, then trying to extract the timestamp
in one of the two default timestamp formats. This procedure is more
reliable than the previous method of making assumptions about how many
period characters should be in a log filename.
Also, when Bro terminates normally, post-terminate now just tries to
archive all log files, instead of only those that were rotated. This
is to avoid missing any logs. This also means that the
stderr.log/stdout.log files are now archived when Bro terminates
normally (instead of only when Bro crashes), which is useful to
capture any error messages from archive-log or Bro.
Also fixed an issue that could occasionally occur when post-terminate
archives an unrotated log file (i.e., no timestamp in the filename)
and a different log with the same base name was archived after
post-terminate started, then the computed start time of the unrotated
log would be later than the end time. Fixed by setting the start time
to equal the end time.
Also added the node name to the subject line in the email sent when
post-terminate fails to archive a log. (Daniel Thayer)
* Add error checking of archive-log timestamp parameters
Check if the format of the timestamp command-line parameters matches
the required format. If not, exit with an error message. This will
prevent archive-log from creating an archived log file with a corrup
filename or in a directory with a corrupt name.
Also simplified the code that gets the current century. (Daniel Thayer)
1.5-5 | 2017-01-26 13:34:37 -0500
* Fix crash-diag script to use the correct debugger, because on some systems
the correct debugger to use is not called "gdb" (currently, this
affects OS X and OpenBSD). (Daniel Thayer)
1.5-2 | 2016-12-06 12:35:40 -0800
* Don't show output of "ulimit -v" in crash reports on OpenBSD; adjusting
it always fails and showing the value only creates confusion. (Daniel Thayer)
1.5 | 2016-11-16 14:51:05 -0800
* Pruning CHANGES a bit. (Daniel Thayer)
* Update broctl.rst using "make doc". (Daniel Thayer)
1.5-beta2 | 2016-11-02 11:08:45 -0700
* Release 1.5-beta2.
1.5-beta-56 | 2016-11-02 13:44:41 -0400
* A number of portability fixes, mostly related to OpenBSD. (Daniel Thayer)
1.5-beta-48 | 2016-11-02 13:38:34 -0400
* Fix bug where standalone bro port isn't recorded to state.db, and
add more test cases. (Daniel Thayer)
1.5-beta-41 | 2016-11-01 09:34:19 -0700
* Add support for local-logger.bro site policy script. (Daniel Thayer)
* Add a few clarifications to broctl documentation. (Daniel Thayer)
1.5-beta-31 | 2016-10-07 14:55:07 -0400
* Improve diag command output. (Daniel Thayer)
* Add new option SitePolicyScripts to replace SitePolicyStandalone.
Also marked SitePolicyManager, SitePolicyWorker, and SitePolicyStandalone
as deprecated in the documentation. (Daniel Thayer)
* Fix a couple of failing tests. (Daniel Thayer)
* Fix a failing test on FreeBSD. (Daniel Thayer)
* Improved the documentation, especially documentation of node attributes,
documentation of broctl commands, and added a section about Bro/BroControl
communication. (Daniel Thayer)
1.5-beta-24 | 2016-09-26 16:24:21 -0400
* Define all BroControl exceptions in the new exceptions.py file.
The broctl client will now handle only those exceptions, showing a
useful error message instead of a stack trace. As before, if a
standard Python exception is raised (this is not expected to occur),
then broctl will terminate with a stack trace, which is useful to
help debug the problem. (Daniel Thayer)
1.5-beta-22 | 2016-09-26 16:11:21 -0400
* Fix crash-diag script to not confuse log files with core files
Fixed the crash-diag script to not include any log filenames that
contain the word "core" in the list of core files. (Daniel Thayer)
* Improve crash-diag script's handling of core filenames
Fixed the script to handle filenames that contain a space. (Daniel Thayer)
1.5-beta-19 | 2016-09-26 15:50:22 -0400
* Fix a bug where broctl loses state of running Bro nodes
If a node name contains uppercase letters, then restarting broctl while
that node is running results in a confusing warning about that node still
running, and broctl discards the PID of that node. Fixed by converting
the node name to lowercase before checking the state database (where all
keys are converted to lowercase).
Addresses BIT-1676. (Daniel Thayer)
* Report an error if a user defines node names differing only by case (such
as "worker-1" and "Worker-1"). This check is needed because keys
in the state db are converted to lowercase. (Daniel Thayer)
* Improve error messages for plugin API functions (Daniel Thayer)
* Removed the restriction that plugin state variables must be string
type, because normal state variables have no such restriction. (Daniel Thayer)
* Fixed the getGlobalOption() function in the plugin API. It did not
convert its argument to lowercase, and could return the value of a
state variable.
Also simplified some code by replacing the config has_attr() function
with a new function get_option(), which helps reduce the number of places
in the code where keys are converted to lowercase. (Daniel Thayer)
* Do not set a plugin state var. with invalid name (Daniel Thayer)
* Improve code that sets plugin option values
Improved error messages to include the name of the plugin, and fixed a
problem where any option with an invalid name was being set (now such
options are skipped). (Daniel Thayer)
* Simplify some broctl cron-related code by using get_state() (Daniel Thayer)
* Fix the subst() function for non-string data types (Daniel Thayer)
* Code simplification involving the config get_state() function
Added an optional default parameter to the config get_state() function,
and changed that function to convert the key to lowercase. These changes
help simplify some code by reducing the number of conversions to lowercase. (Daniel Thayer)
* Simplify code by not converting option values to lowercase (Daniel Thayer)
* Fix problem with custom node keys that are not lowercase (Daniel Thayer)
* Improve documentation of case-sensitive issues in broctl (Daniel Thayer)
* Remove redundant lowercase conversions of state var. names (Daniel Thayer)
1.5-beta-2 | 2016-09-01 12:03:46 -0400
* Improve crash reports by showing Bro plugin info (Daniel Thayer)
1.5-beta | 2016-08-12 13:20:27 -0700
* Release 1.5-beta.
* Fix rsync error message to not show ssh login banner. (Daniel Thayer)
* Run "make doc" to update broctl.rst (Daniel Thayer)
* Pruning CHANGES a bit (Daniel Thayer)
1.4-150 | 2016-08-09 13:38:17 -0400
* Show python stack trace if unexpected exception is raised.
(Daniel Thayer)
* Improve broctl error messages and error handling across the board.
(Daniel Thayer)
* Add a new optional node type "logger" that will handle logging
instead of the manager. (Daniel Thayer)
1.4-132 | 2016-07-14 18:23:27 -0400
* Don't run capstats on interfaces with packet source prefix. (Daniel Thayer)
1.4-130 | 2016-07-13 14:36:34 -0400
* Improve the text of crash reports with instructions on how to
get a backtrace, which should reduce the amount of useless crash
reports mailed to the Bro team. (Daniel Thayer)
1.4-127 | 2016-07-06 08:58:18 -0500
* Ignore packet source prefix of interface name when using capstats. (Jan Grashoefer)
1.4-125 | 2016-07-02 17:53:42 -0500
* New plugin function "broctl_config" so plugin authors can add their own
script code to the autogenerated broctl-config.bro script. (Seth Hall)
1.4-122 | 2016-07-02 12:05:23 -0500
* Follow symlinks to directories when searching for plugins. (Jon Siwek)
1.4-119 | 2016-06-28 11:11:19 -0400
* Fix race condition in reading/writing broctl-config.sh (Daniel Thayer)
1.4-117 | 2016-06-22 12:14:37 -0400
* Improve broctl behavior when unable to stop a node. (Daniel Thayer)
1.4-112 | 2016-06-14 16:14:52 -0700
* Fix a failing test on some platforms and improve its error
message. (Daniel Thayer)
* Add Bro plugin directory to broctl plugin search path. (Daniel Thayer)
* Update test baselines. (Daniel Thayer)