Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

piconv bug of decoding UTF-16 (with fix) #19

Closed
numa2666 opened this issue Mar 26, 2014 · 2 comments
Closed

piconv bug of decoding UTF-16 (with fix) #19

numa2666 opened this issue Mar 26, 2014 · 2 comments

Comments

@numa2666
Copy link

Hi,

I've found a bug in piconv 2.5 included in perl 5.18.2 distribution.
The problem is, it can not handle UTF-16 correctly, especially
when UTF-16BE or UTF-16LE is specified. It is because that
piconv does not slurp the input (necessary when UTF-16/UTF-32
are read), and try to read line by line, which will lead to failed
conversion. They must be treated the same as UTF-16 and
UTF-32 (non-BE/LE versions). Ah, well, piconv also fails to
determine whether slurp is needed or not, by using $to
encoding instead of $from encoding. I've included a patch
below.

The following patch also includes some minor fixes.

  • piconv fails to detect the locale-specific encoding if LANG
    or LC_* environment variables are not set, or even they are set,
    if they don't have any encoding field. It is quite common situation,
    and using Encode::Locale's encoding value instead of environment
    variable is better and portable to (especially) non-Unix systems.
  • The help message is fixed to display backslash-escaped string
    correctly.

Hope this helps,

Toshinori Numata
http://blog.livedoor.jp/numa2666/ (in Japanese)

--- c:/strawberry/perl/bin/piconv   Tue Jan  7 19:19:14 2014
+++ piconv  Sun Mar 23 19:30:29 2014
@@ -5,6 +5,7 @@
 use strict;
 use Encode ;
 use Encode::Alias;
+use Encode::Locale;
 my %Scheme =  map {$_ => 1} qw(from_to decode_encode perlio);

 use File::Basename;
@@ -34,7 +35,7 @@

 $Opt{help} and help();
 $Opt{list} and list_encodings();
-my $locale = $ENV{LC_CTYPE} || $ENV{LC_ALL} || $ENV{LANG};
+my $locale = $Encode::Locale::ENCODING_LOCALE;
 defined $Opt{resolve} and resolve_encoding($Opt{resolve});
 $Opt{from} || $Opt{to} || help();
 my $from = $Opt{from} || $locale or help("from_encoding unspecified");
@@ -68,14 +69,14 @@
 EOT
 }

-my %use_bom = map { $_ => 1 } qw/UTF-16 UTF-32/;
+my %use_bom = map { $_ => 1 } qw/UTF-16 UTF-16BE UTF-16LE UTF-32 UTF-32BE UTF-32LE/;

 # we do not use <> (or ARGV) for the sake of binmode()
 @ARGV or push @ARGV, \*STDIN;

 unless ( $scheme eq 'perlio' ) {
     binmode STDOUT;
-    my $need2slurp = $use_bom{ find_encoding($to)->name };
+    my $need2slurp = $use_bom{ find_encoding($from)->name };
     for my $argv (@ARGV) {
         my $ifh = ref $argv ? $argv : undef;
    $ifh or open $ifh, "<", $argv or warn "Can't open $argv: $!" and next;
@@ -169,7 +170,7 @@
   -D,--debug          show debug information
   -S,--scheme scheme  use the scheme for conversion
 Those are handy when you can only see ASCII characters:
-  -p,--perlqq         transliterate characters missing in encoding to \x{HHHH}
+  -p,--perlqq         transliterate characters missing in encoding to \\x{HHHH}
                       where HHHH is the hexadecimal Unicode code point
   --htmlcref          transliterate characters missing in encoding to &#NNN;
                       where NNN is the decimal Unicode code point
@dankogai
Copy link
Owner

Thank you for your work. And I am sorry I cannot take this patch. Encode is a core module so I cannot depend on non-core module. Encode::Locale is not in core.

I'll come up with other solution…

Dan the Maintainer Thereof

@dankogai
Copy link
Owner

Encode 2.58 released -- with your fixes except for the Encode::Locale dependency.

Dan the Maintainer Thereof

jsonn pushed a commit to jsonn/pkgsrc that referenced this issue Jul 16, 2014
2.62 2014/05/31 12:12:39
! Encode.pm
  s/2013/2014/ on COPYRIGHT section
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode/Makefile_PL.e2x
  Encode.xs
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  Merged from perl.git: "Fix Encode 2.60 with g++"
  http://perl5.git.perl.org/perl.git/commit/89c2544cd3

2.61 2014/05/31 09:48:48
! bin/piconv
  Applied: piconv nit
  + Better error handling when the encoding name is nonexistent
  Message-Id: <537139A0.1000503@iki.fi>
! Encode.xs
  Applied: RT #95466:
   fallback definition of SvIsCOW() is wrong
   (and hence breaks on 5.8.2 and earlier)
  https://rt.cpan.org/Ticket/Display.html?id=95466

2.60 2014/04/29 16:25:06
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode/Makefile_PL.e2x
  Encode/encode.h
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  encengine.c
  Applied: more Fix Windows build (of Encode) with VC++ 6.0
  http://perl5.git.perl.org/perl.git/commit/9e9002efd1609c7d154f98af43a026320df7582c
! Unicode/Unicode.xs
  Addressed: sign extension issue found by Coverity #21
  dankogai/p5-encode#21
! Encode/encode.h Encode.xs Unicode/Unicode.xs
  removed #define U8 U8
  https://rt.perl.org/Ticket/Display.html?id=121554
  http://perl5.git.perl.org/perl.git/commit/2f2b4ff2c154a8e461857f2e82cb815c238d0d94

2.59 2014/04/06 17:23:55
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode.pm
  Encode.xs
  Encode/Makefile_PL.e2x
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  Restored the signature of Encode_XSEncoding() to address RT#94478
  * While dankogai/p5-encode#20
    pulls the symnames via argument thus breaks the compatibility
    with Encode::XX modules with *.ucm, the restored version
    pulls the symanmes via enc->name[0] so the added 2nd argument
    is no longer needed.
  https://rt.cpan.org/Public/Bug/Display.html?id=94478

2.58 2014/03/28 02:37:42
! bin/piconv
  Addressed: piconv bug of decoding UTF-16 (with fix)
  dankogai/p5-encode#19
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode.pm
  Encode.xs
  Encode/Makefile_PL.e2x
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  Pulled: Remap symname [RT #94221]
  dankogai/p5-encode#20
  https://rt.cpan.org/Public/Bug/Display.html?id=94221
! Encode.pm
  Pulled: [doc] clarify that CHECK coderefs return octets #18
  dankogai/p5-encode#18
jsonn pushed a commit to jsonn/pkgsrc that referenced this issue Oct 11, 2014
2.62 2014/05/31 12:12:39
! Encode.pm
  s/2013/2014/ on COPYRIGHT section
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode/Makefile_PL.e2x
  Encode.xs
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  Merged from perl.git: "Fix Encode 2.60 with g++"
  http://perl5.git.perl.org/perl.git/commit/89c2544cd3

2.61 2014/05/31 09:48:48
! bin/piconv
  Applied: piconv nit
  + Better error handling when the encoding name is nonexistent
  Message-Id: <537139A0.1000503@iki.fi>
! Encode.xs
  Applied: RT #95466:
   fallback definition of SvIsCOW() is wrong
   (and hence breaks on 5.8.2 and earlier)
  https://rt.cpan.org/Ticket/Display.html?id=95466

2.60 2014/04/29 16:25:06
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode/Makefile_PL.e2x
  Encode/encode.h
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  encengine.c
  Applied: more Fix Windows build (of Encode) with VC++ 6.0
  http://perl5.git.perl.org/perl.git/commit/9e9002efd1609c7d154f98af43a026320df7582c
! Unicode/Unicode.xs
  Addressed: sign extension issue found by Coverity #21
  dankogai/p5-encode#21
! Encode/encode.h Encode.xs Unicode/Unicode.xs
  removed #define U8 U8
  https://rt.perl.org/Ticket/Display.html?id=121554
  http://perl5.git.perl.org/perl.git/commit/2f2b4ff2c154a8e461857f2e82cb815c238d0d94

2.59 2014/04/06 17:23:55
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode.pm
  Encode.xs
  Encode/Makefile_PL.e2x
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  Restored the signature of Encode_XSEncoding() to address RT#94478
  * While dankogai/p5-encode#20
    pulls the symnames via argument thus breaks the compatibility
    with Encode::XX modules with *.ucm, the restored version
    pulls the symanmes via enc->name[0] so the added 2nd argument
    is no longer needed.
  https://rt.cpan.org/Public/Bug/Display.html?id=94478

2.58 2014/03/28 02:37:42
! bin/piconv
  Addressed: piconv bug of decoding UTF-16 (with fix)
  dankogai/p5-encode#19
! Byte/Makefile.PL
  CN/Makefile.PL
  EBCDIC/Makefile.PL
  Encode.pm
  Encode.xs
  Encode/Makefile_PL.e2x
  JP/Makefile.PL
  KR/Makefile.PL
  Symbol/Makefile.PL
  TW/Makefile.PL
  bin/enc2xs
  Pulled: Remap symname [RT #94221]
  dankogai/p5-encode#20
  https://rt.cpan.org/Public/Bug/Display.html?id=94221
! Encode.pm
  Pulled: [doc] clarify that CHECK coderefs return octets #18
  dankogai/p5-encode#18
notxarb referenced this issue in nanobox-io/nanobox-pkgsrc-lite May 8, 2015
Squashed commit of the following:

commit 130c8eb2df4b63034353a232af8e5fa4526693a5
Merge: 5888b51 a980f97
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Sun Apr 26 23:00:03 2015 +0100

    Merge branch 'joyent/feature/miscfix/2015Q1' into joyent/release/2015Q1

commit a980f97415f48af9407551d6e515a09104079a8d
Merge: 1cbc272 fdbe317
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Sun Apr 26 22:55:29 2015 +0100

    Merge branch 'pkgsrc_2015Q1' into joyent/feature/miscfix/2015Q1

commit fdbe317df493f113aa0a8bdcf1ec1b6b65937bf4
Author: tron <tron>
Date:   Sun Apr 26 08:47:38 2015 +0000

    Pullup tickets #4684, #4685 and #4686.

commit faae9e8bdb9fe7ab45c47c87b904784650e473d4
Author: tron <tron>
Date:   Sun Apr 26 08:47:12 2015 +0000

    Pullup ticket #4686 - requested by taca
    lang/ruby22-base: security update

    Revisions pulled up:
    - lang/ruby/rubyversion.mk                                      1.141
    - lang/ruby22-base/distinfo                                     1.2

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Sun Apr 19 16:25:10 UTC 2015

       Modified Files:
       	pkgsrc/lang/ruby: rubyversion.mk
       	pkgsrc/lang/ruby22-base: distinfo

       Log Message:
       Update ruby22-base and ruby22 package to 2.2.2.

       >From release announce:

       We are pleased to announce the release of Ruby 2.2.2. This is a TEENY version
       release of the stable 2.2 series.

       This release includes the security fix for a OpenSSL extension's hostname
       verification vulnerability.

           CVE-2015-1855: Ruby OpenSSL Hostname Verification

       There are also some bugfixes. See ChangeLog for details.

commit 6e88dba1add92447fc97f35ec9518d1a58ce2484
Author: tron <tron>
Date:   Sun Apr 26 08:36:34 2015 +0000

    Pullup ticket #4685 - requested by taca
    lang/ruby21-base: security update

    Revisions pulled up:
    - lang/ruby/rubyversion.mk                                      1.140
    - lang/ruby21-base/PLIST                                        1.6
    - lang/ruby21-base/distinfo                                     1.14-1.15
    - lang/ruby21-base/patches/patch-configure                      1.3
    - lang/ruby21-base/patches/patch-lib_rdoc_text.rb               deleted
    - lang/ruby21/Makefile                                          1.3

    ---
       Module Name:	pkgsrc
       Committed By:	jperkin
       Date:		Fri Apr  3 09:02:24 UTC 2015

       Modified Files:
       	pkgsrc/lang/ruby21-base: distinfo
       	pkgsrc/lang/ruby21-base/patches: patch-configure

       Log Message:
       Disable CPU detection on Darwin, the result for 32-bit (i486) is incompatible
       with pkgsrc MACHINE_ARCH (i386).  Fixes 32-bit build, no change for 64-bit.

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Sun Apr 19 16:19:00 UTC 2015

       Modified Files:
       	pkgsrc/lang/ruby: rubyversion.mk
       	pkgsrc/lang/ruby21-base: PLIST distinfo
       Removed Files:
       	pkgsrc/lang/ruby21-base/patches: patch-lib_rdoc_text.rb

       Log Message:
       Update ruby21-base and ruby21 packages to 2.1.6.

       >From release announce:

       Ruby 2.1.6 has been released.

       This release includes a security fix for OpenSSL extension. Please view the
       topic below for more details.

           CVE-2015-1855: Ruby OpenSSL Hostname Verification

       And, many bug fixes are also included. See tickets and ChangeLog for details.

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Sun Apr 19 16:20:42 UTC 2015

       Modified Files:
       	pkgsrc/lang/ruby21: Makefile

       Log Message:
       Reset PKGREVISION.

commit 072ebbdcf4981a4582ea66498337b1e25e302b8c
Author: tron <tron>
Date:   Sun Apr 26 08:26:15 2015 +0000

    Pullup ticket #4684 - requested by taca
    lang/ruby200-base: security update

    Revisions pulled up:
    - lang/ruby/rubyversion.mk                                      1.139
    - lang/ruby200-base/distinfo                                    1.23 via patch

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Sun Apr 19 16:12:23 UTC 2015

       Modified Files:
       	pkgsrc/lang/ruby: rubyversion.mk
       	pkgsrc/lang/ruby200-base: distinfo

       Log Message:
       Update ruby200 package to 2.0.0p645 (Ruby 2.0.0-p645).

       >From release announce:

       We are pleased to announce the release of Ruby 2.0.0-p645.

       This release includes a security fix for OpenSSL extension. Please view the
       topic below for more details.

           CVE-2015-1855: Ruby OpenSSL Hostname Verification

       Ruby 2.0.0 is now under the state of the security maintenance phase, until
       Feb. 24th, 2016. After the date, maintenance of Ruby 2.0.0 will be ended. We
       recommend you start planning migration to newer versions of Ruby, such as 2.1
       or 2.2.

       This release includes the security fix mentioned above along with small
       changes required for test environment (that shouldn't affect normal users).

       See ChangeLog for full details.

commit 58cd83802d5111831a785666eefc272c1c43051f
Author: bsiegert <bsiegert>
Date:   Sat Apr 25 20:08:09 2015 +0000

    Pullup ticket 4670

commit 259293bfa47febf9a2b4d4d71c94c3c467f0397f
Author: bsiegert <bsiegert>
Date:   Sat Apr 25 20:08:04 2015 +0000

    Pullup ticket #4670 - requested by joerg
    devel/rsltc: build fix

    Revisions pulled up:
    - devel/rsltc/Makefile                                          1.16

    ---
       Module Name:	pkgsrc
       Committed By:	joerg
       Date:		Sat Apr 18 20:40:46 UTC 2015

       Modified Files:
       	pkgsrc/devel/rsltc: Makefile

       Log Message:
       Let clang ignore the mixed up return use of the K&R code.

commit 5342fbe8148fa3cc6051f8087e07c96b646a573c
Author: bsiegert <bsiegert>
Date:   Sat Apr 25 19:28:44 2015 +0000

    Pullup ticket #4682 - requested by hiramatsu
    inputmethod/librime: build fix

    Revisions pulled up:
    - inputmethod/librime/distinfo                                  1.3
    - inputmethod/librime/patches/patch-src_dict_table.cc           1.2

    ---
       Module Name:	pkgsrc
       Committed By:	hiramatsu
       Date:		Tue Apr 21 13:59:31 UTC 2015

       Modified Files:
       	pkgsrc/inputmethod/librime: distinfo
       	pkgsrc/inputmethod/librime/patches: patch-src_dict_table.cc

       Log Message:
       Fix build break with gcc.

commit 924e69ca4577415c75e6b584de574ca82b51dcc1
Author: hiramatsu <hiramatsu>
Date:   Sat Apr 25 04:06:02 2015 +0000

    pullup #4683

commit ca4f2559fbd0e9f02fa251517d3b9d92ff68d961
Author: hiramatsu <hiramatsu>
Date:   Sat Apr 25 04:04:43 2015 +0000

    Pullup ticket #4683 - requested by joerg
    editors/p5-Wx-Scintilla: link fix patch

    Revisions pulled up:
    - editors/p5-Wx-Scintilla/distinfo                              1.3
    - editors/p5-Wx-Scintilla/patches/patch-aa                      1.3

    ---
       Module Name: pkgsrc
       Committed By:        joerg
       Date:                Fri Apr 17 15:22:44 UTC 2015

       Modified Files:
            pkgsrc/editors/p5-Wx-Scintilla: distinfo
            pkgsrc/editors/p5-Wx-Scintilla/patches: patch-aa

       Log Message:
       Don't depend on ${PREFIX}/lib to be added implicitly by the wrappers.
       Don't use empty arguments to -o.

commit 26b7efffbf80de9703df2d47f9fcd08a5374ec2c
Author: hiramatsu <hiramatsu>
Date:   Fri Apr 24 16:18:39 2015 +0000

    pullup #4654.

commit dc5bb12c74c6a65858d49e497554493bd3f63564
Author: hiramatsu <hiramatsu>
Date:   Fri Apr 24 16:07:10 2015 +0000

    Pullup ticket #4654 - requested by dsainty
    security/zoneminder: security update

    Revisions pulled up:
    - security/zoneminder/Makefile                                  1.18
    - security/zoneminder/distinfo                                  1.7
    - security/zoneminder/MESSAGE                                   1.3
    - security/zoneminder/PLIST                                     1.4
    - security/zoneminder/patches/patch-Makefile_am                 1.3
    - security/zoneminder/patches/patch-configure_ac                1.4
    - security/zoneminder/patches/patch-scripts_ZoneMinder_lib_ZoneMinder_General_pm 1.2
    - security/zoneminder/patches/patch-scripts_zm_in               1.3
    - security/zoneminder/patches/patch-src_Makefile_am             1.3
    - security/zoneminder/patches/patch-src_zm__thread.h            1.4
    - security/zoneminder/patches/patch-src_zm__timer.h             1.3
    - security/zoneminder/patches/patch-src_zm__utils.h             1.2
    - security/zoneminder/patches/patch-src_zm_ffmpeg_camera_cpp    1.1
    - security/zoneminder/patches/patch-src_zm_remote_camera_h      1.4
    - security/zoneminder/patches/patch-src_zm_signal_cpp           1.1
    - security/zoneminder/patches/patch-src_zm_signal_h             deleted
    - security/zoneminder/patches/patch-src_zmf_cpp                 deleted

    ---
       Module Name:    pkgsrc
       Committed By:   dsainty
       Date:           Sun Apr  5 08:51:08 UTC 2015

       Modified Files:
               pkgsrc/security/zoneminder: MESSAGE Makefile PLIST distinfo
               pkgsrc/security/zoneminder/patches: patch-Makefile_am
                   patch-configure_ac
                   patch-scripts_ZoneMinder_lib_ZoneMinder_General_pm
                   patch-scripts_zm_in patch-src_Makefile_am patch-src_zm__thread.h
                   patch-src_zm__timer.h patch-src_zm__utils.h
                   patch-src_zm_remote_camera_h
       Removed Files:
               pkgsrc/security/zoneminder/patches: patch-src_zm_signal_h
                   patch-src_zmf_cpp

       Log Message:
       Update ZoneMinder from 1.25.0 to 1.28.1.

       Numerous changes, documented at:
        https://github.com/ZoneMinder/ZoneMinder/releases

       Addresses two security advisories:

         https://github.com/ZoneMinder/ZoneMinder/releases/tag/v1.28.0
         http://secunia.com/advisories/62918/

       Pkgsrc changes:

       patch-src_zm_signal_h is no longer necessary because zm_signal.h uses
       HAVE_EXECINFO_H.

       patch-src_zmf_cpp appears to be applied upstream.

       patch-configure_ac no longer needs to set PATH_BUILD to
       PREFIX/share/zoneminder, so that zmupdate.pl can locate the database build
       scripts as installed files.  Upstream has now implemented this via the
       ZM_PATH_DATA entry in zm.conf, and adds a ZM_PATH_DATA/db subdirectory.

       src/Makefile.am no longer setuid's zmfix, as zmfix was removed from
       ZoneMinder 1.26.6.

       The code now uses clock_gettime(), which on some systems (like Linux), calls
       for -lrt.  Since the build system isn't aware of this, but Pkgsrc is, just set
       PTHREAD_AUTO_VARS=yes.

       The PHP code now uses PDO for DB access, but it looks like there are some
       straggling dependencies on the raw MySQL driver, so both are pulled in.

    ---
       Module Name:    pkgsrc
       Committed By:   dsainty
       Date:           Tue Apr  7 12:32:59 UTC 2015

       Modified Files:
               pkgsrc/security/zoneminder: Makefile

       Log Message:
       NetBSD's tar (6.1) does an exit(1) on the source archive, complaining:

        Invalid header, starting valid header search.

       As a workaround: EXTRACT_USING=gtar

    ---
       Module Name:    pkgsrc
       Committed By:   dsainty
       Date:           Tue Apr  7 12:54:36 UTC 2015

       Modified Files:
               pkgsrc/security/zoneminder: distinfo
               pkgsrc/security/zoneminder/patches: patch-configure_ac

       Log Message:
       Recognise lower-case "netbsd" as BSD.

       NB: This doesn't fix the threads portability issues yet for NetBSD, but gets
       through configuration.

    ---
       Module Name:    pkgsrc
       Committed By:   dsainty
       Date:           Fri Apr 10 02:53:20 UTC 2015

       Modified Files:
               pkgsrc/security/zoneminder: Makefile

       Log Message:
       New ZoneMinder supports the newer ffmpeg APIs, so switch the ffmpeg
       dependency from ffmpeg010 to ffmpeg2.

    ---
       Module Name:    pkgsrc
       Committed By:   dsainty
       Date:           Fri Apr 10 02:58:49 UTC 2015

       Modified Files:
               pkgsrc/security/zoneminder: Makefile distinfo
               pkgsrc/security/zoneminder/patches: patch-src_zm__thread.h
                   patch-src_zm__timer.h
       Added Files:
               pkgsrc/security/zoneminder/patches: patch-src_zm_ffmpeg_camera_cpp
                   patch-src_zm_signal_cpp

       Log Message:
       Patch up some Linux-specific assumptions in the code.

       Fixes build under NetBSD.

       Bump PKGREVISION for switch to ffmpeg2, and some portability changes that
       aren't expected to affect functionality.

    ---
       Module Name:    pkgsrc
       Committed By:   dsainty
       Date:           Wed Apr 22 15:13:44 UTC 2015

       Modified Files:
               pkgsrc/security/zoneminder: Makefile

       Log Message:
       Fix paths in two missed installed scripts.

       The intention of zmsystemctl.pl is to use bin/pkexec to allow the apache user
       to start and stop the ZoneMinder services on operating systems using systemd
       and newer versions of Polkit than Pkgsrc currently has.

       If the base OS doesn't use systemd (E.g. anything not Linux), this file
       shouldn't be used anyway.

       In Pkgsrc we ignore the potentially absent pkexec interpreter in this file.
       If the base OS uses systemd, it probably also has pkexec in its base
       installation.

       Bump PKGREVISION.

commit 5888b51a0b3b68add698d985f00528ba171625b4
Merge: 0d3b50c 1cbc272
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Thu Apr 23 12:34:40 2015 +0100

    Merge branch 'joyent/feature/miscfix/2015Q1' into joyent/release/2015Q1

commit 1cbc272373cd255cd6a62f723a2d6e3efd36e50e
Merge: ccca9bf cc4bcc6
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Thu Apr 23 12:33:54 2015 +0100

    Merge branch 'pkgsrc_2015Q1' into joyent/feature/miscfix/2015Q1

commit 0d3b50ce89ba6306650de22989cd367056fea258
Merge: a5f8d21 ccca9bf
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Thu Apr 23 10:40:08 2015 +0100

    Merge branch 'joyent/feature/miscfix/2015Q1' into joyent/release/2015Q1

commit ccca9bf3edab64da44b0dc2b92675a00155d6fed
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Thu Apr 23 10:38:13 2015 +0100

    www/curl: backport 7.42.0 update.

commit 9e10d96a16dc43313e438319d39b61658df06820
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Thu Apr 23 10:36:53 2015 +0100

    Backport changes to disable epoll/inotify.

commit cc4bcc659cd1f7d2b5f05c11d767b03ed87f07f4
Author: tron <tron>
Date:   Wed Apr 22 22:55:10 2015 +0000

    Pullup tickets #4680 and #4681.

commit da8721d64da4906e7d9cec770d35c0227d1e8c79
Author: tron <tron>
Date:   Wed Apr 22 22:54:34 2015 +0000

    Pullup ticket #4681 - requested by joerg
    graphics/p5-PerlMagick: bug fix patch

    Revisions pulled up:
    - graphics/p5-PerlMagick/Makefile                               1.110

    ---
       Module Name:	pkgsrc
       Committed By:	joerg
       Date:		Tue Apr 21 17:21:18 UTC 2015

       Modified Files:
       	pkgsrc/graphics/p5-PerlMagick: Makefile

       Log Message:
       Don't use the shipped Magick.pm, it has the wrong bootstrap version.

commit c9a8d86c67b8dbd0b626c588d5210245c521a90f
Author: tron <tron>
Date:   Wed Apr 22 22:43:54 2015 +0000

    Pullup ticket #4680 - requested by taca
    lang/php56: security update

    Revisions pulled up:
    - lang/php/phpversion.mk                                        1.96
    - lang/php56/distinfo                                           1.9

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Fri Apr 17 16:42:43 UTC 2015

       Modified Files:
       	pkgsrc/lang/php: phpversion.mk
       	pkgsrc/lang/php56: distinfo

       Log Message:
       Update php56 to 5.6.8.

       16 Apr 2015, PHP 5.6.8

       - Core:
         . Fixed bug #66609 (php crashes with __get() and ++ operator in some cases).
           (Dmitry, Laruence)
         . Fixed bug #68021 (get_browser() browser_name_regex returns non-utf-8
           characters). (Tjerk)
         . Fixed bug #68917 (parse_url fails on some partial urls). (Wei Dai)
         . Fixed bug #69134 (Per Directory Values overrides PHP_INI_SYSTEM
           configuration options). (Anatol Belski)
         . Additional fix for bug #69152 (Type confusion vulnerability in
           exception::getTraceAsString). (Stas)
         . Fixed bug #69210 (serialize function return corrupted data when sleep has
           non-string values). (Juan Basso)
         . Fixed bug #69212 (Leaking VIA_HANDLER func when exception thrown in
           __call/... arg passing). (Nikita)
         . Fixed bug #69221 (Segmentation fault when using a generator in combination
           with an Iterator). (Nikita)
         . Fixed bug #69337 (php_stream_url_wrap_http_ex() type-confusion
           vulnerability). (Stas)
         . Fixed bug #69353 (Missing null byte checks for paths in various PHP
           extensions). (Stas)

       - Apache2handler:
         . Fixed bug #69218 (potential remote code execution with apache 2.4
           apache2handler). (Gerrit Venema)

       - cURL:
         . Implemented FR#69278 (HTTP2 support). (Masaki Kagaya)
         . Fixed bug #68739 (Missing break / control flow). (Laruence)
         . Fixed bug #69316 (Use-after-free in php_curl related to
           CURLOPT_FILE/_INFILE/_WRITEHEADER). (Laruence)

       - Date:
         . Fixed bug #69336 (Issues with "last day of <monthname>"). (Derick Rethans)

       - Enchant:
         . Fixed bug #65406 (Enchant broker plugins are in the wrong place in windows
           builds). (Anatol)

       - Ereg:
         . Fixed bug #68740 (NULL Pointer Dereference). (Laruence)

       - Fileinfo:
         . Fixed bug #68819 (Fileinfo on specific file causes spurious OOM and/or
           segfault). (Anatol Belski)

       - Filter:
         . Fixed bug #69202: (FILTER_FLAG_STRIP_BACKTICK ignored unless other
           flags are used). (Jeff Welch)
         . Fixed bug #69203 (FILTER_FLAG_STRIP_HIGH doesn't strip ASCII 127). (Jeff
           Welch)

       - OPCache:
         . Fixed bug #69297 (function_exists strange behavior with OPCache on
           disabled function). (Laruence)
         . Fixed bug #69281 (opcache_is_script_cached no longer works). (danack)
         . Fixed bug #68677 (Use After Free). (CVE-2015-1351) (Laruence)

       - OpenSSL
         . Fixed bugs #68853, #65137 (Buffered crypto stream data breaks IO polling
           in stream_select() contexts) (Chris Wright)
         . Fixed bug #69197 (openssl_pkcs7_sign handles default value incorrectly)
           (Daniel Lowrey)
         . Fixed bug #69215 (Crypto servers should send client CA list)
           (Daniel Lowrey)
         . Add a check for RAND_egd to allow compiling against LibreSSL (Leigh)

       - Phar:
         . Fixed bug #64343 (PharData::extractTo fails for tarball created by BSD tar).
           (Mike)
         . Fixed bug #64931 (phar_add_file is too restrictive on filename). (Mike)
         . Fixed bug #65467 (Call to undefined method cli_arg_typ_string). (Mike)
         . Fixed bug #67761 (Phar::mapPhar fails for Phars inside a path containing
           ".tar"). (Mike)
         . Fixed bug #69324 (Buffer Over-read in unserialize when parsing Phar). (Stas)
         . Fixed bug #69441 (Buffer Overflow when parsing tar/zip/phar in
           phar_set_inode). (Stas)

       - Postgres:
         . Fixed bug #68741 (Null pointer dereference). (CVE-2015-1352) (Laruence)

       - SPL:
         . Fixed bug #69227 (Use after free in zval_scan caused by
            spl_object_storage_get_gc). (adam dot scarr at 99designs dot com)

       - SOAP:
         . Fixed bug #69293 (NEW segfault when using SoapClient::__setSoapHeader
            (bisected, regression)). (Laruence)

       - Sqlite3:
         . Fixed bug #68760 (SQLITE segfaults if custom collator throws an exception).
            (Dan Ackroyd)
         . Fixed bug #69287 (Upgrade bundled libsqlite to 3.8.8.3). (Anatol)
         . Fixed bug #66550 (SQLite prepared statement use-after-free). (Sean Heelan)

commit fcea77f015634d2ceccf87aa8cb9d4d6f37c4fb8
Author: tron <tron>
Date:   Tue Apr 21 21:55:20 2015 +0000

    Pullup tickets #4676, #4677, #4678 and #4679.

commit 925ab7098bb6cc3ce910092b754e612ad6aad606
Author: tron <tron>
Date:   Tue Apr 21 21:53:52 2015 +0000

    Pullup ticket #4679 - requested by taca
    lang/php55: security update

    Revisions pulled up:
    - lang/php/phpversion.mk                                        1.95
    - lang/php55/distinfo                                           1.39

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Fri Apr 17 16:40:58 UTC 2015

       Modified Files:
       	pkgsrc/lang/php: phpversion.mk
       	pkgsrc/lang/php55: distinfo

       Log Message:
       Update php55 to 5.5.24.

       16 Apr 2015, PHP 5.5.24

       - Apache2handler:
         . Fixed bug #69218 (potential remote code execution with apache 2.4
           apache2handler). (Gerrit Venema)

       - Core:
         . Fixed bug #66609 (php crashes with __get() and ++ operator in some cases).
           (Dmitry, Laruence)
         . Fixed bug #67626 (User exceptions not properly handled in streams).
           (Julian)
         . Fixed bug #68021 (get_browser() browser_name_regex returns non-utf-8
           characters). (Tjerk)
         . Fixed bug #68917 (parse_url fails on some partial urls). (Wei Dai)
         . Fixed bug #69134 (Per Directory Values overrides PHP_INI_SYSTEM
           configuration options). (Anatol Belski)
         . Additional fix for bug #69152 (Type confusion vulnerability in
           exception::getTraceAsString). (Stas)
         . Fixed bug #69212 (Leaking VIA_HANDLER func when exception thrown in
           __call/... arg passing). (Nikita)
         . Fixed bug #69221 (Segmentation fault when using a generator in combination
           with an Iterator). (Nikita)
         . Fixed bug #69337 (php_stream_url_wrap_http_ex() type-confusion
           vulnerability). (Stas)
         . Fixed bug #69353 (Missing null byte checks for paths in various PHP
           extensions). (Stas)

       - Curl:
         . Implemented FR#69278 (HTTP2 support). (Masaki Kagaya)
         . Fixed bug #69316 (Use-after-free in php_curl related to
           CURLOPT_FILE/_INFILE/_WRITEHEADER). (Laruence)

       - Date:
         . Export date_get_immutable_ce so that it can be used by extensions. (Derick
           Rethans)
         . Fixed bug #69336 (Issues with "last day of <monthname>"). (Derick Rethans)

       - Enchant:
         . Fixed bug #65406 (Enchant broker plugins are in the wrong place in windows
           builds). (Anatol)

       - Fileinfo:
         . Fixed bug #68819 (Fileinfo on specific file causes spurious OOM and/or
           segfault). (Anatol Belski)

       - Filter:
         . Fixed bug #69202 (FILTER_FLAG_STRIP_BACKTICK ignored unless other
           flags are used). (Jeff Welch)
         . Fixed bug #69203 (FILTER_FLAG_STRIP_HIGH doesn't strip ASCII 127). (Jeff
           Welch)

       - Mbstring:
         . Fixed bug #68846 (False detection of CJK Unified Ideographs Extension E).
           (Masaki Kagaya)

       - OPCache
          . Fixed bug #68677 (Use After Free). (CVE-2015-1351) (Laruence)
          . Fixed bug #69281 (opcache_is_script_cached no longer works). (danack)

       - OpenSSL:
         . Fixed bug #67403 (Add signatureType to openssl_x509_parse).
         . Add a check for RAND_egd to allow compiling against LibreSSL (Leigh)

       - Phar:
         . Fixed bug #64343 (PharData::extractTo fails for tarball created by BSD tar).
           (Mike)
         . Fixed bug #64931 (phar_add_file is too restrictive on filename). (Mike)
         . Fixed bug #65467 (Call to undefined method cli_arg_typ_string). (Mike)
         . Fixed bug #67761 (Phar::mapPhar fails for Phars inside a path containing
           ".tar"). (Mike)
         . Fixed bug #69324 (Buffer Over-read in unserialize when parsing Phar). (Stas)
         . Fixed bug #69441 (Buffer Overflow when parsing tar/zip/phar in
           phar_set_inode). (Stas)

       - Postgres:
         . Fixed bug #68741 (Null pointer dereference). (CVE-2015-1352) (Laruence)

       - SPL:
         . Fixed bug #69227 (Use after free in zval_scan caused by
            spl_object_storage_get_gc). (adam dot scarr at 99designs dot com)

       - SOAP:
         . Fixed bug #69293 (NEW segfault when using SoapClient::__setSoapHeader
            (bisected, regression)). (thomas at shadowweb dot org, Laruence)

       - SQLITE:
         . Fixed bug #68760 (SQLITE segfaults if custom collator throws an exception).
            (Dan Ackroyd)
         . Fixed bug #69287 (Upgrade bundled sqlite to 3.8.8.3). (Anatol)

commit 06cfdb83dfa6add7a1890739e737320f7ca3044b
Author: tron <tron>
Date:   Tue Apr 21 21:50:09 2015 +0000

    Pullup ticket #4677 - requested by taca
    lang/php54: security update

    Revisions pulled up:
    - lang/php/phpversion.mk                                        1.94
    - lang/php54/distinfo                                           1.56

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Fri Apr 17 16:39:56 UTC 2015

       Modified Files:
       	pkgsrc/lang/php: phpversion.mk
       	pkgsrc/lang/php54: distinfo

       Log Message:
       Update php54 to 5.4.40.

       16 Apr 2015 PHP 5.4.40

       - Apache2handler:
         . Fixed bug #69218 (potential remote code execution with apache 2.4
           apache2handler). (Gerrit Venema)

       - Core:
         . Additional fix for bug #69152 (Type confusion vulnerability in
           exception::getTraceAsString). (Stas)
         . Fixed bug #69337 (php_stream_url_wrap_http_ex() type-confusion
           vulnerability). (Stas)
         . Fixed bug #69353 (Missing null byte checks for paths in various PHP
           extensions). (Stas)

       - cURL:
         . Fixed bug #69316 (Use-after-free in php_curl related to
           CURLOPT_FILE/_INFILE/_WRITEHEADER). (Laruence)

       - Ereg:
         . Fixed bug #68740 (NULL Pointer Dereference). (Laruence)

       - Fileinfo:
         . Fixed bug #68819 (Fileinfo on specific file causes spurious OOM and/or
           segfault). (Anatol Belski)

       - GD:
         . Fixed bug #68601 (buffer read overflow in gd_gif_in.c). (Remi)

       - Phar:
         . Fixed bug #68901 (use after free). (bugreports at internot dot info)
         . Fixed bug #69324 (Buffer Over-read in unserialize when parsing Phar). (Stas)
         . Fixed bug #69441 (Buffer Overflow when parsing tar/zip/phar in
           phar_set_inode). (Stas)

       - Postgres:
         . Fixed bug #68741 (Null pointer deference) (CVE-2015-1352). (Xinchen Hui)

       - SOAP:
         . Fixed bug #69152 (Type Confusion Infoleak Vulnerability in unserialize()
           with SoapFault). (Dmitry)

       - Sqlite3:
         . Fixed bug #66550 (SQLite prepared statement use-after-free). (Sean Heelan)

commit 259235aeb1391a7d418c7f05567d8fd849ecbf64
Author: tron <tron>
Date:   Tue Apr 21 21:44:22 2015 +0000

    Pullup ticket #4678 - requested by taca
    net/ntp4: security update

    Revisions pulled up:
    - net/ntp4/Makefile                                             1.85
    - net/ntp4/PLIST                                                1.18
    - net/ntp4/distinfo                                             1.21

    ---
       Module Name:	pkgsrc
       Committed By:	taca
       Date:		Wed Apr  8 03:31:34 UTC 2015

       Modified Files:
       	pkgsrc/net/ntp4: Makefile PLIST distinfo

       Log Message:
       Update ntp4 package to 4.2.8p2.

       NTP 4.2.8p2 (Harlan Stenn <stenn@ntp.org>, 2015/04/xx)

       Focus: Security and Bug fixes, enhancements.

       Severity: MEDIUM

       In addition to bug fixes and enhancements, this release fixes the
       following medium-severity vulnerabilities involving private key
       authentication:

       * [Sec 2779] ntpd accepts unauthenticated packets with symmetric key crypto.

           References: Sec 2779 / CVE-2015-1798 / VU#374268
           Affects: All NTP4 releases starting with ntp-4.2.5p99 up to but not
       	including ntp-4.2.8p2 where the installation uses symmetric keys
       	to authenticate remote associations.
           CVSS: (AV:A/AC:M/Au:N/C:P/I:P/A:P) Base Score: 5.4
           Date Resolved: Stable (4.2.8p2) 07 Apr 2015
           Summary: When ntpd is configured to use a symmetric key to authenticate
       	a remote NTP server/peer, it checks if the NTP message
       	authentication code (MAC) in received packets is valid, but not if
       	there actually is any MAC included. Packets without a MAC are
       	accepted as if they had a valid MAC. This allows a MITM attacker to
       	send false packets that are accepted by the client/peer without
       	having to know the symmetric key. The attacker needs to know the
       	transmit timestamp of the client to match it in the forged reply
       	and the false reply needs to reach the client before the genuine
       	reply from the server. The attacker doesn't necessarily need to be
       	relaying the packets between the client and the server.

       	Authentication using autokey doesn't have this problem as there is
       	a check that requires the key ID to be larger than NTP_MAXKEY,
       	which fails for packets without a MAC.
           Mitigation:
               Upgrade to 4.2.8p2, or later, from the NTP Project Download Page
       	or the NTP Public Services Project Download Page
               Configure ntpd with enough time sources and monitor it properly.
           Credit: This issue was discovered by Miroslav Lichvar, of Red Hat.

       * [Sec 2781] Authentication doesn't protect symmetric associations against
         DoS attacks.

           References: Sec 2781 / CVE-2015-1799 / VU#374268
           Affects: All NTP releases starting with at least xntp3.3wy up to but
       	not including ntp-4.2.8p2 where the installation uses symmetric
       	key authentication.
           CVSS: (AV:A/AC:M/Au:N/C:P/I:P/A:P) Base Score: 5.4
           Note: the CVSS base Score for this issue could be 4.3 or lower, and
       	it could be higher than 5.4.
           Date Resolved: Stable (4.2.8p2) 07 Apr 2015
           Summary: An attacker knowing that NTP hosts A and B are peering with
       	each other (symmetric association) can send a packet to host A
       	with source address of B which will set the NTP state variables
       	on A to the values sent by the attacker. Host A will then send
       	on its next poll to B a packet with originate timestamp that
       	doesn't match the transmit timestamp of B and the packet will
       	be dropped. If the attacker does this periodically for both
       	hosts, they won't be able to synchronize to each other. This is
       	a known denial-of-service attack, described at
       	https://www.eecis.udel.edu/~mills/onwire.html .

       	According to the document the NTP authentication is supposed to
       	protect symmetric associations against this attack, but that
       	doesn't seem to be the case. The state variables are updated even
       	when authentication fails and the peers are sending packets with
       	originate timestamps that don't match the transmit timestamps on
       	the receiving side.

       	This seems to be a very old problem, dating back to at least
       	xntp3.3wy. It's also in the NTPv3 (RFC 1305) and NTPv4 (RFC 5905)
       	specifications, so other NTP implementations with support for
       	symmetric associations and authentication may be vulnerable too.
       	An update to the NTP RFC to correct this error is in-process.
           Mitigation:
               Upgrade to 4.2.8p2, or later, from the NTP Project Download Page
       	or the NTP Public Services Project Download Page
               Note that for users of autokey, this specific style of MITM attack
       	is simply a long-known potential problem.
               Configure ntpd with appropriate time sources and monitor ntpd.
       	Alert your staff if problems are detected.
           Credit: This issue was discovered by Miroslav Lichvar, of Red Hat.

       * New script: update-leap
       The update-leap script will verify and if necessary, update the
       leap-second definition file.
       It requires the following commands in order to work:

       	wget logger tr sed shasum

       Some may choose to run this from cron.  It needs more portability testing.

commit 94c1b5104684448d27261c42d658ac54910af9ac
Author: tron <tron>
Date:   Tue Apr 21 21:39:41 2015 +0000

    Pullup ticket #4676 - requested by tron
    net/tigervnc: build fix

    Revisions pulled up:
    - net/tigervnc/Makefile                                         1.4

    ---
       Module Name:	pkgsrc
       Committed By:	joerg
       Date:		Sat Apr 18 20:44:47 UTC 2015

       Modified Files:
       	pkgsrc/net/tigervnc: Makefile

       Log Message:
       Needs explicit libdrm dependency.

commit 9ac98dab09e6d06bb842b68ed5f6011621f1b13f
Author: tron <tron>
Date:   Tue Apr 21 19:07:46 2015 +0000

    Pullup tickets #4669, #4674 and #4675.

commit f4dd44867374ea54f7a364d6262a851839ca6aec
Author: tron <tron>
Date:   Tue Apr 21 19:07:14 2015 +0000

    Pullup ticket #4675 - requested by joerg
    games/ioquake3: build fix

    Revisions pulled up:
    - games/ioquake3/Makefile                                       1.4
    - games/ioquake3/PLIST                                          1.3
    - games/ioquake3/distinfo                                       1.5
    - games/ioquake3/patches/patch-aa                               1.3

    ---
       Module Name:	pkgsrc
       Committed By:	joerg
       Date:		Sat Apr 18 20:44:08 UTC 2015

       Modified Files:
       	pkgsrc/games/ioquake3: Makefile PLIST distinfo
       	pkgsrc/games/ioquake3/patches: patch-aa

       Log Message:
       Allow building on NetBSD/arm.

commit d3ef2ddfc6c5d191f3a29bb4af90aab71356dffa
Author: tron <tron>
Date:   Tue Apr 21 19:00:54 2015 +0000

    Pullup ticket #4674 - requested by joerg
    games/orbital_eunuchs_sniper: packaging fix

    Revisions pulled up:
    - games/orbital_eunuchs_sniper/Makefile                         1.23

    ---
       Module Name:	pkgsrc
       Committed By:	joerg
       Date:		Sat Apr 18 20:43:19 UTC 2015

       Modified Files:
       	pkgsrc/games/orbital_eunuchs_sniper: Makefile

       Log Message:
       Use uname -m output for NetBSD/ARM in the PLIST as the package does.

commit c3e33c1c17b50b1641b75315f564d4f914c8d49e
Author: tron <tron>
Date:   Tue Apr 21 18:55:45 2015 +0000

    Pullup ticket #4669 - requested by manu
    databases/php-ldap: build fix

    Revisions pulled up:
    - databases/php-ldap/files/ldap-ctrl-exop.patch                 deleted
    - databases/php-ldap/files/ldap-ctrl-exop54.patch               1.1-1.2
    - databases/php-ldap/files/ldap-ctrl-exop55.patch               1.1-1.2
    - databases/php-ldap/files/ldap-ctrl-exop56.patch               1.1-1.2
    - databases/php-ldap/options.mk                                 1.4

    ---
       Module Name:    pkgsrc
       Committed By:   manu
       Date:           Mon Apr 13 08:04:17 UTC 2015

       Modified Files:
               pkgsrc/databases/php-ldap: options.mk
       Added Files:
               pkgsrc/databases/php-ldap/files: ldap-ctrl-exop54.patch
                   ldap-ctrl-exop55.patch ldap-ctrl-exop56.patch
       Removed Files:
               pkgsrc/databases/php-ldap/files: ldap-ctrl-exop.patch

       Log Message:
       Update the LDAP EXOP support patch for PHP 5.4, 5.5 and 5.6

    ---
       Module Name:    pkgsrc
       Committed By:   manu
       Date:           Mon Apr 20 03:24:36 UTC 2015

       Modified Files:
               pkgsrc/databases/php-ldap/files: ldap-ctrl-exop54.patch
                   ldap-ctrl-exop55.patch ldap-ctrl-exop56.patch

       Log Message:
       Fix build with SASL disabled

       Do not force-define HAVE_LDAP_SASL, since the PHP configure script
       actually takes care of that one. This lets us build with the exop
       option enabled while sasl is disabled.

commit a5f8d21945363b03bec896fb6af2714201ab919e
Merge: 7d962d8 faf0e76
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Tue Apr 21 14:40:27 2015 +0100

    Merge branch 'joyent/feature/miscfix/2015Q1' into joyent/release/2015Q1

commit faf0e763cdbe20b49f8215cb58530dff6e44d97f
Merge: 86554d2 a295034
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Tue Apr 21 14:40:13 2015 +0100

    Merge branch 'joyent/feature/miscfix/2015Q1' of github.com:joyent/pkgsrc into joyent/feature/miscfix/2015Q1

commit 7d962d8f53c204790d4d7120d6e6dbfe107a7af6
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Tue Apr 21 14:39:38 2015 +0100

    Update 'joyent' and 'wip' submodules.

commit b163d85dc024e7f042e22cfb6f1fc3cc5279114b
Merge: 952134b 86554d2
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Tue Apr 21 14:39:13 2015 +0100

    Merge branch 'joyent/feature/miscfix/2015Q1' into joyent/release/2015Q1

commit 86554d2c5a721a99fd2e7c4b5862b35b69918090
Merge: 0b4c611 6996bfa
Author: Jonathan Perkin <jperkin@joyent.com>
Date:   Tue Apr 21 14:38:23 2015 +0100

    Merge branch 'pkgsrc_2015Q1' into joyent/feature/miscfix/2015Q1

commit a295034d9249a35784c34cc387513f5de3c73570
Author: Filip Hajny <filip@joyent.com>
Date:   Mon Apr 20 17:09:03 2015 +0000

    Update sysutils/cfengine3 to 3.6.5. Add lmdb support option.

    Upstream changelog follows.

    3.6.5

    Features:
    - Introduced "systemd" hard class.
    - Added paths to dtrace, zfs and zpool on FreeBSD in masterfiles.

    Bug fixes:
    - Fixed build error on certain RHEL5 and SLES10 setups.
    - Fixed a bug which caused dangling symlinks not to be removed.
    - Fixed data_readstringarrayidx function not preserving the order of the
      array it's producing.
    - Fixed a bug which sometimes caused CFEngine to kill the wrong daemon
      if both the host and a container inside the host were running
      CFEngine.
    - Made sure the rm_rf_depth bundle also deletes the base directory.
    - Fixed monitord reporting wrongly on open ports.
    - Skip adding the class when its name is longer than 1024 characters.
      Fixed core dump when the name is too large.
    - Fixed detection of stopped process on Solaris.
    - Fixed infinite loop plus a couple more minor
      bugs in edit_xml promises.

    3.6.4

    Features:
    - Introduced users promises support on HP-UX platform.
    - Introduced process promises support on HP-UX platform.

    Bug fixes:
    - Fixed bug on FreeBSD which sometimes led to the wrong process being killed
    - Fixed package version comparison sometimes failing with rpm package manager
    - Fixed a bug in users promises which would sometimes set the wrong
      password hash if the user would also be unlocked at the same time.
    - Fixed a bug on AIX which would occationally kill the wrong process.
    - Improved error message for functions that require an absolute path.
    - Fixed some spelling errors in examples.
    - Fixed error in out-of-tree builds when building cf-upgrade.
    - Fixed a bug which would make cf-agent exit with an error if it was
      built with a custom log directory, and that directory did not exist.
    - Fixed ordering of evaluating promises when depends_on is used.
    - Skip non-empty directories silently when recursively deleting.
    - Fix memory exhaustion with list larger than 4994 items.
    - Fix cf-execd segfault on IP address detection
    - Fix hard class detection of RHEL6 ComputeNode

    3.6.3

    New features:
    - support for HP-UX 11.23 and later
    - experimental support for Red Hat Enterprise Linux 7

    Bug fixes:
    - fix getindices on multi-dimensional arrays
    - fix mustache template method to run in dryrun mode
    - set mailto and mailfrom settings for execd in def.cf
    - fix conflation of multi-index entries in arrays
    - fix promise locking when transferring using update.cf
    - update JSON parser to return an error on truncation
    - fix sys.hardware_addresses not expanded
    - fix opening database txn /var/cfengine/cf_lastseen.lmdb:
      MDB_READERS_FULL when running cf-keys --show-hosts
    - fix segfault (Null pointer dereference) when select_end in
      delete_lines never matches
    - fix max_file_size => "0" not disabling or allowing any size
     - fix ifvarclass, with iteration over list, failing when deleting
      files with time condition
    - fix classes defined with "or" constraint are never set if any value
      doesn't evaluate to a scalar
    - update "mailfrom" default in default policy
    - fix logrotate ambiguity of filename
    - fix parsing JSON files
    - reduce write count activity to /var partition
    - fix files delete attribute incorrectly triggering promise_kept
    - update services bundle output related to chkconfig when run in
      inform mode.
    - fix Solaris serverd tests
    - fix broken bechaviour of merging arrays with readstringarray
    - fix ifelapsed bug with bundle nesting
    - fix handling cf_null in bundlesequence
    - fix maparray reading whole input array when using subarray
     - fix directories being randomly changed to files
    - update defaults promise type to work with classes
    - systemd integration in services promises
    - fix touch attribute ignoring action = warn_only
    - fix 4KB string limit in functions readfile, string_downcase,
      string_head, string_reverse, string_length, string_tail,
      string_upcase

    3.6.2

    Bug fixes:
    - don't regenerate software_packages.csv every time
    - improve verbose message for package_list_command
    - fix missing log output on AIX
    - assorted fixes to dirname() esp on Windows
    - fix package manager detection
    - fix build issues on FreeBSD
    - allow copying of dead symbolic links
    - preserve order in readstringarrayidx
    - fix passing of unexpanded variable references to arrays
     - use entries for new {admin,deny}_{ips,hostnames} constraints in
      the relevant legacy lists
    - cope with ps's numeric fields overflowing to the right
    - interpret failing function calls in ifvarclass as class not set
     - remove unexpanded lists when extending lists
    - infer start-time of a process from elapsed when needed
     - fix input range definition for laterthan() function
    - don't add trailing delimiter when join()'ing lists ending with a
      null-value
    - 9999999999 (ten 9s) or higher has been historically used as an upper
      bound in CFEngine code and policy but because of overflow on 32-bit
      platforms it caused problems with big numbers. Fixed in two ways:
      first change all existing policy uses to 999999999 (nine 9s instead
      of eleven 9s), second fix the C code to not wrap-around in case of
      overflow, but use the LONG_MAX value
    - cf-serverd and other daemons no longer reload their configuration
      every minute if CFEngine is built with an inputs directory outside
      of the work directory (not the default).

    3.6.1

    New features:
    - Introduced Solaris and AIX support into the 3.6 series, with many associated
      build and bug fixes.

    Changes:
    - Short-circuit evaluation of classes promises if class is already set
     - fix to assume all non-specified return codes are failed in commands
      promises
    - cf-serverd logs reconfiguration message to NOTICE (was INFO) so that it's
      always logged in syslog

    Bug fixes:
    - File monitoring has been completely rewritten (changes attribute in files
      promise), which eliminates many bugs, particularly regarding files that
      are deleted. Upgrading will keep all monitoring data, but downgrading again
      will reinitialize the DB, so all files will be reported as if they were new.
     - $(this.promiser) expands in files promises for 'transformer',
      'edit_template', 'copy_from.source', 'file_select.exec_program', 'classes'
      and 'action' bodies
    - 'body changes' notifies about disappeared files in file monitoring
    - Set not-kept classes when files or commands promise should be repaired,
      but is warn-only
    - Fixed CFEngine template producing a zero-sized file
    - Add 0-9 A-Z _ to allowed context of module protocol
    - Extend ps command column width on Solaris and filter on zone
      rather than adding it to the ps output.
    - Fixed strftime() function on Solaris when called with certain specifiers.
    - Fixed users promise bug regarding password hashes in a NIS/NSS setup.
    - Fixed $(sys.uptime), $(sys.systime) and $(sys.sysday) in AIX.
      #5148, #5206)
    - Fixed processes_select complaining about "Unacceptable model uncertainty
      examining processes"
    - ps command for linux has been changed to cope with big rss values
    - Address ps -axo shift on FreeBSD 10 and later
    - methods and services promises respect action_policy => "warn"
     - LMDB should no longer deadlock if an agent is killed on the hub while
      holding the DB lock. Note that the change only affects binary packages
      shipped by CFEngine, since the upstream LMDB project has not yet integrated
      the change.

    3.6.0

    Changes:
    - Changes to logging output
        - add process name and pid in syslog message (GitHub #789)
        - cf-serverd logging levels are now more standardised:
              - INFO logs only failures
              - VERBOSE logs successful requests as well
              - DEBUG logs actual protocol traffic.
        - cf-serverd now logs the relevant client IP address on
          each message.
        - Logging contexts to local database (cf_classes.tcdb) has been deprecated.
        - 'usebundle' promisees are logged for all the bundle promises
        - output from 'reports' promises has nothing prefixed except 'R: '
        - a log line with stack path is generated when the promise type evaluated
          changes
    - LMDB (symas.com/mdb) is the default database for local data storage : use
      version 0.9.9 or later. cf-agent --self-diagnostics (-x) is only
      implemented for TCDB, not for LMDB
    - port argument in readtcp() and selectservers() may be a
      service name (e.g. "http", "pop3").
    - Enable source file in agent copy_from promises to be a relative path.
        - file "changes" reporting now reports with log level "notice", instead
          of "error".
    - process_results default to AND'ing of set attributes if not specified
     - interface is now canonified in sys.hardware_mac[interface] to align with
      sys.ipv4[interface]
    - cf-promises no longer errors on missing bodies when run without
      --full-check (-c)
    - Linux flavor "SUSE" now correctly spelled with all uppercase in variables
      and class names. The "suse" lowercase version is also
      provided for convenience.
    - $(this.promise_filename) and $(..._dirname) variables are now absolute
      paths.
    - including the same file multiple times in 'body control inputs' is
      not an error
    - portnumber in body copy_from now supports service names like "cfengine",
      "pop3" etc, check /etc/services for more.
    - The failsafe.cf policy, run on bootstrap and in some other
      unusual cases, has been extracted from C code into libpromises/failsafe.cf
    - masterfiles
        - cf_promises_validated is now in JSON format
        - timestamp key is timestamp (sec since unix epoch) of last time validated
        - the masterfiles now come from https://github.com/cfengine/masterfiles
          and are not in the core repository
    - cf-serverd calls cf-agent with -Dcfruncommand when executing cf-runagent
      requests
    Mark as removed: promise_notkept_log_include, promise_notkept_log_exclude,
    promise_repaired_log_include, promise_repaired_log_exclude, classes_include,
    classes_exclude, variables_include, variables_exclude attributes from
    report_data_select body (syntax is valid but not functional). They have been
    replaced by the following attributes: promise_handle_include,
    promise_handle_exclude, metatags_include, metatags_exclude.

    New features:
    - New promise type "users" for managing local user accounts.
    - TLS authentication and fully encrypted network protocol.
      Additions specific to the new type of connections:
        - New attribute "allowlegacyconnects" in body server control,
          which enables serving policy via non-latest cfengine protocol,
          to the given list of hosts. If the option is absent, it
          defaults to allow all hosts. To refuse non-TLS connections,
          specify an empty list.
        - New attribute "protocol_version" in body copy_from, and body
          common control, which defines the preferred protocol for
          outgoing connections.. Allowed values at the moment: "0" or
          "undefined", "classic" or "1", "latest" or "2". By leaving the
          copy_from option as undefined the common control option is
          used, and if both are undefined then classic protocol is used
          by default.
        - The new networking protocol uses TLS for authentication,
          after which all dialog is encrypted within the established
          TLS session.  cf-serverd is still able to speak the legacy
          protocol with old agents.
        - The 'skipverify' option in 'body server control' is
          deprecated and only left for compatibility; it does
          nothing
        - cf-serverd does not hang up the connection if some request
          fails, so that the client can add more requests.
        - For the connections using the new protocol, all of the
          paths in bundle server access_rules now differentiate
          between a directory and a file using the trailing
          slash. If the path exists then this is auto-detected and
          trailing slash appended automatically. You have to append
          a trailing slash manually to an inexistent or symbolic
          path (e.g. "/path/to/$(connection.ip)/") to force
          recursive access.
    - New in 'access' promises for 'bundle server access_rules'
        - Attributes "admit_ips", "admit_hostnames", "admit_keys",
          "deny_ips", "deny_hostnames", "deny_keys"
        - "admit_keys" and "deny_keys" add the new functionality
          of controlling access according to host identity,
          regardless of the connecting IP.
        - For these new attributes, regular expressions
          are not allowed, only CIDR notation for "admit/deny_ips", exact
          "SHA=..." strings for "admit/deny_keys", and exact hostnames
          (e.g. "cfengine.com") or subdomains (starting with dot,
          e.g. ".cfengine.com") for "admit/deny"_hostnames. Same rules
          apply to 'deny_*' attributes.
        - These new constraints and the paths in access_rules, can contain
          special variables "$(connection.ip)", "$(connection.hostname)",
          "$(connection.key)", which are expanded dynamically for every
          received connection.
        - For connections using the new protocol, "admit" and "deny"
          constraints in bundle server access_rules are being phased
          out, preferred attributes are now "admit_ips", "deny_ips",
          "admit_hostnames", "deny_hostnames", "admit_keys",
          "deny_keys".
        - New "shortcut" attribute in bundle server access_rules used to
          dynamically expand non-absolute request paths.
    - masterfiles
            - standard library split: lib/3.5 (compatibility) and lib/3.6
              (mainline)
            - many standard library bundles and bodies, especially packages-
              and file-related, were revised and fixed
            - supports both Community and Enterprise
            - new 'inventory/' structure to provide OS, dmidecode, LSB, etc.
              system inventory (configured mainly in def.cf)
            - cf_promises_release_id contains the policy release ID which is
              the GIT HEAD SHA if available or hash of tree
            - a bunch'o'bundles to make starting with CFEngine easier:
            - file-related: file_mustache, file_mustache_jsonstring, file_tidy,
              dir_sync, file_copy, file_link, file_hardlink, file_empty, file_make
            - packages-related: package_absent, package_present, package_latest,
              package_specific_present, package_specific_absent,
              package_specific_latest, package_specific
            - XML-related: xml_insert_tree_nopath, xml_insert_tree, xml_set_value,
              xml_set_attribute
            - VCS-related: git_init, git_add, git_checkout,
              git_checkout_new_branch, git_clean, git_stash, git_stash_and_clean,
              git_commit, git
            - process-related: process_kill
            - other: cmerge, url_ping, logrotate, prunedir
    - New command line options for agent binaries
        - New options to cf-promises
            - '--show-classes' and '--show-vars'
            - '--eval-functions' controls whether cf-promises should evaluate
              functions
        - Colorized output for agent binaries with command line option '--color'
          (auto-enabled if you set CFENGINE_COLOR=1)
    - New language features
        - New variable type 'data' for handling of structured data (ie JSON),
          including supporting functions:
            - 'data_readstringarray' - read a delimited file into a data map
            - 'data_readstringarrayidx' - read a delimited file into a data array
            - 'datastate' - create a data variable with currently set classes
              and variables
            - 'datatype' - determine the type of the top element of a container
            - 'format' - %S can be used to serialize 'data' containers into
              a string
            - 'mergedata' - merge two data containers, slists/ilists/rlists,
              or "classic" arrays into a data container
            - 'parsejson' - create a data container from a JSON string
            - 'readjson' - create a data container from a file that contains JSON
            - 'storejson' - serialize a data container into a string
            - Most functions operating on lists can also operate on data containers
            - pass a data container to a bundle with the @(container) notation
            - the module protocol accepts JSON for data containers with
              the '%' sigil
        - Tagging of classes and variables allows annotating of language
          construct with meta data; supporting functionality:
            - The module protocol in 'commands' promises has been extended
              to allow setting of tags of created variables and classes,
              and the context of created variables
            - 'getclassmetatags' - returns list of meta tags for a class
            - 'getvariablemetatags' - returns list of meta tags for a variable
        - 'body file control' has an 'inputs' attribute to include library files
          and other dependencies
        - bundlesequences can be built with bundlesmatching() based on bundle
          name and tags
    - New attributes in existing promise types and bodies
        - New option 'preserve_all_lines' for insert_type in insert_lines promises
        - Caching of expensive system functions to avoid multiple executions of
          execresult() etc, can be controlled via cache_system_functions
          attribute in body common control
        - New option 'mailsubject' in body executor control allows defining
          the subject in emails sent by CFEngine
        - Support for Mustache templates in 'files' promises; use
          'template_method' and 'template_data' attributes.  Without
          'template_data' specified, uses datastate().
    - New and improved functions
        - 'bundlesmatching' - returns list of defined bundles matching
          a regex and tags
        - 'canonifyuniquely' - converts a string into a unique, legal class name
        - 'classesmatching' - returns list of set classes matching a regex
          and tags
        - 'eval' - evaluates mathematical expressions; knows SI k, m, g
          quantifiers, e.g. "100k"
        - 'findfiles' - list files matching a search pattern; use "**" for
          recursive searches
        - 'makerule' - evaluates whether a target file needs to be rebuilt
          from sources
        - 'max', 'min' - returns maximum and minimum of the numbers in
          a container or list (sorted by a 'sort' method)
        - 'mean' - returns the mean of the numbers in a container or list
        - 'nth' - learned to look up by key in a data container holding a map
        - 'packagesmatching' - returns a filtered list of installed packages.
        - 'readfile' - learned to read system files of unknown size like those
          in /proc
        - 'sort' - can sort lexicographically, numerically (int or real), by IP,
          or by MAC
        - 'string_downcase', 'string_upcase' - returns the lower-/upper-case
          version of a string
        - 'string_head', 'string_tail' - returns the beginning/end of a string
        - 'string_length' - returns the length of a string
        - 'string_reverse' - reverses a string
        - 'string_split' - improved implementation, deprecates 'splitstring'
        - 'variablesmatching' - returns a list of variables matching a regex
          and tags
        - 'variance' - returns the variance of numbers in a list or container
    - New hard classes
        - Introduced alias 'policy_server' for context 'am_policy_hub'
          (the latter will be deprecated)
        - all the time-based classes have GMT equivalents
    - New variables
        - 'sys.bindir' - the location of the CFEngine binaries
        - 'sys.failsafe_policy_path' - the location of the failsafe policy file
        - 'sys.inputdir' - the directory where CFEngine searches for policy files
        - 'sys.key_digest' - the digest of the host's cryptographic key
        - 'sys.libdir', 'sys.local_libdir' - the location of the CFEngine libraries
        - 'sys.logdir' - the directory where the CFEngine log files are saved
        - 'sys.masterdir' - the location of masterfiles on the policy server
        - 'sys.piddir' - the directory where the daemon pid files are saved
        - 'sys.sysday' - the number of days since the beginning of the UNIX epoch
        - 'sys.systime' - the number of seconds since the beginning of the UNIX epoch
        - 'sys.update_policy_path' - the name of the update policy file
        - 'sys.uptime' - the number of minutes the host has been online
        - 'this.promise_dirname' - the name of the file in which the current promise
          is defined
        - 'this.promiser_uid' - the ID of the user running cf-agent
        - 'this.promiser_gid' - the group ID of the user running cf-agent
        - 'this.promiser_ppid' - the ID of the  parent process running cf-agent

    Deprecations:
    - 'splitstring' - deprecated by 'string_split'
    - 'track_value'
    - 'skipverify'

    Bug fixes: for a complete list of fixed bugs, see https://cfengine.com/dev
    - various fixes in evaluation and variable resolution
    - Improve performance of list iteration
    - Removed limitation of input length to internal buffer sizes
        - directories ending with "/" are not ignored
        - lsdir() always return a list now, never a scalar
    - 'abortclasses' fixed to work in common bundles and other cases
    - namespaced 'edit_line' bundles now work
    - lists are interpolated in correct order
    - cf-serverd reloads policies properly when they change
    - lots of leaks (memory and file descriptor) fixed

    3.5.3

    Changes:
    - Improved security checks of symlink ownership. A symlink created by a user
      pointing to resources owned by a different user will no longer be followed.
    - Changed the way package versions are compared in package promises.
      In previous versions the comparison was inconsistent. This has been fixed,
      but may also lead to behavior changes in certain cases.

    Bug fixes:
    - fix cf-monitord crash due to incorrect array initialization
    - fix cf-serverd stat()'ing the file tree every second
    - correctly populate sys.hardware_addresses variable
    - add support for Debian's GNU/kfreebsd to build system
    - fix possible stack corruption in guest_environments promises
    - work-around hostname trunctation in HP-UX's uname
    - fix body copy purging of empty directories
    - make discovery and loading of avahi libraries more robust
    - compile and packaging fixes for HP-UX, AIX and Solaris
    - fix fatal error in lsdir() when directory doesn't exist
    - fix epoch calculation for stime inrange calculation

    3.5.2

    Bug fixes:
    - fix delayed abortclasses checking
    - fix maplist arguments bug
    - fix segfaults in cf-pomises
    - fix build on Solaris 10/SmartOS
    - sanitize characters from /etc/issue in sys.flavor for Debian
    - Fix segfault when dealing with files or data > 4K
    - Don't truncate keys to 126 characters in getindices
    - files created via log_* actions now have mode 600
    - fix wrong log message when a promise is ignored due to 'ifvarclass'
      not matching
    - fix lifetime of persistent classes
    - fix segfault when process_select body had no process_result attribute
      Default to AND'ed expression of all specified attributes
    - include system message in output when acl promises fail
    - fix invocation of standard_services bundle and corresponding promise
      compliance

    3.5.1

    Changes:
    - file changes are logged with log level Notice, not Error
    - the CFEngine Standard Library in masterfiles/libraries is now split into
      promise-type specific policy files, and lives in a version-specific
      directory. This should have no impact on current code, but allows more
      granular include of needed stdlib elements

    Bug fixes:
    - fix recursive copying of files
    - respect classes in templates
    - fix timestamps on Windows
    - fix non-root cf-agent flooding syslog
    - fix email flood from cf-execd due to timestamps in agent output
    - Preserve security context when editing or copying local files
    - fix path for sys.crontab on redhat systems
    - prevent incorrect "insert_lines promise uses the same select_line_matching
      anchor" warning
    - Fix regression of setting VIPADDRESS to 127.0.0.1
    - Fix "changes" promise not receiving status when file is missing
    - Fix symlinks being destroyed when editing them
    - Fix missing "promise kept" status for the last line in a file

    3.5.0

    New features:
    - classes promises now take an optional scope constraint.
    - new built-in functions: every, none, some, nth, sublist, uniq, filter
      - every
      - none
      - some
      - nth
      - sublist
      - uniq
      - filter
      - classesmatching
      - strftime
      - filestat
      - ifelse
      - maparray
      - format
    - cf-promises flag --parse-tree is replaced by --policy-output-format=,
      requiring the
      user to specify the output format (none, cf, json)
    - cf-promises allows partial check of policy (without body common control)
      without integrity check; --full-check enforces integrity check
    - agent binaries support JSON input format (.json file as generated by
      cf-promises)
    - cf-key: new options --trust-key/-t and --print-digest/-p
    - Class "failsafe_fallback" is defined in failsafe.cf when main policy
      contains errors and failsafe is run because of this
    - add scope attribute for body classes
    - Better diagnostics of parsing errors
    - Error messages from parser now show the context of error
    - new cf-agent option: --self-diagnostics
    - new output format, and --legacy-output
    - warnings for cf-promises.
    - Enable zeroconf-discovery of policy hubs for automatic bootstrapping
      if Avahi is present
    - Support for sys.cpus on more platforms than Linux & HPUX

    Changes:
    - parser no longer allows ',' after promiser or promisee. must be either
      ';' or lval
    - Make parser output in GCC compatible format the only supported format
      (remove --gcc-brief-format flag)
    - Silence license warnings in Enterprise Free25 installations
    - action_policy => "warn" causes not_kept classes to be set on promise
      needing repair.
    - command line option version (-V) now prints a shorter parsable version
      without graphic
    - implicit execution of server and common bundles taking arguments is
      skipped in cf-serverd.
    - WARNING: option --policy-server removed, require option to --bootstrap
      instead
    - process promises don't log if processes are out of rang…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants