r70952 - in /branches/upstream/libencode-locale-perl: ./ current/ current/Changes current/MANIFEST current/META.yml current/Makefile.PL current/README current/lib/ current/lib/Encode/ current/lib/Encode/Locale.pm current/t/ current/t/arg.t current/t/env.t
periapt-guest at users.alioth.debian.org
periapt-guest at users.alioth.debian.org
Wed Mar 9 14:56:46 UTC 2011
Author: periapt-guest
Date: Wed Mar 9 14:56:33 2011
New Revision: 70952
URL: http://svn.debian.org/wsvn/pkg-perl/?sc=1&rev=70952
Log:
[svn-inject] Installing original source of libencode-locale-perl (1.01)
Added:
branches/upstream/libencode-locale-perl/
branches/upstream/libencode-locale-perl/current/
branches/upstream/libencode-locale-perl/current/Changes
branches/upstream/libencode-locale-perl/current/MANIFEST
branches/upstream/libencode-locale-perl/current/META.yml
branches/upstream/libencode-locale-perl/current/Makefile.PL
branches/upstream/libencode-locale-perl/current/README
branches/upstream/libencode-locale-perl/current/lib/
branches/upstream/libencode-locale-perl/current/lib/Encode/
branches/upstream/libencode-locale-perl/current/lib/Encode/Locale.pm
branches/upstream/libencode-locale-perl/current/t/
branches/upstream/libencode-locale-perl/current/t/arg.t
branches/upstream/libencode-locale-perl/current/t/env.t
Added: branches/upstream/libencode-locale-perl/current/Changes
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/Changes?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/Changes (added)
+++ branches/upstream/libencode-locale-perl/current/Changes Wed Mar 9 14:56:33 2011
@@ -1,0 +1,66 @@
+## 2011-02-22 Release 1.01
+
+Avoid compilation warnings (RT#65975). Thanks to Goro Fuji.
+
+
+## 2011-01-23 Release 1.00
+
+Documentation tweaks.
+
+
+## 2010-10-23 Release 0.04
+
+Look up the ANSI code page on Windows and alias this one as the "locale"
+encoding.
+
+Providide the reinit() function to be called if something is changed
+in the locale and you need to re-initialize the encodings set up by
+this module.
+
+Improved documentation.
+
+
+## 2010-10-20 Release 0.03
+
+Changed the default for decode_argv() to match Encode's default.
+It became too strange to tell people to pass FB_DEFAULT to get the
+non-default behaviour.
+
+Changed $ENCODING_FS into $ENCODING_LOCALE_FS (as already documented below),
+but not implemented as such.
+
+Workaround for test failure where the Encode does not know about the "646"
+encoding alias.
+
+Documentation tweaks.
+
+
+## 2010-10-13 Release 0.02
+
+...where I realized that I could not get away with a single locale encoding.
+Now `Encode::Locale` provides 4 encoding names that often will map to the same
+underlying encoding. I've used the following names:
+
+ locale $ENCODING_LOCALE
+ locale_fs $ENCODING_LOCALE_FS
+ console_in $ENCODING_CONSOLE_IN
+ console_out $ENCODING_CONSOLE_OUT
+
+The first one is the encoding specified by the POSIX locale (or the equivalent
+on Windows). This can be set by the user. The second one (`locale_fs`) is the
+encoding that should be used when interfacing with the file system, that is the
+encoding of file names. For some systems (like Mac OS X) this is fixed system
+wide and the same for all users. Last; some systems allow the input and output
+encoding for data aimed at the console to differ so there are separate entries
+for these. For classic POSIX systems all 4 of these will all denote the same
+encoding.
+
+This release also introduce the function env() as a Unicode interface to the
+%ENV hash (the process environment variables). We don't want to decode the ENV
+%values in-place because this also affects what the child processes
+observes. The %ENV hash should always contain byte strings.
+
+
+## 2010-10-11 Release 0.01
+
+Initial release
Added: branches/upstream/libencode-locale-perl/current/MANIFEST
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/MANIFEST?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/MANIFEST (added)
+++ branches/upstream/libencode-locale-perl/current/MANIFEST Wed Mar 9 14:56:33 2011
@@ -1,0 +1,8 @@
+Changes
+lib/Encode/Locale.pm
+Makefile.PL
+MANIFEST This list of files
+README
+t/arg.t
+t/env.t
+META.yml Module meta-data (added by MakeMaker)
Added: branches/upstream/libencode-locale-perl/current/META.yml
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/META.yml?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/META.yml (added)
+++ branches/upstream/libencode-locale-perl/current/META.yml Wed Mar 9 14:56:33 2011
@@ -1,0 +1,29 @@
+--- #YAML:1.0
+name: Encode-Locale
+version: 1.01
+abstract: Determine the locale encoding
+author:
+ - Gisle Aas <gisle at activestate.com>
+license: perl
+distribution_type: module
+configure_requires:
+ ExtUtils::MakeMaker: 0
+build_requires:
+ Test: 0
+requires:
+ Encode: 2
+ Encode::Alias: 0
+ perl: 5.008
+resources:
+ repository: http://github.com/gisle/encode-locale
+no_index:
+ directory:
+ - t
+ - inc
+generated_by: ExtUtils::MakeMaker version 6.56
+meta-spec:
+ url: http://module-build.sourceforge.net/META-spec-v1.4.html
+ version: 1.4
+recommends:
+ I18N::Langinfo: 0
+ Win32::Console: 0
Added: branches/upstream/libencode-locale-perl/current/Makefile.PL
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/Makefile.PL?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/Makefile.PL (added)
+++ branches/upstream/libencode-locale-perl/current/Makefile.PL Wed Mar 9 14:56:33 2011
@@ -1,0 +1,50 @@
+require 5.008;
+use ExtUtils::MakeMaker;
+
+WriteMakefile(
+ NAME => 'Encode::Locale',
+ VERSION_FROM => 'lib/Encode/Locale.pm',
+ ABSTRACT_FROM => "lib/Encode/Locale.pm",
+ AUTHOR => 'Gisle Aas <gisle at activestate.com>',
+ LICENSE => "perl",
+ MIN_PERL_VERSION => 5.008,
+ PREREQ_PM => {
+ 'Encode' => 2,
+ 'Encode::Alias' => 0,
+ },
+ META_MERGE => {
+ resources => {
+ repository => 'http://github.com/gisle/encode-locale',
+ },
+ recommends => {
+ 'I18N::Langinfo' => 0,
+ 'Win32::Console' => 0,
+ },
+ },
+ BUILD_REQUIRES => {
+ Test => 0,
+ },
+);
+
+BEGIN {
+ # compatibility with older versions of MakeMaker
+ my $developer = -d ".git";
+ my %mm_req = (
+ LICENCE => 6.31,
+ META_MERGE => 6.45,
+ META_ADD => 6.45,
+ MIN_PERL_VERSION => 6.48,
+ BUILD_REQUIRES => 6.56,
+ );
+ undef(*WriteMakefile);
+ *WriteMakefile = sub {
+ my %arg = @_;
+ for (keys %mm_req) {
+ unless (eval { ExtUtils::MakeMaker->VERSION($mm_req{$_}) }) {
+ warn "$_ $@" if $developer;
+ delete $arg{$_};
+ }
+ }
+ ExtUtils::MakeMaker::WriteMakefile(%arg);
+ };
+}
Added: branches/upstream/libencode-locale-perl/current/README
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/README?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/README (added)
+++ branches/upstream/libencode-locale-perl/current/README Wed Mar 9 14:56:33 2011
@@ -1,0 +1,11 @@
+Encode-Locale
+==============
+
+The purpose of this Perl module is try determine what encodings should be used
+when interfacing to various external interfaces. You will need perl-5.8 or
+better to use this module.
+
+© 2010 Gisle Aas `<gisle at aas.no>`.
+
+This library is free software; you can redistribute it and/or
+modify it under the same terms as Perl itself.
Added: branches/upstream/libencode-locale-perl/current/lib/Encode/Locale.pm
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/lib/Encode/Locale.pm?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/lib/Encode/Locale.pm (added)
+++ branches/upstream/libencode-locale-perl/current/lib/Encode/Locale.pm Wed Mar 9 14:56:33 2011
@@ -1,0 +1,324 @@
+package Encode::Locale;
+
+use strict;
+our $VERSION = "1.01";
+
+use base 'Exporter';
+our @EXPORT_OK = qw(
+ decode_argv env
+ $ENCODING_LOCALE $ENCODING_LOCALE_FS
+ $ENCODING_CONSOLE_IN $ENCODING_CONSOLE_OUT
+);
+
+use Encode ();
+use Encode::Alias ();
+
+our $ENCODING_LOCALE;
+our $ENCODING_LOCALE_FS;
+our $ENCODING_CONSOLE_IN;
+our $ENCODING_CONSOLE_OUT;
+
+sub DEBUG () { 0 }
+
+sub _init {
+ if ($^O eq "MSWin32") {
+ # Try to obtain what the Windows ANSI code page is
+ eval {
+ require Win32::API;
+ if (Win32::API->Import('kernel32', 'int GetACP()')) {
+ my $cp = GetACP();
+ $ENCODING_LOCALE = "cp$cp" if $cp;
+ }
+ };
+
+ # If we have the Win32::Console module installed we can ask
+ # it for the code set to use
+ eval {
+ require Win32::Console;
+ my $cp = Win32::Console::InputCP();
+ $ENCODING_CONSOLE_IN = "cp$cp" if $cp;
+ $cp = Win32::Console::OutputCP();
+ $ENCODING_CONSOLE_OUT = "cp$cp" if $cp;
+ };
+ # Invoking the 'chcp' program might also work
+ if (!$ENCODING_CONSOLE_IN && qx(chcp) =~ /^Active code page: (\d+)/) {
+ $ENCODING_CONSOLE_IN = "cp$1";
+ }
+ }
+
+ unless ($ENCODING_LOCALE) {
+ eval {
+ require I18N::Langinfo;
+ $ENCODING_LOCALE = I18N::Langinfo::langinfo(I18N::Langinfo::CODESET());
+
+ # Workaround of Encode < v2.25. The "646" encoding alias was
+ # introducted in Encode-2.25, but we don't want to require that version
+ # quite yet. Should avoid the CPAN testers failure reported from
+ # openbsd-4.7/perl-5.10.0 combo.
+ $ENCODING_LOCALE = "ascii" if $ENCODING_LOCALE eq "646";
+ };
+ $ENCODING_LOCALE ||= $ENCODING_CONSOLE_IN;
+ }
+
+ if ($^O eq "darwin") {
+ $ENCODING_LOCALE_FS ||= "UTF-8";
+ }
+
+ # final fallback
+ $ENCODING_LOCALE ||= $^O eq "MSWin32" ? "cp1252" : "UTF-8";
+ $ENCODING_LOCALE_FS ||= $ENCODING_LOCALE;
+ $ENCODING_CONSOLE_IN ||= $ENCODING_LOCALE;
+ $ENCODING_CONSOLE_OUT ||= $ENCODING_CONSOLE_IN;
+
+ unless (Encode::find_encoding($ENCODING_LOCALE)) {
+ die "The locale codeset ($ENCODING_LOCALE) isn't one that perl can decode, stopped";
+ }
+}
+
+_init();
+Encode::Alias::define_alias(sub {
+ no strict 'refs';
+ no warnings 'once';
+ return ${"ENCODING_" . uc(shift)};
+}, "locale");
+
+sub _flush_aliases {
+ no strict 'refs';
+ for my $a (keys %Encode::Alias::Alias) {
+ if (defined ${"ENCODING_" . uc($a)}) {
+ delete $Encode::Alias::Alias{$a};
+ warn "Flushed alias cache for $a" if DEBUG;
+ }
+ }
+}
+
+sub reinit {
+ $ENCODING_LOCALE = shift;
+ $ENCODING_LOCALE_FS = shift;
+ $ENCODING_CONSOLE_IN = $ENCODING_LOCALE;
+ $ENCODING_CONSOLE_OUT = $ENCODING_LOCALE;
+ _init();
+ _flush_aliases();
+}
+
+sub decode_argv {
+ die if defined wantarray;
+ for (@ARGV) {
+ $_ = Encode::decode(locale => $_, @_);
+ }
+}
+
+sub env {
+ my $k = Encode::encode(locale => shift);
+ my $old = $ENV{$k};
+ if (@_) {
+ my $v = shift;
+ if (defined $v) {
+ $ENV{$k} = Encode::encode(locale => $v);
+ }
+ else {
+ delete $ENV{$k};
+ }
+ }
+ return Encode::decode(locale => $old) if defined wantarray;
+}
+
+1;
+
+__END__
+
+=head1 NAME
+
+Encode::Locale - Determine the locale encoding
+
+=head1 SYNOPSIS
+
+ use Encode::Locale;
+ use Encode;
+
+ $string = decode(locale => $bytes);
+ $bytes = encode(locale => $string);
+
+ if (-t) {
+ binmode(STDIN, ":encoding(console_in)");
+ binmode(STDOUT, ":encoding(console_out)");
+ binmode(STDERR, ":encoding(console_out)");
+ }
+
+ # Processing file names passed in as arguments
+ my $uni_filename = decode(locale => $ARGV[0]);
+ open(my $fh, "<", encode(locale_fs => $uni_filename))
+ || die "Can't open '$uni_filename': $!";
+ binmode($fh, ":encoding(locale)");
+ ...
+
+=head1 DESCRIPTION
+
+In many applications it's wise to let Perl use Unicode for the strings it
+processes. Most of the interfaces Perl has to the outside world is still byte
+based. Programs therefore needs to decode byte strings that enter the program
+from the outside and encode them again on the way out.
+
+The POSIX locale system is used to specify both the language conventions
+requested by the user and the preferred character set to consume and
+output. The C<Encode::Locale> module looks up the charset and encoding (called
+a CODESET in the locale jargon) and arrange for the L<Encode> module to know
+this encoding under the name "locale". It means bytes obtained from the
+environment can be converted to Unicode strings by calling C<<
+Encode::encode(locale => $bytes) >> and converted back again with C<<
+Encode::decode(locale => $string) >>.
+
+Where file systems interfaces pass file names in and out of the program we also
+need care. The trend is for operating systems to use a fixed file encoding
+that don't actually depend on the locale; and this module determines the most
+appropriate encoding for file names. The L<Encode> module will know this
+encoding under the name "locale_fs". For traditional Unix systems this will
+be an alias to the same encoding as "locale".
+
+For programs running in a terminal window (called a "Console" on some systems)
+the "locale" encoding is usually a good choice for what to expect as input and
+output. Some systems allows us to query the encoding set for the terminal and
+C<Encode::Locale> will do that if available and make these encodings known
+under the C<Encode> aliases "console_in" and "console_out". For systems where
+we can't determine the terminal encoding these will be aliased as the same
+encoding as "locale". The advice is to use "console_in" for input known to
+come from the terminal and "console_out" for output known to go from the
+terminal.
+
+In addition to arranging for various Encode aliases the following functions and
+variables are provided:
+
+=over
+
+=item decode_argv( )
+
+=item decode_argv( Encode::FB_CROAK )
+
+This will decode the command line arguments to perl (the C<@ARGV> array) in-place.
+
+The function will by default replace characters that can't be decoded by
+"\x{FFFD}", the Unicode replacement character.
+
+Any argument provided is passed as CHECK to underlying Encode::decode() call.
+Pass the value C<Encode::FB_CROAK> to have the decoding croak if not all the
+command line arguments can be decoded. See L<Encode/"Handling Malformed Data">
+for details on other options for CHECK.
+
+=item env( $uni_key )
+
+=item env( $uni_key => $uni_value )
+
+Interface to get/set environment variables. Returns the current value as a
+Unicode string. The $uni_key and $uni_value arguments are expected to be
+Unicode strings as well. Passing C<undef> as $uni_value deletes the
+environment variable named $uni_key.
+
+The returned value will have the characters that can't be decoded replaced by
+"\x{FFFD}", the Unicode replacement character.
+
+There is no interface to request alternative CHECK behavior as for
+decode_argv(). If you need that you need to call encode/decode yourself.
+For example:
+
+ my $key = Encode::encode(locale => $uni_key, Encode::FB_CROAK);
+ my $uni_value = Encode::decode(locale => $ENV{$key}, Encode::FB_CROAK);
+
+=item reinit( )
+
+=item reinit( $encoding )
+
+Reinitialize the encodings from the locale. You want to call this function if
+changed anything in the environment that might influence the locale.
+
+This function will croak if the determined encoding isn't recognized by
+the Encode module.
+
+With argument force $ENCODING_... variables to set to the given value.
+
+=item $ENCODING_LOCALE
+
+The encoding name determined to be suitable for the current locale.
+L<Encode> know this encoding as "locale".
+
+=item $ENCODING_LOCALE_FS
+
+The encoding name determined to be suiteable for file system interfaces
+involving file names.
+L<Encode> know this encoding as "locale_fs".
+
+=item $ENCODING_CONSOLE_IN
+
+=item $ENCODING_CONSOLE_OUT
+
+The encodings to be used for reading and writing output to the a console.
+L<Encode> know these encodings as "console_in" and "console_out".
+
+=back
+
+=head1 NOTES
+
+This table summarizes the mapping of the encodings set up
+by the C<Encode::Locale> module:
+
+ Encode | | |
+ Alias | Windows | Mac OS X | POSIX
+ ------------+---------+--------------+------------
+ locale | ANSI | nl_langinfo | nl_langinfo
+ locale_fs | ANSI | UTF-8 | nl_langinfo
+ console_in | OEM | nl_langinfo | nl_langinfo
+ console_out | OEM | nl_langinfo | nl_langinfo
+
+=head2 Windows
+
+Windows has basically 2 sets of APIs. A wide API (based on passing UTF-16
+strings) and a byte based API based a character set called ANSI. The
+regular Perl interfaces to the OS currently only uses the ANSI APIs.
+Unfortunately ANSI is not a single character set.
+
+The encoding that corresponds to ANSI varies between different editions of
+Windows. For many western editions of Windows ANSI corresponds to CP-1252
+which is a character set similar to ISO-8859-1. Conceptually the ANSI
+character set is a similar concept to the POSIX locale CODESET so this module
+figures out what the ANSI code page is and make this available as
+$ENCODING_LOCALE and the "locale" Encoding alias.
+
+Windows systems also operate with another byte based character set.
+It's called the OEM code page. This is the encoding that the Console
+takes as input and output. It's common for the OEM code page to
+differ from the ANSI code page.
+
+=head2 Mac OS X
+
+On Mac OS X the file system encoding is always UTF-8 while the locale
+can otherwise be set up as normal for POSIX systems.
+
+File names on Mac OS X will at the OS-level be converted to
+NFD-form. A file created by passing a NFC-filename will come
+in NFD-form from readdir(). See L<Unicode::Normalize> for details
+of NFD/NFC.
+
+Actually, Apple does not follow the Unicode NFD standard since not all
+character ranges are decomposed. The claim is that this avoids problems with
+round trip conversions from old Mac text encodings. See L<Encode::UTF8Mac> for
+details.
+
+=head2 POSIX (Linux and other Unixes)
+
+File systems might vary in what encoding is to be used for
+filenames. Since this module has no way to actually figure out
+what the is correct it goes with the best guess which is to
+assume filenames are encoding according to the current locale.
+Users are adviced to always specify UTF-8 as the locale charset.
+
+=head1 SEE ALSO
+
+L<I18N::Langinfo>, L<Encode>
+
+=head1 AUTHOR
+
+Copyright 2010 Gisle Aas <gisle at aas.no>.
+
+This library is free software; you can redistribute it and/or
+modify it under the same terms as Perl itself.
+
+=cut
Added: branches/upstream/libencode-locale-perl/current/t/arg.t
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/t/arg.t?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/t/arg.t (added)
+++ branches/upstream/libencode-locale-perl/current/t/arg.t Wed Mar 9 14:56:33 2011
@@ -1,0 +1,30 @@
+#!perl -w
+
+use strict;
+use Test;
+plan tests => 1;
+
+use Encode::Locale qw($ENCODING_LOCALE decode_argv);
+
+print "# ENCODING_LOCALE is $ENCODING_LOCALE\n";
+decode_argv();
+
+my $i;
+for my $arg (@ARGV) {
+ print "# ", ++$i, ": \"";
+ for (split(//, $arg)) {
+ if (ord() > 32 && ord() < 128) {
+ print $_;
+ }
+ elsif (ord() < 256) {
+ printf "\\x%02X", ord();
+ }
+ else {
+ printf "\\x{%04X}", ord();
+ }
+ }
+ print "\"\n";
+}
+
+# fake it :-)
+ok(1);
Added: branches/upstream/libencode-locale-perl/current/t/env.t
URL: http://svn.debian.org/wsvn/pkg-perl/branches/upstream/libencode-locale-perl/current/t/env.t?rev=70952&op=file
==============================================================================
--- branches/upstream/libencode-locale-perl/current/t/env.t (added)
+++ branches/upstream/libencode-locale-perl/current/t/env.t Wed Mar 9 14:56:33 2011
@@ -1,0 +1,25 @@
+#!perl -w
+
+use strict;
+use Test;
+plan tests => 13;
+
+use Encode::Locale qw(env);
+
+$ENV{foo} = "bar";
+ok(env("foo"), "bar");
+ok(env("foo", "baz"), "bar");
+ok(env("foo"), "baz");
+ok($ENV{foo}, "baz");
+ok(env("foo", undef), "baz");
+ok(env("foo"), undef);
+ok(!exists $ENV{foo});
+
+Encode::Locale::reinit("cp1252");
+$ENV{"m\xf6ney"} = "\x80uro";
+ok(env("m\xf6ney", "\x{20AC}"), "\x{20AC}uro");
+ok(env("m\xf6ney"), "\x{20AC}");
+ok($ENV{"m\xf6ney"}, "\x80");
+ok(env("\x{20AC}", 1), undef);
+ok(env("\x{20AC}"), 1);
+ok($ENV{"\x80"}, 1);
More information about the Pkg-perl-cvs-commits
mailing list