r55449 - in /trunk/libjson-perl: Changes MANIFEST META.yml README debian/changelog eg/ lib/JSON.pm lib/JSON/PP.pm t/e14_decode_prefix.t
gregoa at users.alioth.debian.org
gregoa at users.alioth.debian.org
Sun Apr 4 16:39:48 UTC 2010
Author: gregoa
Date: Sun Apr 4 16:39:31 2010
New Revision: 55449
URL: http://svn.debian.org/wsvn/pkg-perl/?sc=1&rev=55449
Log:
New upstream release.
Added:
trunk/libjson-perl/eg/
- copied from r55448, branches/upstream/libjson-perl/current/eg/
trunk/libjson-perl/t/e14_decode_prefix.t
- copied unchanged from r55448, branches/upstream/libjson-perl/current/t/e14_decode_prefix.t
Modified:
trunk/libjson-perl/Changes
trunk/libjson-perl/MANIFEST
trunk/libjson-perl/META.yml
trunk/libjson-perl/README
trunk/libjson-perl/debian/changelog
trunk/libjson-perl/lib/JSON.pm
trunk/libjson-perl/lib/JSON/PP.pm
Modified: trunk/libjson-perl/Changes
URL: http://svn.debian.org/wsvn/pkg-perl/trunk/libjson-perl/Changes?rev=55449&op=diff
==============================================================================
--- trunk/libjson-perl/Changes (original)
+++ trunk/libjson-perl/Changes Sun Apr 4 16:39:31 2010
@@ -15,6 +15,15 @@
!! Since 2.16, PP's relaxed option caused an infinite loop in some condition.
!! Recommend to update old versions.
+
+2.20 Fri Apr 2 12:50:08 2010
+ [JSON]
+ - added eg/bench_pp_xs.pl for benchmark sample
+ - updated 'INCREMENTAL PARSING' section
+ [JSON::PP]
+ - decode_prefix() didn't count a consumed text length properly.
+ - enhanced XS compatibilty
+ in the case of decoding a white space garbaged text.
2.19 Tue Mar 30 13:40:24 2010
[JSON]
Modified: trunk/libjson-perl/MANIFEST
URL: http://svn.debian.org/wsvn/pkg-perl/trunk/libjson-perl/MANIFEST?rev=55449&op=diff
==============================================================================
--- trunk/libjson-perl/MANIFEST (original)
+++ trunk/libjson-perl/MANIFEST Sun Apr 4 16:39:31 2010
@@ -1,4 +1,5 @@
Changes
+eg/bench_pp_xs.pl
lib/JSON.pm
lib/JSON/PP.pm
lib/JSON/PP/Boolean.pm
@@ -47,6 +48,7 @@
t/e11_conv_blessed_univ.t
t/e12_upgrade.t
t/e13_overloaded_eq.t
+t/e14_decode_prefix.t
t/x00_load.t
t/x02_error.t
t/x12_blessed.t
Modified: trunk/libjson-perl/META.yml
URL: http://svn.debian.org/wsvn/pkg-perl/trunk/libjson-perl/META.yml?rev=55449&op=diff
==============================================================================
--- trunk/libjson-perl/META.yml (original)
+++ trunk/libjson-perl/META.yml Sun Apr 4 16:39:31 2010
@@ -1,6 +1,6 @@
--- #YAML:1.0
name: JSON
-version: 2.19
+version: 2.20
abstract: JSON (JavaScript Object Notation) encoder/decoder
author:
- Makamaka Hannyaharamitu, E<lt>makamaka[at]cpan.orgE<gt>
Modified: trunk/libjson-perl/README
URL: http://svn.debian.org/wsvn/pkg-perl/trunk/libjson-perl/README?rev=55449&op=diff
==============================================================================
--- trunk/libjson-perl/README (original)
+++ trunk/libjson-perl/README Sun Apr 4 16:39:31 2010
@@ -1,4 +1,4 @@
-JSON version 2.19
+JSON version 2.20
=================
INSTALLATION
@@ -45,7 +45,7 @@
# recommend to use (en|de)code_json.
VERSION
- 2.19
+ 2.20
This version is compatible with JSON::XS 2.27 and later.
@@ -266,7 +266,7 @@
# open( my $fh, '<:encoding(cp932)', 'json.data' );
# $json_text = <$fh>;
- In this case, $json_text is UNICODE string. So you can not use
+ In this case, $json_text is UNICODE string. So you cannot use
"decode_json" nor "JSON" module object with "utf8" enable. Instead of
them, you use "JSON" module object with "utf8" disable or "from_json".
@@ -286,13 +286,18 @@
print $json->utf8->encode( $perl_scalar );
And if $perl_scalar does not contain UNICODE, then your characters are
- latin1 for perl. So you can not use "encode_json" nor "JSON" module
+ latin1 for perl. So you cannot use "encode_json" nor "JSON" module
object with "utf8" enable. Instead of them, you use "JSON" module object
with "utf8" disable or "to_json".
- $outer_json_text = $json->utf8(0)->encode( $perl_scalar );
+ $json_text = $json->utf8(0)->encode( $perl_scalar );
# or
- $outer_json_text = to_json( $perl_scalar );
+ $json_text = to_json( $perl_scalar );
+
+ # and then print or convert to UNICODE
+ print $json_text;
+
+ $unicode_json_text = encode( $encoding, $json_text );
See to Encode, perluniintro.
@@ -808,71 +813,109 @@
$flag_hashref = $json->property();
INCREMENTAL PARSING
- In JSON::XS 2.2, incremental parsing feature of JSON texts was
- implemented. Please check to "INCREMENTAL PARSING" in JSON::XS.
-
- [void, scalar or list context] = $json->incr_parse ([$string])
- This is the central parsing function. It can both append new text
- and extract objects from the stream accumulated so far (both of
- these functions are optional).
-
- If $string is given, then this string is appended to the already
- existing JSON fragment stored in the $json object.
-
- After that, if the function is called in void context, it will
- simply return without doing anything further. This can be used to
- add more text in as many chunks as you want.
-
- If the method is called in scalar context, then it will try to
- extract exactly *one* JSON object. If that is successful, it will
- return this object, otherwise it will return "undef". If there is a
- parse error, this method will croak just as "decode" would do (one
- can then use "incr_skip" to skip the errornous part). This is the
- most common way of using the method.
-
- And finally, in list context, it will try to extract as many objects
- from the stream as it can find and return them, or the empty list
- otherwise. For this to work, there must be no separators between the
- JSON objects or arrays, instead they must be concatenated
- back-to-back. If an error occurs, an exception will be raised as in
- the scalar context case. Note that in this case, any
- previously-parsed JSON texts will be lost.
-
- $lvalue_string = $json->incr_text
- This method returns the currently stored JSON fragment as an lvalue,
- that is, you can manipulate it. This *only* works when a preceding
- call to "incr_parse" in *scalar context* successfully returned an
- object. Under all other circumstances you must not call this
- function (I mean it. although in simple tests it might actually
- work, it *will* fail under real world conditions). As a special
- exception, you can also call this method before having parsed
- anything.
-
- This function is useful in two cases: a) finding the trailing text
- after a JSON object or b) parsing multiple JSON objects separated by
- non-JSON text (such as commas).
-
- In Perl 5.005, "lvalue" attribute is not available. You must write
- codes like the below:
-
- $string = $json->incr_text;
- $string =~ s/\s*,\s*//;
- $json->incr_text( $string );
-
- $json->incr_skip
- This will reset the state of the incremental parser and will remove
- the parsed text from the input buffer. This is useful after
- "incr_parse" died, in which case the input buffer and incremental
- parser state is left unchanged, to skip the text parsed so far and
- to reset the parse state.
-
- $json->incr_reset
- This completely resets the incremental parser, that is, after this
- call, it will be as if the parser had never parsed anything.
-
- This is useful if you want ot repeatedly parse JSON objects and want
- to ignore any trailing data, which means you have to reset the
- parser after each successful decode.
+ Most of this section are copied and modified from "INCREMENTAL PARSING"
+ in JSON::XS.
+
+ In some cases, there is the need for incremental parsing of JSON texts.
+ This module does allow you to parse a JSON stream incrementally. It does
+ so by accumulating text until it has a full JSON object, which it then
+ can decode. This process is similar to using "decode_prefix" to see if a
+ full JSON object is available, but is much more efficient (and can be
+ implemented with a minimum of method calls).
+
+ The backend module will only attempt to parse the JSON text once it is
+ sure it has enough text to get a decisive result, using a very simple
+ but truly incremental parser. This means that it sometimes won't stop as
+ early as the full parser, for example, it doesn't detect parenthese
+ mismatches. The only thing it guarantees is that it starts decoding as
+ soon as a syntactically valid JSON text has been seen. This means you
+ need to set resource limits (e.g. "max_size") to ensure the parser will
+ stop parsing in the presence if syntax errors.
+
+ The following methods implement this incremental parser.
+
+ incr_parse
+ $json->incr_parse( [$string] ) # void context
+
+ $obj_or_undef = $json->incr_parse( [$string] ) # scalar context
+
+ @obj_or_empty = $json->incr_parse( [$string] ) # list context
+
+ This is the central parsing function. It can both append new text and
+ extract objects from the stream accumulated so far (both of these
+ functions are optional).
+
+ If $string is given, then this string is appended to the already
+ existing JSON fragment stored in the $json object.
+
+ After that, if the function is called in void context, it will simply
+ return without doing anything further. This can be used to add more text
+ in as many chunks as you want.
+
+ If the method is called in scalar context, then it will try to extract
+ exactly *one* JSON object. If that is successful, it will return this
+ object, otherwise it will return "undef". If there is a parse error,
+ this method will croak just as "decode" would do (one can then use
+ "incr_skip" to skip the errornous part). This is the most common way of
+ using the method.
+
+ And finally, in list context, it will try to extract as many objects
+ from the stream as it can find and return them, or the empty list
+ otherwise. For this to work, there must be no separators between the
+ JSON objects or arrays, instead they must be concatenated back-to-back.
+ If an error occurs, an exception will be raised as in the scalar context
+ case. Note that in this case, any previously-parsed JSON texts will be
+ lost.
+
+ Example: Parse some JSON arrays/objects in a given string and return
+ them.
+
+ my @objs = JSON->new->incr_parse ("[5][7][1,2]");
+
+ incr_text
+ $lvalue_string = $json->incr_text
+
+ This method returns the currently stored JSON fragment as an lvalue,
+ that is, you can manipulate it. This *only* works when a preceding call
+ to "incr_parse" in *scalar context* successfully returned an object.
+ Under all other circumstances you must not call this function (I mean
+ it. although in simple tests it might actually work, it *will* fail
+ under real world conditions). As a special exception, you can also call
+ this method before having parsed anything.
+
+ This function is useful in two cases: a) finding the trailing text after
+ a JSON object or b) parsing multiple JSON objects separated by non-JSON
+ text (such as commas).
+
+ $json->incr_text =~ s/\s*,\s*//;
+
+ In Perl 5.005, "lvalue" attribute is not available. You must write codes
+ like the below:
+
+ $string = $json->incr_text;
+ $string =~ s/\s*,\s*//;
+ $json->incr_text( $string );
+
+ incr_skip
+ $json->incr_skip
+
+ This will reset the state of the incremental parser and will remove the
+ parsed text from the input buffer. This is useful after "incr_parse"
+ died, in which case the input buffer and incremental parser state is
+ left unchanged, to skip the text parsed so far and to reset the parse
+ state.
+
+ incr_reset
+ $json->incr_reset
+
+ This completely resets the incremental parser, that is, after this call,
+ it will be as if the parser had never parsed anything.
+
+ This is useful if you want ot repeatedly parse JSON objects and want to
+ ignore any trailing data, which means you have to reset the parser after
+ each successful decode.
+
+ See to "INCREMENTAL PARSING" in JSON::XS for examples.
JSON::PP SUPPORT METHODS
The below methods are JSON::PP own methods, so when "JSON" works with
Modified: trunk/libjson-perl/debian/changelog
URL: http://svn.debian.org/wsvn/pkg-perl/trunk/libjson-perl/debian/changelog?rev=55449&op=diff
==============================================================================
--- trunk/libjson-perl/debian/changelog (original)
+++ trunk/libjson-perl/debian/changelog Sun Apr 4 16:39:31 2010
@@ -1,3 +1,9 @@
+libjson-perl (2.20-1) UNRELEASED; urgency=low
+
+ * New upstream release.
+
+ -- gregor herrmann <gregoa at debian.org> Sun, 04 Apr 2010 18:38:24 +0200
+
libjson-perl (2.19-1) unstable; urgency=low
* New upstream release
Modified: trunk/libjson-perl/lib/JSON.pm
URL: http://svn.debian.org/wsvn/pkg-perl/trunk/libjson-perl/lib/JSON.pm?rev=55449&op=diff
==============================================================================
--- trunk/libjson-perl/lib/JSON.pm (original)
+++ trunk/libjson-perl/lib/JSON.pm Sun Apr 4 16:39:31 2010
@@ -7,7 +7,7 @@
@JSON::EXPORT = qw(from_json to_json jsonToObj objToJson encode_json decode_json);
BEGIN {
- $JSON::VERSION = '2.19';
+ $JSON::VERSION = '2.20';
$JSON::DEBUG = 0 unless (defined $JSON::DEBUG);
}
@@ -608,7 +608,7 @@
=head1 VERSION
- 2.19
+ 2.20
This version is compatible with JSON::XS B<2.27> and later.
@@ -845,7 +845,7 @@
# $json_text = <$fh>;
In this case, C<$json_text> is UNICODE string.
-So you can B<not> use C<decode_json> nor C<JSON> module object with C<utf8> enable.
+So you B<cannot> use C<decode_json> nor C<JSON> module object with C<utf8> enable.
Instead of them, you use C<JSON> module object with C<utf8> disable or C<from_json>.
$perl_scalar = $json->utf8(0)->decode( $json_text );
@@ -863,12 +863,18 @@
print $json->utf8->encode( $perl_scalar );
And if C<$perl_scalar> does not contain UNICODE, then your characters are latin1 for perl.
-So you can B<not> use C<encode_json> nor C<JSON> module object with C<utf8> enable.
+So you B<cannot> use C<encode_json> nor C<JSON> module object with C<utf8> enable.
Instead of them, you use C<JSON> module object with C<utf8> disable or C<to_json>.
- $outer_json_text = $json->utf8(0)->encode( $perl_scalar );
+ $json_text = $json->utf8(0)->encode( $perl_scalar );
# or
- $outer_json_text = to_json( $perl_scalar );
+ $json_text = to_json( $perl_scalar );
+
+ # and then print or convert to UNICODE
+ print $json_text;
+
+ $unicode_json_text = encode( $encoding, $json_text );
+
See to L<Encode>, L<perluniintro>.
@@ -1419,12 +1425,33 @@
=head1 INCREMENTAL PARSING
-In JSON::XS 2.2, incremental parsing feature of JSON texts was implemented.
-Please check to L<JSON::XS/INCREMENTAL PARSING>.
-
-=over 4
-
-=item [void, scalar or list context] = $json->incr_parse ([$string])
+Most of this section are copied and modified from L<JSON::XS/INCREMENTAL PARSING>.
+
+In some cases, there is the need for incremental parsing of JSON texts.
+This module does allow you to parse a JSON stream incrementally.
+It does so by accumulating text until it has a full JSON object, which
+it then can decode. This process is similar to using C<decode_prefix>
+to see if a full JSON object is available, but is much more efficient
+(and can be implemented with a minimum of method calls).
+
+The backend module will only attempt to parse the JSON text once it is sure it
+has enough text to get a decisive result, using a very simple but
+truly incremental parser. This means that it sometimes won't stop as
+early as the full parser, for example, it doesn't detect parenthese
+mismatches. The only thing it guarantees is that it starts decoding as
+soon as a syntactically valid JSON text has been seen. This means you need
+to set resource limits (e.g. C<max_size>) to ensure the parser will stop
+parsing in the presence if syntax errors.
+
+The following methods implement this incremental parser.
+
+=head2 incr_parse
+
+ $json->incr_parse( [$string] ) # void context
+
+ $obj_or_undef = $json->incr_parse( [$string] ) # scalar context
+
+ @obj_or_empty = $json->incr_parse( [$string] ) # list context
This is the central parsing function. It can both append new text and
extract objects from the stream accumulated so far (both of these
@@ -1452,7 +1479,13 @@
case. Note that in this case, any previously-parsed JSON texts will be
lost.
-=item $lvalue_string = $json->incr_text
+Example: Parse some JSON arrays/objects in a given string and return them.
+
+ my @objs = JSON->new->incr_parse ("[5][7][1,2]");
+
+=head2 incr_text
+
+ $lvalue_string = $json->incr_text
This method returns the currently stored JSON fragment as an lvalue, that
is, you can manipulate it. This I<only> works when a preceding call to
@@ -1466,6 +1499,8 @@
JSON object or b) parsing multiple JSON objects separated by non-JSON text
(such as commas).
+ $json->incr_text =~ s/\s*,\s*//;
+
In Perl 5.005, C<lvalue> attribute is not available.
You must write codes like the below:
@@ -1473,14 +1508,18 @@
$string =~ s/\s*,\s*//;
$json->incr_text( $string );
-=item $json->incr_skip
+=head2 incr_skip
+
+ $json->incr_skip
This will reset the state of the incremental parser and will remove the
parsed text from the input buffer. This is useful after C<incr_parse>
died, in which case the input buffer and incremental parser state is left
unchanged, to skip the text parsed so far and to reset the parse state.
-=item $json->incr_reset
+=head2 incr_reset
+
+ $json->incr_reset
This completely resets the incremental parser, that is, after this call,
it will be as if the parser had never parsed anything.
@@ -1489,7 +1528,8 @@
ignore any trailing data, which means you have to reset the parser after
each successful decode.
-=back
+See to L<JSON::XS/INCREMENTAL PARSING> for examples.
+
=head1 JSON::PP SUPPORT METHODS
Modified: trunk/libjson-perl/lib/JSON/PP.pm
URL: http://svn.debian.org/wsvn/pkg-perl/trunk/libjson-perl/lib/JSON/PP.pm?rev=55449&op=diff
==============================================================================
--- trunk/libjson-perl/lib/JSON/PP.pm (original)
+++ trunk/libjson-perl/lib/JSON/PP.pm Sun Apr 4 16:39:31 2010
@@ -11,7 +11,7 @@
use B ();
#use Devel::Peek;
-$JSON::PP::VERSION = '2.27002';
+$JSON::PP::VERSION = '2.27003';
@JSON::PP::EXPORT = qw(encode_json decode_json from_json to_json);
@@ -662,6 +662,7 @@
# $opt flag
# 0x00000001 .... decode_prefix
+ # 0x10000000 .... incr_parse
sub PP_decode_json {
my ($self, $opt); # $opt is an effective flag during this decode_json.
@@ -670,8 +671,8 @@
($at, $ch, $depth) = (0, '', 0);
- if (!defined $text or ref $text) {
- decode_error("malformed text data.");
+ if ( !defined $text or ref $text ) {
+ decode_error("malformed JSON string, neither array, object, number, string or atom");
}
my $idx = $self->{PROPS};
@@ -710,30 +711,34 @@
: (!$octets[2] ) ? 'UTF-32LE'
: 'unknown';
-# my $result = value();
-
- my $eof = !( my ( $result ) = value() ); # $eof for incr_parse
-
- if ( $eof && ( $opt & 0x00000001 ) ) {
- return undef;
- }
-
- if (!$idx->[ P_ALLOW_NONREF ] and !ref $result) {
+ white(); # remove head white space
+
+ my $valid_start = defined $ch; # Is there a first character for JSON structure?
+
+ my $result = value();
+
+ return undef if ( !$result && ( $opt & 0x10000000 ) ); # for incr_parse
+
+ decode_error("malformed JSON string, neither array, object, number, string or atom") unless $valid_start;
+
+ if ( !$idx->[ P_ALLOW_NONREF ] and !ref $result ) {
decode_error(
'JSON text must be an object or array (but found number, string, true, false or null,'
. ' use allow_nonref to allow this)', 1);
}
- if ($len >= $at) {
- my $consumed = $at - 1;
- white();
- if ($ch) {
- decode_error("garbage after JSON object") unless ($opt & 0x00000001);
- return ($result, $consumed);
- }
- }
-
- $result;
+ Carp::croak('something wrong.') if $len < $at; # we won't arrive here.
+
+ my $consumed = defined $ch ? $at - 1 : $at; # consumed JSON text length
+
+ white(); # remove tail white space
+
+ if ( $ch ) {
+ return ( $result, $consumed ) if ($opt & 0x00000001); # all right if decode_prefix
+ decode_error("garbage after JSON object");
+ }
+
+ ( $opt & 0x00000001 ) ? ( $result, $consumed ) : $result;
}
@@ -1481,7 +1486,7 @@
$self->{incr_p} = $restore;
$self->{incr_c} = $p;
- my ( $obj, $tail ) = $coder->decode_prefix( substr( $self->{incr_text}, 0, $p ) );
+ my ( $obj, $tail ) = $coder->PP_decode_json( substr( $self->{incr_text}, 0, $p ), 0x10000001 );
$self->{incr_text} = substr( $self->{incr_text}, $p );
$self->{incr_p} = 0;
@@ -1838,16 +1843,35 @@
($perl_scalar, $characters) = $json->decode_prefix($json_text)
-
=head1 INCREMENTAL PARSING
-In JSON::XS 2.2, incremental parsing feature of JSON
-texts was experimentally implemented.
-Please check to L<JSON::XS/INCREMENTAL PARSING>.
-
-=over 4
-
-=item [void, scalar or list context] = $json->incr_parse ([$string])
+Most of this section are copied and modified from L<JSON::XS/INCREMENTAL PARSING>.
+
+In some cases, there is the need for incremental parsing of JSON texts.
+This module does allow you to parse a JSON stream incrementally.
+It does so by accumulating text until it has a full JSON object, which
+it then can decode. This process is similar to using C<decode_prefix>
+to see if a full JSON object is available, but is much more efficient
+(and can be implemented with a minimum of method calls).
+
+This module will only attempt to parse the JSON text once it is sure it
+has enough text to get a decisive result, using a very simple but
+truly incremental parser. This means that it sometimes won't stop as
+early as the full parser, for example, it doesn't detect parenthese
+mismatches. The only thing it guarantees is that it starts decoding as
+soon as a syntactically valid JSON text has been seen. This means you need
+to set resource limits (e.g. C<max_size>) to ensure the parser will stop
+parsing in the presence if syntax errors.
+
+The following methods implement this incremental parser.
+
+=head2 incr_parse
+
+ $json->incr_parse( [$string] ) # void context
+
+ $obj_or_undef = $json->incr_parse( [$string] ) # scalar context
+
+ @obj_or_empty = $json->incr_parse( [$string] ) # list context
This is the central parsing function. It can both append new text and
extract objects from the stream accumulated so far (both of these
@@ -1875,7 +1899,13 @@
case. Note that in this case, any previously-parsed JSON texts will be
lost.
-=item $lvalue_string = $json->incr_text
+Example: Parse some JSON arrays/objects in a given string and return them.
+
+ my @objs = JSON->new->incr_parse ("[5][7][1,2]");
+
+=head2 incr_text
+
+ $lvalue_string = $json->incr_text
This method returns the currently stored JSON fragment as an lvalue, that
is, you can manipulate it. This I<only> works when a preceding call to
@@ -1889,6 +1919,8 @@
JSON object or b) parsing multiple JSON objects separated by non-JSON text
(such as commas).
+ $json->incr_text =~ s/\s*,\s*//;
+
In Perl 5.005, C<lvalue> attribute is not available.
You must write codes like the below:
@@ -1896,15 +1928,27 @@
$string =~ s/\s*,\s*//;
$json->incr_text( $string );
-=item $json->incr_skip
+=head2 incr_skip
+
+ $json->incr_skip
This will reset the state of the incremental parser and will remove the
parsed text from the input buffer. This is useful after C<incr_parse>
died, in which case the input buffer and incremental parser state is left
unchanged, to skip the text parsed so far and to reset the parse state.
-=back
-
+=head2 incr_reset
+
+ $json->incr_reset
+
+This completely resets the incremental parser, that is, after this call,
+it will be as if the parser had never parsed anything.
+
+This is useful if you want ot repeatedly parse JSON objects and want to
+ignore any trailing data, which means you have to reset the parser after
+each successful decode.
+
+See to L<JSON::XS/INCREMENTAL PARSING> for examples.
=head1 JSON::PP OWN METHODS
More information about the Pkg-perl-cvs-commits
mailing list