[Git][java-team/jruby-joni][master] 26 commits: Provide encoding for group name exception
Hideki Yamane (@henrich)
gitlab at salsa.debian.org
Tue Jul 4 13:29:36 BST 2023
Hideki Yamane pushed to branch master at Debian Java Maintainers / jruby-joni
Commits:
a074777a by Charles Oliver Nutter at 2023-02-08T00:52:46+01:00
Provide encoding for group name exception
Without an encoding provided, this code assumes the bytes are in
the default character set. When they are not, the decoded string
will be corrupt. This is the cause of a failure in the strscan
library on Windows.
This commit adds encoding to the parameters for named group lookup
so that it can be decoded properly if an exception must be raised.
- - - - -
e05f92ca by Luke Nezda at 2023-05-22T07:48:21-05:00
DRY out various test methods, add a couple taking p, end (offsets of patterns in a super string), and WarnCallback
- - - - -
629799ba by Luke Nezda at 2023-05-22T08:02:52-05:00
[#63] add test cases demonstrating StringIndexOutOfBoundsException
- - - - -
fe6de011 by Luke Nezda at 2023-05-22T08:02:52-05:00
fix [#63] and related issue
- - - - -
ba09a833 by Luke Nezda at 2023-05-22T08:49:58-05:00
[#36] allow changing almost everything in Config using System Properties
* note: used underscores instead of periods used in the joni.debug* options
- - - - -
eede23d6 by Luke Nezda at 2023-05-22T11:55:54-05:00
fix indent
- - - - -
3603d698 by Luke Nezda at 2023-05-22T12:00:35-05:00
add un-negated OPTIMIZE to initialize with per code review
- - - - -
dd5b7ec2 by Luke Nezda at 2023-05-22T13:03:25-05:00
deprecated double-negated DONT_OPTIMIZE per code review
- - - - -
80767dc9 by Luke Nezda at 2023-05-22T13:19:38-05:00
fix build status badge in readme (build was moved from Travis CI -> GitHub Action)
- - - - -
78dcf3ac by Marcin Mielżyński at 2023-05-22T23:53:48+02:00
Merge pull request #66 from nezda/fix-issue-63
Fix issue 63 (and 2 related issues)
- - - - -
994871e8 by Thomas E Enebo at 2023-05-23T14:48:35+00:00
Merge pull request #68 from nezda/fix-build-status-badge-in-readme
fix build status badge in readme (build was moved from Travis CI -> GitHub Actions)
- - - - -
9dc79353 by Charles Oliver Nutter at 2023-05-23T10:51:43-04:00
Merge pull request #67 from nezda/fix-issue-36
[#36] allow changing almost everything in Config using System Properties
- - - - -
ad9d9fa8 by Charles Oliver Nutter at 2023-05-23T10:18:18-05:00
Make Region fields private
This is a follow-up to #58 and will release in 2.2.
- - - - -
c1a138b6 by Charles Oliver Nutter at 2023-05-23T10:18:20-05:00
Only allocate a single array for beg/end
- - - - -
44204acb by Charles Oliver Nutter at 2023-05-23T10:21:33-05:00
Add a lightweight region for numRegs == 1
Whie benchmarking the CSV library in JRuby, one of the largest
groups of allocated objects was int[] primarily for Region
objects, some of which most likely only have one regionand do not
need the flexibility of an array of regions. This commit splits
Region into multi and single forms, so that single-region objects
can be as compact as possible and not require a second (or third)
object for the beg/end arrays.
- - - - -
d1afcaf4 by Charles Oliver Nutter at 2023-05-23T10:31:44-05:00
Drop numRegs field
It's equivalent to begEnd.length / 2.
- - - - -
d2a0fed8 by Charles Oliver Nutter at 2023-05-23T11:57:56-04:00
Merge pull request #59 from headius/private_region_fields
Make Region fields private
- - - - -
45172907 by Charles Oliver Nutter at 2023-05-23T11:59:09-04:00
Merge pull request #61 from headius/group_name_encoding
Provide encoding for group name exception
- - - - -
35c22059 by Charles Oliver Nutter at 2023-05-25T09:20:57-05:00
Update bug tracker in pom
- - - - -
04beab6b by Charles Oliver Nutter at 2023-06-06T16:32:23-05:00
[maven-release-plugin] prepare release joni-2.2.0
- - - - -
3f891850 by Charles Oliver Nutter at 2023-06-06T16:32:25-05:00
[maven-release-plugin] prepare for next development iteration
- - - - -
e2358a13 by Thomas E. Enebo at 2023-06-07T12:08:09-05:00
Make java 8 compilable again. Use slow search on //i due to bad logic in choosing fast algo (to be fixed later)
- - - - -
92d391e5 by Thomas E Enebo at 2023-06-07T17:29:37+00:00
Merge pull request #72 from jruby/hack_fast_i_broken
Use slow search on //i due to bad logic in choosing fast algo (to be fixed later)
- - - - -
3812f8b1 by Charles Oliver Nutter at 2023-06-07T12:49:40-05:00
[maven-release-plugin] prepare release joni-2.2.1
- - - - -
e5f37619 by Charles Oliver Nutter at 2023-06-07T12:49:43-05:00
[maven-release-plugin] prepare for next development iteration
- - - - -
6a0d556f by Sebastian Thomschke at 2023-06-14T12:56:24+02:00
minor code cleanup (#73)
* style: remove unnecessary semicolons
* style: add missing `@Override` annotations
* style: access Thread.interrupted() statically
* fix: transitive dependency compiler warning: in org.joni.Regex class
"The type `Encoding` from module org.jruby.jcodings may not be accessible to clients due to missing 'requires transitive'"
- - - - -
26 changed files:
- README.md
- pom.xml
- src/module-info.java
- src/org/joni/ApplyCaseFold.java
- src/org/joni/BitSet.java
- src/org/joni/ByteCodeMachine.java
- src/org/joni/CodeRangeBuffer.java
- src/org/joni/Config.java
- src/org/joni/ConfigSupport.java
- src/org/joni/Lexer.java
- + src/org/joni/MultiRegion.java
- src/org/joni/NameEntry.java
- src/org/joni/Regex.java
- src/org/joni/Region.java
- src/org/joni/Search.java
- + src/org/joni/SingleRegion.java
- src/org/joni/StackMachine.java
- src/org/joni/UnsetAddrList.java
- src/org/joni/WarnCallback.java
- src/org/joni/ast/Node.java
- src/org/joni/ast/QuantifierNode.java
- test/org/joni/test/Test.java
- test/org/joni/test/TestA.java
- test/org/joni/test/TestError.java
- test/org/joni/test/TestInterrupt.java
- test/org/joni/test/TestU.java
Changes:
=====================================
README.md
=====================================
@@ -2,7 +2,7 @@ joni
====
[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.jruby.joni%22)
-[](http://travis-ci.org/jruby/joni)
+[](https://github.com/jruby/joni/actions)
Java port of Oniguruma regexp library
=====================================
pom.xml
=====================================
@@ -4,7 +4,7 @@
<groupId>org.jruby.joni</groupId>
<artifactId>joni</artifactId>
<packaging>jar</packaging>
- <version>2.1.49-SNAPSHOT</version>
+ <version>2.2.2-SNAPSHOT</version>
<name>Joni</name>
<description>
Java port of Oniguruma: http://www.geocities.jp/kosako3/oniguruma
@@ -22,8 +22,8 @@
</parent>
<issueManagement>
- <system>JIRA</system>
- <url>http://jira.codehaus.org/browse/JRUBY</url>
+ <system>GitHub</system>
+ <url>https://github.com/jruby/joni/issues</url>
</issueManagement>
<scm>
=====================================
src/module-info.java
=====================================
@@ -3,5 +3,5 @@ open module org.jruby.joni {
exports org.joni.constants;
exports org.joni.exception;
- requires org.jruby.jcodings;
+ requires transitive org.jruby.jcodings;
}
\ No newline at end of file
=====================================
src/org/joni/ApplyCaseFold.java
=====================================
@@ -28,6 +28,7 @@ import org.joni.ast.StringNode;
final class ApplyCaseFold implements ApplyAllCaseFoldFunction {
// i_apply_case_fold
+ @Override
public void apply(int from, int[]to, int length, Object o) {
ApplyCaseFoldArg arg = (ApplyCaseFoldArg)o;
=====================================
src/org/joni/BitSet.java
=====================================
@@ -110,6 +110,7 @@ public final class BitSet {
}
private static final int BITS_TO_STRING_WRAP = 4;
+ @Override
public String toString() {
StringBuilder buffer = new StringBuilder();
buffer.append("BitSet");
=====================================
src/org/joni/ByteCodeMachine.java
=====================================
@@ -57,6 +57,7 @@ class ByteCodeMachine extends StackMachine {
this.code = regex.code;
}
+ @Override
public void interrupt() {
interrupted = true;
// might have no effect on the executing thread but worth a try
@@ -153,6 +154,7 @@ class ByteCodeMachine extends StackMachine {
return true;
}
+ @Override
protected final int matchAt(int _range, int _sstart, int _sprev, boolean interrupt) throws InterruptedException {
range = _range;
sstart = _sstart;
@@ -444,7 +446,7 @@ class ByteCodeMachine extends StackMachine {
private void handleInterrupted(final boolean checkThreadInterrupt) throws InterruptedException {
if (interrupted || (checkThreadInterrupt && Thread.currentThread().isInterrupted())) {
- Thread.currentThread().interrupted();
+ Thread.interrupted();
throw new InterruptedException();
}
interruptCheckEvery = Math.min(interruptCheckEvery << 1, MAX_INTERRUPT_CHECK_EVERY);
=====================================
src/org/joni/CodeRangeBuffer.java
=====================================
@@ -89,6 +89,7 @@ public final class CodeRangeBuffer {
if (used < u) used = u;
}
+ @Override
public CodeRangeBuffer clone() {
return new CodeRangeBuffer(this);
}
@@ -374,6 +375,7 @@ public final class CodeRangeBuffer {
return pbuf;
}
+ @Override
public String toString() {
StringBuilder buf = new StringBuilder();
buf.append("CodeRange");
=====================================
src/org/joni/Config.java
=====================================
@@ -22,54 +22,57 @@ package org.joni;
import java.io.PrintStream;
public interface Config extends org.jcodings.Config {
- final int CHAR_TABLE_SIZE = 256;
- final boolean USE_NO_INVALID_QUANTIFIER = true;
- final int SCANENV_MEMNODES_SIZE = 8;
-
- final boolean USE_NAMED_GROUP = true;
- final boolean USE_SUBEXP_CALL = true;
- final boolean USE_PERL_SUBEXP_CALL = true;
- final boolean USE_BACKREF_WITH_LEVEL = true; /* \k<name+n>, \k<name-n> */
-
- final boolean USE_MONOMANIAC_CHECK_CAPTURES_IN_ENDLESS_REPEAT = true; /* /(?:()|())*\2/ */
- final boolean USE_NEWLINE_AT_END_OF_STRING_HAS_EMPTY_LINE = true; /* /\n$/ =~ "\n" */
- final boolean USE_WARNING_REDUNDANT_NESTED_REPEAT_OPERATOR = true;
-
- final boolean CASE_FOLD_IS_APPLIED_INSIDE_NEGATIVE_CCLASS = true;
-
- final boolean USE_MATCH_RANGE_MUST_BE_INSIDE_OF_SPECIFIED_RANGE = false;
- final boolean USE_CAPTURE_HISTORY = false;
- final boolean USE_VARIABLE_META_CHARS = true;
- final boolean USE_WORD_BEGIN_END = true; /* "\<": word-begin, "\>": word-end */
- final boolean USE_FIND_LONGEST_SEARCH_ALL_OF_RANGE = true;
- final boolean USE_SUNDAY_QUICK_SEARCH = true;
- final boolean USE_CEC = false;
- final boolean USE_DYNAMIC_OPTION = false;
- final boolean USE_BYTE_MAP = OptExactInfo.OPT_EXACT_MAXLEN <= CHAR_TABLE_SIZE;
- final boolean USE_INT_MAP_BACKWARD = false;
-
- final int NREGION = 10;
- final int MAX_BACKREF_NUM = 1000;
- final int MAX_CAPTURE_GROUP_NUM = 32767;
- final int MAX_REPEAT_NUM = 100000;
- final int MAX_MULTI_BYTE_RANGES_NUM = 10000;
+ final int CHAR_TABLE_SIZE = ConfigSupport.getInt("joni.char_table_size", 256);
+ final boolean USE_NO_INVALID_QUANTIFIER = ConfigSupport.getBoolean("joni.use_no_invalid_quantifier", true);
+ final int SCANENV_MEMNODES_SIZE = ConfigSupport.getInt("joni.scanenv_memnodes_size", 8);
+
+ final boolean USE_NAMED_GROUP = ConfigSupport.getBoolean("joni.use_named_group", true);
+ final boolean USE_SUBEXP_CALL = ConfigSupport.getBoolean("joni.use_subexp_call", true);
+ final boolean USE_PERL_SUBEXP_CALL = ConfigSupport.getBoolean("joni.use_perl_subexp_call", true);
+ final boolean USE_BACKREF_WITH_LEVEL = ConfigSupport.getBoolean("joni.use_backref_with_level", true); /* \k<name+n>, \k<name-n> */
+
+ final boolean USE_MONOMANIAC_CHECK_CAPTURES_IN_ENDLESS_REPEAT = ConfigSupport.getBoolean("joni.use_monomaniac_check_captures_in_endless_repeat", true); /* /(?:()|())*\2/ */
+ final boolean USE_NEWLINE_AT_END_OF_STRING_HAS_EMPTY_LINE = ConfigSupport.getBoolean("joni.use_newline_at_end_of_string_has_empty_line", true); /* /\n$/ =~ "\n" */
+ final boolean USE_WARNING_REDUNDANT_NESTED_REPEAT_OPERATOR = ConfigSupport.getBoolean("joni.use_warning_redundant_nested_repeat_operator", true);
+
+ final boolean CASE_FOLD_IS_APPLIED_INSIDE_NEGATIVE_CCLASS = ConfigSupport.getBoolean("joni.case_fold_is_applied_inside_negative_cclass", true);
+
+ final boolean USE_MATCH_RANGE_MUST_BE_INSIDE_OF_SPECIFIED_RANGE = ConfigSupport.getBoolean("joni.use_match_range_must_be_inside_of_specified_range", false);
+ final boolean USE_CAPTURE_HISTORY = ConfigSupport.getBoolean("joni.use_capture_history", false);
+ final boolean USE_VARIABLE_META_CHARS = ConfigSupport.getBoolean("joni.use_variable_meta_chars", true);
+ final boolean USE_WORD_BEGIN_END = ConfigSupport.getBoolean("joni.use_word_begin_end", true); /* "\<": word-begin, "\>": word-end */
+ final boolean USE_FIND_LONGEST_SEARCH_ALL_OF_RANGE = ConfigSupport.getBoolean("joni.use_find_longest_search_all_of_range", true);
+ final boolean USE_SUNDAY_QUICK_SEARCH = ConfigSupport.getBoolean("joni.use_sunday_quick_search", true);
+ final boolean USE_CEC = ConfigSupport.getBoolean("joni.use_cec", false);
+ final boolean USE_DYNAMIC_OPTION = ConfigSupport.getBoolean("joni.use_dynamic_option", false);
+ final boolean USE_BYTE_MAP = ConfigSupport.getBoolean("joni.use_byte_map", OptExactInfo.OPT_EXACT_MAXLEN <= CHAR_TABLE_SIZE);
+ final boolean USE_INT_MAP_BACKWARD = ConfigSupport.getBoolean("joni.use_int_map_backward", false);
+
+ final int NREGION = ConfigSupport.getInt("joni.nregion", 10);
+ final int MAX_BACKREF_NUM = ConfigSupport.getInt("joni.max_backref_num", 1000);
+ final int MAX_CAPTURE_GROUP_NUM = ConfigSupport.getInt("joni.max_capture_group_num", 32767);
+ final int MAX_REPEAT_NUM = ConfigSupport.getInt("joni.max_multi_byte_ranges_num", 100000);
+ final int MAX_MULTI_BYTE_RANGES_NUM = ConfigSupport.getInt("joni.max_multi_byte_ranges_num", 10000);
// internal config
- final boolean USE_OP_PUSH_OR_JUMP_EXACT = true;
- final boolean USE_QTFR_PEEK_NEXT = true;
+ final boolean USE_OP_PUSH_OR_JUMP_EXACT = ConfigSupport.getBoolean("joni.use_op_push_or_jump_exact", true);
+ final boolean USE_QTFR_PEEK_NEXT = ConfigSupport.getBoolean("joni.use_qtfr_peek_next", true);
- final int INIT_MATCH_STACK_SIZE = 64;
+ final int INIT_MATCH_STACK_SIZE = ConfigSupport.getInt("joni.init_match_stack_size", 64);
- final boolean DONT_OPTIMIZE = false;
+ final boolean OPTIMIZE = ConfigSupport.getBoolean("joni.optimize", true);
+ @Deprecated
+ final boolean DONT_OPTIMIZE = !OPTIMIZE;
- final boolean USE_STRING_TEMPLATES = true; // use embedded string templates in Regex object as byte arrays instead of compiling them into int bytecode array
+ // use embedded string templates in Regex object as byte arrays instead of compiling them into int bytecode array
+ final boolean USE_STRING_TEMPLATES = ConfigSupport.getBoolean("joni.use_string_templates", true);
- final int MAX_CAPTURE_HISTORY_GROUP = 31;
+ final int MAX_CAPTURE_HISTORY_GROUP = ConfigSupport.getInt("joni.max_capture_history_group", 31);
- final int CHECK_STRING_THRESHOLD_LEN = 7;
- final int CHECK_BUFF_MAX_SIZE = 0x4000;
+ final int CHECK_STRING_THRESHOLD_LEN = ConfigSupport.getInt("joni.check_string_threshold_len", 7);
+ final int CHECK_BUFF_MAX_SIZE = ConfigSupport.getInt("joni.check_buff_max_size", 0x4000);
final PrintStream log = System.out;
final PrintStream err = System.err;
=====================================
src/org/joni/ConfigSupport.java
=====================================
@@ -5,4 +5,9 @@ public class ConfigSupport {
String value = System.getProperty(property, def ? "true" : "false");
return !value.equals("false");
}
+
+ static int getInt(String property, int def) {
+ String value = System.getProperty(property);
+ return value != null ? Integer.parseInt(value) : def;
+ }
}
=====================================
src/org/joni/Lexer.java
=====================================
@@ -1324,7 +1324,7 @@ class Lexer extends ScannerSupport {
protected final void syntaxWarn(String message) {
if (env.warnings != WarnCallback.NONE) {
- env.warnings.warn(message + ": /" + new String(bytes, getBegin(), getEnd()) + "/");
+ env.warnings.warn(message + ": /" + new String(bytes, getBegin(), getEnd() - getBegin()) + "/");
}
}
}
=====================================
src/org/joni/MultiRegion.java
=====================================
@@ -0,0 +1,72 @@
+/*
+ * Permission is hereby granted, free of charge, to any person obtaining a copy of
+ * this software and associated documentation files (the "Software"), to deal in
+ * the Software without restriction, including without limitation the rights to
+ * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+ * of the Software, and to permit persons to whom the Software is furnished to do
+ * so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+package org.joni;
+
+import java.util.Arrays;
+
+public final class MultiRegion extends Region {
+ private final int[] begEnd;
+
+ public MultiRegion(int num) {
+ this.begEnd = new int[num * 2];
+ }
+
+ public MultiRegion(int begin, int end) {
+ this.begEnd = new int[]{begin, end};
+ }
+
+ @Override
+ public final int getNumRegs() {
+ return begEnd.length / 2;
+ }
+
+ @Override
+ public MultiRegion clone() {
+ MultiRegion region = new MultiRegion(getNumRegs());
+ System.arraycopy(begEnd, 0, region.begEnd, 0, begEnd.length);
+ if (getCaptureTree() != null) region.setCaptureTree(getCaptureTree().cloneTree());
+ return region;
+ }
+
+ @Override
+ public int getBeg(int index) {
+ return begEnd[index * 2];
+ }
+
+ @Override
+ public int setBeg(int index, int value) {
+ return begEnd[index * 2] = value;
+ }
+
+ @Override
+ public int getEnd(int index) {
+ return begEnd[index * 2 + 1];
+ }
+
+ @Override
+ public int setEnd(int index, int value) {
+ return begEnd[index * 2 + 1] = value;
+ }
+
+ @Override
+ void clear() {
+ Arrays.fill(begEnd, REGION_NOTPOS);
+ }
+}
=====================================
src/org/joni/NameEntry.java
=====================================
@@ -79,6 +79,7 @@ public final class NameEntry {
}
}
+ @Override
public String toString() {
StringBuilder buff = new StringBuilder(new String(name, nameP, nameEnd - nameP) + " ");
if (backNum == 0) {
=====================================
src/org/joni/Regex.java
=====================================
@@ -24,11 +24,13 @@ import static org.joni.Config.USE_SUNDAY_QUICK_SEARCH;
import static org.joni.Option.isCaptureGroup;
import static org.joni.Option.isDontCaptureGroup;
+import java.nio.charset.Charset;
import java.util.Collections;
import java.util.Iterator;
import org.jcodings.CaseFoldCodeItem;
import org.jcodings.Encoding;
+import org.jcodings.EncodingDB;
import org.jcodings.specific.ASCIIEncoding;
import org.jcodings.specific.UTF8Encoding;
import org.jcodings.util.BytesHash;
@@ -88,6 +90,18 @@ public final class Regex {
byte[][]templates; /* fixed pattern strings not embedded in bytecode */
int templateNum;
+ private static final Encoding DEFAULT_ENCODING;
+ static {
+ Encoding defaultEncoding;
+ EncodingDB.Entry entry = EncodingDB.getEncodings().get(Charset.defaultCharset().name().getBytes());
+ if (entry == null) {
+ defaultEncoding = UTF8Encoding.INSTANCE;
+ } else {
+ defaultEncoding = entry.getEncoding();
+ }
+ DEFAULT_ENCODING = defaultEncoding;
+ }
+
public Regex(CharSequence cs) {
this(cs.toString());
}
@@ -232,9 +246,13 @@ public final class Regex {
}
public int nameToBackrefNumber(byte[]name, int nameP, int nameEnd, Region region) {
+ return nameToBackrefNumber(name, nameP, nameEnd, DEFAULT_ENCODING, region);
+ }
+
+ public int nameToBackrefNumber(byte[]name, int nameP, int nameEnd, Encoding nameEncoding, Region region) {
NameEntry e = nameToGroupNumbers(name, nameP, nameEnd);
if (e == null) throw new ValueException(ErrorMessages.UNDEFINED_NAME_REFERENCE,
- new String(name, nameP, nameEnd - nameP));
+ new String(name, nameP, nameEnd - nameP, nameEncoding.getCharset()));
switch(e.backNum) {
case 0:
@@ -352,7 +370,9 @@ public final class Regex {
if (e.length >= 3 || (e.length >= 2 && allowReverse)) {
forward = enc.toLowerCaseTable() != null ? Search.SLOW_IC_SB_FORWARD : Search.SLOW_IC_FORWARD;
if (!setupBMSkipMap(true)) {
- forward = allowReverse ? Search.BM_IC_FORWARD : Search.BM_NOT_REV_IC_FORWARD;
+ forward = allowReverse ? (enc.toLowerCaseTable() != null ? Search.SLOW_IC_SB_FORWARD : Search.SLOW_IC_FORWARD) : Search.BM_NOT_REV_IC_FORWARD;
+ // FIXME: put above line in place to work around some failures. Either BM_IC_FORWARD is broken here or we are choosing it when we shouldn't.
+ //forward = allowReverse ? Search.BM_IC_FORWARD : Search.BM_NOT_REV_IC_FORWARD;
} else {
forward = enc.toLowerCaseTable() != null ? Search.SLOW_IC_SB_FORWARD : Search.SLOW_IC_FORWARD;
}
=====================================
src/org/joni/Region.java
=====================================
@@ -19,100 +19,48 @@
*/
package org.joni;
-public final class Region {
+public abstract class Region {
static final int REGION_NOTPOS = -1;
- @Deprecated
- public final int numRegs;
- @Deprecated
- public final int[] beg;
- @Deprecated
- public final int[] end;
- @Deprecated
- public CaptureTreeNode historyRoot;
+ protected CaptureTreeNode historyRoot;
- @SuppressWarnings("deprecation")
public static Region newRegion(int num) {
- return new Region(num);
+ if (num == 1) return new SingleRegion(num);
+ return new MultiRegion(num);
}
- @SuppressWarnings("deprecation")
public static Region newRegion(int begin, int end) {
- return new Region(begin, end);
+ return new SingleRegion(begin, end);
}
- @Deprecated
- @SuppressWarnings("deprecation")
- public Region(int num) {
- this.numRegs = num;
- this.beg = new int[num];
- this.end = new int[num];
- }
+ @Override
+ public abstract Region clone();
- @Deprecated
- @SuppressWarnings("deprecation")
- public Region(int begin, int end) {
- this.numRegs = 1;
- this.beg = new int[]{begin};
- this.end = new int[]{end};
- }
+ public abstract int getNumRegs();
- @SuppressWarnings("deprecation")
- public Region clone() {
- Region region = new Region(numRegs);
- System.arraycopy(beg, 0, region.beg, 0, beg.length);
- System.arraycopy(end, 0, region.end, 0, end.length);
- if (historyRoot != null) region.historyRoot = historyRoot.cloneTree();
- return region;
- }
+ public abstract int getBeg(int index);
- @SuppressWarnings("deprecation")
- public int getNumRegs() {
- return numRegs;
- }
+ public abstract int setBeg(int index, int value);
- @SuppressWarnings("deprecation")
- public int getBeg(int index) {
- return beg[index];
- }
+ public abstract int getEnd(int index);
- @SuppressWarnings("deprecation")
- public int setBeg(int index, int value) {
- return beg[index] = value;
- }
+ public abstract int setEnd(int index, int value);
- @SuppressWarnings("deprecation")
- public int getEnd(int index) {
- return end[index];
- }
-
- @SuppressWarnings("deprecation")
- public int setEnd(int index, int value) {
- return end[index] = value;
- }
-
- @SuppressWarnings("deprecation")
+ @Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("Region: \n");
- for (int i=0; i<beg.length; i++) sb.append(" " + i + ": (" + beg[i] + "-" + end[i] + ")");
+ for (int i=0; i<getNumRegs(); i++) sb.append(" " + i + ": (" + getBeg(i) + "-" + getEnd(i) + ")");
return sb.toString();
}
- @SuppressWarnings("deprecation")
CaptureTreeNode getCaptureTree() {
return historyRoot;
}
- @SuppressWarnings("deprecation")
CaptureTreeNode setCaptureTree(CaptureTreeNode ctn) {
return this.historyRoot = ctn;
}
- @SuppressWarnings("deprecation")
- void clear() {
- for (int i=0; i<beg.length; i++) {
- beg[i] = end[i] = REGION_NOTPOS;
- }
- }
+ abstract void clear();
}
=====================================
src/org/joni/Search.java
=====================================
@@ -297,7 +297,7 @@ final class Search {
s--;
}
return -1;
- };
+ }
};
static final Forward BM_FORWARD = new Forward() {
@@ -421,10 +421,10 @@ final class Search {
int targetP = regex.exactP;
int targetEnd = regex.exactEnd;
- int end, s, tlen1;
+ int end, s;
int tail = targetEnd - 1;
+ int tlen1 = tail - targetP;
if (USE_SUNDAY_QUICK_SEARCH) {
- tlen1 = tail - targetP;
end = textRange + tlen1;
s = textP + tlen1;
} else {
=====================================
src/org/joni/SingleRegion.java
=====================================
@@ -0,0 +1,75 @@
+/*
+ * Permission is hereby granted, free of charge, to any person obtaining a copy of
+ * this software and associated documentation files (the "Software"), to deal in
+ * the Software without restriction, including without limitation the rights to
+ * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+ * of the Software, and to permit persons to whom the Software is furnished to do
+ * so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+package org.joni;
+
+public class SingleRegion extends Region {
+ int beg;
+ int end;
+
+ public SingleRegion(int num) {
+ if (num != 1) throw new IndexOutOfBoundsException(""+num);
+ }
+
+ public SingleRegion(int begin, int end) {
+ this.beg = begin;
+ this.end = end;
+ }
+
+ @Override
+ public int getNumRegs() {
+ return 1;
+ }
+
+ @Override
+ public SingleRegion clone() {
+ SingleRegion region = new SingleRegion(beg, end);
+ if (getCaptureTree() != null) region.setCaptureTree(getCaptureTree().cloneTree());
+ return region;
+ }
+
+ @Override
+ public int getBeg(int index) {
+ if (index != 0) throw new IndexOutOfBoundsException(""+index);
+ return beg;
+ }
+
+ @Override
+ public int setBeg(int index, int value) {
+ if (index != 0) throw new IndexOutOfBoundsException(""+index);
+ return beg = value;
+ }
+
+ @Override
+ public int getEnd(int index) {
+ if (index != 0) throw new IndexOutOfBoundsException(""+index);
+ return end;
+ }
+
+ @Override
+ public int setEnd(int index, int value) {
+ if (index != 0) throw new IndexOutOfBoundsException(""+index);
+ return end = value;
+ }
+
+ @Override
+ void clear() {
+ beg = end = REGION_NOTPOS;
+ }
+}
=====================================
src/org/joni/StackMachine.java
=====================================
@@ -132,6 +132,7 @@ abstract class StackMachine extends Matcher implements StackType {
// STATE_CHECK_BUFF_INIT
private static final int STATE_CHECK_BUFF_MALLOC_THRESHOLD_SIZE = 16;
+ @Override
protected final void stateCheckBuffInit(int strLength, int offset, int stateNum) {
if (stateNum > 0 && strLength >= Config.CHECK_STRING_THRESHOLD_LEN) {
int size = ((strLength + 1) * stateNum + 7) >>> 3;
@@ -156,6 +157,7 @@ abstract class StackMachine extends Matcher implements StackType {
}
}
+ @Override
protected final void stateCheckBuffClear() {
stateCheckBuff = null;
stateCheckBuffSize = 0;
@@ -549,7 +551,7 @@ abstract class StackMachine extends Matcher implements StackType {
isNull = 0;
break;
} else if (endp != s) {
- isNull = -1;; /* empty, but position changed */
+ isNull = -1; /* empty, but position changed */
}
}
k++;
=====================================
src/org/joni/UnsetAddrList.java
=====================================
@@ -55,6 +55,7 @@ public final class UnsetAddrList {
}
}
+ @Override
public String toString() {
StringBuilder value = new StringBuilder();
if (num > 0) {
=====================================
src/org/joni/WarnCallback.java
=====================================
@@ -24,12 +24,14 @@ package org.joni;
*/
public interface WarnCallback {
WarnCallback DEFAULT = new WarnCallback() {
- public void warn(String message) {
+ @Override
+ public void warn(String message) {
System.err.println(message);
}
};
WarnCallback NONE = new WarnCallback() {
+ @Override
public void warn(String message) {
}
};
=====================================
src/org/joni/ast/Node.java
=====================================
@@ -44,7 +44,7 @@ public abstract class Node implements NodeType {
protected Node getChild(){
// default definition
return null;
- };
+ }
public void replaceWith(Node with) {
with.parent = parent;
@@ -59,6 +59,7 @@ public abstract class Node implements NodeType {
return getName() + ":0x" + Integer.toHexString(System.identityHashCode(this));
}
+ @Override
public final String toString() {
StringBuilder s = new StringBuilder();
s.append("<" + getAddressName() + " (" + (parent == null ? "NULL" : parent.getAddressName()) + ")>");
=====================================
src/org/joni/ast/QuantifierNode.java
=====================================
@@ -238,11 +238,11 @@ public final class QuantifierNode extends StateNode {
case ASIS:
break;
case DEL:
- env.warnings.warn("regular expression has redundant nested repeat operator " + PopularQStr[targetQNum] + " /" + new String(bytes, p, end) + "/");
+ env.warnings.warn("regular expression has redundant nested repeat operator " + PopularQStr[targetQNum] + " /" + new String(bytes, p, end - p) + "/");
break;
default:
env.warnings.warn("nested repeat operator '" + PopularQStr[targetQNum] + "' and '" + PopularQStr[nestQNum] +
- "' was replaced with '" + ReduceQStr[REDUCE_TABLE[targetQNum][nestQNum].ordinal()] + "' in regular expression " + "/" + new String(bytes, p, end) + "/");
+ "' was replaced with '" + ReduceQStr[REDUCE_TABLE[targetQNum][nestQNum].ordinal()] + "' in regular expression " + "/" + new String(bytes, p, end - p) + "/");
}
}
} // USE_WARNING_REDUNDANT_NESTED_REPEAT_OPERATOR
=====================================
test/org/joni/test/Test.java
=====================================
@@ -77,13 +77,27 @@ public abstract class Test {
xerr(pattern.getBytes(testEncoding()), msg, option());
}
+ public void xerrs(String patternIsSubstring, int p, int end, String msg, WarnCallback warnCallback) throws Exception {
+ xerrs(patternIsSubstring, p, end, msg, option(), warnCallback);
+ }
+
public void xerrs(String pattern, String msg, int option) throws Exception {
xerr(pattern.getBytes(testEncoding()), msg, option);
}
+ public void xerrs(String patternIsSubstring, int p, int end, String msg, int option, WarnCallback warnCallback) throws Exception {
+ p = length(patternIsSubstring.substring(0, p).getBytes(testEncoding()));
+ end = length(patternIsSubstring.substring(0, end).getBytes(testEncoding()));
+ xerr(patternIsSubstring.getBytes(testEncoding()), p, end, msg, option, warnCallback);
+ }
+
public void xerr(byte[] pattern, String msg, int option) throws Exception {
+ xerr(pattern, 0, length(pattern), msg, option, WarnCallback.NONE);
+ }
+
+ public void xerr(byte[] pattern, int p, int end, String msg, int option, WarnCallback warnCallback) throws Exception {
try {
- new Regex(pattern, 0, length(pattern), option, encoding(), syntax(), WarnCallback.NONE);
+ new Regex(pattern, p, end, option, encoding(), syntax(), warnCallback);
nfail++;
} catch (JOniException je) {
nsucc++;
@@ -91,7 +105,6 @@ public abstract class Test {
} catch (CharacterPropertyException cpe) {
nsucc++;
assertEquals(cpe.getMessage(), msg);
-
}
}
@@ -115,10 +128,14 @@ public abstract class Test {
}
public int xx(byte[] pattern, byte[] str, int gpos, int searchStart, int from, int to, int mem, boolean not, int option) throws InterruptedException {
+ return xx(pattern, str, 0, length(pattern), gpos, searchStart, from, to, mem, not, option, WarnCallback.NONE);
+ }
+
+ public int xx(byte[] pattern, byte[] str, int p, int end, int gpos, int searchStart, int from, int to, int mem, boolean not, int option, WarnCallback warnCallback) throws InterruptedException {
Regex reg;
try {
- reg = new Regex(pattern, 0, length(pattern), option, encoding(), syntax(), WarnCallback.NONE);
+ reg = new Regex(pattern, p, end, option, encoding(), syntax(), warnCallback);
} catch (JOniException je) {
Config.err.println(reprTest(pattern, str, option));
je.printStackTrace(Config.err);
@@ -140,7 +157,7 @@ public abstract class Test {
return check(reg, pattern, str, option, gpos, searchStart, from, to, mem, not);
}
- private int check(Regex reg, byte[]pattern, byte[]str, int option, int gpos, int searchStart, int from, int to, int mem, boolean not) throws InterruptedException {
+ private int check(Regex reg, byte[]pattern, byte[] str, int option, int gpos, int searchStart, int from, int to, int mem, boolean not) throws InterruptedException {
Matcher m = reg.matcher(str, 0, length(str));
final Region region;
final int result;
@@ -221,11 +238,7 @@ public abstract class Test {
public void xxs(String pattern, String str, int from, int to, int mem, boolean not, int option)
throws InterruptedException {
- try {
- xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), from, to, mem, not, option);
- } catch (UnsupportedEncodingException uee) {
- uee.printStackTrace();
- }
+ x2s(pattern, 0, pattern.length(), str, 0, 0, from, to, mem, not, option, WarnCallback.NONE);
}
public int x2s(String pattern, String str, int from, int to) throws InterruptedException {
@@ -233,12 +246,7 @@ public abstract class Test {
}
public int x2s(String pattern, String str, int from, int to, int option) throws InterruptedException {
- try {
- return xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), from, to, 0, false, option);
- } catch (UnsupportedEncodingException uee) {
- uee.printStackTrace();
- return Matcher.FAILED;
- }
+ return x2s(pattern, str, 0, 0, from, to, option);
}
public int x2s(String pattern, String str, int gpos, int searchStart, int from, int to) throws InterruptedException {
@@ -246,8 +254,21 @@ public abstract class Test {
}
public int x2s(String pattern, String str, int gpos, int searchStart, int from, int to, int option) throws InterruptedException {
+ return x2s(pattern, 0, pattern.length(), str, gpos, searchStart, from, to, 0, false, option, WarnCallback.NONE);
+ }
+
+ public int x2s(String patternIsSubstring, int p, int end, String str, int from, int to, boolean not, WarnCallback warnCallback) throws InterruptedException {
+ return x2s(patternIsSubstring, p, end, str, 0, 0, from, to, 0, not, option(), warnCallback);
+ }
+
+ public int x2s(String patternIsSubstring, int p, int end, String str, int gpos, int searchStart, int from, int to, int mem, boolean not, int option, WarnCallback warnCallback) throws InterruptedException {
try {
- return xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), gpos, searchStart, from, to, 0, false, option);
+ byte[] patternBytes = patternIsSubstring.substring(p, end).getBytes(testEncoding());
+ // convert char p and end to byte
+ byte[] prefixBytes = patternIsSubstring.substring(0, p).getBytes(testEncoding());
+ p = prefixBytes.length == 0 ? 0 : length(prefixBytes);
+ end = p + length(patternBytes);
+ return xx(patternIsSubstring.getBytes(testEncoding()), str.getBytes(testEncoding()), p, end, gpos, searchStart, from, to, mem, not, option, warnCallback);
} catch (UnsupportedEncodingException uee) {
uee.printStackTrace();
return Matcher.FAILED;
@@ -259,11 +280,7 @@ public abstract class Test {
}
public void x3s(String pattern, String str, int from, int to, int mem, int option) throws InterruptedException {
- try {
- xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), from, to, mem, false, option);
- } catch (UnsupportedEncodingException uee) {
- uee.printStackTrace();
- }
+ x2s(pattern, 0, pattern.length(), str, 0, 0, from, to, mem, false, option, WarnCallback.NONE);
}
public void ns(String pattern, String str) throws InterruptedException {
@@ -275,11 +292,7 @@ public abstract class Test {
}
public void ns(String pattern, String str, int gpos, int option) throws InterruptedException {
- try {
- xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), gpos, 0, 0, 0, 0, true, option);
- } catch (UnsupportedEncodingException uee) {
- uee.printStackTrace();
- }
+ x2s(pattern, 0, pattern.length(), str, gpos, 0, 0, 0, 0, true, option, WarnCallback.NONE);
}
@org.junit.Test
=====================================
test/org/joni/test/TestA.java
=====================================
@@ -23,6 +23,7 @@ import org.jcodings.Encoding;
import org.jcodings.specific.ASCIIEncoding;
import org.joni.Option;
import org.joni.Syntax;
+import org.joni.WarnCallback;
import org.joni.exception.ErrorMessages;
public class TestA extends Test {
@@ -165,6 +166,7 @@ public class TestA extends Test {
x2s("xyz\\Z", "xyz", 0, 3);
x2s("xyz\\z", "xyz", 0, 3);
x2s("a\\Z", "a", 0, 1);
+ x2s("0\\gA", 1, 4, "doesntMatter", 0, 0, true, WarnCallback.DEFAULT);
x2s("\\Gaz", "az", 0, 2);
ns("\\Gz", "bza");
ns("az\\G", "az");
=====================================
test/org/joni/test/TestError.java
=====================================
@@ -23,6 +23,7 @@ import org.jcodings.Encoding;
import org.jcodings.specific.UTF8Encoding;
import org.joni.Option;
import org.joni.Syntax;
+import org.joni.WarnCallback;
import org.joni.exception.ErrorMessages;
public class TestError extends Test {
@@ -62,6 +63,8 @@ public class TestError extends Test {
xerrs("\\((", ErrorMessages.END_PATTERN_WITH_UNMATCHED_PARENTHESIS);
xerrs("(|", ErrorMessages.END_PATTERN_WITH_UNMATCHED_PARENTHESIS);
xerrs("'/g\\\u00ff\u00ff\u00ff\u00ff&))", ErrorMessages.UNMATCHED_CLOSE_PARENTHESIS);
+ xerrs("0'/g\\\u00ff\u00ff\u00ff\u00ff&))", 1, 12, ErrorMessages.UNMATCHED_CLOSE_PARENTHESIS, WarnCallback.DEFAULT);
+ xerrs("0\\1**?", 1, 6, ErrorMessages.INVALID_BACKREF, WarnCallback.DEFAULT);
xerrs("[0-0-\u00ff ", ErrorMessages.PREMATURE_END_OF_CHAR_CLASS); // \xe2
xerrs("\\p{foobarbaz}", ErrorMessages.ERR_INVALID_CHAR_PROPERTY_NAME.replace("%n", "foobarbaz"));
//xerrs("\\p{あ}", ErrorMessages.ERR_INVALID_CHAR_PROPERTY_NAME.replace("%n", "あ"));
=====================================
test/org/joni/test/TestInterrupt.java
=====================================
@@ -58,8 +58,10 @@ public class TestInterrupt extends Test {
}
@org.junit.Test
+ @Override
public void test() throws Exception {
interruptAfter(new InterruptibleRunnable() {
+ @Override
public void run() throws InterruptedException {
x2s("a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaa", 0, 0);
@@ -69,6 +71,7 @@ public class TestInterrupt extends Test {
final int status[] = new int[1];
interruptAfter(new InterruptibleRunnable() {
+ @Override
public void run() throws InterruptedException {
try {
x2s("a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
=====================================
test/org/joni/test/TestU.java
=====================================
@@ -69,13 +69,16 @@ public class TestU extends Test {
return sb.toString();
}
+ @Override
protected String repr(byte[]bytes) {
return uconv(bytes, ulen(bytes));
}
+ @Override
protected int length(byte[]bytes) {
return ulen(bytes);
}
+
@Override
public void test() throws Exception {
x2s("\000\000", "\000\000", 0, 0);
View it on GitLab: https://salsa.debian.org/java-team/jruby-joni/-/compare/92292ed34f66500b87504c43babd8cb4a72a54c1...6a0d556f7c24d77b0735e3d4fcfc2e3207174b4d
--
View it on GitLab: https://salsa.debian.org/java-team/jruby-joni/-/compare/92292ed34f66500b87504c43babd8cb4a72a54c1...6a0d556f7c24d77b0735e3d4fcfc2e3207174b4d
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-java-commits/attachments/20230704/d6f7ee19/attachment.htm>
More information about the pkg-java-commits
mailing list