Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NoSuchElementException when selecting Elements #2088

Closed
justinleinbach-wk opened this issue Dec 21, 2023 · 2 comments
Closed

NoSuchElementException when selecting Elements #2088

justinleinbach-wk opened this issue Dec 21, 2023 · 2 comments
Assignees
Labels
bug Confirmed bug that we should fix fixed
Milestone

Comments

@justinleinbach-wk
Copy link

Hi,

My team and I have encountered NoSuchElementException when parsing and selecting elements.

Our caller uses the parser to parse HTML into a document, selects one of the elements and attempts to access sub-elements through a select call with a passed in evaluator. All interactions with the document, including parsing, element selection, and sub-element access, are confined to a solitary thread.

The stack trace from the caller is as follows:

 java.util.NoSuchElementException: null
 at org.jsoup.nodes.NodeIterator.next(NodeIterator.java:69)
 at org.jsoup.select.StructuralEvaluator$Has.matches(StructuralEvaluator.java:73)
 at org.jsoup.select.CombiningEvaluator$And.matches(CombiningEvaluator.java:84)
 at org.jsoup.select.StructuralEvaluator$ImmediateParentRun.matches(StructuralEvaluator.java:218)
 at org.jsoup.select.CombiningEvaluator$Or.matches(CombiningEvaluator.java:125)
 at org.jsoup.select.Evaluator.lambda$asPredicate$0(Evaluator.java:38)
 at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
 at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
 at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
 at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
 at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
 at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
 at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
 at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
 at org.jsoup.select.Collector.collect(Collector.java:29)
 at org.jsoup.select.Selector.select(Selector.java:111)
 at org.jsoup.nodes.Element.select(Element.java:463)

Tracing that, it appears to be throwing here, which shouldn't be possible because of the null check in the caller here.

We haven't been able to reproduce from a standalone test case, but have seen it occur intermittently twice now in a long-running process.

Our service currently uses jsoup 1.17.1 on Amazon Corretto 17 images. Additional details below

openjdk version "17.0.9" 2023-10-17 LTS
OpenJDK Runtime Environment Corretto-17.0.9.8.1 (build 17.0.9+8-LTS)
OpenJDK 64-Bit Server VM Corretto-17.0.9.8.1 (build 17.0.9+8-LTS, mixed mode, sharing)

Are there any ideas or explanation for what might be wrong?
If there's any additional information we can provide please let me know.

@jhy
Copy link
Owner

jhy commented Dec 22, 2023

Hi, thanks for the report.

To confirm -- your entry point is:

    public Elements select(Evaluator evaluator) {
        return Selector.select(evaluator, this);
    }

Right?

I am wondering if you are using that Evaluator object across multiple concurrent threads?

I think that would explain it - the Iterator is created when the Evaluator is created and then is getting hit concurrently. And then in the .hasNext() / .next() calls there is a race on the next object.

The evaluator is supposed to be threadsafe and other instance fields are ThreadLocals. So I missed making the Iterator a threadlocal.

The iteration method in the :has evaluator changed in 1.17.1 which introduced this local Iterator, so you wouldn't have seen the issue in previous versions.

@jhy jhy self-assigned this Dec 22, 2023
@jhy jhy added the bug Confirmed bug that we should fix label Dec 22, 2023
@jhy jhy added this to the 1.17.2 milestone Dec 22, 2023
@jhy jhy closed this as completed in 374ded2 Dec 22, 2023
@jhy jhy added the fixed label Dec 22, 2023
jhy added a commit that referenced this issue Dec 22, 2023
@justinleinbach-wk
Copy link
Author

Hi, thanks for the report.

To confirm -- your entry point is:

    public Elements select(Evaluator evaluator) {
        return Selector.select(evaluator, this);
    }

Right?

I am wondering if you are using that Evaluator object across multiple concurrent threads?

I think that would explain it - the Iterator is created when the Evaluator is created and then is getting hit concurrently. And then in the .hasNext() / .next() calls there is a race on the next object.

The evaluator is supposed to be threadsafe and other instance fields are ThreadLocals. So I missed making the Iterator a threadlocal.

The iteration method in the :has evaluator changed in 1.17.1 which introduced this local Iterator, so you wouldn't have seen the issue in previous versions.

That's correct. We are sharing the evaluator between threads so that seems to us like the correct solution to the issue we were seeing. Thanks!

benkard added a commit to benkard/mulkcms2 that referenced this issue Dec 29, 2023
This MR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [flow-bin](https://github.com/flowtype/flow-bin) ([changelog](https://github.com/facebook/flow/blob/master/Changelog.md)) | devDependencies | minor | [`^0.224.0` -> `^0.225.0`](https://renovatebot.com/diffs/npm/flow-bin/0.224.0/0.225.1) |
| [org.liquibase.ext:liquibase-hibernate5](https://github.com/liquibase/liquibase-hibernate/wiki) ([source](https://github.com/liquibase/liquibase-hibernate)) | build | patch | `4.25.0` -> `4.25.1` |
| [org.liquibase:liquibase-maven-plugin](http://www.liquibase.org/liquibase-maven-plugin) ([source](https://github.com/liquibase/liquibase)) | build | patch | `4.25.0` -> `4.25.1` |
| [org.jsoup:jsoup](https://jsoup.org/) ([source](https://github.com/jhy/jsoup)) | compile | patch | `1.17.1` -> `1.17.2` |
| [io.hypersistence:hypersistence-utils-hibernate-62](https://github.com/vladmihalcea/hypersistence-utils) | compile | minor | `3.6.1` -> `3.7.0` |
| [io.quarkus:quarkus-maven-plugin](https://github.com/quarkusio/quarkus) | build | patch | `3.6.3` -> `3.6.4` |
| [io.quarkus:quarkus-universe-bom](https://github.com/quarkusio/quarkus-platform) | import | patch | `3.6.3` -> `3.6.4` |
| [org.apache.maven.plugins:maven-compiler-plugin](https://maven.apache.org/plugins/) | build | minor | `3.11.0` -> `3.12.1` |

---

### Release Notes

<details>
<summary>flowtype/flow-bin</summary>

### [`v0.225.1`](flow/flow-bin@62a43fb...23ec616)

[Compare Source](flow/flow-bin@62a43fb...23ec616)

### [`v0.225.0`](flow/flow-bin@e6104a1...62a43fb)

[Compare Source](flow/flow-bin@e6104a1...62a43fb)

</details>

<details>
<summary>liquibase/liquibase</summary>

### [`v4.25.1`](https://github.com/liquibase/liquibase/blob/HEAD/changelog.txt#Liquibase-4251-is-a-patch-release)

[Compare Source](liquibase/liquibase@v4.25.0...v4.25.1)

</details>

<details>
<summary>jhy/jsoup</summary>

### [`v1.17.2`](https://github.com/jhy/jsoup/blob/HEAD/CHANGES.md#&#8203;1172-2023-Dec-29)

##### Improvements

-   **Attribute object accessors**: Added `Element.attribute(String)` and `Attributes.attribute(String)` to more simply
    obtain an `Attribute` object. [2069](jhy/jsoup#2069)
-   **Attribute source tracking**: If source tracking is on, and an Attribute's key is changed (
    via `Attribute.setKey(String)`), the source range is now still tracked
    in `Attribute.sourceRange()`. [2070](jhy/jsoup#2070)
-   **Wildcard attribute selector**: Added support for the `[*]` element with any attribute selector. And also restored
    support for selecting by an empty attribute name prefix (`[^]`). [2079](jhy/jsoup#2079)

##### Bug Fixes

-   **Mixed-cased source position**: When tracking the source position of attributes, if the source attribute name was
    mix-cased but the parser was lower-case normalizing attribute names, the source position for that attribute was not
    tracked correctly. [2067](jhy/jsoup#2067)
-   **Source position NPE**: When tracking the source position of a body fragment parse, a null pointer
    exception was thrown. [2068](jhy/jsoup#2068)
-   **Multi-point emoji entity**: A multi-point encoded emoji entity may be incorrectly decoded to the replacement
    character. [2074](jhy/jsoup#2074)
-   **Selector sub-expressions**: (Regression) in a selector like `parent [attr=va], other`, the `, OR` was binding
    to `[attr=va]` instead of `parent [attr=va]`, causing incorrect selections. The fix includes a EvaluatorDebug class
    that generates a sexpr to represent the query, allowing simpler and more thorough query parse
    tests. [2073](jhy/jsoup#2073)
-   **XML CData output**: When generating XML-syntax output from parsed HTML, script nodes containing (pseudo) CData
    sections would have an extraneous CData section added, causing script execution errors. Now, the data content is
    emitted in a HTML/XML/XHTML polyglot format, if the data is not already within a CData
    section. [2078](jhy/jsoup#2078)
-   **Thread safety**: The `:has` evaluator held a non-thread-safe Iterator, and so if an Evaluator object was
    shared across multiple concurrent threads, a NoSuchElement exception may be thrown, and the selected results may be
    incorrect. Now, the iterator object is a thread-local. [2088](jhy/jsoup#2088)

***

Older changes for versions 0.1.1 (2010-Jan-31) through 1.17.1 (2023-Nov-27) may be found in
[change-archive.txt](./change-archive.txt).

</details>

<details>
<summary>vladmihalcea/hypersistence-utils</summary>

### [`v3.7.0`](https://github.com/vladmihalcea/hypersistence-utils/blob/HEAD/changelog.txt#Version-370---December-18-2023)

\================================================================================

Oracle Interval Type does not support negative intervals [#&#8203;682](vladmihalcea/hypersistence-utils#682)

Return original object if target and original are the same when merging [#&#8203;677](vladmihalcea/hypersistence-utils#677)

Add a hypersistence-utils-hibernate-63 module for Hibernate 6.3 [#&#8203;657](vladmihalcea/hypersistence-utils#657)

</details>

<details>
<summary>quarkusio/quarkus</summary>

### [`v3.6.4`](https://github.com/quarkusio/quarkus/releases/tag/3.6.4)

[Compare Source](quarkusio/quarkus@3.6.3...3.6.4)

##### Complete changelog

-   [#&#8203;37808](quarkusio/quarkus#37808) - CLI - Rework how missing commands are detected
-   [#&#8203;37803](quarkusio/quarkus#37803) - Dev mode: add null checks to TimestampSet.isRestartNeeded()
-   [#&#8203;37798](quarkusio/quarkus#37798) - Only update ~/.docker/config.json if it exists
-   [#&#8203;37787](quarkusio/quarkus#37787) - Take priority into account in ConfigurationImpl
-   [#&#8203;37775](quarkusio/quarkus#37775) - Docs: fix typo in rabbitmq reference documentation
-   [#&#8203;37770](quarkusio/quarkus#37770) - Add SequencedCollection to BANNED_INTERFACE_TYPES
-   [#&#8203;37768](quarkusio/quarkus#37768) - Running application build with JDK21 and target Java 17 crash with NoClassDefFoundError: java/util/SequencedCollection
-   [#&#8203;37731](quarkusio/quarkus#37731) - Query logging is being done in io.quarkus.mongodb.panache.common.runtime.MongoOperations
-   [#&#8203;37723](quarkusio/quarkus#37723) - Do not use CSRF cookie as the next token value
-   [#&#8203;37717](quarkusio/quarkus#37717) - Docs: Fix incorrect link reference in Cross-Site Request Forgery Prevention guide
-   [#&#8203;37714](quarkusio/quarkus#37714) - Remove the driver property in the documentation for Cloud SQL
-   [#&#8203;37710](quarkusio/quarkus#37710) - Use NoStackTraceException in metrics
-   [#&#8203;37677](quarkusio/quarkus#37677) - Bump io.quarkus:quarkus-platform-bom-maven-plugin from 0.0.100 to 0.0.101
-   [#&#8203;37654](quarkusio/quarkus#37654) - Make sure dev mode is properly written in doc
-   [#&#8203;36848](quarkusio/quarkus#36848) - CSRF Token with HMAC signature gets double signed

</details>

<details>
<summary>quarkusio/quarkus-platform</summary>

### [`v3.6.4`](quarkusio/quarkus-platform@3.6.3...3.6.4)

[Compare Source](quarkusio/quarkus-platform@3.6.3...3.6.4)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox.

👻 **Immortal**: This MR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4yNC4wIiwidXBkYXRlZEluVmVyIjoiMzQuMjQuMCJ9-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Confirmed bug that we should fix fixed
Projects
None yet
Development

No branches or pull requests

2 participants