Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

com.arcadedb.graph.ImmutableVertex cannot be cast to class com.arcadedb.graph.Edge when trying to remove whole DB #1476

Closed
ExtReMLapin opened this issue Feb 21, 2024 · 14 comments
Assignees
Labels
bug Something isn't working fixed
Milestone

Comments

@ExtReMLapin
Copy link
Contributor

Hello,

ArcadeDB Version:

Latest release 24.1.1

OS and JDK Version:

Tried on both windows and ubuntu

Running on Windows 10 10.0 - Java HotSpot(TM) 64-Bit Server VM 21.0.1

Expected behavior

whole db to be wiped

Actual behavior

com.arcadedb.exception.CommandParsingException: Error on executing Cypher query
        at com.arcadedb.cypher.query.CypherQueryEngine.command(CypherQueryEngine.java:80)
        at com.arcadedb.database.LocalDatabase.command(LocalDatabase.java:1348)
        at com.arcadedb.server.ServerDatabase.command(ServerDatabase.java:472)
        at com.arcadedb.server.http.handler.PostCommandHandler.executeCommand(PostCommandHandler.java:131)
        at com.arcadedb.server.http.handler.PostCommandHandler.execute(PostCommandHandler.java:110)
        at com.arcadedb.server.http.handler.DatabaseAbstractHandler.execute(DatabaseAbstractHandler.java:100)
        at com.arcadedb.server.http.handler.AbstractServerHttpHandler.handleRequest(AbstractServerHttpHandler.java:127)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
        at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ClassCastException: class com.arcadedb.graph.ImmutableVertex cannot be cast to class com.arcadedb.graph.Edge (com.arcadedb.graph.ImmutableVertex and com.arcadedb.graph.Edge are in unnamed module of loader 'app')
        at com.arcadedb.database.RID.asEdge(RID.java:154)
        at com.arcadedb.graph.EdgeIterator.next(EdgeIterator.java:84)
        at com.arcadedb.graph.EdgeIterator.next(EdgeIterator.java:30)
        at com.arcadedb.graph.GraphEngine.deleteVertex(GraphEngine.java:358)
        at com.arcadedb.database.LocalDatabase.deleteRecordNoLock(LocalDatabase.java:1005)
        at com.arcadedb.database.LocalDatabase.lambda$deleteRecord$20(LocalDatabase.java:973)
        at com.arcadedb.database.LocalDatabase.executeInReadLock(LocalDatabase.java:1415)
        at com.arcadedb.database.LocalDatabase.deleteRecord(LocalDatabase.java:972)
        at com.arcadedb.gremlin.ArcadeGraph.deleteElement(ArcadeGraph.java:449)
        at com.arcadedb.gremlin.ArcadeElement.remove(ArcadeElement.java:89)
        at org.apache.tinkerpop.gremlin.process.traversal.step.filter.DropStep.filter(DropStep.java:67)
        at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:41)
        at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:155)
        at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:192)
        at com.arcadedb.gremlin.ArcadeGremlin$2.hasNext(ArcadeGremlin.java:112)
        at com.arcadedb.query.sql.executor.IteratorResultSet.hasNext(IteratorResultSet.java:37)
        at com.arcadedb.cypher.ArcadeCypher.execute(ArcadeCypher.java:75)
        at com.arcadedb.cypher.query.CypherQueryEngine.command(CypherQueryEngine.java:77)
        ... 13 more

Steps to reproduce

db_test.zip

download db linked in message, and run match (n) detach delete n cypher command on DB

It's my coworker that has this issue, she claims she's been running this command for ages and it's only now that it started being a problem.

@lvca lvca self-assigned this Feb 21, 2024
@lvca lvca added this to the 24.2.1 milestone Feb 21, 2024
@lvca
Copy link
Contributor

lvca commented Feb 21, 2024

Reproduced, thanks for the database!

@lvca lvca added bug Something isn't working in progress The team is actively working on this issue labels Feb 21, 2024
@lvca
Copy link
Contributor

lvca commented Feb 21, 2024

I see the RID inside the edge list contain vertices instead of edges. How this database was populated? Did you import it from another database?

@ExtReMLapin
Copy link
Contributor Author

Thanks for the quick answer,

Tbh i'm just doing the admin sys role here, while i wrote the python rest driver she's using, I didn't toy with the ArcadeDB myself.

The database was created only using cypher requests.

I'll ask my coworker tomorrow (it's 19:44 here), is it possible that what you're talking about is an human error on her side, like the way she populated the database ? Or could it be caused by some other bug ?

@lvca
Copy link
Contributor

lvca commented Feb 21, 2024

It could be possible that the Cypher API have an issue and are creating edges in the wrong way.

@ExtReMLapin
Copy link
Contributor Author

I'll be honest, she's a junior, so it's possible she did something wrong.

I'll add a log in our driver so we can keep a track of the requests.

@ExtReMLapin
Copy link
Contributor Author

I have the confirmation from her that she created nodes exactly this way in Neo4J and she never had issue with it.

@lvca
Copy link
Contributor

lvca commented Mar 12, 2024

Is there a way I can reproduce this error? I want to test it against the latest 24.2.1. Thanks in advance.

@ExtReMLapin
Copy link
Contributor Author

You mean like a code that could produce the same db ?

I’m not allowed to share it.

nothing too specific but my coworker was probably creating edges and nodes and multiple short requests, very low milisec delay between then.

I am out of office in lapland for 7days, sorry for delay

@ExtReMLapin
Copy link
Contributor Author

I gave a try with fuzzing, creating a lot of nodes and edges. Which cannot reproduce the bug.

What we'll do is we will logs all commands sent to the rest api and try to reproduce it with that

@ExtReMLapin
Copy link
Contributor Author

ExtReMLapin commented Mar 22, 2024

FINALLY ! @lvca

Here are two files to reproduce the error.

You'll need to edit the python file to set the correct IP and EMPTY database name.
Please note the database name has to be empty and fresh to be sure the id created in the logs are the same as the ones created in the new one, creating a lookup tables of new and old id is a mess because they were not always returned after creation.

you might need to install python and do pip install requests , there is about 6k requests so it should not take more than one minute.

login for database is set to root:rootroot in script

do not hesitate to get back to me if you meet any difficulty while trying to run this script.

import json
import requests

db_name = "bugPierre42"
file_history = "logs_final.txt"
import requests
import json
headers = {"Content-Type": "application/json"}


url = "http://192.168.25.25:2480/api/v1/query/"+db_name
def subhandler(response: requests.Response, return_headers: bool=False):
        if response.status_code >= 400 :
            print(response.text)
            json_decoded_data = response.json()
            exc = json_decoded_data["exception"] if "detail" not in json_decoded_data else json_decoded_data["detail"]
            raise ValueError(exc)
        
        response.raise_for_status()
        if return_headers is False:
            if len(response.text) > 0:
                try:
                    return response.json()["result"]
                except:
                    return response.text
            else:
                return
        else:
            return  response.headers

def send_request(payload):
    response = requests.post(
        url,
        data=json.dumps(payload),
        auth=('root', 'rootroot'),
    )
    return subhandler(response, return_headers=False)

def parse_request(line):
    start_payload = 10
    end_payload = line.find("=>")
    payload = line[start_payload:end_payload]
    payload_obj = json.loads(payload)
    returned_data = line[end_payload+3:]
    returned_data_obj = json.loads(returned_data)

    return payload_obj, returned_data_obj

lines_db = []
with open(file_history, "r") as file:
    lines = file.readlines()
    for line in lines:
        payload_obj, old_returned_data_obj = parse_request(line)
        lines_db.append((payload_obj, old_returned_data_obj))


for payload_obj, old_returned_data_obj in lines_db:
    real_now_returned_data = send_request(payload_obj)
    if len(real_now_returned_data) != len(old_returned_data_obj):
        print("Error in the data, len mismatch")
        print(payload_obj)
        print(old_returned_data_obj)
        print(real_now_returned_data)
        break

logs_final.txt

The script will open logs_final.txt and run all the cypher command in the same order. On our server, the created RIDs from a run to another are the same, I hope it will be the same on your end.

@lvca
Copy link
Contributor

lvca commented Mar 23, 2024

@ExtReMLapin thanks for the test case. I was able to reproduce the issue.

@ExtReMLapin
Copy link
Contributor Author

Hooray !

@lvca
Copy link
Contributor

lvca commented Mar 23, 2024

OK, seems there is nothing wrong with the insertion or the database itself. Not even with Cypher. The issue is with the delete: while the vertices/edges are deleted, the iterator points to the wrong link (containing the connected vertex instead of the edge). Working on it.

lvca added a commit that referenced this issue Mar 23, 2024
@lvca
Copy link
Contributor

lvca commented Mar 23, 2024

Thanks so much for the test case. I was able to isolate the issue and fix it. The deletion of vertices now postpones to the end the deletion of the edges, so no concurrency anymore during the deletion of connected vertices in the same transaction.

@lvca lvca closed this as completed Mar 23, 2024
@lvca lvca added fixed and removed waiting for user in progress The team is actively working on this issue labels Mar 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working fixed
Projects
None yet
Development

No branches or pull requests

2 participants