Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory corruption with slighthly bigger alphabets. #74

Open
c4f3a0ce opened this issue Dec 8, 2019 · 1 comment · May be fixed by #75
Open

Memory corruption with slighthly bigger alphabets. #74

c4f3a0ce opened this issue Dec 8, 2019 · 1 comment · May be fixed by #75

Comments

@c4f3a0ce
Copy link

c4f3a0ce commented Dec 8, 2019

When I try to use Trie with larger alphabet I start to experience different memory related issue. Details differ from Python version to Python version, and environment, but in general following:

import datrie

xs = [chr(i) for i in range(500)]

datrie.Trie(xs[:250])  # Works fine
datrie.Trie(xs[250:])  # Works fine

datrie.Trie(xs)  # Fails miserably

fails when executed as script with:

malloc(): invalid size (unsorted)
[1]    19260 abort      python foo.py

In IPython I got:

corrupted double-linked list                                                                                                                                                    
[1]    19270 abort      ipython

or

malloc(): invalid size (unsorted)
[1]    19364 abort      ipython

and some other, less common failures.

This specific outcomes tested with

  • datrie==0.8
  • Python 3.7.3
  • IPython 7.10.1

Is this something expected?
@c4f3a0ce
Copy link
Author

c4f3a0ce commented Dec 8, 2019

After further investigation it seems it might be related to tlwg/libdatrie#6.

I've updated and rebuild the package from fa3cfc3 by updating libdatrie submodule to tlwg/libdatrie@6ef4485 and the problem disappears.

c4f3a0ce added a commit to c4f3a0ce/datrie that referenced this issue Dec 8, 2019
@c4f3a0ce
Update libdatatrie to 6ef4485474890606946ed208ff453231b581cdf1 and
0cca86e 
provide tests for pytries#74

This resolves pytries#74
@c4f3a0ce c4f3a0ce linked a pull request Dec 8, 2019 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update libdatatrie to 6ef4485474890606946ed208ff453231b581cdf1 c4f3a0ce/datrie
1 participant
@c4f3a0ce