Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java.lang.IllegalArgumentException: Comparison method violates its general contract! #1628

Open
w296488320 opened this issue Aug 11, 2022 · 26 comments
Assignees
Labels
bug GUI Issues in jadx-gui module
Milestone

Comments

@w296488320
Copy link

Please describe what you did before the error occurred.
IMPORTANT! If the error occurs with a specific APK file please attach or provide link to apk file!

  • Jadx version: 1.4.3
  • Java version: 11.0.12
  • Java VM: JetBrains s.r.o OpenJDK 64-Bit Server VM
  • Platform: Windows 10 (10.0 amd64)
  • Max heap size: 30688 MB
  • Program args: -Xms128M -XX:MaxRAMPercentage=70.0 -XX:+UseG1GC -Dawt.useSystemAAFontSettings=lcd -Dswing.aatext=true
java.lang.IllegalArgumentException: Comparison method violates its general contract!
	at java.base/java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:870)
	at java.base/java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:487)
	at java.base/java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:413)
	at java.base/java.util.ComparableTimSort.sort(ComparableTimSort.java:213)
	at java.base/java.util.Arrays.sort(Arrays.java:1249)
	at java.desktop/sun.awt.shell.Win32ShellFolderManager2.get(Win32ShellFolderManager2.java:313)
	at java.desktop/sun.awt.shell.ShellFolder.get(ShellFolder.java:259)
	at java.desktop/javax.swing.plaf.metal.MetalFileChooserUI$DirectoryComboBoxModel.addItem(MetalFileChooserUI.java:1029)
	at java.desktop/javax.swing.plaf.metal.MetalFileChooserUI.doDirectoryChanged(MetalFileChooserUI.java:717)
	at java.desktop/javax.swing.plaf.metal.MetalFileChooserUI$5.propertyChange(MetalFileChooserUI.java:806)
	at java.desktop/java.beans.PropertyChangeSupport.fire(PropertyChangeSupport.java:341)
	at java.desktop/java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:333)
	at java.desktop/java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:266)
	at java.desktop/java.awt.Component.firePropertyChange(Component.java:8754)
	at java.desktop/javax.swing.JFileChooser.setCurrentDirectory(JFileChooser.java:608)
	at java.desktop/javax.swing.JFileChooser.<init>(JFileChooser.java:362)
	at java.desktop/javax.swing.JFileChooser.<init>(JFileChooser.java:338)
	at jadx.gui.ui.dialog.FileDialog$FileChooser.<init>(FileDialog.java:156)
	at jadx.gui.ui.dialog.FileDialog.buildFileChooser(FileDialog.java:138)
	at jadx.gui.ui.dialog.FileDialog.show(FileDialog.java:74)
	at jadx.gui.ui.MainWindow.openFileOrProject(MainWindow.java:295)
	at jadx.gui.ui.MainWindow.processCommandLineArgs(MainWindow.java:252)
	at jadx.gui.ui.MainWindow.init(MainWindow.java:247)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:776)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:746)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 11, 2022

Same issue here, see #1606

@w296488320
Copy link
Author

Same issue here, see #1606
Is there a better solution right now ,
Very strange, yesterday or good to use, today suddenly appeared this problem, the computer restart also doesn't work

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 11, 2022

It's likely to be a bug introduced in a recent Windows 11 update. I did a temporary fix: https://github.com/xxr0ss/jadx/tree/fix-broken-filedialog.

@jpstotz
Copy link
Collaborator

jpstotz commented Aug 11, 2022

@xxr0ss But this time it is not Windows 11 but 10 according to the bug report. Unfortunately I wasn't able to reproduce this bug on both Windows versions. The affected systems seem to have a special property like a special connected device (e.g. via USB) or a installed software.

@skylot
Copy link
Owner

skylot commented Aug 11, 2022

So as @xxr0ss in his fix uses java.awt.FileDialog, I think I will add it as an option to use as a workaround for this issue.
This change is also related to #1213 and commit 79405f9 in branch file-dialog.

@skylot skylot self-assigned this Aug 11, 2022
@skylot skylot added bug GUI Issues in jadx-gui module labels Aug 11, 2022
@skylot skylot added this to the TBD milestone Aug 11, 2022
@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 11, 2022

@jpstotz @skylot
I've debugged FileDialog and it turned out to be a bug in javax.swing.JFileChooser.

Here's the quick fix:
Add -Djava.util.Arrays.useLegacyMergeSort=true to JVM_OPTS

@jpstotz
Copy link
Collaborator

jpstotz commented Aug 11, 2022

@xxr0ss Thanks for your investigation. But I am not sure If I understand the result you have posted. If the bug is caused by JFileChooser how can switching the sort algorithm prevent it from happening? Can you give us more details on the problem or are you preparing a OoenJDK bug report?

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 11, 2022

So here's the detail @jpstotz :
ShellFolder is being sorted when the exception throws. JFileChooser use Arrays.sort() to sort ShellFolder. The NEW TimSort is introduced in JDK 7, which requires strict comparison: 0 should be returned when two comparables are equal.
Let's see how sun.awt.shell.ShellFolder implements comparasion:

    private static final Comparator<File> FILE_COMPARATOR = new Comparator<File>() {
        public int compare(File f1, File f2) {
            ShellFolder sf1 = null;
            ShellFolder sf2 = null;

            if (f1 instanceof ShellFolder) {
                sf1 = (ShellFolder) f1;
                if (sf1.isFileSystem()) {
                    sf1 = null;
                }
            }
            if (f2 instanceof ShellFolder) {
                sf2 = (ShellFolder) f2;
                if (sf2.isFileSystem()) {
                    sf2 = null;
                }
            }

            if (sf1 != null && sf2 != null) {
                return sf1.compareTo(sf2);
            } else if (sf1 != null) {
                // Non-file shellfolders sort before files
                return -1;
            } else if (sf2 != null) {
                return 1;
            } else {
                String name1 = f1.getName();
                String name2 = f2.getName();

                // First ignore case when comparing
                int diff = name1.compareToIgnoreCase(name2);
                if (diff != 0) {
                    return diff;
                } else {
                    // May differ in case (e.g. "mail" vs. "Mail")
                    // We need this test for consistent sorting
                    return name1.compareTo(name2);
                }
            }
        }
    };

You can see the return value can only be 1 or -1. And that's what the TimSort is unhappy about.
So a quick fix is to force Arrays.sort() to use legacy sort.

    public static void sort(Object[] a) {
        if (LegacyMergeSort.userRequested)
            legacyMergeSort(a);
        else
            ComparableTimSort.sort(a, 0, a.length, null, 0, 0);
    }

To me this is quite unusual😂!!! I've been using Java 11 for a long time, but I didn't have this problem until recently.

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 11, 2022

@jpstotz I don't have an account in OpenJDK Community and I cann't even find where to sign up one😓. And I've being a bit busy lately. It'll be nice of you to report this bug to OpenJDK!

@jpstotz
Copy link
Collaborator

jpstotz commented Aug 11, 2022

@xxr0ss Thanks for the detailed explanation, I think I got the point but I have to look at the code myself because name1.compareToIgnoreCase and name1.compareTo can return 0.

Last time I checked the Java bug tracker was still "ruled" (provided) by Oracle and they don't provide accounts unless you have contributed code to OpenJDK. Therefore for Java developers there is only an email address where you can send bug reports which will create a new issue. But without account you can't comment or anything else.

I will try to create a minimal reproduceable example to demonstrate the bug.

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 11, 2022

Understood👍@jpstotz

I will also try to find out what's so special about the ShellFolders on my computer that causes TimSort to fail, just when I have time.

@jpstotz
Copy link
Collaborator

jpstotz commented Aug 12, 2022

@xxr0ss Debugging Jadx and the code at java.desktop/sun.awt.shell.Win32ShellFolderManager2.get(Win32ShellFolderManager2.java:313) I have to admit that I don't see how sun.awt.shell.ShellFolder.FILE_COMPARATOR is used. May be you have discovered a second code path of the same bug?

According to the stack trace the error starts here:
Win32ShellFolderManager2

public Object get(String key)
    ...
    File[] secondLevelFolders = checkFiles(desktop.listFiles());
    Arrays.sort(secondLevelFolders);

At run-time if I look into secondLevelFolders it contains at the beginning special folders like This PC, Network, Libraries and then all files on my desktop.
I don't think the real files on the desktop cause problems, so most likely the special folders are the problem. Is it possible that on your system is one special folder twice in this list?

I tried to create a simple example of an Array containing special folder entries. Unfortunately the module system of Java does not allow to use/invoke ShellFolder and related classes outside of the module java.desktop even with reflection. At such times you really want to throw this damn module system against the next wall :(

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 12, 2022

@jpstotz 😵I'm sorry I messed things up!! I debugged again and FILE_COMPARATOR is not used. The comparison that is used is in sun.awt.shell.Win32ShellFolderManager2

static int compareShellFolders(Win32ShellFolder2 sf1, Win32ShellFolder2 sf2)

Duplicated folder entries do exist on my computer! I logged Arrays.sort()

// sun.awt.shell.Win32ShellFolderManager2
        folders.add(desktop);
        // Add all second level folders
        File[] secondLevelFolders = checkFiles(desktop.listFiles());
log>    Arrays.sort(secondLevelFolders);
        for (File secondLevelFolder : secondLevelFolders) {
Documents, ::{20D04FE0-3AEA-1069-A2D8-08002B30309D}, ::{F02C1A0D-BE21-4350-88B0-7367FC96EF3C}, 
::{031E4825-7B94-4DC3-B131-E946B44C8DD5}, OneDrive, ShellFolder: , Music, Downloads, Pictures, 
Videos, Desktop, Desktop, Downloads, Documents, Pictures, XXX

You can see Desktop, Downloads, Pictures, Documents are duplicated.

And I'm not sure if the return value of comparison for TimSort is requied be no other than -1, 0, 1?

// sun.awt.shell.Win32ShellFolder2
    public int compareTo(File file2) {
        ...
        return Win32ShellFolderManager2.compareShellFolders(this, (Win32ShellFolder2) file2);
    }

This comparison returns value like below, which

14, 5, 3, -1, 25932, 25923, -10

@jpstotz
Copy link
Collaborator

jpstotz commented Aug 12, 2022

@xxr0ss values other than [-1, 0, 1] returned by compareTo should not be a problem:

Return a negative integer, zero, or a positive integer as this object is less than, equal to, or greater than the specified object.

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 12, 2022

@jpstotz are there any ShellFolders on your computer that have the same name, while one is a special entry and the other not?

I logged isSpecial() for ShellFolders being compared. See here:
image
0 is returned here, but I do not think a non-special ShellFolder should be equal to a special one when they have the same name.

@jpstotz
Copy link
Collaborator

jpstotz commented Aug 13, 2022

@xxr0ss That is indeed strange and might be the cause. I assume you are printing the elements using toString(). I noticed that special folders return a different value (usually the used CLSID) using getPath() (and getPath() is used by `Win32ShellFolder2.equals).

Can you identify which element on your Windows Desktop is creating this special documents folder entry?

@xxr0ss
Copy link
Contributor

xxr0ss commented Aug 13, 2022

@jpstotz Oh! I just stopped getting the exception after I removed all special folders (Documents, Pictures ...) from Quick Access.

And I went to C:\Users\<UserName> and added them all (Documents, Pictures...) back to Quick Access, I debugged again and the special and non-special Documents both exist and the code here was executed, however, the exception does not show up any more.

image

@uckelman
Copy link

I came across this issue while troubleshooting the same problem for a user on a different project. Here is quite a long thread detailing that effort. If you scroll all the way to the bottom, you'll find two posts where I explain the cause.

The short version is: This line in sun.awt.shell.Win32ShellFolderManager2.compareShellFolders() is wrong. The condition should be a conjunction instead of a disjunction.

I submitted a bug report to OpenJDK earlier today. I'd encourage you to do the same; maybe we can light a fire under them to get this fixed.

@mbien
Copy link

mbien commented Aug 3, 2023

looks like the JDK tracks it as https://bugs.openjdk.org/browse/JDK-8305072

@edemen
Copy link

edemen commented Aug 7, 2023

looks like the JDK tracks it as https://bugs.openjdk.org/browse/JDK-8305072

They seem to have decided to close it being unable to reproduce.

@mbien
Copy link

mbien commented Aug 7, 2023

@edemen if someone knows how to reproduce it.. let them know. The test code is in the bug report, what is missing is the right combination of "special" and "non special" windows folders - whatever that is.

@uckelman
Copy link

uckelman commented Aug 7, 2023

Nice. They don't need to be able to reproduce the problem to see what's wrong---it should be obvious just from reading the code. Argh.

@edemen
Copy link

edemen commented Aug 7, 2023

@edemen if someone knows how to reproduce it.. let them know. The test code is in the bug report, what is missing is the right combination of "special" and "non special" windows folders - whatever that is.

For a long time I had two desktop.ini files on my Desktop. Somehow then one of them disappeared months later.
I suspect this has something to do with mixing shortcuts/settings that are configured for the individual user, and the ones shared for all users. I don't have the time to verify this yet though. Specifically, the user may not always be able to override shared shortcuts/settings, so perhaps this can lead to this multiple files with the same name in a virtual/combined special folder, like Desktop? Idk.

@uwemock
Copy link

uwemock commented Feb 22, 2024

Just a quick note to let you know I'm facing this same problem with someone using OpenJDK 21. Is there anything I can do to help?

@skylot
Copy link
Owner

skylot commented Feb 22, 2024

@uwemock workaround for this issue (legacy sort JVM flag: -Djava.util.Arrays.useLegacyMergeSort=true) already in master, so you can use latest unstable build.
This issue kept open to track progress on JDK fix, but looks like fix will not be applyied because this issue is hard to reproduce 🙁

@uckelman
Copy link

My fix was merged: openjdk/jdk#18126

It ought to appear in the next JDK release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug GUI Issues in jadx-gui module
Projects
None yet
Development

No branches or pull requests

8 participants