Selection not in cache #73

stef204 · 2018-04-03T23:15:20Z

I have used clipmenu only for a couple of days but have run into one particular issue:

At some point, it will stop pasting what I select and only paste the top item. I usually try many times, and try to use other lines, etc., and it seems erratic, pasting some selections but not others, etc.

I see this: "${CM_MAX_CLIPS=1000}" (pasted here with clipmenu, mind you!)

Is it 1000 lines or number of clips/snippets, something else?

This could be happening only when clipboard is full but not sure yet.

The text was updated successfully, but these errors were encountered:

cdown · 2018-04-03T23:47:00Z

Maybe this is #71 or #70. Thanks for the clear description.

1000 is the number of clips, ie. we only keep 1000 clipboard entries in cache. I'll do some testing in a bit.

stef204 · 2018-04-04T09:53:35Z

Happy to test and help troubleshoot, just let me know what is needed.

cdown · 2018-04-04T11:05:00Z

Thanks, I've not been able to repro thus far. The best thing to do would be to show bash -x "$(command -v clipmenu)", select a clip, and give me an example where it selected the wrong thing.

stef204 · 2018-04-05T10:59:31Z

If I run above command, it brings up rofi/clipmenu. I then either use a clip by pressing enter. and It returns to command line, or presse Escape, exiting in both cases.

IOW, you want to me to stop using clipmenu and substitute that for bash -x "$(command -v clipmenu)" every single time until the problem occurs. (Best would be I modify the KB shortcut I set up.) Is that correct?

cdown · 2018-04-05T11:00:29Z

Yeah, that would be best if it's not reproducible every time.

You might want to redirect the output to a file so that you can get it afterwards.

stef204 · 2018-04-05T12:07:20Z

You're going to have to help me out; having trouble redirecting to a file (sorry, I cannot take the time to RTFM bash at the moment due to work.)

I am using a script (for which I have created a KB shortcut):
Note: the KB shortcut obviously doesn't work since you'd need a shell opened to run the script. But I can just keep a shell open in a small floating window and run: cdb which is in my path.

#!/bin/sh

bash -x "$(command -v clipmenu)" >> $HOME/cdb.log
exit

It does not redirect to the file which I created using touch cdb.log

I tried a number of pipes, etc., but I am obviously missing the info on this very basic point.

I tried 2>&1 > cdb.log but no joy there either.

How do I amend the script to capture output in a way it appends and does not overwrite?

Edit1: not actually sure we need to append, actually. but you chime in.

Edit2: this below is where I am now, does not exit shell cleanly but captures..... But set me straight please....

cdb 2>&1 | tee cdb.log

stef204 · 2018-04-05T12:16:04Z

Also, I think I may have found something, clipmenu may have trouble with symbols or path names, etc. That is how the "bug" showed up first. And it is now back again:

Here is the log on one instance.

And here on another instance.

I should say though that I was just able to paste the url to those logs using clipmenu so there is more to it than meets the eye.

So when this issue occurs, it pastes something else instead of the selection.

cdown · 2018-04-05T14:45:39Z

When you next hit this, it would be ace to check if there is any file at all that contains similar contents. For example in this case:

# replace 2>&1 with whatever it said didn't exist
grep -FR '2>&1' /run/user/1000/clipmenu.5.user

Doing it now may be too late as we may have legitimately purged it

cdown · 2018-04-06T20:16:12Z

(comment deleted upon request)

stef204 · 2018-04-06T22:54:25Z

Here is one.

And here is the grep -FR.

So again, the result is that clipmenu pastes the wrong clip/snippet.

Could have something to do with escaping, quote marks, etc.?

cdown · 2018-04-07T10:18:03Z

Thanks! After seeing that, I think I have a suspicion about what's happening here.

For CM_MAX_CLIPS, the logic is that we should delete the clip if it falls too far from the end of the file. This logic was fine when there was a single file, but now that we have multiple files, it's possible that this clip also exists in the other selection and therefore should not be deleted.

Hmm... this is not super easy to mitigate in the current model. Let me have a think about it.

Would this possibly explain your situation? :-)

Fixes #73.

cdown · 2018-04-07T10:57:01Z

I believe 0364087 fixes this. Can you test it out for a few days and let me know if it resolves the issue? :-) Thanks!

cdown · 2018-04-07T10:57:17Z

Also cc @kaihendry who was having similar issues

kaihendry · 2018-04-07T11:02:15Z

Thanks Chris! Do you have a quick tip to modify https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=clipmenu so that I can test this branch?

cdown · 2018-04-07T11:06:53Z

You should be able to test with this PKGBUILD:

# Maintainer: Chris Down <chris@chrisdown.name>

pkgname=clipmenu
pkgver=5.1.0
pkgrel=1
pkgdesc='Clipboard management using dmenu'
url='http://github.com/cdown/clipmenu'
arch=('any')
license=('Public Domain')
depends=(dmenu xsel clipnotify)
conflicts=(clipmenu-git)  # Old package from pre-AUR4 days

source=("https://github.com/cdown/clipmenu/archive/deletion_bug_1.zip")
md5sums=('6fd9b534127012d6b8202aa3aedc1844')

package() {
    install -D -m755 \
        "${srcdir?}/$pkgname-deletion_bug_1/clipmenu" \
        "${pkgdir?}/usr/bin/clipmenu"
    install -D -m755 \
        "${srcdir?}/$pkgname-deletion_bug_1/clipmenud" \
        "${pkgdir?}/usr/bin/clipmenud"
    install -D -m644 \
        "${srcdir?}/$pkgname-deletion_bug_1/init/clipmenud.service" \
        "${pkgdir?}/usr/lib/systemd/user/clipmenud.service"
}

stef204 · 2018-04-07T16:21:09Z

Thanks Chris. Have installed debug version and testing now. Will report back.

kaihendry · 2018-04-08T10:11:02Z

Same selection issue FATAL: .. not in cache

[hendry@t480s ~]$ cd /run/user/1000/clipmenu.5.hendry/
[hendry@t480s clipmenu.5.hendry]$ grep -r CSC .
./177862433 12:CSC Loyang(
./1074766712 9:CSC Loya
./line_cache_primary:1523097259506416042 CSC Loyang(
./line_cache_primary:1523097260711142961 CSC Loyang
./line_cache_clipboard:1523097262864155235 CSC Loyang
[hendry@t480s clipmenu.5.hendry]$ pacman -Ql clipmenu
clipmenu /usr/
clipmenu /usr/bin/
clipmenu /usr/bin/clipmenu
clipmenu /usr/bin/clipmenud
clipmenu /usr/lib/
clipmenu /usr/lib/systemd/
clipmenu /usr/lib/systemd/user/
clipmenu /usr/lib/systemd/user/clipmenud.service
[hendry@t480s clipmenu.5.hendry]$ cd
[hendry@t480s ~]$ vim /usr/bin/clipmenu
[hendry@t480s ~]$ cd
[hendry@t480s ~]$ cd
[hendry@t480s ~]$ cd tmp/dwm/
[hendry@t480s dwm]$ vim config.h
[hendry@t480s dwm]$ cd
[hendry@t480s ~]$ whereis clipmenu
clipmenu: /usr/bin/clipmenu
[hendry@t480s ~]$ clipmenu
FATAL: CSC Loyang not in cache
[hendry@t480s ~]$ pacman -Qi clipmenu
Name            : clipmenu
Version         : 5.1.0-1
Description     : Clipboard management using dmenu
Architecture    : any
URL             : http://github.com/cdown/clipmenu
Licenses        : Public Domain
Groups          : None
Provides        : None
Depends On      : dmenu  xsel  clipnotify
Optional Deps   : None
Required By     : None
Optional For    : None
Conflicts With  : clipmenu-git
Replaces        : None
Installed Size  : 11.00 KiB
Packager        : Unknown Packager
Build Date      : Sun 08 Apr 2018 08:34:18 AM
Install Date    : Sun 08 Apr 2018 08:34:19 AM
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : None

[hendry@t480s ~]$

cdown · 2018-04-08T13:44:38Z

As mentioned, I need to see bash -x "$(command -v clipmenu)" instead of just clipmenu when this happens, as otherwise it's impossible to know what it actually expected vs. what was available

cdown · 2018-04-08T13:46:11Z

I'll also add it to the FATAL message for an upcoming release

kaihendry · 2018-04-09T00:32:27Z

[hendry@t480s ~]$ bash -x "$(command -v clipmenu)"
+ : dmenu
+ : /run/user/1000
+ major_version=5
+ shopt -s nullglob
+ cache_dir=/run/user/1000/clipmenu.5.hendry
+ cache_file_prefix=/run/user/1000/clipmenu.5.hendry/line_cache
+ [[ '' == --help ]]
+ [[ '' == -h ]]
+ [[ dmenu == rofi ]]
+ [[ dmenu == rofi-script ]]
++ list_clips
++ dmenu -l 8
++ LC_ALL=C
++ sort -rnk 1
++ cat /run/user/1000/clipmenu.5.hendry/line_cache_clipboard /run/user/1000/clipmenu.5.hendry/line_cache_primary /dev/null
++ cut '-d ' -f2-
++ awk '!seen[$0]++'
+ chosen_line='CSC Loyang'
+ [[ -n CSC Loyang ]]
++ cksum
+ file='/run/user/1000/clipmenu.5.hendry/619706596 11'
+ [[ -f /run/user/1000/clipmenu.5.hendry/619706596 11 ]]
+ printf 'FATAL: %s not in cache\n' 'CSC Loyang'
FATAL: CSC Loyang not in cache
+ exit 2
[hendry@t480s ~]$
[hendry@t480s ~]$ cat /run/user/1000/clipmenu.5.hendry/619706596
cat: /run/user/1000/clipmenu.5.hendry/619706596: No such file or directory
[hendry@t480s ~]$ grep 'CSC Loyang' /run/user/1000/clipmenu.5.hendry/*
/run/user/1000/clipmenu.5.hendry/1044873404 35:./line_cache_primary:1523097259506416042 CSC Loyang(
/run/user/1000/clipmenu.5.hendry/1044873404 35:./line_cache_primary:1523097260711142961 CSC Loyang
/run/user/1000/clipmenu.5.hendry/1044873404 35:./line_cache_clipboard:1523097262864155235 CSC Loyang
/run/user/1000/clipmenu.5.hendry/1044873404 35:FATAL: CSC Loyang not in cache
/run/user/1000/clipmenu.5.hendry/177862433 12:CSC Loyang(
/run/user/1000/clipmenu.5.hendry/2490308208 62:+ chosen_line='CSC Loyang'
/run/user/1000/clipmenu.5.hendry/2490308208 62:+ [[ -n CSC Loyang ]]
/run/user/1000/clipmenu.5.hendry/2490308208 62:+ printf 'FATAL: %s not in cache\n' 'CSC Loyang'
/run/user/1000/clipmenu.5.hendry/2490308208 62:FATAL: CSC Loyang not in cache
/run/user/1000/clipmenu.5.hendry/3203152914 46:+ chosen_line='CSC Loyang'
/run/user/1000/clipmenu.5.hendry/3203152914 46:+ [[ -n CSC Loyang ]]
/run/user/1000/clipmenu.5.hendry/3203152914 46:+ printf 'FATAL: %s not in cache\n' 'CSC Loyang'
/run/user/1000/clipmenu.5.hendry/3203152914 46:FATAL: CSC Loyang not in cache
/run/user/1000/clipmenu.5.hendry/45617049 25: CSC Loyang not in cache
/run/user/1000/clipmenu.5.hendry/619706596 11:CSC Loyang
/run/user/1000/clipmenu.5.hendry/line_cache_clipboard:1523097262864155235 CSC Loyang
/run/user/1000/clipmenu.5.hendry/line_cache_clipboard:1523182198206060452 FATAL: CSC Loyang not in cache
/run/user/1000/clipmenu.5.hendry/line_cache_clipboard:1523233748215523210 CSC Loyang
/run/user/1000/clipmenu.5.hendry/line_cache_primary:1523097259506416042 CSC Loyang(
/run/user/1000/clipmenu.5.hendry/line_cache_primary:1523097260711142961 CSC Loyang
/run/user/1000/clipmenu.5.hendry/line_cache_primary:1523182197539370431 FATAL: CSC Loyang not in cache
/run/user/1000/clipmenu.5.hendry/line_cache_primary:1523182208016423861 CSC Loyang
/run/user/1000/clipmenu.5.hendry/line_cache_primary:1523233748262785289 CSC Loyang
[hendry@t480s ~]$ cat '/run/user/1000/clipmenu.5.hendry/619706596 11'
CSC Loyang[hendry@t480s ~]$ [[ -f /run/user/1000/clipmenu.5.hendry/619706596 11 ]] && echo yay
-bash: syntax error in conditional expression
-bash: syntax error near `11'

Don't quite understand '/run/user/1000/clipmenu.5.hendry/619706596 11' What does the 11 mean? Why is there a space? Why is there a need for a _cache?

stef204 · 2018-04-09T23:36:29Z

Quick interim report: seems OK so far, don't believe have run into same problem (although there might have been an instance today but could not check it.)
Will continue to run the debug version and report here. So far, loads better.

cdown · 2018-04-10T08:30:40Z

@kaihendry

Don't quite understand '/run/user/1000/clipmenu.5.hendry/619706596 11' What does the 11 mean?

Quoting is not rendered properly with bash -x and [[. This is direct output from cksum, 11 is the length.

Something weird is going on in your case. What's the output of these?

stat '/run/user/1000/clipmenu.5.hendry/619706596 11'
[[ -f '/run/user/1000/clipmenu.5.hendry/619706596 11' ]]; echo $?

@stef204 Great!

kaihendry · 2018-04-10T08:51:13Z

[hendry@t480s ~]$ stat '/run/user/1000/clipmenu.5.hendry/619706596 11'
  File: /run/user/1000/clipmenu.5.hendry/619706596 11
  Size: 10              Blocks: 8          IO Block: 4096   regular file
Device: 33h/51d Inode: 5763489     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  hendry)   Gid: (  985/   users)
Access: 2018-04-09 08:29:17.061985488 +0800
Modify: 2018-04-09 08:29:08.266884746 +0800
Change: 2018-04-09 08:29:08.266884746 +0800
 Birth: -
[hendry@t480s ~]$ cat '/run/user/1000/clipmenu.5.hendry/619706596 11'
CSC Loyang[hendry@t480s ~]$ [[ -f '/run/user/1000/clipmenu.5.hendry/619706596 11' ]]; echo $?
0
[hendry@t480s ~]$

hmmm.... just tried again:

[hendry@t480s ~]$ bash -x "$(command -v clipmenu)"
+ : dmenu
+ : /run/user/1000
+ major_version=5
+ shopt -s nullglob
+ cache_dir=/run/user/1000/clipmenu.5.hendry
+ cache_file_prefix=/run/user/1000/clipmenu.5.hendry/line_cache
+ [[ '' == --help ]]
+ [[ '' == -h ]]
+ [[ dmenu == rofi ]]
+ [[ dmenu == rofi-script ]]
++ list_clips
++ dmenu -l 8
++ LC_ALL=C
++ sort -rnk 1
++ cut '-d ' -f2-
++ awk '!seen[$0]++'
++ cat /run/user/1000/clipmenu.5.hendry/line_cache_clipboard /run/user/1000/clipmenu.5.hendry/line_cache_primary /dev/null
+ chosen_line='CSC Loyang'
+ [[ -n CSC Loyang ]]
++ cksum
+ file='/run/user/1000/clipmenu.5.hendry/619706596 11'
+ [[ -f /run/user/1000/clipmenu.5.hendry/619706596 11 ]]
+ for selection in clipboard primary
+ xsel --logfile /dev/stderr -i --clipboard
+ for selection in clipboard primary
+ xsel --logfile /dev/stderr -i --primary
[hendry@t480s ~]$ CSC Loyang

This would help with #73.

stef204 · 2018-04-10T10:55:13Z

Still present, unfortunately. I couldn't get the output of bash -x "$(command -v clipmenu)" to paste or link to but basically looks same as before. Will do it again likely today and make sure to get output + grep -FR and link to it here.

kaihendry · 2018-04-12T06:58:14Z

I have the feeling it gets triggered when I have two similar selections..

cdown · 2018-04-12T10:26:37Z

It's possible that there is also a bug somewhere in the possible partial detection/mitigation code. I'll take a look later today.

kaihendry · 2018-04-13T01:37:06Z

Same problem again: https://s.natalian.org/2018-04-13/1523582975.mp4

Need to figure out the ~/.config/systemd/user/ systemctl --user status clipmenud environment variable override with CM_DEBUG set.

cdown · 2018-04-13T11:00:07Z

[Service]
...
ENVIRONMENT="CM_DEBUG=1"

cdown · 2018-04-13T11:03:46Z

In the video I also see other errors about not being able to remove files because they never existed: this makes me even more suspicious that this directory is being cleaned externally

kaihendry · 2018-04-18T04:57:17Z

CM_MAX_CLIPS caught me by surprise. This definitely seems indicative of something that's wrong https://s.natalian.org/2018-04-18/clipmenud.log

Is there an explanation of line_cache_clipboard and line_cache_primary ? I'm confused how it can even show up in dmenu see. Couldn't this extra state be avoided?

$ cat /home/hendry/.config/systemd/user/clipmenud.service.d/override.conf
[Service]
Environment=CM_DEBUG=1
Environment=CM_MAX_CLIPS=0

cdown · 2018-04-19T10:00:45Z

Can you test with the merge from #76 applied? :-)

kaihendry · 2018-04-20T01:48:55Z

Testing with

pkgname=clipmenu
pkgver=5.1.0.r7.g95cf774
pkgrel=1
pkgdesc='Clipboard management using dmenu'
url='http://github.com/cdown/clipmenu'
arch=('any')
license=('Public Domain')
depends=(dmenu xsel clipnotify)
conflicts=(clipmenu-git)  # Old package from pre-AUR4 days

source=("git://github.com/cdown/clipmenu.git#branch=develop")
md5sums=('SKIP')

pkgver() {
    cd "$pkgname"
    git describe --long --tags | sed -r 's/([^-]*-g)/r\1/;s/-/./g'
}

package() {
    install -D -m755 \
        "${srcdir?}/$pkgname/clipmenu" \
        "${pkgdir?}/usr/bin/clipmenu"
    install -D -m755 \
        "${srcdir?}/$pkgname/clipmenud" \
        "${pkgdir?}/usr/bin/clipmenud"
    install -D -m644 \
        "${srcdir?}/$pkgname/init/clipmenud.service" \
        "${pkgdir?}/usr/lib/systemd/user/clipmenud.service"
}

kaihendry · 2018-04-22T03:24:26Z

It's looking good after a day of use. I would merge and release if I were you! Thank you @Gravemind and Chris!

cdown · 2018-04-22T09:22:18Z

5.3.0 released and on AUR. Thanks!

stef204 changed the title ~~Clipmenu stops pasing selection~~ Clipmenu stops pasting selection Apr 3, 2018

cdown added the unconfirmed bug label Apr 3, 2018

Repository owner deleted a comment from stef204 Apr 6, 2018

cdown added a commit that referenced this issue Apr 7, 2018

Improve handling of CM_MAX_CLIPS with duplicates between cache files

0364087

Fixes #73.

This was referenced Apr 7, 2018

Doesn't copy into clipboard the right selection #71

Closed

FATAL: $SELECTION not in cache #70

Closed

cdown added bug and removed unconfirmed bug labels Apr 7, 2018

cdown changed the title ~~Clipmenu stops pasting selection~~ Selection not in cache Apr 7, 2018

cdown added the fix ready label Apr 7, 2018

cdown added a commit that referenced this issue Apr 10, 2018

Add more debugging information when cache entry is missing

d82337d

This would help with #73.

cdown mentioned this issue Apr 19, 2018

Fix wrong file deleted after partial selection #76

Merged

cdown closed this as completed Apr 22, 2018

cdown mentioned this issue Apr 26, 2018

Clipmenu fuzzer #78

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Selection not in cache #73

Selection not in cache #73

stef204 commented Apr 3, 2018 •

edited

Loading

cdown commented Apr 3, 2018

stef204 commented Apr 4, 2018

cdown commented Apr 4, 2018

stef204 commented Apr 5, 2018

cdown commented Apr 5, 2018

stef204 commented Apr 5, 2018 •

edited

Loading

stef204 commented Apr 5, 2018 •

edited

Loading

cdown commented Apr 5, 2018 •

edited

Loading

cdown commented Apr 6, 2018

stef204 commented Apr 6, 2018 •

edited

Loading

cdown commented Apr 7, 2018

cdown commented Apr 7, 2018

cdown commented Apr 7, 2018

kaihendry commented Apr 7, 2018

cdown commented Apr 7, 2018

stef204 commented Apr 7, 2018

kaihendry commented Apr 8, 2018

cdown commented Apr 8, 2018 •

edited

Loading

cdown commented Apr 8, 2018

kaihendry commented Apr 9, 2018

stef204 commented Apr 9, 2018

cdown commented Apr 10, 2018 •

edited

Loading

kaihendry commented Apr 10, 2018

stef204 commented Apr 10, 2018

kaihendry commented Apr 12, 2018

cdown commented Apr 12, 2018

kaihendry commented Apr 13, 2018

cdown commented Apr 13, 2018 •

edited

Loading

cdown commented Apr 13, 2018

kaihendry commented Apr 18, 2018 •

edited

Loading

cdown commented Apr 19, 2018

kaihendry commented Apr 20, 2018

kaihendry commented Apr 22, 2018

cdown commented Apr 22, 2018

Selection not in cache #73

Selection not in cache #73

Comments

stef204 commented Apr 3, 2018 • edited Loading

cdown commented Apr 3, 2018

stef204 commented Apr 4, 2018

cdown commented Apr 4, 2018

stef204 commented Apr 5, 2018

cdown commented Apr 5, 2018

stef204 commented Apr 5, 2018 • edited Loading

stef204 commented Apr 5, 2018 • edited Loading

cdown commented Apr 5, 2018 • edited Loading

cdown commented Apr 6, 2018

stef204 commented Apr 6, 2018 • edited Loading

cdown commented Apr 7, 2018

cdown commented Apr 7, 2018

cdown commented Apr 7, 2018

kaihendry commented Apr 7, 2018

cdown commented Apr 7, 2018

stef204 commented Apr 7, 2018

kaihendry commented Apr 8, 2018

cdown commented Apr 8, 2018 • edited Loading

cdown commented Apr 8, 2018

kaihendry commented Apr 9, 2018

stef204 commented Apr 9, 2018

cdown commented Apr 10, 2018 • edited Loading

kaihendry commented Apr 10, 2018

stef204 commented Apr 10, 2018

kaihendry commented Apr 12, 2018

cdown commented Apr 12, 2018

kaihendry commented Apr 13, 2018

cdown commented Apr 13, 2018 • edited Loading

cdown commented Apr 13, 2018

kaihendry commented Apr 18, 2018 • edited Loading

cdown commented Apr 19, 2018

kaihendry commented Apr 20, 2018

kaihendry commented Apr 22, 2018

cdown commented Apr 22, 2018

stef204 commented Apr 3, 2018 •

edited

Loading

stef204 commented Apr 5, 2018 •

edited

Loading

stef204 commented Apr 5, 2018 •

edited

Loading

cdown commented Apr 5, 2018 •

edited

Loading

stef204 commented Apr 6, 2018 •

edited

Loading

cdown commented Apr 8, 2018 •

edited

Loading

cdown commented Apr 10, 2018 •

edited

Loading

cdown commented Apr 13, 2018 •

edited

Loading

kaihendry commented Apr 18, 2018 •

edited

Loading