Broken pipe

[CitronSorcier]

Hi, I've just installed and ran oil, and got this:

Traceback (most recent call last):
File "/usr/local/bin/buku", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/buku.py", line 5481, in main
bdb.print_rec(0)
File "/usr/local/lib/python3.8/dist-packages/buku.py", line 1748, in print_rec
print(format_json(resultset, field_filter=self.field_filter))
BrokenPipeError: [Errno 32] Broken pipe

[AndreiUlmeyda]

Hey there,

I am not up to speed yet after not being able to maintain this project for a long, long while. Therefore I can't try to replicate your error at the moment. At first glance it seems to me the underlying error has to do more with python/buku than oil.

Could you try to verify that your buku installation is working correctly? For instance, what's
the output of the command?
buku --print --json

[AndreiUlmeyda]

Hey there, sorry to keep you waiting. I am currently trying to replicate the problem.
The easiest approach for the moment was spinning up a vagrant vm like so:

# Try to replicate the issue https://github.com/AndreiUlmeyda/oil/issues/24

$replicateOilError = <<-SCRIPT

# Full system update, mainly to get the latest certificates and enable subsequent installs.
pacman --noconfirm -Syyu

# Install a few dependencies
pacman --noconfirm --needed -S base-devel git jq xdg-utils python-setuptools

# Install an AUR-helper.
runuser -l vagrant -c 'git clone https://aur.archlinux.org/yay.git; cd yay; yes | makepkg -si'

# # Install the package in question.
runuser -l vagrant -c 'yay -S --noconfirm buku-git oil'

#  
runuser -l vagrant -c 'buku --add https://www.youtube.com'

SCRIPT

Vagrant.configure("2") do |config|
  config.vm.box = "archlinux/archlinux"
  config.vm.provision "shell", inline: $replicateOilError
end

Running oil or oil -p inside the box seems to behave well.

I will now try the same using your OS.

[ReneFroger]

I have the same issue as well. When I try buku --print --json, I get the following

[
    {
        "description": "",
        "index": 164,
        "tags": "food,health",
        "title": "Life long when using asprin and calis",
        "uri": "https://en.m.wikipedia.org/wiki/Walter_Breuning"
    },
    {
        "description": "4.0k votes, 129 comments. 761k members in the intermittentfasting community. Intermittent Fasting(IF) is way of eating that restricts *when* you \u2026",
        "index": 165,
        "tags": "health",
        "title": "Intermittent Fasting examples",
        "uri": "https://www.reddit.com/r/intermittentfasting/comments/myxs6b/body_and_face_gains_102lbs_down_and_finally_at_a/?sort=top"
    },
    {
        "description": "For middle-aged men like me, the right time to do an ice bath is before a workout--not after.",
        "index": 166,
        "tags": "health",
        "title": "Boost testeron",
        "uri": "https://medium.com/morozko-method/ice-bath-testosterone-af07fb4e3514"
    }
]

There are more bookmarks, but I paste only 3 bookmarks to get an idea of the structure here.
I made a small script on Ubuntu 20.04.2 LTS to install Buku and Oil, and encountered this issue. You can run this little Bash script to see if you could be able to reproduce it:

#!/bin/bash

# Buku dependencies
sudo add-apt-repository -y ppa:twodopeshaggy/jarun
sudo apt update

sudo apt install -y ca-certificates python3-urllib3 python3-cryptography python3-bs4
sudo apt install -y buku

# install CLI bookmarks
sudo apt install -y jq                         # JSON parser for oil, filter buku bookmark 

# install Peco for filtering results from buku with Oil
wget -O /tmp/peco.tar.gz https://github.com/peco/peco/releases/download/v0.5.2/peco_linux_amd64.tar.gz
sudo tar -xvzf /tmp/peco.tar.gz -C /tmp/
sudo cp -va /tmp/peco_linux_amd64/peco /usr/bin/
# Copy binaries for oil (for buku), for the case they cannot find the binaries for awk and jq in PATH
sudo /bin/rsync -av /usr/bin/jq /bin/
sudo /bin/rsync -av /usr/bin/awk /bin/

# install oil for bookmarks 
git clone --depth 1 https://github.com/AndreiUlmeyda/oil /tmp/oil
cd /tmp/oil && sudo make install

This installs Oil, Peco and Buku. Buku works fine, Oil gives problems however.

Python 3 version is: 3.810
Python 2 version is 2.7.18
Ubuntu 20.04.2 LTS as OS

[ReneFroger]

@AndreiUlmeyda If you need more information, or you wasn't able to reproduce it, could you let me know?

[AndreiUlmeyda]

Hey there @ReneFroger, first of all, excellent list of steps to reproduce the problem. Secondly, I was not able to install buku from the PPA you mentioned, but installing the first thing apt found was enough. I've got the error and have a strong suspicion.

In both auxiliary scripts 'format-columns.awk' and 'json-to-line.jq'
the call to 'env' needs to have a -S if the command afterwards has flags of its own.
so the line
#!/usr/bin/env jq -fr
should probably be
#!/usr/bin/env -S jq -fr
and, respectively, the line
#!/usr/bin/env awk -f
should be
#!/usr/bin/env -S awk -f

In my virtual machines this fixes the issue. I will merge this soon.
If you are so inclined you could try out this fix on your machine and report back if it works.

If that was the issue then it came in with the last pull request and I failed to test properly. Then again, it is a bit weird it works on arch. Maybe env works differently there, who knows.

[ReneFroger]

I modified the lines in format-columns.awk and json-to-line.jq with adding the -S flag to the lines. And reïnstalled Oil, and it worked again like a charm.

Could you merge the changes, so that other users could benefit from it too? Thanks in advance! Your help is really appreciated.

[AndreiUlmeyda]

I've created a new tag containing the fix. If it works for you and @CitronSorcier I will close the issue. Thanks a bunch for the cooperation. Have a fine day.

[alimoezzi]

I was facing this issue and fixed it by using
#!/usr/bin/env -S awk -f -> #!/usr/bin/awk -f
#!/usr/bin/env -S jq -fr -> #!/usr/bin/ jq -fr