Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid argument on log file #412

Closed
vikshiv123 opened this issue Sep 25, 2024 · 15 comments
Closed

Invalid argument on log file #412

vikshiv123 opened this issue Sep 25, 2024 · 15 comments

Comments

@vikshiv123
Copy link

Hi, I'm running PGGB on HPRC haplotypes of a single chromosome. I tried Singularity to run PGGB:
singularity run -e -B ../chr19_test/:/data ../pggb_latest.sif pggb -i /data/chr19_renamed.fa -t 16 -V 'chm13#1#chr19' -o /data/pggb
However, I get an error without much output info:
/usr/local/bin/pggb: line 487: /data/pggb/chr19_renamed.fa.bf3285f.11fba48.07bc622.smooth.09-25-2024_19:19:29.log: Invalid argument
The output directory is empty (no log files etc.).

I get the same error when I try to run the test data from the Singularity section of the README:
singularity run -e -B ${PWD}/data:/data ../pggb_latest.sif pggb -i /data/HLA/DRB1-3123.fa.gz -p 70 -s 3000 -n 10 -t 16 -o /data/out
/usr/local/bin/pggb: line 487: /data/out/DRB1-3123.fa.gz.7bf7864.11fba48.5180c7d.smooth.09-25-2024_19:20:13.log: Invalid argument

I figured this might be an issue linking directories through singularity, and installed PGGB through Conda instead:
mamba create -n pggb -c bioconda pggb
mamba activate pggb

I get a similar (but slightly different) error for this command:
pggb -i chr19_renamed.fa -t 16 -V 'chm13#1#chr19' -o pggb
/conda/envs/pggb/bin/pggb: line 437: pggb/chr19_renamed.fa.bf3285f.11fba48.07bc622.smooth.09-25-2024_19:22:34.log: Invalid argument

I'm running it on haplotypes of chr19 from HPRC, one sequence per line in the fasta. Here is a snippet of the fasta index for reference:

chm13#1#chr19   61707364        15      61707364        61707365
HG00438#1#chr19 60772275        61707397        60772275        60772276
HG00438#2#chr19 61080741        122479690       61080741        61080742
HG00621#1#chr19 61442093        183560449       61442093        61442094
HG00621#2#chr19 61506976        245002560       61506976        61506977
HG00673#1#chr19 63033057        306509554       63033057        63033058
HG00673#2#chr19 62744279        369542629       62744279        62744280
HG00733#1#chr19 61934365        432286926       61934365        61934366
HG00733#2#chr19 61328042        494221309       61328042        61328043
HG00735#1#chr19 60499191        555549369       60499191        60499192
HG00735#2#chr19 61874577        616048578       61874577        61874578
HG00741#1#chr19 61637809        677923173       61637809        61637810

I'm not sure what the issue is here, I've checked the read/write permissions in the directory (which I've had no issue with setting up and manipulating the input FASTAs). Any help would be appreciated!

@nicmoya
Copy link
Contributor

nicmoya commented Oct 15, 2024

I am facing the same issue. Running pggb using conda install.

$ pggb -i CE_chr1_5hifi.rff.fa -o 5hifi -t 16 -p 80 -s 5k -V 'N2:1000' -n 1
/home/nmoya1/mambaforge/envs/pggb/bin/pggb: line 437: 5hifi/CE_chr1_5hifi.rff.fa.304cbb4.11fba48.69afeb3.smooth.10-15-2024_14:17:00.log: Invalid argument

I am using newly assembled C. elegans genomes, single chromosome, single haplotype.

ECA738#1#I      15574207        13      15574207        15574208
N2#1#I  15072434        15574229        15072434        15072435
QG4021#1#I      15321380        30646676        15321380        15321381
QX1794#1#I      15402706        45968069        15402706        15402707
XZ1516#1#I      15720420        61370788        15720420        15720421

@AndreaGuarracino
Copy link
Member

@vikshiv , @nicmoya , could you please try again with pggb v0.7.0? I would appreciate feedback about the Docker/Singularity/Bioconda installation. The latter just worked on my laptop.

@vikshiv123
Copy link
Author

Hi, I get the same issue on v0.7.0. I updated on Bioconda and get this error (same as above, I believe):

(pggb) $ pggb --version
pggb 0.7.0
(pggb) $ pggb -i chr19_renamed.fa -o chr19_pggb -t 16
/conda/envs/pggb/bin/pggb: line 488: chr19_pggb/chr19_renamed.fa.bf3285f.11fba48.07bc622.smooth.10-17-2024_22:43:28.log: Invalid argument

Also occurs with Singularity (after re-pulling the image and the repo to the latest commit, tagged 0.7.0)

(pggb) $ singularity run -e -B ./:/data ../pggb_latest.sif pggb -i /data/chr19_renamed.fa -t 16 -o /data/chr19_pggb
/usr/local/bin/pggb: line 487: /data/chr19_pggb/chr19_renamed.fa.bf3285f.11fba48.07bc622.smooth.10-17-2024_23:16:56.log: Invalid argument

@AndreaGuarracino
Copy link
Member

@vikshiv, thank you for trying again! Could you please start from scratch and report all commands you execute? For me, it was a simple

conda install -c bioconda -c conda-forge pggb

and then I was able to run pggb without issues. I am using Ubuntu 22.10.

Please also report your operating system information and the output of conda list!

@nicmoya
Copy link
Contributor

nicmoya commented Oct 21, 2024

@AndreaGuarracino I tried a fresh install using the conda command you shared. I am running into the same issue (line 488 this time instead of 437 in my first issue):

pggb -i CE_chr1_5hifi.rff.fa -o 5hifi -t 8 -p 80 -s 5k -V 'N2:1000' -n 1
/home/nmoya1/mambaforge/envs/pggb_env/bin/pggb: line 488: 5hifi/CE_chr1_5hifi.rff.fa.9e2a976.11fba48.69afeb3.smooth.10-21-2024_12:18:44.log: Invalid argument

I am on Rocky Linux 8.8 (Green Obsidian). I'm deploying the environment with Mamba 1.5.5. Here is my list output:

# packages in environment at /home/nmoya1/mambaforge/envs/pggb_env:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
annotated-types           0.7.0              pyhd8ed1ab_0    conda-forge
arpack                    3.9.1           nompi_h77f6705_101    conda-forge
bc                        1.07.1               h7f98852_0    conda-forge
bcftools                  1.21                 h8b25389_0    bioconda
brotli                    1.1.0                hb9d3cd8_2    conda-forge
brotli-bin                1.1.0                hb9d3cd8_2    conda-forge
brotli-python             1.1.0           py312h2ec8cdc_2    conda-forge
bzip2                     1.0.8                h4bc722e_7    conda-forge
c-ares                    1.34.2               heb4867d_0    conda-forge
ca-certificates           2024.8.30            hbcca054_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py312h06ac9bb_0    conda-forge
charset-normalizer        3.4.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
cmake                     3.30.5               hf9cb763_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
coloredlogs               15.0.1             pyhd8ed1ab_3    conda-forge
colormath                 3.0.0                      py_2    conda-forge
contourpy                 1.3.0           py312h68727a3_2    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
eigen                     3.2                           3    bioconda
expat                     2.6.3                h5888daf_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_3    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.54.1          py312h66e93f0_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
future                    1.0.0              pyhd8ed1ab_0    conda-forge
gettext                   0.22.5               he02047a_3    conda-forge
gettext-tools             0.22.5               he02047a_3    conda-forge
gfaffix                   0.1.5b               h031d066_0    bioconda
glpk                      5.0                  h445213a_0    conda-forge
gmp                       6.3.0                hac33072_2    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
htslib                    1.21                 h5efdd21_0    bioconda
humanfriendly             10.0               pyhd8ed1ab_6    conda-forge
humanize                  4.11.0             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       75.1                 he02047a_0    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
importlib-metadata        8.5.0              pyha770c72_0    conda-forge
jemalloc                  4.5.0                         0    bioconda
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jsoncpp                   1.9.6                h84d6215_0    conda-forge
kaleido-core              0.2.1                h3644ca4_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.7           py312h68727a3_0    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.43                 h712a8e2_1    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libasprintf               0.22.5               he8f35ee_3    conda-forge
libasprintf-devel         0.22.5               he8f35ee_3    conda-forge
libblas                   3.9.0           24_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hb9d3cd8_2    conda-forge
libbrotlidec              1.1.0                hb9d3cd8_2    conda-forge
libbrotlienc              1.1.0                hb9d3cd8_2    conda-forge
libcblas                  3.9.0           24_linux64_openblas    conda-forge
libcurl                   8.10.1               hbbe4b11_0    conda-forge
libdeflate                1.21                 h4bc722e_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.3                h5888daf_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.2.0               h77fa898_1    conda-forge
libgcc-ng                 14.2.0               h69a702a_1    conda-forge
libgettextpo              0.22.5               he02047a_3    conda-forge
libgettextpo-devel        0.22.5               he02047a_3    conda-forge
libgfortran               14.2.0               h69a702a_1    conda-forge
libgfortran-ng            14.2.0               h69a702a_1    conda-forge
libgfortran5              14.2.0               hd5240d6_1    conda-forge
libgomp                   14.2.0               h77fa898_1    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libidn2                   2.3.7                hd590300_0    conda-forge
libjemalloc               5.3.0                h5888daf_1    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           24_linux64_openblas    conda-forge
libnghttp2                1.63.0               h161d5f1_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.27          pthreads_hac2b453_1    conda-forge
libpng                    1.6.44               hadc24fc_0    conda-forge
libsqlite                 3.46.1               hadc24fc_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx                 14.2.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.2.0               h4852527_1    conda-forge
libtiff                   4.7.0                h6565414_0    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libuv                     1.49.2               hb9d3cd8_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.17.0               h8a09558_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.7               he7c6b58_4    conda-forge
libzlib                   1.3.1                hb9d3cd8_2    conda-forge
lzstring                  1.0.4                   py_1001    conda-forge
markdown                  3.6                pyhd8ed1ab_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                3.0.2           py312h178313f_0    conda-forge
mathjax                   2.7.7                ha770c72_3    conda-forge
matplotlib-base           3.9.2           py312hd3ec401_1    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
multiqc                   1.22               pyhdfd78af_0    bioconda
munkres                   1.0.7                      py_1    bioconda
ncurses                   6.5                  he02047a_1    conda-forge
networkx                  3.4.1              pyhd8ed1ab_0    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.105                hd34e28f_0    conda-forge
numpy                     2.1.2           py312h58c1407_0    conda-forge
odgi                      0.9.0           py312ha1f7cf2_0    bioconda
openjpeg                  2.5.2                h488ebb8_0    conda-forge
openssl                   3.3.2                hb9d3cd8_0    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
perl                      5.32.1          7_hd590300_perl5    conda-forge
pggb                      0.7.0                h9ee0642_0    bioconda
pigz                      2.8                  h2797004_0    conda-forge
pillow                    11.0.0          py312h7b63e92_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
plotly                    5.24.1             pyhd8ed1ab_0    conda-forge
pthread-stubs             0.4               hb9d3cd8_1002    conda-forge
pyaml-env                 1.2.1              pyhd8ed1ab_0    conda-forge
pybind11                  2.13.6             pyh085cc03_1    conda-forge
pybind11-global           2.13.6             pyh085cc03_1    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pydantic                  2.9.2              pyhd8ed1ab_0    conda-forge
pydantic-core             2.23.4          py312h12e396e_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.2.0              pyhd8ed1ab_1    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.12.7          hc5c86c4_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-igraph             0.11.5          py312h4f72774_1    conda-forge
python-kaleido            0.2.1              pyhd8ed1ab_0    conda-forge
python_abi                3.12                    5_cp312    conda-forge
pyyaml                    6.0.2           py312h66e93f0_1    conda-forge
qhull                     2020.2               h434a139_5    conda-forge
readline                  8.2                  h8228510_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
rhash                     1.4.5                hb9d3cd8_0    conda-forge
rich                      13.9.2             pyhd8ed1ab_0    conda-forge
rich-click                1.8.3              pyhd8ed1ab_0    conda-forge
samtools                  1.21                 h50ea8bc_0    bioconda
seqwish                   0.7.11               h43eeafb_0    bioconda
setuptools                75.1.0             pyhd8ed1ab_0    conda-forge
simplejson                3.19.3          py312h66e93f0_1    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smoothxg                  0.8.0                h40c17d1_0    bioconda
spectra                   0.0.11                     py_1    conda-forge
sqlite                    3.46.1               h9eae976_0    conda-forge
tabix                     1.11                 hdfd78af_0    bioconda
tabixpp                   1.1.2                h27d5293_3    bioconda
tenacity                  9.0.0              pyhd8ed1ab_0    conda-forge
texttable                 1.7.0              pyhd8ed1ab_0    conda-forge
time                      1.9                  hd590300_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
typeguard                 4.3.0              pyhd8ed1ab_1    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
urllib3                   2.2.3              pyhd8ed1ab_0    conda-forge
vcfbub                    0.1.1                h715e4b3_1    bioconda
vcflib                    1.0.10               hdcf5f25_1    bioconda
vg                        1.59.0               h9ee0642_0    bioconda
wfa2-lib                  2.3.5                h4ac6f70_2    bioconda
wfmash                    0.14.0               h11f254b_0    bioconda
wget                      1.21.4               hda4d442_0    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               hb9d3cd8_1    conda-forge
xorg-libxdmcp             1.1.5                hb9d3cd8_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zipp                      3.20.2             pyhd8ed1ab_0    conda-forge
zlib                      1.3.1                hb9d3cd8_2    conda-forge
zstandard                 0.23.0          py312hef9b889_1    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

@vikshiv123
Copy link
Author

Sure. Here is the exact process from scratch:

$ mamba create -n pggb; mamba activate pggby
$ mamba install -c bioconda -c conda-forge pggb
$ pggb -i chr19_renamed.fa -o chr19_pggb -t 16
/path/to/conda/envs/pggb/bin/pggb: line 488: chr19_pggb/chr19_renamed.fa.bf3285f.11fba48.07bc622.smooth.10-21-2024_15:22:21.log: Invalid argument

Same exact issue as above. I am also on Rocky Linux (8.8) GreenObsidian. x86_64 cpu. Mamba version 1.4.2, conda version 23.3.1. I have an identical conda list as above.

I just checked with @nicmoya and we both happen to be using the same cluster at the same institution by coincidence. So that might be the root issue here. Let me know if we can be of some help tracking this down!

@subwaystation
Copy link
Member

What sometimes happened to me is that although I created an environment from scratch, the a Bioconda package was not updated, because it was cached. What does pggb --version show? But I guess, this is not the problem here.

The line in question is

echo -e "Command: $cmd\n" >> "$log_file"

Maybe on your cluster $cmd is already taken by another command?

@nicmoya
Copy link
Contributor

nicmoya commented Oct 22, 2024

Version returns:

$ pggb --version
pggb 0.7.0

And echo $cmd returns nothing.

@nicmoya
Copy link
Contributor

nicmoya commented Oct 22, 2024

Just to clarify, using alternative syntax mamba install bioconda::pggb, updates pggb to 0.7.1, but the problem persists:

$ pggb --version
pggb 0.7.1
$ pggb -i CE_chr1_5hifi.rff.fa -o 5hifi -t 16 -p 80 -s 5k -V 'N2:1000' -n 1
/home/nmoya1/mambaforge/envs/pggb2/bin/pggb: line 488: 5hifi/CE_chr1_5hifi.rff.fa.9e2a976.11fba48.69afeb3.smooth.10-22-2024_12:32:46.log: Invalid argument

@nicmoya
Copy link
Contributor

nicmoya commented Oct 22, 2024

After testing some minor tweaks, it seems that removing $date from the log filenames resolves the issue. This is likely because of the use of : in the time stamp, which is an illegal character in this computer cluster.

@vikshiv123
Copy link
Author

Can confirm, removing the : from the $date definition fixes this issue! Thanks @nicmoya.

@AndreaGuarracino
Copy link
Member

@nicmoya, 9000+ thank you! Could you, or @vikshiv, please make a PR with the minimal change to apply to have a happy pggb?

@nicmoya
Copy link
Contributor

nicmoya commented Oct 22, 2024

No problem, I can do the PR sometime tomorrow. Thank you.

@nicmoya
Copy link
Contributor

nicmoya commented Oct 23, 2024

PR at #419

@AndreaGuarracino
Copy link
Member

Long live to @nicmoya!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants