panic: runtime error: invalid memory address or nil pointer dereference

[CalounovaT]

Hello,
I got an error running this seqkit split command on my data file:

seqkit split databases/mgnify.fasta -s 10000 --two-pass -O "databases/mgnify_chunks" --threads 6 2> logs/split_database_mgnify.log

This is the log and error, it seems that no sequences were loaded although there are 113,795,414 sequences in the databases/mgnify.fasta.

[INFO] create and read FASTA index ...
[INFO] create FASTA index for databases/mgnify.fasta
[INFO] read sequence IDs from FASTA index ...
[INFO] 0 sequences loaded
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7358f9]

goroutine 1 [running]:
github.com/shenwei356/xopen.(*Writer).Flush(0x0)
        /home/shenwei/shenwei/scripts/go/pkg/mod/github.com/shenwei356/xopen@v0.2.2/xopen.go:147 +0x19
github.com/shenwei356/xopen.(*Writer).Close(0x0)
        /home/shenwei/shenwei/scripts/go/pkg/mod/github.com/shenwei356/xopen@v0.2.2/xopen.go:131 +0x1e
github.com/shenwei356/seqkit/v2/seqkit/cmd.glob..func31(0x11eec20?, {0xc00057a380, 0x1, 0x8})
        /home/shenwei/shenwei/scripts/go/src/github.com/shenwei356/seqkit/seqkit/cmd/split.go:332 +0x704a
github.com/spf13/cobra.(*Command).execute(0x11eec20, {0xc00057a300, 0x8, 0x8})
        /home/shenwei/shenwei/scripts/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:860 +0x663
github.com/spf13/cobra.(*Command).ExecuteC(0x11ed0a0)
        /home/shenwei/shenwei/scripts/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
        /home/shenwei/shenwei/scripts/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902
github.com/shenwei356/seqkit/v2/seqkit/cmd.Execute()
        /home/shenwei/shenwei/scripts/go/src/github.com/shenwei356/seqkit/seqkit/cmd/root.go:63 +0x25
main.main()
        /home/shenwei/shenwei/scripts/go/src/github.com/shenwei356/seqkit/seqkit/main.go:58 +0x17

The seqkit version is 2.3.1 installed with conda.

I will be happy for any help how I could solve this problem.
Best regards,
Tereza

[shenwei356]

It might be due to some blank line at the end of sequences. Try format the fasta file and replace the original one.

seqkit seq seq.fasta > seq.f.fasta

Also try the new binaries: #360 (comment)

[CalounovaT]

I tried to run

seqkit seq seq.fasta > seq.f.fasta

but I got the same error.

[shenwei356]

I see, it's due to the empty fasta index file databases/mgnify.fasta.seqkit.fai, which may be generated for an older databases/mgnify.fasta file with zero or un-formatted sequences.

To solve this, Just delete it and try again. I'll also add some codes to check this.

A similar case: #364

[shenwei356]

I've added a flag to recreate .fai file, improved the log info, and updated the help message.

• seqkit faidx/sort/shuffle/split/subseq:
  • new flag -U/--update-faidx: update the FASTA index file if it exists.
  • improve log info and update help message.

Please try the new binaries:

• seqkit_linux_amd64.tar.gz
• seqkit_darwin_arm64.tar.gz
• seqkit_darwin_amd64.tar.gz

[shenwei356]

Besides, --threads 6 does not help, the default value 4 is fast enough.

Also, recommend using seqkit split2 -s , and it's faster and does not require a two-pass mode