File sizes can be incorrectly reported by ls but are correct when data is read

[Martin2112]

Hi again!

I think there might still be another file size issue after you fixed the main one. It's with flac -> flac transcoding as before. The database says the track file should be 27992970 bytes and it's finished.

sqlite> select filename,finished,predicted_filesize,encoded_filesize from cache_entry where filename like '%to stone%';
/export/cdrip/flac/Music/Electric Light Orchestra/Out of the Blue/Electric Light Orchestra - Out of the Blue.flac.tracks/01. Electric Light Orchestra - Turn to Stone [03-46.893].flac|2|446962320|27992970

And if you read the file the data you get matches this length:

[martin@zen ~]$ wc /export/cdrip/flactrack/Music/Electric\ Light\ Orchestra/Out\ of\ the\ Blue/Electric\ Light\ Orchestra\ -\ Out\ of\ the\ Blue.flac.tracks/01.\ Electric\ Light\ Orchestra\ -\ Turn\ to\ Stone\ \[03-46.893\].flac 
  121136   646290 27992970 /export/cdrip/flactrack/Music/Electric Light Orchestra/Out of the Blue/Electric Light Orchestra - Out of the Blue.flac.tracks/01. Electric Light Orchestra - Turn to Stone [03-46.893].flac

But ls thinks it's shorter at 24014389 bytes:

[martin@zen ~]$ ls -l /export/cdrip/flactrack/Music/Electric\ Light\ Orchestra/Out\ of\ the\ Blue/Electric\ Light\ Orchestra\ -\ Out\ of\ the\ Blue.flac.tracks/01.\ Electric\ Light\ Orchestra\ -\ Turn\ to\ Stone\ \[03-46.893\].flac 
-rwxr--r--. 1 martin martin 24014389 Mar 12 17:31 '/export/cdrip/flactrack/Music/Electric Light Orchestra/Out of the Blue/Electric Light Orchestra - Out of the Blue.flac.tracks/01. Electric Light Orchestra - Turn to Stone [03-46.893].flac'

This confuses players into finishing playback early.

[nschlia]

Sorry but this can't be fixed. The real file size will not be known unless the file had been fully decoded. This is because most formats use variable bit rates and the final file size heavily depends on the audio/video contents.

So the file sizes are just a guess at first.

[nschlia]

Closed as not fixable.

[Martin2112]

OK but hasn't it already done that work? The file size should no longer be a guess?

[nschlia]

OK but hasn't it already done that work? The file size should no longer be a guess?

The size will always be a guess. Even for WAV or MP3, where the size can precisely be calculated for the audio portion. But there are meta data parts that make even that impossible.

[Martin2112]

Sorry, I'm still not understanding why the size isn't coming from the sql db or the cache file once transcoding is complete?

I understand why it would be a guess for files that haven't been transcoded yet.

[nschlia]

I think now I've got it: Until the files have been recoded, the size shown by ls -l is a guess. As explained, this cannot be fixed.

After recoding, the size is correct, and properly stored in the database. When ffmpegfs is restarted, all track file sizes go back to the guess. The original file retains the correct size, on the other hand. Only the virtual tracks have wrong sizes.

Appears that the sizes of virtual tracks are not read from the database like for "real" files.

I'll check into that.

[Martin2112]

OK, thanks for the time spent looking into the issues I've reported. If you visit London I'll buy you a beer!

[nschlia]

OK, thanks for the time spent looking into the issues I've reported.

Thanks for your time spent reporting these issues and helping with fixes. I appreciate that.

If you visit London I'll buy you a beer!

That's dangerous. I might get back to it one day :)

[nschlia]

The issue has grown bigger than I first thought:

Same problem applies to other virtual directories, DVD, VCD, Bluray and even HLS also. Same effect: After recode, the file size is correct. After ffmpegfs restart it is back to the guess, although the real value is stored in the database.

This has been around ever since the features have been added as the code that would retrieve the size does simply not exist...

Thanks for insisting on the topic.

[Martin2112]

Looks good so far, will do more testing.

[nschlia]

Looks good so far, will do more testing.

For cuesheets this was actually working, but failed for some formats, FLAC being one of those. Happened when the file size could not be predicted (as for FLAC), after I fixed that, the size problem was gone as well :)
For DVD, Bluray, Video CD and HLS this never worked. I forgot to implement the functionality in the first place...
Anways, for DVD, Bluray, Video CD, HLS and cuesheets the cache file size now is used. This is the correct size, as the cached file is sent to the reading application. This is better than the database entry, as it may be wrong, and if the cache file should be gone (deleted) the size falls back to a guess. The source will be recoded in that case.