Skip to content

Commit

Permalink
Bugfix (maybe): properly clean up old decodes
Browse files Browse the repository at this point in the history
This requires some more thoughts -- the shape of the problem is that the
encoder and decoder are running out of band with each other (very
intentionally -- it's the whole point). We want to "keep up" with what
the encoder throws at us, but we also need to clean up our list of
in-progress (and finished!) decodes at some point -- we don't want it to
be unbounded.

The bug this fixes should manifest as files refusing to
progress/download -- the decoder (app) has already reserved the "stream
slot", finished a download, but failed to clean up the mess to let it be
reclaimed. The workaround for CFC is to close and re-open the app, which
will throw out all in progress downloads and start from a clean slate.
  • Loading branch information
sz3 committed Feb 4, 2022
1 parent f752be4 commit c5f7676
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 7 deletions.
2 changes: 1 addition & 1 deletion src/exe/cimbar_send/send.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ int main(int argc, char** argv)
if (!initialize_GL(window_size, window_size))
{
std::cerr << "failed to create window :(" << std::endl;
return 50;
return 70;
}

configure(colorBits, ecc, compressionLevel);
Expand Down
2 changes: 1 addition & 1 deletion src/lib/fountain/FountainMetadata.h
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ class FountainMetadata
to_uint8_arr(encode_id, size, d);
}

unsigned id() const
uint64_t id() const
{
return _data;
}
Expand Down
13 changes: 8 additions & 5 deletions src/lib/fountain/fountain_decoder_sink.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,10 @@ class fountain_decoder_sink
return true;
}

void mark_done(uint64_t id)
void mark_done(const FountainMetadata& md)
{
_done.insert(id);
auto it = _streams.find(id);
_done.insert(md.id());
auto it = _streams.find(stream_slot(md));
if (it != _streams.end())
_streams.erase(it);
}
Expand Down Expand Up @@ -106,7 +106,7 @@ class fountain_decoder_sink
return false;

if (store(md, *finished))
mark_done(md.id());
mark_done(md);
return true;
}

Expand All @@ -122,7 +122,7 @@ class fountain_decoder_sink
}

protected:
// streams is limited to at most 8 decoders at a time. Current, we just use the lower bits of the encode_id.
// streams is limited to at most 8 decoders at a time. Currently, we just use the lower bits of the encode_id.
uint8_t stream_slot(const FountainMetadata& md) const
{
return md.encode_id() & 0x7;
Expand All @@ -137,6 +137,9 @@ class fountain_decoder_sink
std::string _dataDir;
unsigned _chunkSize;

// maybe instead of unordered_map+set, something where we can "age out" old streams?
// e.g. most recent 16/8, or something?
// question is what happens to _done/_streams when we wrap for continuous data streaming...
std::unordered_map<uint8_t, fountain_decoder_stream> _streams;
// track the uint64_t combo of (encode_id,size) to avoid redundant work
std::set<uint64_t> _done;
Expand Down

0 comments on commit c5f7676

Please sign in to comment.