Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is this a corrupt database? #1082

Closed
adamretter opened this issue Apr 16, 2016 · 1 comment
Closed

Is this a corrupt database? #1082

adamretter opened this issue Apr 16, 2016 · 1 comment
Labels
abandoned-or-aged-out

Comments

@adamretter
Copy link
Collaborator

adamretter commented Apr 16, 2016 •
edited
Loading

I have a database with Rocks that I have been using. However when I start up now, it appears that my database is empty somehow.

At startup it now looks like the recovery process runs:

2016-04-16 16:17:39,991 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Compression algorithms supported: 
2016-04-16 16:17:39,991 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   Snappy supported: 1 
2016-04-16 16:17:39,991 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   Zlib supported: 1 
2016-04-16 16:17:39,991 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   Bzip supported: 1 
2016-04-16 16:17:39,992 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   LZ4 supported: 0 
2016-04-16 16:17:39,992 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Fast CRC32 supported: 1 
2016-04-16 16:17:39,992 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Recovering from manifest file: MANIFEST-000188

2016-04-16 16:17:39,992 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [default]:

2016-04-16 16:17:39,993 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [XML_DOM_STORE]:

2016-04-16 16:17:39,993 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [SYMBOL_STORE]:

2016-04-16 16:17:39,994 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [COLLECTION_STORE]:

2016-04-16 16:17:39,994 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [NAME_ID_INDEX]:

2016-04-16 16:17:39,994 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [NAME_INDEX]:

2016-04-16 16:17:39,994 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [SORT_NAME_INDEX]:

2016-04-16 16:17:39,995 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [SORT_INDEX]:

2016-04-16 16:17:39,995 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [NGRAM_INDEX]:

2016-04-16 16:17:39,995 [main] INFO  (RocksLoggerAdapter.java [log]:53) - --------------- Options for column family [METADATA_STORE]:

2016-04-16 16:17:39,995 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   (skipping printing options)

2016-04-16 16:17:39,996 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   (skipping printing options)

2016-04-16 16:17:39,996 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   (skipping printing options)

2016-04-16 16:17:39,996 [main] INFO  (RocksLoggerAdapter.java [log]:53) -   (skipping printing options)

2016-04-16 16:17:39,997 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Recovered from manifest file:/Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/MANIFEST-000188 succeeded,manifest_file_number is 188, next_file_number is 210, last_sequence is 40955535, log_number is 0,prev_log_number is 0,max_column_family is 13

2016-04-16 16:17:39,998 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [default] (ID 0), log number is 187

2016-04-16 16:17:39,998 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [XML_DOM_STORE] (ID 1), log number is 187

2016-04-16 16:17:39,998 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [SYMBOL_STORE] (ID 2), log number is 187

2016-04-16 16:17:39,998 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [COLLECTION_STORE] (ID 3), log number is 187

2016-04-16 16:17:39,998 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [NAME_ID_INDEX] (ID 4), log number is 187

2016-04-16 16:17:39,999 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [NAME_INDEX] (ID 5), log number is 187

2016-04-16 16:17:39,999 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [SORT_NAME_INDEX] (ID 6), log number is 187

2016-04-16 16:17:39,999 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [SORT_INDEX] (ID 7), log number is 187

2016-04-16 16:17:39,999 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [NGRAM_INDEX] (ID 8), log number is 187

2016-04-16 16:17:39,999 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [METADATA_STORE] (ID 9), log number is 187

2016-04-16 16:17:39,999 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [METADATA_INDEX] (ID 10), log number is 187

2016-04-16 16:17:40,000 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [UUID_STORE] (ID 11), log number is 187

2016-04-16 16:17:40,000 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [UUID_URI_INDEX] (ID 12), log number is 187

2016-04-16 16:17:40,000 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Column family [URI_UUID_INDEX] (ID 13), log number is 187

2016-04-16 16:17:40,001 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837860001011, "job": 1, "event": "recovery_started", "log_files": [199, 209]} 
2016-04-16 16:17:40,001 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Recovering log #199 mode 0 skip-recovery 0 
2016-04-16 16:18:00,238 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [NAME_INDEX] [WriteLevel0TableForRecovery] Level-0 table #210: started 
2016-04-16 16:18:02,317 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [NAME_INDEX] [WriteLevel0TableForRecovery] Level-0 table #210: 66777884 bytes OK 
2016-04-16 16:18:02,317 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837882317688, "cf_name": "NAME_INDEX", "job": 1, "event": "table_file_creation", "file_number": 210, "file_size": 66777884, "table_properties": {"data_size": 66213551, "index_size": 628663, "filter_size": 0, "raw_key_size": 82480425, "raw_average_key_size": 22, "raw_value_size": 14732488, "raw_average_value_size": 4, "num_data_blocks": 16238, "num_entries": 3683122, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:02,413 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Recovering log #209 mode 0 skip-recovery 0 
2016-04-16 16:18:22,234 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [NAME_INDEX] [WriteLevel0TableForRecovery] Level-0 table #211: started 
2016-04-16 16:18:24,303 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [NAME_INDEX] [WriteLevel0TableForRecovery] Level-0 table #211: 66777884 bytes OK 
2016-04-16 16:18:24,303 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904303854, "cf_name": "NAME_INDEX", "job": 1, "event": "table_file_creation", "file_number": 211, "file_size": 66777884, "table_properties": {"data_size": 66213551, "index_size": 628663, "filter_size": 0, "raw_key_size": 82480425, "raw_average_key_size": 22, "raw_value_size": 14732488, "raw_average_value_size": 4, "num_data_blocks": 16238, "num_entries": 3683122, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,405 [main] INFO  (RocksLoggerAdapter.java [log]:53) - Creating manifest 212

2016-04-16 16:18:24,409 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [NAME_ID_INDEX] [WriteLevel0TableForRecovery] Level-0 table #213: started 
2016-04-16 16:18:24,410 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [NAME_ID_INDEX] [WriteLevel0TableForRecovery] Level-0 table #213: 63250 bytes OK 
2016-04-16 16:18:24,411 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904410993, "cf_name": "NAME_ID_INDEX", "job": 1, "event": "table_file_creation", "file_number": 213, "file_size": 63250, "table_properties": {"data_size": 62074, "index_size": 623, "filter_size": 0, "raw_key_size": 19457, "raw_average_key_size": 27, "raw_value_size": 50135, "raw_average_value_size": 69, "num_data_blocks": 16, "num_entries": 718, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,413 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [SORT_NAME_INDEX] [WriteLevel0TableForRecovery] Level-0 table #214: started 
2016-04-16 16:18:24,436 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [SORT_NAME_INDEX] [WriteLevel0TableForRecovery] Level-0 table #214: 484107 bytes OK 
2016-04-16 16:18:24,436 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904436652, "cf_name": "SORT_NAME_INDEX", "job": 1, "event": "table_file_creation", "file_number": 214, "file_size": 484107, "table_properties": {"data_size": 480188, "index_size": 3773, "filter_size": 0, "raw_key_size": 540855, "raw_average_key_size": 17, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 118, "num_entries": 31815, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,440 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [UUID_STORE] [WriteLevel0TableForRecovery] Level-0 table #215: started 
2016-04-16 16:18:24,441 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [UUID_STORE] [WriteLevel0TableForRecovery] Level-0 table #215: 668 bytes OK 
2016-04-16 16:18:24,442 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904441964, "cf_name": "UUID_STORE", "job": 1, "event": "table_file_creation", "file_number": 215, "file_size": 668, "table_properties": {"data_size": 33, "index_size": 35, "filter_size": 0, "raw_key_size": 9, "raw_average_key_size": 9, "raw_value_size": 8, "raw_average_value_size": 8, "num_data_blocks": 1, "num_entries": 1, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,444 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [UUID_URI_INDEX] [WriteLevel0TableForRecovery] Level-0 table #216: started 
2016-04-16 16:18:24,446 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [UUID_URI_INDEX] [WriteLevel0TableForRecovery] Level-0 table #216: 66443 bytes OK 
2016-04-16 16:18:24,446 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904446594, "cf_name": "UUID_URI_INDEX", "job": 1, "event": "table_file_creation", "file_number": 216, "file_size": 66443, "table_properties": {"data_size": 65318, "index_size": 575, "filter_size": 0, "raw_key_size": 17232, "raw_average_key_size": 24, "raw_value_size": 46272, "raw_average_value_size": 64, "num_data_blocks": 17, "num_entries": 718, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,449 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [URI_UUID_INDEX] [WriteLevel0TableForRecovery] Level-0 table #217: started 
2016-04-16 16:18:24,451 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [URI_UUID_INDEX] [WriteLevel0TableForRecovery] Level-0 table #217: 31558 bytes OK 
2016-04-16 16:18:24,451 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904451753, "cf_name": "URI_UUID_INDEX", "job": 1, "event": "table_file_creation", "file_number": 217, "file_size": 31558, "table_properties": {"data_size": 30297, "index_size": 676, "filter_size": 0, "raw_key_size": 52016, "raw_average_key_size": 72, "raw_value_size": 11488, "raw_average_value_size": 16, "num_data_blocks": 8, "num_entries": 718, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,456 [main] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904456005, "job": 1, "event": "recovery_finished"} 
2016-04-16 16:18:24,456 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 2] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks//MANIFEST-000188 type=3 #188 -- OK

2016-04-16 16:18:24,471 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 2] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks//000209.log type=0 #209 -- OK

2016-04-16 16:18:24,485 [main] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 2] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks//000199.log type=0 #199 -- OK

2016-04-16 16:18:24,493 [main] INFO  (RocksLoggerAdapter.java [log]:53) - DB pointer 0x7ff422afc800 
2016-04-16 16:18:24,494 [Thread-6] INFO  (RocksLoggerAdapter.java [log]:53) - [UUID_URI_INDEX] [JOB 6] Compacting 2@0 + 1@1 files to L1, score 1.00 
2016-04-16 16:18:24,494 [Thread-7] INFO  (RocksLoggerAdapter.java [log]:53) - [NAME_INDEX] [JOB 5] Compacting 2@0 + 1@1 files to L1, score 1.00 
2016-04-16 16:18:24,495 [Thread-10] INFO  (RocksLoggerAdapter.java [log]:53) - [NAME_INDEX] Compaction start summary: Base version 33 Base level 0, inputs: [211(63MB) 210(63MB)], [205(63MB)]

2016-04-16 16:18:24,495 [Thread-11] INFO  (RocksLoggerAdapter.java [log]:53) - [NAME_ID_INDEX] Compaction start summary: Base version 32 Base level 0, inputs: [213(61KB) 191(62KB)], [179(462KB)]

2016-04-16 16:18:24,495 [Thread-5] FATAL (RocksLoggerAdapter.java [log]:53) - ------- DUMPING STATS ------- 
2016-04-16 16:18:24,495 [Thread-12] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904495643, "job": 5, "event": "compaction_started", "files_L0": [211, 210], "files_L1": [205], "score": 1, "input_data_size": 200333577} 
2016-04-16 16:18:24,496 [Thread-13] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904495738, "job": 4, "event": "compaction_started", "files_L0": [213, 191], "files_L1": [179], "score": 1, "input_data_size": 600797} 
2016-04-16 16:18:24,496 [Thread-15] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904495899, "job": 6, "event": "compaction_started", "files_L0": [216, 197], "files_L1": [177], "score": 1, "input_data_size": 608122} 
2016-04-16 16:18:24,496 [Thread-14] FATAL (RocksLoggerAdapter.java [log]:53) - 
** Compaction Stats [default] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sum      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [XML_DOM_STORE] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L1      3/0     157.49   0.3      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      3/0     157.49   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [SYMBOL_STORE] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      1/0       0.00   0.5      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
  L1      3/0       0.03   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      4/0       0.03   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [COLLECTION_STORE] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sum      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [NAME_ID_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      2/0       0.12   1.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0     30.2         0         1    0.002       0      0
  L1      1/0       0.45   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      3/0       0.57   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     30.2         0         1    0.002       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     30.2         0         1    0.002       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [NAME_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      2/0     127.37   1.0      0.0     0.0      0.0       0.1      0.1       0.0   0.0      0.0     30.7         4         2    2.075       0      0
  L1      1/0      63.68   0.1      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      3/0     191.05   0.0      0.0     0.0      0.0       0.1      0.1       0.0   1.0      0.0     30.7         4         2    2.075       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.1      0.1       0.0   1.0      0.0     30.7         4         2    2.075       0      0
Flush(GB): cumulative 0.124, interval 0.124
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [SORT_NAME_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      1/0       0.46   0.5      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0     19.9         0         1    0.023       0      0
  L1      1/0       0.46   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      2/0       0.92   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     19.9         0         1    0.023       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     19.9         0         1    0.023       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [SORT_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sum      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [NGRAM_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sum      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [METADATA_STORE] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L1      1/0       0.49   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      1/0       0.49   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [METADATA_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L1      1/0       0.19   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      1/0       0.19   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [UUID_STORE] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      1/0       0.00   0.5      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.5         0         1    0.001       0      0
  L1      1/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      2/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0      0.5         0         1    0.001       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0      0.5         0         1    0.001       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [UUID_URI_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      2/0       0.13   1.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0     24.1         0         1    0.003       0      0
  L1      1/0       0.45   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      3/0       0.58   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     24.1         0         1    0.003       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     24.1         0         1    0.003       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** Compaction Stats [URI_UUID_INDEX] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      2/0       0.06   1.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0     13.0         0         1    0.002       0      0
  L1      1/0       0.21   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum      3/0       0.27   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     13.0         0         1    0.002       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     13.0         0         1    0.002       0      0
Flush(GB): cumulative 0.000, interval 0.000
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

** DB Stats **
Uptime(secs): 44.5 total, 44.5 interval
Cumulative writes: 0 writes, 0 keys, 0 batches, 0.0 writes per batch, ingest: 0.00 GB, 0.00 MB/s
Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s
Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 0 writes, 0 keys, 0 batches, 0.0 writes per batch, ingest: 0.00 MB, 0.00 MB/s
Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval stall: 00:00:0.000 H:M:S, 0.0 percent

2016-04-16 16:18:24,499 [Thread-16] INFO  (RocksLoggerAdapter.java [log]:53) - [URI_UUID_INDEX] [JOB 3] Compacting 2@0 + 1@1 files to L1, score 1.00 
2016-04-16 16:18:24,499 [main] INFO  (RocksDatabase.java [open]:121) - Opened RocksDB: /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks 
2016-04-16 16:18:24,499 [Thread-17] INFO  (RocksLoggerAdapter.java [log]:53) - [URI_UUID_INDEX] Compaction start summary: Base version 41 Base level 0, inputs: [217(30KB) 198(31KB)], [178(214KB)]

2016-04-16 16:18:24,500 [Thread-18] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904500096, "job": 3, "event": "compaction_started", "files_L0": [217, 198], "files_L1": [178], "score": 1, "input_data_size": 282942} 
2016-04-16 16:18:24,507 [Thread-19] INFO  (RocksLoggerAdapter.java [log]:53) - [UUID_URI_INDEX] [JOB 6] Generated table #222: 4340 keys, 475188 bytes 
2016-04-16 16:18:24,508 [Thread-20] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904507889, "cf_name": "UUID_URI_INDEX", "job": 6, "event": "table_file_creation", "file_number": 222, "file_size": 475188, "table_properties": {"data_size": 471554, "index_size": 3484, "filter_size": 0, "raw_key_size": 104160, "raw_average_key_size": 24, "raw_value_size": 356981, "raw_average_value_size": 82, "num_data_blocks": 117, "num_entries": 4340, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,508 [Thread-22] INFO  (RocksLoggerAdapter.java [log]:53) - [UUID_URI_INDEX] [JOB 6] Compacted 2@0 + 1@1 files to L1 => 475188 bytes 
2016-04-16 16:18:24,508 [Thread-23] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904508537, "cf_name": "NAME_ID_INDEX", "job": 4, "event": "table_file_creation", "file_number": 221, "file_size": 474145, "table_properties": {"data_size": 468917, "index_size": 5051, "filter_size": 0, "raw_key_size": 165514, "raw_average_key_size": 38, "raw_value_size": 368110, "raw_average_value_size": 84, "num_data_blocks": 110, "num_entries": 4342, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,509 [Thread-24] INFO  (RocksLoggerAdapter.java [log]:53) - [NAME_ID_INDEX] [JOB 4] Compacted 2@0 + 1@1 files to L1 => 474145 bytes 
2016-04-16 16:18:24,510 [Thread-25] INFO  (RocksLoggerAdapter.java [log]:53) - (Original Log Time 2016/04/16-16:18:24.509723) [UUID_URI_INDEX] compacted to: base level 1 max bytes base 536870912 files[0 1 0 0 0 0 0] max score 0.00, MB/sec: 50.3 rd, 39.3 wr, level 1, files in(2, 1) out(1) MB in(0.1, 0.5) out(0.5), read-write-amplify(8.1) write-amplify(3.6) OK, records in: 5778, records dropped: 1438

2016-04-16 16:18:24,510 [Thread-26] INFO  (RocksLoggerAdapter.java [log]:53) - (Original Log Time 2016/04/16-16:18:24.510125) [NAME_ID_INDEX] compacted to: base level 1 max bytes base 536870912 files[0 1 0 0 0 0 0] max score 0.00, MB/sec: 46.9 rd, 37.0 wr, level 1, files in(2, 1) out(1) MB in(0.1, 0.5) out(0.5), read-write-amplify(8.5) write-amplify(3.7) OK, records in: 5782, records dropped: 1440

2016-04-16 16:18:24,510 [Thread-27] INFO  (RocksLoggerAdapter.java [log]:53) - (Original Log Time 2016/04/16-16:18:24.509780) EVENT_LOG_v1 {"time_micros": 1460837904509738, "job": 6, "event": "compaction_finished", "compaction_time_micros": 12101, "output_level": 1, "num_output_files": 1, "total_output_size": 475188, "num_input_records": 5778, "num_output_records": 4340, "num_subcompactions": 1, "lsm_state": [0, 1, 0, 0, 0, 0, 0]} 
2016-04-16 16:18:24,510 [Thread-28] INFO  (RocksLoggerAdapter.java [log]:53) - (Original Log Time 2016/04/16-16:18:24.510167) EVENT_LOG_v1 {"time_micros": 1460837904510139, "job": 4, "event": "compaction_finished", "compaction_time_micros": 12808, "output_level": 1, "num_output_files": 1, "total_output_size": 474145, "num_input_records": 5782, "num_output_records": 4342, "num_subcompactions": 1, "lsm_state": [0, 1, 0, 0, 0, 0, 0]} 
2016-04-16 16:18:24,511 [Thread-29] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 6] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000216.sst type=2 #216 -- OK

2016-04-16 16:18:24,511 [Thread-30] INFO  (RocksLoggerAdapter.java [log]:53) - [URI_UUID_INDEX] [JOB 3] Generated table #224: 4340 keys, 220047 bytes 
2016-04-16 16:18:24,511 [Thread-31] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 4] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000213.sst type=2 #213 -- OK

2016-04-16 16:18:24,511 [Thread-32] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904511615, "job": 6, "event": "table_file_deletion", "file_number": 216} 
2016-04-16 16:18:24,511 [Thread-33] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904511680, "cf_name": "URI_UUID_INDEX", "job": 3, "event": "table_file_creation", "file_number": 224, "file_size": 220047, "table_properties": {"data_size": 214261, "index_size": 5384, "filter_size": 0, "raw_key_size": 391701, "raw_average_key_size": 90, "raw_value_size": 69440, "raw_average_value_size": 16, "num_data_blocks": 54, "num_entries": 4340, "filter_policy_name": "", "kDeletedKeys": "0"}} 
2016-04-16 16:18:24,512 [Thread-34] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904512025, "job": 4, "event": "table_file_deletion", "file_number": 213} 
2016-04-16 16:18:24,512 [Thread-35] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 6] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000197.sst type=2 #197 -- OK

2016-04-16 16:18:24,512 [Thread-37] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 4] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000191.sst type=2 #191 -- OK

2016-04-16 16:18:24,513 [Thread-38] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904512838, "job": 6, "event": "table_file_deletion", "file_number": 197} 
2016-04-16 16:18:24,513 [Thread-39] INFO  (RocksLoggerAdapter.java [log]:53) - (Original Log Time 2016/04/16-16:18:24.513273) [URI_UUID_INDEX] compacted to: base level 1 max bytes base 536870912 files[0 1 0 0 0 0 0] max score 0.00, MB/sec: 24.3 rd, 18.9 wr, level 1, files in(2, 1) out(1) MB in(0.1, 0.2) out(0.2), read-write-amplify(7.9) write-amplify(3.5) OK, records in: 5778, records dropped: 1438

2016-04-16 16:18:24,514 [Thread-40] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904513751, "job": 4, "event": "table_file_deletion", "file_number": 191} 
2016-04-16 16:18:24,514 [Thread-42] INFO  (RocksLoggerAdapter.java [log]:53) - (Original Log Time 2016/04/16-16:18:24.513460) EVENT_LOG_v1 {"time_micros": 1460837904513287, "job": 3, "event": "compaction_finished", "compaction_time_micros": 11662, "output_level": 1, "num_output_files": 1, "total_output_size": 220047, "num_input_records": 5778, "num_output_records": 4340, "num_subcompactions": 1, "lsm_state": [0, 1, 0, 0, 0, 0, 0]} 
2016-04-16 16:18:24,514 [Thread-45] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904514731, "job": 6, "event": "table_file_deletion", "file_number": 177} 
2016-04-16 16:18:24,515 [Thread-43] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 4] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000179.sst type=2 #179 -- OK

2016-04-16 16:18:24,515 [Thread-44] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 3] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000217.sst type=2 #217 -- OK

2016-04-16 16:18:24,515 [Thread-46] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904515312, "job": 4, "event": "table_file_deletion", "file_number": 179} 
2016-04-16 16:18:24,515 [Thread-47] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904515506, "job": 3, "event": "table_file_deletion", "file_number": 217} 
2016-04-16 16:18:24,516 [Thread-48] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 3] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000198.sst type=2 #198 -- OK

2016-04-16 16:18:24,518 [Thread-49] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904516713, "job": 3, "event": "table_file_deletion", "file_number": 198} 
2016-04-16 16:18:24,519 [Thread-50] DEBUG (RocksLoggerAdapter.java [log]:53) - [JOB 3] Delete /Users/aretter/code/exist-rocks/webapp/WEB-INF/data/rocks/000178.sst type=2 #178 -- OK

2016-04-16 16:18:24,519 [Thread-51] INFO  (RocksLoggerAdapter.java [log]:53) - EVENT_LOG_v1 {"time_micros": 1460837904519668, "job": 3, "event": "table_file_deletion", "file_number": 178}

Is the recovery process deleting my data somehow?
@lookforsandy lookforsandy mentioned this issue May 28, 2016
Closed
@gfosco
Copy link
Contributor

gfosco commented Jan 10, 2018

Closing this via automation due to lack of activity. If discussion is still needed here, please re-open or create a new/updated issue.

@gfosco gfosco closed this as completed Jan 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
abandoned-or-aged-out
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gfosco @adamretter