Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis server crash when adding lots of hash keys #12

Closed
Kortenbach opened this issue Mar 21, 2018 · 23 comments
Closed

Redis server crash when adding lots of hash keys #12

Kortenbach opened this issue Mar 21, 2018 · 23 comments
Labels
bug win-port-only Related only to Windows port of Redis

Comments

@Kortenbach
Copy link

My Redis server is crashing at seemingly random points. Mostly when I'm setting a lot of Hash keys, but it could be something else. So far the server has crashed 5 times. Below is the report I copied from the server console:
[14676] 21 Mar 14:21:37.249 # fork operation complete
[14676] 21 Mar 14:21:37.250 * Background saving terminated with success
[14676] 21 Mar 14:21:51.051 * Starting automatic rewriting of AOF on 100% growth

[14676] 21 Mar 14:21:51.052 * Background append only file rewriting started by p
id 13768

=== REDIS BUG REPORT START: Cut & paste starting from here ===
Redis version: 4.0.2.1
[13768] 21 Mar 14:21:51.130 # --- EXCEPTION_ACCESS_VIOLATION
[13768] 21 Mar 14:21:51.131 # --- STACK TRACE
redis-server.exe!UnhandledExceptiontHandler(d:\dev\github\redis\src\win32_intero
p\win32_stacktrace.cpp:185)(0x00000001, 0x00000000, 0x00000001, 0x1400497A2)
kernel32.dll!UnhandledExceptionFilter(d:\dev\github\redis\src\win32_interop\win3
2_stacktrace.cpp:185)(0x0012E530, 0x00000006, 0x00000000, 0x100000001)
ntdll.dll!EtwEventSetInformation(d:\dev\github\redis\src\win32_interop\win32_sta
cktrace.cpp:185)(0x0012F170, 0x0334D8F8, 0x140170830, 0x00000000)
ntdll.dll!C_specific_handler(d:\dev\github\redis\src\win32_interop\win32_stackt
race.cpp:185)(0x00130000, 0x0012FF90, 0x0012FF90, 0x778EF818)
ntdll.dll!RtlDecodePointer(d:\dev\github\redis\src\win32_interop\win32_stacktrac
e.cpp:185)(0x00130000, 0x7779DD88, 0x000127EC, 0x001C0080)
ntdll.dll!RtlUnwindEx(d:\dev\github\redis\src\win32_interop\win32_stacktrace.cpp
:185)(0x0012F170, 0x0012EC80, 0x00000000, 0x00000000)
ntdll.dll!KiUserExceptionDispatcher(d:\dev\github\redis\src\win32_interop\win32

stacktrace.cpp:185)(0x14019AEA4, 0x1400DA5E7, 0x14019AEA4, 0x00000002)
redis-server.exe!rioWriteBulkObject(d:\dev\github\redis\src\aof.c:796)(0x0000000
0, 0x0000000E, 0x00000002, 0x14007DB77)
redis-server.exe!rioWriteHashIteratorCursor(d:\dev\github\redis\src\aof.c:979)(0
x0012F56E, 0x0012F520, 0x00000002, 0x0012F5D0)
redis-server.exe!rewriteHashObject(d:\dev\github\redis\src\aof.c:998)(0x0012F5D0
, 0x0012F5C8, 0x14019AEA6, 0x7E112AE6CC0)
redis-server.exe!rewriteAppendOnlyFile(d:\dev\github\redis\src\aof.c:1100)(0x025
70000, 0x02570000, 0x00000000, 0x02570000)
redis-server.exe!QForkChildInit(d:\dev\github\redis\src\win32_interop\win32_qfor
k.cpp:350)(0x0039B1E0, 0x00000178, 0x003A78E0, 0x1401B8D18)
redis-server.exe!QForkStartup(d:\dev\github\redis\src\win32_interop\win32_qfork.
cpp:525)(0x0039B1E0, 0x00000000, 0x00000005, 0x00000006)
redis-server.exe!main(d:\dev\github\redis\src\win32_interop\win32_qfork.cpp:1284
)(0x00000000, 0x00000000, 0x1401BAD98, 0x00000000)
redis-server.exe!__scrt_common_main_seh(f:\dd\vctools\crt\vcstartup\src\startup
exe_common.inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
kernel32.dll!BaseThreadInitThunk(f:\dd\vctools\crt\vcstartup\src\startup\exe_com
mon.inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\vcstartup\src\startup\exe_common.
inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\vcstartup\src\startup\exe_common.
inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
[13768] 21 Mar 14:21:51.153 # --- INFO OUTPUT
[14676] 21 Mar 14:22:04.652 # fork operation failed
[14676] 21 Mar 14:22:04.658 # Background AOF rewrite terminated by signal 1
[14676] 21 Mar 14:22:04.759 * Starting automatic rewriting of AOF on 110% growth

[14676] 21 Mar 14:22:04.759 * Background append only file rewriting started by p
id 14272

=== REDIS BUG REPORT START: Cut & paste starting from here ===
Redis version: 4.0.2.1
[14272] 21 Mar 14:22:04.798 # --- EXCEPTION_ACCESS_VIOLATION
[14272] 21 Mar 14:22:04.798 # --- STACK TRACE
redis-server.exe!UnhandledExceptiontHandler(d:\dev\github\redis\src\win32_intero
p\win32_stacktrace.cpp:185)(0x00000001, 0x00000000, 0x00000001, 0x1400497A2)
kernel32.dll!UnhandledExceptionFilter(d:\dev\github\redis\src\win32_interop\win3

This repeats until I close the server

@tporadowski
Copy link
Owner

Can you observe and share some statistics about usage of Redis - in terms of number of keys, data ingestion rate, and so on? I'm looking for some clues how to reproduce this :)

@Kortenbach
Copy link
Author

Kortenbach commented Mar 21, 2018 via email

@tporadowski
Copy link
Owner

tomasz@poradowski.com (it's also in my profile)

@tporadowski
Copy link
Owner

@Kortenbach please ping me here when you send it as it might go to spam by accident

@Kortenbach
Copy link
Author

ping

@tporadowski
Copy link
Owner

Pong! As expected, landed in spam folder due to suspicious attachment ;)

@tporadowski
Copy link
Owner

@Kortenbach I've just sent you fixed version for testing via email. If it also works fine for you - I will commit the changes and release version 4.0.2.2. Thanks!

@Kortenbach
Copy link
Author

Kortenbach commented Mar 23, 2018 via email

@tporadowski tporadowski added bug win-port-only Related only to Windows port of Redis labels Mar 23, 2018
@wywdstc
Copy link

wywdstc commented Apr 11, 2018

test

=== REDIS BUG REPORT START: Cut & paste starting from here ===
Redis version: 4.0.2.2
[3468] 11 Apr 15:53:59.670 # --- EXCEPTION_ACCESS_VIOLATION
[3468] 11 Apr 15:53:59.675 # --- STACK TRACE
redis-server.exe!UnhandledExceptiontHandler(d:\dev\github\redis\src\win32_intero
p\win32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000001, 0x140049903)
KERNELBASE.dll!UnhandledExceptionFilter(d:\dev\github\redis\src\win32_interop\wi
n32_stacktrace.cpp:185)(0x00000003, 0x7FFB84541AD0, 0x00000000, 0x7FFB86FE8736)
ntdll.dll!memset(d:\dev\github\redis\src\win32_interop\win32_stacktrace.cpp:185)
(0x0013EE70, 0x7FA71903A9AA, 0x140170830, 0x7FA600000000)
ntdll.dll!_C_specific_handler(d:\dev\github\redis\src\win32_interop\win32_stackt
race.cpp:185)(0x00000000, 0x0013E3B0, 0x0013EE70, 0x0013EE70)
ntdll.dll!chkstk(d:\dev\github\redis\src\win32_interop\win32_stacktrace.cpp:185
)(0x00000001, 0x7FFB86FB0000, 0x00000000, 0x7FFB870E6204)
ntdll.dll!RtlRaiseException(d:\dev\github\redis\src\win32_interop\win32_stacktra
ce.cpp:185)(0x7FA69AED1614, 0x7FA64A505D17, 0x000003E0, 0x7FA6981ABD3A)
ntdll.dll!KiUserExceptionDispatcher(d:\dev\github\redis\src\win32_interop\win32

stacktrace.cpp:185)(0x03C07040, 0x7FA6D3E36290, 0x0013F240, 0x7FA6D3E1D923)
redis-server.exe!dictNext(d:\dev\github\redis\src\dict.c:600)(0x7FA6D3E36290, 0x
0000000F, 0x0013F20F, 0x0000039E)
redis-server.exe!rdbSaveObject(d:\dev\github\redis\src\rdb.c:787)(0x0013F240, 0x
00000004, 0x0013F240, 0x0013F240)
redis-server.exe!rdbSaveRio(d:\dev\github\redis\src\rdb.c:958)(0x001A6C30, 0x01A
A0000, 0x01EA2760, 0x00000D8C)
redis-server.exe!rdbSave(d:\dev\github\redis\src\rdb.c:1038)(0x01AA0000, 0x00000
005, 0x01AA0000, 0x00000010)
redis-server.exe!QForkChildInit(d:\dev\github\redis\src\win32_interop\win32_qfor
k.cpp:344)(0x00000000, 0x001AB500, 0x001A6AB0, 0x1401B8E70)
redis-server.exe!QForkStartup(d:\dev\github\redis\src\win32_interop\win32_qfork.
cpp:525)(0x001AB500, 0x00000000, 0x00000005, 0x00000006)
redis-server.exe!main(d:\dev\github\redis\src\win32_interop\win32_qfork.cpp:1284
)(0x140045300, 0x00000000, 0x1401BAEF8, 0x7FF5FFFF4000)
redis-server.exe!__scrt_common_main_seh(f:\dd\vctools\crt\vcstartup\src\startup
exe_common.inl:283)(0x140045328, 0x7FF5FFFF4000, 0x00000000, 0x00000000)
KERNEL32.DLL!BaseThreadInitThunk(f:\dd\vctools\crt\vcstartup\src\startup\exe_com
mon.inl:283)(0x7FFB851213B0, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\vcstartup\src\startup\exe_common.
inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\vcstartup\src\startup\exe_common.
inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
[3468] 11 Apr 15:53:59.990 # --- INFO OUTPUT
[3316] 11 Apr 15:54:00.299 # Error writing to socket fd

Hi,version 4.0.2.2 still has this bug。

@tporadowski
Copy link
Owner

@wywdstc can you provide some steps to reproduce it?

@wywdstc
Copy link

wywdstc commented Apr 11, 2018

@tporadowski
Also when adding lots of hash key..

@tporadowski
Copy link
Owner

Could you please be a little more specific? :) What is "lots of hash key"? Are you adding new hashes or setting many entries in one or more hashes?

@wywdstc
Copy link

wywdstc commented Apr 11, 2018

taobao
Accurately,I use hmset to set lots of hash keys like this, by pipeline.
Every 50,000 keys, i has slept 3 second.
my cluster has 5 master nodes and 5 slave nodes.

@wywdstc
Copy link

wywdstc commented Apr 11, 2018

The bug machion is slave node,os is win2012,

错误应用程序名称: redis-server.exe,版本: 4.0.2.2,时间戳: 0x5ab8cd85
错误模块名称: redis-server.exe,版本: 4.0.2.2,时间戳: 0x5ab8cd85
异常代码: 0xc0000005
错误偏移量: 0x00000000000fad7c
错误进程 ID: 0xc94
错误应用程序启动时间: 0x01d3d16a48de62ac
错误应用程序路径: F:\redis\Redis-x64-4.0.2.2\13229\redis-server.exe
错误模块路径: F:\redis\Redis-x64-4.0.2.2\13229\redis-server.exe
报告 ID: bfa06d2f-3d5d-11e8-80de-1c872c778c17
错误程序包全名:
错误程序包相对应用程序 ID:

@wywdstc
Copy link

wywdstc commented Apr 11, 2018

192.168.0.229:14229> info

Server

redis_version:4.0.2.2
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:d5d2b1e1e93d2953
redis_mode:cluster
os:Windows
arch_bits:64
multiplexing_api:WinSock_IOCP
atomicvar_api:pthread-mutex
process_id:1244
run_id:5afdd61c8abd931e131ee25739a7780402734c72
tcp_port:14229
uptime_in_seconds:0
uptime_in_days:0
hz:10
lru_clock:13489262
executable:F:\redis\Redis-x64-4.0.2.2\14229\redis-server
config_file:F:\redis\Redis-x64-4.0.2.2\14229\redis.windows.conf

Clients

connected_clients:1
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0

Memory

used_memory:3225813168
used_memory_human:3.00G
used_memory_rss:0
used_memory_rss_human:0B
used_memory_peak:3225813168
used_memory_peak_human:3.00G
used_memory_peak_perc:100.00%
used_memory_overhead:631571310
used_memory_startup:1194872
used_memory_dataset:2594241858
used_memory_dataset_perc:80.45%
total_system_memory:0
total_system_memory_human:0B
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:0.00
mem_allocator:jemalloc-3.6.0
active_defrag_running:0
lazyfree_pending_objects:0

Persistence

loading:1
rdb_changes_since_last_save:10724999
rdb_bgsave_in_progress:0
rdb_last_save_time:1523438702
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
loading_start_time:1523438702
loading_total_bytes:6375656325
loading_loaded_bytes:2045130219
loading_loaded_perc:32.08
loading_eta_seconds:476

Stats

total_connections_received:1
total_commands_processed:1
instantaneous_ops_per_sec:0
total_net_input_bytes:31
total_net_output_bytes:10162
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0

Replication

role:master
connected_slaves:0
master_replid:77aec56bd8a57dce36f8cd42a294a802c8f2509a
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:67108864
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

CPU

used_cpu_sys:10.08
used_cpu_user:212.36
used_cpu_sys_children:0.00
used_cpu_user_children:0.00

Cluster

cluster_enabled:1

Keyspace

db0:keys=10724999,expires=0,avg_ttl=0
192.168.0.229:14229>

@tporadowski
Copy link
Owner

Thanks for the details. I am currently feeding my test instance with 1M hashes with 100 items each (random string values of 40 characters), so far no issue at all. I'll see what happens in cluster mode. Could you please share your configuration (if not here, then via email from my profile)?

@wywdstc
Copy link

wywdstc commented Apr 11, 2018

ok,here is it.

wywdstc-issue12.txt

@wywdstc
Copy link

wywdstc commented Apr 11, 2018

i have restarted this node,bug still occurs

.168.0.226:13226@23226 master
[2416] 11 Apr 22:09:43.276 # WSA_IO_PENDING writing to socket fd 138
[2416] 11 Apr 22:09:43.276 # clusterWriteDone written 2568 fd 138

=== REDIS BUG REPORT START: Cut & paste starting from here ===
Redis version: 4.0.2.2
[2416] 11 Apr 22:09:43.589 # --- EXCEPTION_ACCESS_VIOLATION
[2416] 11 Apr 22:09:43.604 # --- STACK TRACE
redis-server.exe!UnhandledExceptiontHandler(d:\dev\github\redis\src\win32_intero
p\win32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000001, 0x140049903)
KERNELBASE.dll!UnhandledExceptionFilter(d:\dev\github\redis\src\win32_interop\wi
n32_stacktrace.cpp:185)(0x00000003, 0x7FFB02B41AD0, 0x00000000, 0x7FFB05588736)
ntdll.dll!memset(d:\dev\github\redis\src\win32_interop\win32_stacktrace.cpp:185)
(0x0013E170, 0x00000025, 0x140170830, 0x00000000)
ntdll.dll!_C_specific_handler(d:\dev\github\redis\src\win32_interop\win32_stackt
race.cpp:185)(0x00000000, 0x0013D6B0, 0x0013E170, 0x0013E170)
ntdll.dll!chkstk(d:\dev\github\redis\src\win32_interop\win32_stacktrace.cpp:185
)(0x00000001, 0x7FFB05550000, 0x00000000, 0x7FFB05686204)
ntdll.dll!RtlRaiseException(d:\dev\github\redis\src\win32_interop\win32_stacktra
ce.cpp:185)(0x00000000, 0x00000000, 0x0000000E, 0x0101BA5A)
ntdll.dll!KiUserExceptionDispatcher(d:\dev\github\redis\src\win32_interop\win32

stacktrace.cpp:185)(0x7FA626C00250, 0x1400356B6, 0x7FA6A9733B80, 0x1400306A2)
redis-server.exe!raxInsert(d:\dev\github\redis\src\rax.c:419)(0x7FA62700C160, 0x
00000018, 0x00000015, 0x7FA6A972F9F4)
redis-server.exe!slotToKeyUpdateKey(d:\dev\github\redis\src\db.c:1379)(0x7FA6270
6EF00, 0x7FA62706EF00, 0x0013E541, 0x0013E541)
redis-server.exe!dbAdd(d:\dev\github\redis\src\db.c:170)(0x0013E9F0, 0x7FA62706E
F00, 0x0013E9F0, 0x00000013)
redis-server.exe!rdbLoadRio(d:\dev\github\redis\src\rdb.c:1656)(0x14019A814, 0x0
013EAA0, 0x00000000, 0x14001CAD3)
redis-server.exe!rdbLoad(d:\dev\github\redis\src\rdb.c:1697)(0xAEE434C5, 0xFFFFF
FFFFFFFFFFF, 0xFFFFFFFFFFFFFFFF, 0x00000000)
redis-server.exe!readSyncBulkPayload(d:\dev\github\redis\src\replication.c:1379)
(0x00000001, 0x7FA62700B500, 0x7FA62706F9C0, 0x20C49BA5E353F7CF)
redis-server.exe!aeMain(d:\dev\github\redis\src\ae.c:499)(0x1D3D19E0456FA91, 0x5
699306E5A2EE, 0x0013FC29, 0x00000000)
redis-server.exe!redis_main(d:\dev\github\redis\src\server.c:4027)(0x001A3A00, 0
x00000005, 0x00000005, 0x00000002)
redis-server.exe!main(d:\dev\github\redis\src\win32_interop\win32_qfork.cpp:1285
)(0x140045300, 0x00000000, 0x1401BAEF8, 0x7FF5FFFF3000)
redis-server.exe!__scrt_common_main_seh(f:\dd\vctools\crt\vcstartup\src\startup
exe_common.inl:283)(0x140045328, 0x7FF5FFFF3000, 0x00000000, 0x00000000)
KERNEL32.DLL!BaseThreadInitThunk(f:\dd\vctools\crt\vcstartup\src\startup\exe_com
mon.inl:283)(0x7FFB04C513B0, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\vcstartup\src\startup\exe_common.
inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\vcstartup\src\startup\exe_common.
inl:283)(0x00000000, 0x00000000, 0x00000000, 0x00000000)
[2416] 11 Apr 22:09:44.012 # --- INFO OUTPUT

Server

redis_version:4.0.2.2
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:d5d2b1e1e93d2953
redis_mode:cluster
os:Windows
arch_bits:64
multiplexing_api:WinSock_IOCP
atomicvar_api:pthread-mutex
process_id:2416
run_id:e32a49c9822f222d044d929d40a1a1f87bd0942b
tcp_port:14229
uptime_in_seconds:703
uptime_in_days:0
hz:10
lru_clock:13506235
executable:F:\redis\Redis-x64-4.0.2.2\14229\redis-server
config_file:F:\redis\Redis-x64-4.0.2.2\14229\redis.windows.conf

Clients

connected_clients:0
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0

Memory

used_memory:2206297760
used_memory_human:2.05G
used_memory_rss:5923085760
used_memory_rss_human:5.52G
used_memory_peak:5923085920
used_memory_peak_human:5.52G
used_memory_peak_perc:37.25%
used_memory_overhead:774367576
used_memory_startup:1194872
used_memory_dataset:1431930184
used_memory_dataset_perc:64.94%
total_system_memory:0
total_system_memory_human:0B
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:2.68
mem_allocator:jemalloc-3.6.0
active_defrag_running:0
lazyfree_pending_objects:0

Persistence

loading:1
rdb_changes_since_last_save:19628588
rdb_bgsave_in_progress:0
rdb_last_save_time:1523455080
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
loading_start_time:1523455735
loading_total_bytes:2934191301
loading_loaded_bytes:513802175
loading_loaded_perc:17.51
loading_eta_seconds:230

Stats

total_connections_received:0
total_commands_processed:0
instantaneous_ops_per_sec:0
total_net_input_bytes:2934191301
total_net_output_bytes:0
instantaneous_input_kbps:43199.56
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0

Replication

role:slave
master_host:192.168.0.228
master_port:13228
master_link_status:down
master_last_io_seconds_ago:-1
master_sync_in_progress:1
slave_repl_offset:1
master_sync_left_bytes:0
master_sync_last_io_seconds_ago:108
master_link_down_since_seconds:1523455783
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:1c5520eb858ee5052dbaf990ed498cb82d38eabd
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:67108864
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

CPU

used_cpu_sys:45.31
used_cpu_user:476.78
used_cpu_sys_children:0.00
used_cpu_user_children:0.00

Commandstats

Cluster

cluster_enabled:1

Keyspace

db0:keys=5907544,expires=0,avg_ttl=0
[2416] 11 Apr 22:09:45.130 #
=== REDIS BUG REPORT END. Make sure to include from START to END. ===

   Please report this bug by following the instructions at:

 http://github.com/MSOpenTech/redis/wiki/Submitting-an-Issue

Suspect RAM error? Use redis-server --test-memory to verify it.

@wywdstc
Copy link

wywdstc commented Apr 12, 2018

It seems memory full caused this issue?
This node only has 32G memory.
Other nodes which have more memory work well.

@tporadowski
Copy link
Owner

I suppose that's quite likely. Yesterday I tried a 3+3 cluster with a million of hashes (200 keys with 10 random characters as values) and with save 60 500 setting to dump the database quite often - and could not run into this issue at all. When I have some time I'll check this full-memory scenario.

@ChandraModemGM
Copy link

ChandraModemGM commented Sep 24, 2020

Hi,
I'm using windows 3.2.100 version of Redis and am trying to test inserting about million records from kafka stream. In about 10-15 tests I've performed, I'm seen missing entries in some cases (a max of abt 3000 keys or so) and have seen my Redis server crash a few times. Is this version of Redis considered stable? And are there any recommendations as to what needs to be done? Appreciate your inputs. Thx much


Crash details

Faulting application name: redis-server.exe, version: 0.0.0.0, time stamp: 0x57767e0b
Faulting module name: redis-server.exe, version: 0.0.0.0, time stamp: 0x57767e0b
Exception code: 0xc0000005
Fault offset: 0x00000000000b27e8
Faulting process id: 0x1774
Faulting application start time: 0x01d69208b2d19698
Faulting application path: C:\Program Files\Redis\redis-server.exe
Faulting module path: C:\Program Files\Redis\redis-server.exe
Report Id: 978cefba-af9a-4bd9-8bf7-b155364d0c4f
Faulting package full name:
Faulting package-relative application ID:

Additional Info

host:port> dbsize
(integer) 998684

host:port>info memory

Memory

used_memory:1789559872
used_memory_human:1.67G
used_memory_rss:1789522936
used_memory_rss_human:1.67G
used_memory_peak:1804151688
used_memory_peak_human:1.68G
total_system_memory:0
total_system_memory_human:0B
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:3221225472
maxmemory_human:3.00G
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.00
mem_allocator:jemalloc-3.6.0

@tporadowski
Copy link
Owner

@ChandraModemGM version 3.2.100 was the last released by MS OpenTech team in July 2016 and then it got suddenly stopped. You can try the latest v5.0.9 from this fork (https://github.com/tporadowski/redis/releases) and if you still run into issues - report them in issues.

@ChandraModemGM
Copy link

@tporadowski Thanks much, will take a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug win-port-only Related only to Windows port of Redis
Projects
None yet
Development

No branches or pull requests

4 participants