-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Thread::try_create(): pthread_create failed with error 11 #139
Comments
looks like lmtp process is running out of memory: Can you check, that you increased the system limits (stack_size and open files) for the dovecot process with: |
Thanks, the tuning solved that error but it is still failing afterward. May 9 18:00:34 ceph-all-in-one dovecot: lmtp(joewong)<3417>: Debug: value is: mail_storage |
It looks like the plugin cannot create the connection to rados storage. This may be due to insufficient user rights. Can you check that your dovecot user can read the ceph.conf and the client keyring. e.g. if you are using the defaults: ceph.client.admin.keyring. Can you connect with the ceph admin client via rados or ceph command? |
Thanks for the hints, after adjusting the setup, Dovecot can talk through the plugin with CEPH. There are missing configuration or typo in WIKI that I have to read Dovecot / Plugin Source code in order to put the correct settings in place: 20-imap.conf: mail_plugins = $mail_plugins quota storage_rbox 90-plugin.conf: rbox_cluster_name = ceph The bold values above is the adjustment I made which is differ from WIKI |
thank you for your feedback. i will have a look at it. |
I updated the WIKI, so that the documentation should now be up to date. |
I am following the wiki to setup ceph-plugin, at last dovecot (2.3.1) lmtp get assertion error as follow:
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: Thread::try_create(): pthread_create failed with error 11/home/jenkins-build/build/wo
rkspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/cep
h-12.2.5/src/common/Thread.cc: In function 'void Thread::create(const char*, size_t)' thread 7f1ab9f59740 time 2018-05-08 22:39:52.236250
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAI
LABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/ceph-12.2.5/src/common/Thread.cc: 152: FAILED assert(ret ==
0)
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7
f1aacf32d50]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 2: (Thread::create(char const*, unsigned long)+0xba) [0x7f1aad17702a]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 3: (CephContext::CephContext(unsigned int, code_environment_t, int)+0x27b) [0x7f1aad
16f19b]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 4: (common_preinit(CephInitParameters const&, code_environment_t, int)+0xa7) [0x7f1a
ad168ae7]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 5: (()+0x81e60) [0x7f1ab61e9e60]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 6: (rados_create2()+0x92) [0x7f1ab61ea442]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 7: (librmb::RadosClusterImpl::init(std::string const&, std::string const&)+0x82) [0x
7f1ab670823c]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 8: (librmb::RadosStorageImpl::open_connection(std::string const&, std::string const&
, std::string const&)+0x85) [0x7f1ab670974d]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 9: (rbox_open_rados_connection()+0x138) [0x7f1ab695291b]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 10: (rbox_mail_storage_copy(mail_save_context*, mail*)+0x9f) [0x7f1ab694650a]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 11: (rbox_mail_copy(mail_save_context*, mail*)+0xfa) [0x7f1ab69455fd]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 12: (()+0x373a) [0x7f1ab9b4273a]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 13: (()+0x4cf96) [0x7f1ab9838f96]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 14: (mail_deliver_save()+0x1a9) [0x7f1ab9b43a49]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 15: (mail_deliver()+0x216) [0x7f1ab9b440b6]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 16: (lmtp_local_data()+0x6eb) [0x56191027f7ab]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 17: (cmd_data_continue()+0x233) [0x56191027e293]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 18: (()+0x4f7e0) [0x7f1ab94c17e0]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 19: (io_loop_call_io()+0x65) [0x7f1ab9556015]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 20: (io_loop_handler_run_internal()+0x10f) [0x7f1ab955790f]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 21: (io_loop_handler_run()+0x52) [0x7f1ab9556112]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 22: (io_loop_run()+0x38) [0x7f1ab9556338]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 23: (master_service_run()+0x13) [0x7f1ab94d26c3]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 24: (main()+0x229) [0x56191027d319]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 25: (__libc_start_main()+0xf5) [0x7f1ab90d0c05]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: 26: (()+0x5445) [0x56191027d445]
May 8 22:39:52 ceph-all-in-one dovecot: lmtp: Error: NOTE: a copy of the executable, or
objdump -rdS <executable>
is needed to interpret this.I then rerun with strace, found it reaches ENOMEM after mmap then pthread_create failed:
[pid 2935] 16:48:33.051508 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0 <unfinished ...>
[pid 2935] 16:48:33.051549 <... mmap resumed> ) = -1 ENOMEM (Cannot allocate memory) <0.000036>
[pid 2935] 16:48:33.051615 rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
[pid 2935] 16:48:33.051787 <... rt_sigprocmask resumed> NULL, 8) = 0 <0.000164>
[pid 2935] 16:48:33.051841 write(2, "Thread::try_create(): pthread_create failed with error 11", 57 <unfinished ...>
[pid 2935] 16:48:33.051896 <... write resumed> ) = 57 <0.000049>
I am running ceph 12.2.5 on Centos 7 VM with 8G RAM.
Any idea how to solve this?
The text was updated successfully, but these errors were encountered: