Mac OSX support incomplete #3
Comments
|
We haven't tried it yet. Thrift and Folly, our main depencies, are pretty much working under OS X. This line was basically to document that it is simply untested at this point. |
|
Awesome. When I have time I'll test on OSX. |
|
Building the dependencies is a bit tricky, at least on Yosemite. @dcsommer, would a guide for building on Mac be something worth investing time in? |
|
Perhaps even better than a guide would be a Homebrew formula. Then everyone could just A brief search turned up Homebrew/legacy-homebrew#33086. |
|
I did the build via docker on yosemite , it will works as long as you set virtual box memory to 3 Gigs. |
|
@rezacute Building via docker isn't really what we're looking for here. We want to get a native compile on OSX working. |
|
There is no support for numa on OSX. I get this error configure: error: Please install numa library The rest of the dependencies can be installed via brew |
|
May be the dependencies can be detected like this But this might be a overkill. May be do a bunch of wget of the dependencies from a site and do plain old configure, make and make install . The dependencies versions should be known ahead. |
|
I have succeeded (but still can not run the echo samples) building proxygen on osx yosemite by lot of hacks especially on fbthrift (removing NUMA, and the compiler from the build). Syahs-MacBook-Air:echo syahriza$ ./echo_server
dyld: Symbol not found: __ZTIN6apache6thrift9transport16THeaderTransportE
Referenced from: /Users/syahriza/Documents/private/Proxygen_dev/proxygen/proxygen/fbthrift/thrift/lib/cpp/.libs//libthrift.14.dylib
Expected in: flat namespace
in /Users/syahriza/Documents/private/Proxygen_dev/proxygen/proxygen/fbthrift/thrift/lib/cpp/.libs//libthrift.14.dylib
Trace/BPT trap: 5Anyone have suggestion? |
|
@rezacute how did you get folly working on osx? |
|
Maybe try again now that the dependency on fbthrift has been removed? |
|
I was able to build the latest versions of folly and proxygen on Mac OS 10.10 using downloaded llvm and Boost was installed using I have tried various other combinations of compilers and c++ libraries but only this configuration have worked and produced valid binaries. I also struggled with building libproxygenssl using native darwin toolset (ar not gar) With this setup "make check" in folly passes all tests except for one: Details on the failed test in thread-local group: make check proxygen also has failures in util tests: util test details: Not sure if this a big deal and did not have time to look into the test code itself. But the real problem is the fact the echo sample consistently crashes within seconds with assertion in folly: I tried linking both folly and proxygen with and without jemalloc - the same effect. Can somebody explain the nature of this assertion and exception? Thanks |
|
@jwatzman - do you might have an idea? |
|
I don't, sorry. I have been poking at HHVM's dependencies (of which proxygen and folly are two) as well as HHVM itself to try to get it to build on OS X, but haven't quite succeeded yet. So there are likely issues actually running the code I have yet to get to :) Nothing in the traces above jumps out at me as an obvious likely culprit, sorry. |
|
My guess is that's just a bug. That particular code is doing some pretty platform-specific nonsense (e.g. comments about glibc internals) so I wouldn't be surprised if its just not portable. We can probably rip out the necessary bits to get a reasonable repro. There might be a strong hint if you could pull |
Summary: This diff strengthens the preconditions for attempting to use optimize by madvising away idle stacks, because it seemed to be causing problems on OS X (facebook/proxygen#3). Test Plan: unit tests Reviewed By: ldbrandy@fb.com Subscribers: folly-diffs@, yfeldblum, chalfant FB internal diff: D2129100 Signature: t1:2129100:1433458268:0b6b3696dde6c2bd13b89bc7ec58f0b1898be458
|
@yfinkelstein: Thanks again for this. I just pushed a new version (43) to folly that has a fix for this. |
…on closed Summary: Add emitTraceEvents() to TraceEventObserver. Reviewed By: @yangchi Differential Revision: D2429577
|
I've spent a few hours now trying to get a valid OS X build working. However I was unable to get a valid build for neither folly nor wangle. Both fail to build from the latest master. Wangle has two issues outstanding regarding OS X: And this issue on folly which seems very similar to errors I was getting has no replies: From the issues I ran into with folly, supporting OS X will require some significant changes (such as removing the use of TLS). Would that sort of fix make sense on the private internal branch to support OS X? |
|
I'm also very interested into any plans for supporting OS X. |
|
https://github.com/hhvm/homebrew-hhvm/ could be a useful reference - HHVM uses Folly, Wangle, and Proxygen, and builds on OSX via homebrew with that. |
|
The specific bits of folly and proxygen that HHVM needs to build and work on OS X, but notably that is not all of folly, and doesn't include any of the folly or proxygen binaries or tests. So it's possible to use as a library as-is, but not without some serious finagling. |
|
I have worked on compiling Proxygen on OS X and I've been successful after a few modifications along the way. I've blogged about it over here and I'd like to know if you're having any success reproducing my results. I hope this may help to bring full OS X support to the Proxygen library eventually. |
|
I have tried to build Proxygen on OSX, but still blocked due to missing libcap. Is there a way to workaround the "libcap" dependency? |
|
Simply comment out the else clause in the configure script where it spits an error. That way it will keep going without generating the #define HAVE_LIBCAP 1 instruction which is all fine to experiment with Proxygen on OS X. |
|
Not to resurrect old issues or anything, but just in case someone else wanted some steps for building proxygen on OS X, here's a gist of what I did that got everything up and running pretty quickly. Only real oddities are that you have to patch the configuration and Makefiles to link against P.S. There are some test failures for both Wangle and Proxygen when building against OS X, so you are in a bit of uncharted territory, but running and compiling the Proxygen samples seems to work like a charm. |
|
For what it’s worth, it’s been used as part of HHVM on Mac for a few years now, without any issues I remember - though we extended HHVM’s cmake-based build system to build it, rather than using proxygen’s one |
Summary:
Exposed by UBSAN:
```lang=bash
proxygen/lib/http/codec/SPDYConstants.cpp:67:11: runtime error: load of value 314159, which is not a valid value for type 'proxygen::spdy::ResetStatusCode'
#0 0x7f81bb4b39d9 in proxygen::spdy::rstToErrorCode(proxygen::spdy::ResetStatusCode) proxygen/lib/http/codec/SPDYConstants.cpp:67
#1 0x7f81bb443d31 in proxygen::SPDYCodec::onRstStream(unsigned int) proxygen/lib/http/codec/SPDYCodec.cpp:1331
#2 0x7f81bb43f559 in proxygen::SPDYCodec::onControlFrame(folly::io::Cursor&) proxygen/lib/http/codec/SPDYCodec.cpp:376
#3 0x7f81bb43c305 in proxygen::SPDYCodec::parseIngress(folly::IOBuf const&) proxygen/lib/http/codec/SPDYCodec.cpp:309
#4 0x7f81bb43a4ec in proxygen::SPDYCodec::onIngress(folly::IOBuf const&) proxygen/lib/http/codec/SPDYCodec.cpp:243
#5 0x7f81cdf12854 in proxygen::PassThroughHTTPCodecFilter::onIngress(folly::IOBuf const&) proxygen/lib/http/codec/HTTPCodecFilter.cpp:170
#6 0x7f81cdf12854 in proxygen::PassThroughHTTPCodecFilter::onIngress(folly::IOBuf const&) proxygen/lib/http/codec/HTTPCodecFilter.cpp:170
#7 0x7f81cdf12854 in proxygen::PassThroughHTTPCodecFilter::onIngress(folly::IOBuf const&) proxygen/lib/http/codec/HTTPCodecFilter.cpp:170
#8 0x7f81cdf12854 in proxygen::PassThroughHTTPCodecFilter::onIngress(folly::IOBuf const&) proxygen/lib/http/codec/HTTPCodecFilter.cpp:170
#9 0x7f81d0da0239 in proxygen::HTTPSession::processReadData() proxygen/lib/http/session/HTTPSession.cpp:487
#10 0x7f81d0d9fe39 in proxygen::HTTPSession::readDataAvailable(unsigned long) proxygen/lib/http/session/HTTPSession.cpp:440
#11 0x7f81e6daea85 in folly::AsyncSocket::handleRead() folly/io/async/AsyncSocket.cpp:1900
#12 0x7f81e6da93f9 in folly::AsyncSocket::ioReady(unsigned short) folly/io/async/AsyncSocket.cpp:1694
#13 0x7f81e6dc14ee in folly::AsyncSocket::IoHandler::handlerReady(unsigned short) folly/io/async/AsyncSocket.h:976
#14 0x7f81e6e5e91b in folly::EventHandler::libeventCallback(int, short, void*) folly/io/async/EventHandler.cpp:160
#15 0x7f81de739b11 in event_process_active /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:390
#16 0x7f81de739b11 in event_base_loop /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/event.c:532
#17 0x7f81e6e11864 in folly::EventBase::loopBody(int) folly/io/async/EventBase.cpp:313
#18 0x7f81e6e10726 in folly::EventBase::loop() folly/io/async/EventBase.cpp:252
#19 0x7f81e6e14e85 in folly::EventBase::loopForever() folly/io/async/EventBase.cpp:450
#20 0x7f81c4141a7b in proxygen::WorkerThread::runLoop() proxygen/lib/services/WorkerThread.cpp:137
#21 0x7f81c4143afe in proxygen::WorkerThread::start()::$_0::operator()() proxygen/lib/services/WorkerThread.cpp:41
#22 0x7f81c4143a34 in void std::_Bind_simple<proxygen::WorkerThread::start()::$_0 ()>::_M_invoke<>(std::_Index_tuple<>) third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/functional:1530
#23 0x7f81c4143a04 in std::_Bind_simple<proxygen::WorkerThread::start()::$_0 ()>::operator()() third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/functional:1520
#24 0x7f81c41438d8 in std::thread::_Impl<std::_Bind_simple<proxygen::WorkerThread::start()::$_0 ()> >::_M_run() third-party-buck/gcc-5-glibc-2.23/build/libgcc/include/c++/trunk/thread:115
#25 0x7f81da7ef170 in execute_native_thread_routine /home/engshare/third-party2/libgcc/5.x/src/gcc-5/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:84
#26 0x7f81d7e1f7a8 in start_thread /home/engshare/third-party2/glibc/2.23/src/glibc-2.23/nptl/pthread_create.c:333
#27 0x7f81d71f9a7c in __clone /home/engshare/third-party2/glibc/2.23/src/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:109
SUMMARY: UndefinedBehaviorSanitizer: invalid-enum-load proxygen/lib/http/codec/SPDYConstants.cpp:67:11
```
Here we have a typical function that converts from one enum type to another. There is a special validation logic in place, if the original enum value is not among predetermined values the result is a "special" error value. UBSAN detected that the argumemnt value which was provided was invalid for the original enum type. To prevent the UB and potential overflow use an int type.
Reviewed By: w-o-o
Differential Revision: D7185566
fbshipit-source-id: 613c8c6aa64d24c48889810e56a79fd370cdb32b
|
well, I am happy to say that this should be fixed now. |
|
I tried running I tried building every dependency manually and I had to disable building the tests for some dependencies. I managed to compile proxygen but couldn't link it because of some linking errors related to libfolly. It might not be helpful to consider this though as I may have made a mistake while building everything manually. Better focus on the dependency problem with |
|
ok, let me reopen and look at this. thanks! |
Summary: This should fix the Travis CI builds. It adds rust toolchain support inside docker and sets the required THRIFT env variable. Pull Request resolved: facebookexperimental/rust-shed#3 Reviewed By: krallin Differential Revision: D18905608 Pulled By: lukaspiatkowski fbshipit-source-id: 5db1eff6f215a6617d8acaa0c99a62d45225956b
|
Any updates on this? |
|
@benstadin there are two build script that should help you get a mac build.
1 uses homebrew to pull dependencies Let me know if you hit any build issue |
|
@lnicco Hi, I'm running into an issue with building Proxygen when getting to Folly: However I installed Folly using brew so I am not sure why Proxygen is rebuilding it. Here is a link to more output: link. I'm on OS X High Sierra (10.13.3) |
|
closing this issue again. please refer to the scripts mentioned above for building on osx. @MathBunny thanks for reporting that build failure, it does look like a problem with the folly setup though. |
Summary: Generic WebTransport interface Principles: 1. It should be easy to write simple applications 2. The backpressure and error handling APIs should be understandable 3. The same generic API should work with traditional async code and coroutines Futures is the best way to implement #3 because they can also be awaited in a coroutine, without introducing a dependency on coroutines yet. Reviewed By: lnicco Differential Revision: D47422753 fbshipit-source-id: 6aad62cd399a9e8112cab37f4979003d560caf8a
As per the readme, OS X support is incomplete. What work needs to be done to achieve full compatibility?
The text was updated successfully, but these errors were encountered: