Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unused options error in make_docker_context.py #23

Closed
jeroen-dhollander opened this issue Mar 19, 2018 · 2 comments
Closed

Unused options error in make_docker_context.py #23

jeroen-dhollander opened this issue Mar 19, 2018 · 2 comments

Comments

@jeroen-dhollander
Copy link

Issue Description

When running make_docker_context.py, it fails with an Unused options error

Environment

  • tag or commit hash on which this occured
    master
  • OS version: <e.g. ubuntu-16.04>
    CentOS 7
    ...

Minimal test code / Steps to reproduce the issue

Just run ./make_docker_context.py in build/fbcode_builder

What's the actual result?

/tmp/openr/build/fbcode_builder (master %=) > ./make_docker_context.py 
INFO: Docker ccache not enabled
Traceback (most recent call last):
  File "./make_docker_context.py", line 173, in <module>
    config['github_project'],
  File "./make_docker_context.py", line 154, in make_docker_context
    dockerfile = builder.render(get_steps_fn(builder))
  File "/tmp/openr/build/fbcode_builder/fbcode_builder.py", line 136, in render
    .format(unused_options)
RuntimeError: Unused options: set([u'prefix', u'make_parallelism', u'projects_dir']) -- please check if you made a typo in any of them. Those that are truly not useful should be not be set so that this typo detection can be useful.

What's the expected result?

No error and the Dockerfile is generated.

Note: If I simply remove that unused options check the Dockerfile is generated (but I think it then ignores those 3 options).
@jstrizich
Copy link
Contributor

this is a bit of ugliness. you need to run that command from the parent directory (openr/build/) to pick up the fbcode_builder_config.py that lives there. The one in openr/build/fbcode_builder is usesd so the command works for --help

@jeroen-dhollander
Copy link
Author

Thank you, running it from the parent directory solved the errors.

@jeroen-dhollander jeroen-dhollander mentioned this issue Mar 21, 2018
Closed
facebook-github-bot pushed a commit to facebook/fbthrift that referenced this issue Mar 23, 2018
@jstrizich @facebook-github-bot
add better error for passing config with no steps
e406b7b 
Summary: in response to facebook/openr#23

Reviewed By: snarkmaster

Differential Revision: D7342371

fbshipit-source-id: 15737f79900722ce016494f994932aa26e5acad5
facebook-github-bot pushed a commit to facebookarchive/fbzmq that referenced this issue Mar 23, 2018
@jstrizich @facebook-github-bot
add better error for passing config with no steps
8327697 
Summary: in response to facebook/openr#23

Reviewed By: snarkmaster

Differential Revision: D7342371

fbshipit-source-id: 15737f79900722ce016494f994932aa26e5acad5
facebook-github-bot pushed a commit to facebook/folly that referenced this issue Mar 23, 2018
@jstrizich @facebook-github-bot
add better error for passing config with no steps
e37d52f 
Summary: in response to facebook/openr#23

Reviewed By: snarkmaster

Differential Revision: D7342371

fbshipit-source-id: 15737f79900722ce016494f994932aa26e5acad5
facebook-github-bot pushed a commit to facebookarchive/bistro that referenced this issue Mar 23, 2018
@jstrizich @facebook-github-bot
add better error for passing config with no steps
39ef3bb 
Summary: in response to facebook/openr#23

Reviewed By: snarkmaster

Differential Revision: D7342371

fbshipit-source-id: 15737f79900722ce016494f994932aa26e5acad5
facebook-github-bot pushed a commit that referenced this issue Mar 23, 2018
@jstrizich @facebook-github-bot
add better error for passing config with no steps
9fe7daf 
Summary: in response to #23

Reviewed By: snarkmaster

Differential Revision: D7342371

fbshipit-source-id: 15737f79900722ce016494f994932aa26e5acad5
facebook-github-bot pushed a commit to facebook/proxygen that referenced this issue Mar 23, 2018
@jstrizich @facebook-github-bot
add better error for passing config with no steps
0ae75f9 
Summary: in response to facebook/openr#23

Reviewed By: snarkmaster

Differential Revision: D7342371

fbshipit-source-id: 15737f79900722ce016494f994932aa26e5acad5
facebook-github-bot pushed a commit that referenced this issue Jun 22, 2020
@lukaspiatkowski @facebook-github-bot
GitHub Actions: fix running out of space on Ubuntu build workflow (#23)
99db467 
Summary:
As per actions/runner-images#709 there started to be some issies with Ubuntu envs running out of space. This should fix it.

Also our Cargo builds use a lot of space, changing them to be non-incremental and removing debug symbols keeps the build fast, but greatly reduces the disk space usage leaving us enough space on GitHub Actions virtual machines.
Pull Request resolved: facebook/sapling#23

Reviewed By: farnz

Differential Revision: D22160020

Pulled By: lukaspiatkowski

fbshipit-source-id: c23393e310c15ebf5a18b80f0bb5f1f894d24849
facebook-github-bot pushed a commit that referenced this issue Oct 12, 2020
@xiangxu1121 @facebook-github-bot
Bugfix: fix kvstore stuck issue during desctruction sequence
04ea157 
Summary:
As titled, several different approaches have been added to address original issue of crash when eventbase is destructed. See P135204939

```
(gdb) bt
#0  0x000000000187ea8e in re2::RE2::Set::Match (this=0x2, text=..., v=0x7fffd8c0d500, error_info=0x0) at re2/set.cc:110
#1  0x000000000072d304 in openr::KeyPrefix::keyMatch (this=<optimized out>, key=...) at openr/common/Util.cpp:50
#2  0x00000000006d3649 in openr::KvStoreFilters::keyMatch (this=0x7fb22c613660, key=..., value=...) at openr/kvstore/KvStore.cpp:69
#3  0x00000000006d60e4 in openr::KvStore::mergeKeyValues (kvStore=..., keyVals=..., filters=...) at openr/kvstore/KvStore.cpp:246
#4  0x00000000006e3813 in openr::KvStoreDb::mergePublication (this=0x7fb232fbad28, rcvdPublication=..., senderId=...)
    at openr/kvstore/KvStore.cpp:2835
#5  0x00000000006e32d7 in openr::KvStoreDb::processThriftSuccess (this=0x7fb232fbad28, peerName=..., pub=..., timeDelta=...)
    at openr/kvstore/KvStore.cpp:1395
#6  0x000000000070487b in openr::KvStoreDb::requestThriftPeerSync()::$_19::operator()(openr::thrift::Publication&&) const (
    this=<optimized out>, pub=...) at openr/kvstore/KvStore.cpp:1338
#7  folly::futures::detail::wrapInvoke<openr::thrift::Publication, openr::KvStoreDb::requestThriftPeerSync()::$_19>(folly::Try<openr::thrift::Publication>&&, openr::KvStoreDb::requestThriftPeerSync()::$_19&&)::{lambda()#1}::operator()() const (this=<optimized out>)
    at folly/futures/Future-inl.h:99
#8  folly::futures::detail::InvokeResultWrapper<void>::wrapResult<folly::futures::detail::wrapInvoke<openr::thrift::Publication, openr::KvStoreDb::requestThriftPeerSync()::$_19>(folly::Try<openr::thrift::Publication>&&, openr::KvStoreDb::requestThriftPeerSync()::$_19&&)::{lambda()#1}>(folly::futures::detail::wrapInvoke<openr::thrift::Publication, openr::KvStoreDb::requestThriftPeerSync()::$_19>(folly::Try<openr::thrift::Publication>&&, openr::KvStoreDb::requestThriftPeerSync()::$_19&&)::{lambda()#1}) (fn=...) at folly/futures/Future-inl.h:91
#9  folly::futures::detail::wrapInvoke<openr::thrift::Publication, openr::KvStoreDb::requestThriftPeerSync()::$_19>(folly::Try<openr::thrift::Publication>&&, openr::KvStoreDb::requestThriftPeerSync()::$_19&&) (t=..., f=...) at folly/futures/Future-inl.h:109
#10 folly::Future<openr::thrift::Publication>::thenValue<openr::KvStoreDb::requestThriftPeerSync()::$_19>(openr::KvStoreDb::requestThriftPeerSync()::$_19&&) &&::{lambda(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<openr::thrift::Publication>&&)#1}::operator()(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<openr::thrift::Publication>&&) (this=<optimized out>, t=...)
    at folly/futures/Future-inl.h:1033
#11 folly::futures::detail::CoreCallbackState<folly::Unit, folly::Future<openr::thrift::Publication>::thenValue<openr::KvStoreDb::requestThriftPeerSync()::$_19>(openr::KvStoreDb::requestThriftPeerSync()::$_19&&) &&::{lambda(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<openr::thrift::Publication>&&)#1}>::invoke<folly::Executor::KeepAlive<folly::Executor>, folly::Try<openr::thrift::Publication> >(folly::Executor::KeepAlive<folly::Executor>&&, folly::Try<openr::thrift::Publication>&&) (this=<optimized out>, args=..., args=...)
    at folly/futures/Future-inl.h:145
...
#19 0x00000000015d00ee in folly::detail::function::FunctionTraits<void ()>::operator()() (this=0x7fb22c63b1f0) at folly/Function.h:416
#20 folly::EventBase::FunctionLoopCallback::runLoopCallback (this=0x7fb22c63b1c0) at folly/io/async/EventBase.h:188
#21 folly::EventBase::runLoopCallbacks (this=<optimized out>) at folly/io/async/EventBase.cpp:703
#22 folly::EventBase::loopBody (this=0x7fb23ecb9410, flags=1, ignoreKeepAlive=false) at folly/io/async/EventBase.cpp:402
#23 0x00000000015cdc60 in folly::EventBase::loopOnce (this=0x7fb23ecb9410, flags=0) at folly/io/async/EventBase.cpp:330
#24 folly::EventBase::~EventBase (this=0x7fb23ecb9410, vtt=<optimized out>) at folly/io/async/EventBase.cpp:211
#25 0x00000000006f284e in openr::KvStore::~KvStore (this=0x7fb23ecb9400) at openr/kvstore/KvStore.h:532
```

To fix this, we introduced map to hold every individual future from thrift client. However, this occasionally will make KvStore destruction stuck when waiting for all futures to be fulfilled.

We should NOT track every individual future, which is NOT necessary at all.

From the crash trace, clearly, we are doing `mergePublications()` when invoking `processThriftPublication()` before checking if the peer is valid or NOT.

Fix:
Ignore the rest logic of callback `processThriftPublication()` if peerName is NOT valid.

Reviewed By: saifhhasan

Differential Revision: D24262710

fbshipit-source-id: fa69aaa5c6e43cfc861de7431b9c1e26195684a0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jstrizich @jeroen-dhollander