Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some tests in a Massively Parallel Forecasting Architecture #115

Closed
antoinecarme opened this issue Oct 21, 2019 · 3 comments
Closed

Comments

@antoinecarme
Copy link
Owner

PyAF uses a parallel training process with 4 sub-processes by default , which is OK for a standard PC with about 8 cores. the number of these sub-processes is configurable.

self.mNbCores = 8;

Need to see what happens and what can be improved when one has hunderds of Cores.

The Xeon-Phi Architecture, recently made EOL by Intel, is a good candidate for these tests. Each Xeon-Phi processor has at least 64 Xeon-like cores or 256 concurrent threads.

@antoinecarme
Copy link
Owner Author

setup a xeon-phi debian machine for the tests

New repository for debian-related config data :

https://github.com/antoinecarme/xeon-phi-data

@antoinecarme
Copy link
Owner Author

Hierarchical modeling is now parallelized differently. The number of cores needed is the number of nodes in the hierarchy, All individual node models are trained in parallel.

The same for forecasting. All individual node models are forecast in parallel.

@antoinecarme
Copy link
Owner Author

Closing.

Will be officially available in release 2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant