FAQ
Yes, you do. The batch processing
- is still a fundamental and mission-critical component
- has significant advantages to perform repetitive logic
- remains a viable and strategic option for bulk processing
- needs to be run at any time during the day, in parallel with the OLTP window
- balanced blend of Batch and Online processing
(quoted from Java Batch for Cost Optimized Efficiency slides in SlideShare)
The following picture will explain all logical layer which should be a part of batch execution environment:
And now, let's see what is really covered by JEM, the BEE:
JEM the BEE is a environment where batches can be executed, in a secured way. In few words, we can think about it as an application server for batches
Definitely no. Usually a scheduler submit jobs when a date and time or an event occur. An event could also be the end of another job. But the scheduler shouldn't take care about the runtime environment. JEM is the runtime environment for batches and it is able to manage the submitted jobs by a scheduler. A native integration is already provided with Quartz.
Because on open system environments, batch processing is coming but there is
- Poor culture
- Not enough standardized
- Lack of application standards (some concepts are not still well-knows)
- Tough management on huge distributed environments
- Centralized management is often missing
- Access to the machines, for trouble shooting, could be complex
- Reference Models are still poor
- Few products and guide lines
Here are 2 possible use cases:
-
Batch modernization
* Increase development speed & time to market
- Traditional batch programming languages are not adequate for new requirements * Decrease of maintenance & development costs
- Leveraging on new programming languages * Manage risks & costs
- Necessary skills to maintain and use the current technology are no longer available * Modernize software development
- SOA integration
- Cloud application awareness
-
Mainframe offloading
* Optimization MIPS usage
- Optimize batch processes, run 24x7 to help with the strategy to reduce batch development, operating and runtime costs * Many implemented MF entities to help on open environments
- JCL implemention
- GDGs and datasets concepts
- Resource serialization engine
- Swarm implementation
JCL (Job control language) is a language which describes a job and its steps (mandatory parts of a job). Inside it, you can configure your job and all steps giving to business logic programs all necessary resources and configuration to be executed correctly. Currently JEM is able to manage 2 job control languages, Apache ANT and VMWare Spring Batch.
JEM is written to be run:
- on any OS
- in cluster mode (NO master/slave pattern)
- in elastic environment, adding and removing nodes without any configuration
- using global file systems to share data among nodes (access to local storage is not allowed)
- using all infrastructure resources (i.e. files) by relative path or by common name(equals on all nodes and OS)
No, JEM is able to use and manage jobs which needs other frameworks and platform. Currently JEM is integrated with JPPF, for parallel computing, adding some features like the capabilities to chunks datasets passing to runnable code for JPPF.
About bigdata, JEM is already able to use Apache HDFS as global file system and then to use the same data created by Apache Hadoop applications. Coming soon JEM is able to perform Apache Hadoop jobs starting from a JCL. With this features, you can integrate traditional and bidata jobs on the same environment.
Currently JEM has been tested with:
- FreeNas
- Apache HDFS by NFS gateway
- GlusterFS