 |
|
Workload Automation in Distributed Networks Batch Processing in the Mainframe World The history of batch processing begins with the birth of the mainframe computer. Before online systems became ubiquitous, it was redundant to speak of “batch” processing. Manual submission of work to the computer in batches, representing ad hoc user requests or operator-initiated work schedules was simply the way data center operations were done. In order to save labor costs, reduce costly errors and re-runs and improve data center efficiency, the efficient automation of batch processing was key priority in data center operations. Over 40 years of mainframe evolution has resulted in job scheduling becoming a very structured and robust discipline. Batch Processing in the Distributed World Midrange and server computing platforms were originally developed to implement real-time, online processing. This segment of the computer industry has never existed in a time where batch processing was the only processing method available. As a result, midrange systems are often thought of as existing completely in a real-time environment. Native tools for automation and job management are very limited in their utility and flexibility. UNIX and Linux based systems include the simple time-based job submission tools ‘cron’ and ‘at.’ These programs are universally available, and are the only native tools available to submit work to a system without hands-on operator involvement. It is easy to see why the idea of “batch” processing in a UNIX world has become so tightly associated with time-based scheduling, and thus with cron. In a world without networks and a world where jobs always ran without errors, cron would be a pretty good operational tool. An administrator can write a script to sequentially perform the tasks to be done on a daily, weekly and monthly basis, and create crontab entries to execute those scripts. In fact, this is how batch work in the majority of today’s systems is managed. But of course, we don’t live in such a simplistic IT world. The real world has networks of hundreds or thousands of servers, doing real-time, scheduled and ad hoc processing. Real world operations exceeded the capabilities of cron and at a long time ago. Real-World Workload Automation “Batch” processing originated in single-user, single-threaded, single-system processing environments. And while today’s environment is multi-user, multi-threaded and networked, the reality is that batch processing is just as critical to efficient, secure and reliable operations as it is to mainframe users. The demands of modern distributed systems, supporting e-commerce, Web-enabled applications and high-availability systems has forced the evolution of batch processing into what we now call “workload automation systems” (WAS). Workload Automation Systems are available now that provide all of the time-based scheduling functions of native tools, but incorporate features to provide centralized, efficient management of today’s multi-vendor, multi-site networks. Some of these tools are components of multi-function management application suites, while others, like Xi-Batch™ from Taricon Technologies, are point solutions, focused on the unique requirements of workload automation. Features of Workload Automation Solutions (Why You May Need One) Job Dependency Chaining Some jobs need to run at a specific time (like tonight at 11:00), or on a specific repetition schedule (like the last business day of every month). More often, though, a job doesn’t depend on the clock, but on what else is going on in the network. For example, your database update in Chicago can’t run until the transaction files from Denver have been received. Or the end-of-day report jobs at headquarters have to wait until all of the warehouses have finished their individual end-of-day processing. A Workload Automation System can manage multiple chaining dependencies across the network, even across different UNIX and Linux variants. Load Control If your systems bog down when dozens of users submit their jobs as they head out the door for lunch or at the end of the day, you know what it’s like to operate without load control. A Workload Automation System lets you prioritize and manage ad hoc processing without impacting your production applications. Consolidated Monitoring In any computing environment, it is important for operators to be aware of what jobs are running, and what additional work is waiting to be activated. Many Workload Automation Systems provide a consolidated view of all jobs on all hosts in the network. The best system allow you to modify and customize views so that jobs can be easily managed as needed at the individual or department level as well as by a central operator. Systems are available that offer character and graphical operator interfaces, as well as Windows and Web-browser based management tools. Remote Job Submission A key benefit of Workload Automation Systems is the ability to submit jobs on one system for execution on remote systems. This allows automatic allocation of available computing resources for peak processing demands and maintains central control and monitoring of jobs running on the entire network. The Bottom Line As server networks expand, it becomes nearly impossible for a single operations staff to effectively manage complex workflow requirements without workload automation. This results in increased expenses in two ways. First, additional operators must be hired to keep up with the workload. Secondly, the lack of effective networked workflow management tools prevents full utilization of existing networked resources. The inability to effectively manage existing systems is often manifested as a lack of computing power, leading to purchase of additional servers. It’s easy to see how this situation can spiral out of control, with increasing numbers of servers and users leading to less effective server utilization, which in turn can lead to higher investments in additional servers. Breaking the Cycle Networked workload automation tools allow centralized monitoring, management and control of networked server resources, regardless of geographic location, operating system or time zone. These products provide all of the time-based scheduling functions of traditional UNIX and Linux tools, but add features necessary for job chaining and load management in server networks. Workload automation lowers IT costs through
Taricon Technologies is the North American distributor of Xi-Batch, an enterprise-class networked workload automation system supported on all major UNIX and Linux platforms. Xi-Batch also provides agent support for Windows NT/2000/XP servers. |
