IBM InfoSphere DataStage

The External Source Stage is specifically designed to read data output from external programs or processes. It acts as an input source that can consume data generated by source programs running outside DataStage. External Target Stage writes data out, Complex Flat File handles flat files, and Internal File Stage isn't a standard DataStage component.

DataStage follows a clear three-level hierarchy: Projects sit at the top level, Jobs exist within projects, and Categories organize jobs within each project. This structure allows for logical grouping and management of jobs. The other options either reverse this order or place categories above projects incorrectly.

Lookup, Join, and Merge stages differ in three key aspects: (1) Memory usage - Lookup loads entire reference dataset into memory while Join/Merge process streaming data, (2) Sorting requirements - Join and Merge require sorted inputs but Lookup doesn't, and (3) Unmatched key handling - Lookup drops unmatched rows, Join drops them, but Merge can preserve them depending on settings. All three represent valid differences.

DataStage server components include the Engine (executes jobs), Metadata Repository (stores job metadata and definitions), and Package Installer (manages deployment packages). Options B lists client tools (Administrator, Designer, Director) which are applications that connect to the server but aren't server components themselves. The server runs the backend services that these tools interact with.

Runtime Column Propagation (RCP) requires two configuration steps: enabling it globally in DataStage Administrator and enabling it at individual stage levels where needed. This two-tier approach allows global control while letting specific jobs opt-in. RCP is NOT configured through table definitions in Manager or via environmental parameters at runtime - those are incorrect mechanisms.

Job Sequences can pass parameters to Activity stages through three valid methods: ExecCommand Activity (passes parameters to command scripts), UserVariables Activity (defines and passes variable values), and Routine Activity (passes parameters to called routines). Sequencer Activity controls flow but doesn't pass parameters, and Nested Condition Activity evaluates conditions without parameter passing capability.

Lookup stages consume more memory than Merge and Join because they load the entire reference dataset into RAM for fast lookup operations. Merge and Join stages process data in streaming fashion without requiring full dataset loading. The primary input link to a Lookup stage does NOT require sorting - it's Join and Merge that have sorting requirements on input links.

When shared containers are modified, all jobs that use them automatically reflect those changes - this is their core purpose as reusable components. Option A is also technically true (shared containers ARE reusable elements), but option C specifically describes the modification propagation behavior which is a key distinguishing characteristic. Options B and D are incorrect.

DataStage jobs can be executed through four valid methods: from DataStage Director UI (Job -> Run Now), from command line using dsjob command, from DataStage routines using the DsRunJob command, and through Job Sequencers. All four methods are valid execution mechanisms in DataStage, making option D correct.

Preserving partitioning minimizes data copying between parallel stage operators because when stages maintain the same partitioning scheme, data doesn't need to be repartitioned or moved between processing nodes. Combinability mode and Execution Mode control stage behavior but don't directly address data copying, while Don't combine mode isn't a standard DataStage term for minimizing data movement.

In the Transformer stage, records that fail to meet the output constraint condition are routed to the reject link. This handles data validation failures - if a record doesn't satisfy the condition specified for an output link, it gets sent to reject instead. Mismatched metadata would cause job failure, null records may be handled depending on settings, and the reject link specifically captures condition failures.

The record delimiter string specifies the character(s) that mark the end of each record in a file. This is different from the field delimiter, which separates fields within a record. The record prefix and final delimiter are not standard DataStage stage properties.

SQL (Structured Query Language) is a standard database query language, while DML (Data Manipulation Language), RUN, and MP are all specific to DataStage or its processing paradigms. SQL stands out as it's a universal standard rather than a DataStage-specific component.

The External Source Stage in DataStage is specifically designed to read data from the output of external programs or scripts. It executes external source programs and captures their stdout as input data. The External Target Stage writes to external programs, not reads from them.

DataStage uses a three-level hierarchy: Project at the top level, Jobs within projects, and Categories to organize jobs. Categories provide logical grouping within projects. Jobs cannot exist without projects, and categories are organizational structures within projects.

Lookup, Join, and Merge stages differ in multiple ways: Lookup loads reference data entirely into memory (high memory usage), Join requires sorted input on both links, and they handle unmatched rows differently (Lookup drops, Merge keeps, Join can drop or keep). All three are valid distinguishing characteristics.

DataStage server components include the Data Stage Engine (execution core), Metadata Repository (stores all metadata), and Package Installer (deploys packages). The Designer, Director, and Administrator are client tools, not server components.

Runtime Column Propagation (RCP) must be enabled in two places: globally in DataStage Administrator, and at the individual stage level where you want to use it. RCP allows propagating column definitions dynamically without explicit schema definitions. Table definition attachment and environmental parameters are not RCP requirements.

In Job Sequences, parameters can be passed to Activity stages through ExecCommand (with parameter substitution), UserVariables Activity (defines sequence variables), and Routine Activity (parameters can be job parameters). Sequencer Activity controls flow and Nested Condition handles logic - neither passes parameters.

Lookup stages load the entire reference dataset into memory for fast matching, requiring significantly more RAM than Merge or Join stages which process data in streams. The primary input link does NOT need to be sorted for Lookup - that's a requirement for Join stages.