How does odi work

Repositories are the central component of Oracle Data Integrator architecture. Objects developed or configured through the user interfaces are stored in the repositories.

There are two types of repositories in ODI: 1. Master Repository and 2. Work Repository. A work repository containing only the execution information is known as an Execution Repository, usually is used in a production environment. It is used for designing ODI metadata and mapping objects. Some of the metadata components defined in a designer are as follows:. Operator In the operator navigator, you can monitor the execution of interfaces, packages, scenarios or load plans. The step by step session monitoring helps in performing debugging as well.

Topology Manager The topology manager is used to describe the logical and physical architecture of the information system. The topology manager reads and writes only to the master repository as it maintains the technologies, data servers, schemas, contexts and other related information for each of the physical environments.

This enables ODI to execute the same integration interfaces across different physical environments. Users and Profiles can be created here and privileges can be assigned to these users or profiles.

The security manager metadata that is defined will be stored in the master repository. The agents retrieve the code from the repository and orchestrate the execution of the code on the data servers.

It also retrieves the return codes and messages for the execution. Simple CDC, as the name indicates, is a simple implementation of the infrastructure described above. This infrastructure works fine if you have:. You can immediately see that this implementation works for basic cases, but it is very limited when you want to expand your infrastructure. Parent records must be processed first, or child records cannot be inserted they would be referencing invalid foreign keys.

ODI needs to mark the records that are about to be processed This is the logical lock mentioned earlier , and then process them. But as we are processing the parent records, changes to additional parent and children can be written to the CDC tables.

The challenge is that by the time we lock the children records in order to process them, the parent records for the last arrived changes have not been processed yet. Figure 7 below illustrates this: if ODI starts processing the changes in the Orders table at , and then starts processing the changes in the Order Lines table at , the parent record for order lines 4 and 5 is missing in the target environment: order 3 had not arrived yet when the O rders changes were processed.

When you define the parameters for consistent set CDC, you have to define the parent-child relationship between the tables. To do so, you have to edit the Model that contains these tables and select the Journalized Tables tab. You can either use the Reorganize button to have ODI compute the dependencies for you based on the foreign keys available in the model, or you can manually set the order.

Parent tables should be at the top, children tables the ones referencing the parents should be at the bottom. In Figure 8 we see a Diagram that was created under the model that hosts the journalized tables to represent the relationships between the tables.

To reproduce this, create a Diagram under your Model , then drag and drop the selected tables in that diagram: the foreign keys will automatically be represented as arrows by the ODI Studio.

From then on it is safe to process the parent records, followed by the children records knowing that none of these children are missing their references. This next iteration can be anywhere from a few seconds later to hours later, depending on latency requirements. When the records of a set parents and children records are about to be processed, children records and parent records are logically locked. The KMs do the following operations:. These operations are performed in the packages before processing the interfaces where CDC data is processed as shown in Figure To add these operations to a package, drag and drop the Model that contains the CDC set, then set the Type to Journalizing Model and select the appropriate options in the consumption section, as shown in Figure You can drag and drop the model twice to separate the two operations.

Note that in this case, the Extend Window operation must be done before the Lock Subscriber operation. Simple CDC requires only 2 tables and 2 views. The views are used to either see the changes from the ODI Studio, or to consume the changes in the mappings. The subscribers table is created in the Work Schema of the Default Schema for the data server. To identify the Default Schema , look under the data server definition in the physical architecture of the Topology Navigator: the Default Schema is marked with a checkmark.

If you edit the schema, the Default checkbox is selected. As such, there will be a single, shared subscribers table for all the schemas on that server. These are created in the Work Schema associated to the Physical Schema where the source table is located. When the IKMs lock the records, the value is changed to 1.

In that case, even though the last event in the source system is an update, the operation that is needed on the target side is an insert with the latest values found in the source system. This shows that ODI does not replicate the transactions; it does an integration of the data as they are at the time the integration process runs. Oracle GoldenGate replicates the transactions as they occur on the source system. Again the logic here is that we want to replicate values as they are currently in the source database.

This architecture allows ODI to use the power of the target datasource to execute the transformations. In a nutshell, KMs are generic code templates that are applied to an ODI mapping and use a substitution language to input metadata from within the mapping to produce executable code at runtime. KMs can even be customized or created from scratch to suit your specific data integration needs, adding to the overall flexibility of the tool.

With KMs, you can change how your mappings are physically implemented allowing for the ultimate flexibility. The great part about customizing Oracle Data Integrator is that you can make it adapt to your data warehouse - and not the other way around. With this level of access, you can perform almost every task that can be completed within ODI Studio.

Imagine, you need to create source to staging mappings, all one-for-one column mappings. The power of the SDK is generally found when there is a need for a batch creation or modification of objects.