Alternate Start Load Order
This chapter describes running an initial data load to instantiate the replication environment.
This chapter includes the following sections:
16.1 Overview of the Initial-Load Procedure
Alternate index are the additional index that are created for KSDS/ESDS datasets in addition to their primary index. An alternate index provides access to records by using more than one key. The key of alternate index can be a non-unique key, it can have duplicates. Save time accessing the websites you visit every day. Go beyond bookmarks with our advanced dashboard features. Use across all your devices and browsers. At work, at school or at home.
You can use Oracle GoldenGate to:
Perform a standalone batch load to populate database tables for migration or other purposes.
Load data into database tables as part of an initial synchronization run in preparation for change synchronization with Oracle GoldenGate.
16.1.1 Improving the Performance of an Initial Load
For all initial load methods except those performed with a database utility, you can load large databases more quickly by using parallel Oracle GoldenGate processes. To use parallel processing, take the following steps.
Follow the directions in this chapter for creating an initial-load Extract and an initial-load Replicat for each set of parallel processes that you want to use.
With the
TABLE
andMAP
parameters, specify a different set of tables for each pair of Extract-Replicat processes, or you can use theSQLPREDICATE
option ofTABLE
to partition the rows of large tables among the different Extract processes.
For all initial load methods, testing has shown that using the TCPBUFSIZE
option in the RMTHOST
parameter produced three times faster throughput than loads performed without it. Do not use this parameter if the target system is NonStop.
16.1.2 Prerequisites for Initial Load
Verify that you meet the following prerequisites for executing an initial load.
16.1.2.1 Disable DDL Processing
Before executing an initial load, disable DDL extraction and replication. DDL processing is controlled by the DDL
parameter in the Extract and Replicat parameter files.
16.1.2.2 Prepare the Target Tables
The following are suggestions that can make the load go faster and help you to avoid errors.
Data: Make certain that the target tables are empty. Otherwise, there may be duplicate-row errors or conflicts between existing rows and rows that are being loaded.
Constraints: Disable foreign-key constraints and check constraints. Foreign-key constraints can cause errors, and check constraints can slow down the loading process. Constraints can be reactivated after the load concludes successfully.
Indexes: Remove indexes from the target tables. Indexes are not necessary for inserts. They will slow down the loading process significantly. For each row that is inserted into a table, the database will update every index on that table. You can add back the indexes after the load is finished.
Note:
A primary index is required for all applications that access DB2 for z/OS target tables. You can delete all other indexes from the target tables, except for the primary index.Keys: For Oracle GoldenGate to reconcile the replicated incremental data changes with the results of the load, each target table must have a primary or unique key. If you cannot create a key through your application, use the
KEYCOLS
option of theTABLE
andMAP
parameters to specify columns as a substitute key for Oracle GoldenGate's purposes. A key helps identify which row to process. If you cannot create keys, the source database must be quiesced for the load.
16.1.2.3 Configure the Manager Process
On the source and target systems, configure and start a Manager process. One Manager can be used for the initial-load processes and the change-synchronization processes. See Chapter 3, 'Configuring Manager and Network Communications' for more information.
16.1.2.4 Create a for more information.
16.1.2.5 Create Change-synchronization Groups
To prepare for the capture and replication of transactional changes during the initial load, create online Extract and Replicat groups. You will start these groups during the load procedure. See Section 14, 'Configuring Online Change Synchronization' for more information.
Note:
If the load is performed from a quiet source database and will not be followed by continuous change synchronization, you can omit these groups.Do not start the Extract or Replicat groups until instructed to do so in the initial-load instructions. Change synchronization keeps track of transactional changes while the load is being applied, and then the target tables are reconciled with those changes.
Note:
The first time that Extract starts in a new Oracle GoldenGate configuration, any open transactions will be skipped. Only transactions that begin after Extract starts are captured.16.1.2.6 Sharing Parameters between Process Groups
Some of the parameters that you use in a change-synchronization parameter file also are required in an initial-load Extract and initial-load Replicat parameter file. You can copy those parameters from one parameter file to another, or you can store them in a central file and use the OBEY
parameter in each parameter file to retrieve them. Alternatively, you can create an Oracle GoldenGate macro for the shared parameters and then call the macro from each parameter file with the MACRO
parameter.
See Chapter 4, 'Getting Started with the Oracle GoldenGate Process Interfaces' for more information about using OBEY
and using macros.
16.2 Loading Data with a Database Utility
To use a database copy utility to establish the target data, you start a change-synchronization Extract group to extract ongoing data changes while the database utility makes and applies a static copy of the data. When the copy is finished, you start the change-synchronization Replicat group to re-synchronize rows that were changed while the copy was being applied. From that point forward, both Extract and Replicat continue running to maintain data synchronization. This method does not involve any special initial-load Extract or Replicat processes.
Description of the illustration initsync_copyutil.jpg
Note:
The objects and data types being loaded in this method must be supported by Oracle GoldenGate for your database and also by the database utility that is being used. For items that are supported for your database, see the Oracle GoldenGate installation and configuration documentation for that database. For items that are supported by the database utility, see the database vendor's documentation.Make certain that you have addressed the requirements in Section 16.1.2, 'Prerequisites for Initial Load.'
On the source and target systems, run GGSCI and start the Manager process.
Note:
In a Windows cluster, start the Manager resource from the Cluster Administrator.On the source system, start change extraction.
Where:
group
is the name of the Extract group.(Oracle, if replicating sequences) Issue the
DBLOGIN
command as the user who hasEXECUTE
privilege onupdate.Sequence
.(Oracle, if replicating sequences) Issue the following command to update each source sequence and generate redo. From the redo, Replicat performs initial synchronization of the sequences on the target. You can use an asterisk wildcard for any or all characters in the name of a sequence (but not the owner).
On the source system, start making the copy.
Wait until the copy is finished and record the time of completion.
View the Replicat parameter file to make certain that the
HANDLECOLLISIONS
parameter is listed. If not, add the parameter to the file.Caution:
Do not use theVIEW PARAMS
orEDIT PARAMS
command to view or edit an existing parameter file that is in a character set other than that of the local operating system (such as one where theCHARSET
option was used to specify a different character set). View the parameter file from outside GGSCI if this is the case; otherwise, the contents may become corrupted.On the target system, start change replication.
Where:
group
is the name of the Replicat group.On the target system, issue the following command to verify the status of change replication.
Continue to issue the
INFO REPLICAT
command until you have verified that change replication has posted all of the change data that was generated during the initial load. Reference the time of completion that you recorded. For example, if the copy stopped at 12:05, make sure change replication has posted data up to that point.On the target system, issue the following command to turn off the
HANDLECOLLISIONS
parameter and disable the initial-load error handling.On the target system, edit the Replicat parameter file to remove the
HANDLECOLLISIONS
parameter. This preventsHANDLECOLLISIONS
from being enabled again the next time Replicat starts.Caution:
Do not use theVIEW PARAMS
orEDIT PARAMS
command to view or edit an existing parameter file that is in a character set other than that of the local operating system (such as one where theCHARSET
option was used to specify a different character set). View the parameter file from outside GGSCI if this is the case; otherwise, the contents may become corrupted.Save and close the parameter file.
From this point forward, Oracle GoldenGate continues to synchronize data changes.
16.3 Loading data with Oracle Data Pump
This method uses the Oracle Data Pump utility to establish the target data. After you apply the copy to the target, you record the SCN at which the copy stopped. You then start Extract, data pumps, and Replicat at the specified SCN. Transactions that were included in the copy are skipped to avoid collisions from integrity violations. From the process start point, Oracle GoldenGate maintains data synchronization. No initial-load Oracle GoldenGate processes are required for this method.
To perform instantiation with Oracle Data Pump, see My Oracle Support document 1276058.1. To obtain this document, do the following:
Go to http://support.oracle.com.
Under Sign In, select your language and then log in with your Oracle Single Sign-On (SSO).
On the Dashboard, expand the Knowledge Base heading.
Under Enter Search Terms, paste or type the document ID of
1276058.1
and then click Search.In the search results, select Oracle GoldenGate Best Practices: Instantiation from an Oracle Source Database [Article ID 1276058.1].
Click the link under Attachments to open the article.
16.4 Loading Data from File to Replicat
To use Replicat to establish the target data, you use an initial-load Extract to extract source records from the source tables and write them to an extract file in canonical format. From the file, an initial-load Replicat loads the data using the database interface. During the load, the change-synchronization groups extract and replicate incremental changes, which are then reconciled with the results of the load.
During the load, the records are applied to the target database one record at a time, so this method is considerably slower than any of the other initial load methods. This method permits data transformation to be done on either the source or target system.
Description of the illustration initsyncreplicat.jpg
To load data from file to Replicat
Make certain that you have addressed the requirements in Section 16.1.2, 'Prerequisites for Initial Load.'
On the source and target systems, run GGSCI and start Manager.
Note:
In a Windows cluster, start the Manager resource from the Cluster Administrator.On the source system, issue the following command to create an initial-load Extract parameter file.
Enter the parameters listed in Table 16-1 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Extract parameter file for loading data from file to Replicat.
Table 16-1 Initial-load Extract Parameters
Parameter Description Designates Extract as an initial load process extracting records directly from the source tables.
Specifies database connection information.
SOURCEDB
specifies the source data source name (DSN) or SQL/MX catalog. See Reference for Oracle GoldenGate for Windows and UNIX for more information.USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.Specifies the target system, the port where Manager is running, and optional encryption of data across TCP/IP.
Encrypts the data in the remote file. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.
path
is the relative or fully qualified name of the file.MAXFILES
creates a series of files that are aged as needed. Use if the file could exceed the operating system's file size limitations.MEGABYTES
designates the size of each file.
Specifies the extract file to which the load data will be written. Oracle GoldenGate creates this file during the load. Checkpoints are not maintained with
RMTFILE
.Note that the size of an extract file cannot exceed 2GB.
Specifies the fully qualified name of an object or a fully qualified wildcarded specification for multiple objects. If the database is an Oracle multitenant container database or a SQL/MX database, the object name must include the name of the container or catalog unless
SOURCECATALOG
is used.See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.
CATALOGEXCLUDE
SCHEMAEXCLUDE
TABLEEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific objects from a wildcard specification in the associated
TABLE
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional Extract parameters listed in the Reference for Oracle GoldenGate for Windows and UNIX.
Save and close the parameter file.
On the target system, issue the following command to create an initial-load Replicat parameter file.
Enter the parameters listed in Table 16-2 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Replicat parameter file for loading data from file to Replicat.
Table 16-2 Initial-load Replicat parameters
Parameter Description Implements the initial-load Replicat as a one-time run that does not use checkpoints.
Directs the initial-load Replicat to terminate when the load is finished.
Specifies database connection information.
TARGETDB
specifies the target datasource name (DSN) or SQL/MX catalog. See Reference for Oracle GoldenGate for Windows and UNIX for more information.USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.path
is the relative or fully qualified name of the file.
Specifies the input extract file specified with the Extract parameter
RMTFILE
.Use
SOURCEDEFS
if the source and target tables have different definitions. Specify the relative or fully qualified name of the source-definitions file generated by DEFGEN.Use
ASSUMETARGETDEFS
if the source and target tables have the same definitions.
Specifies how to interpret data definitions.
For more information about data definitions files, see Chapter 13, 'Associating Replicated Data with Metadata.'
SOURCECATALOG
Specifies a default source Oracle container or source SQL/MX catalog for subsequent
MAP
statements. Enables the use of two-part names (schema.object
) where three-part names otherwise would be required for those databases. You can use multiple instances of this parameter to specify different default containers or catalogs for different sets ofMAP
parameters.Specifies a relationship between a source object or objects and a target object or objects.
MAP
specifies the source object, andTARGET
specifies the target object.For the source object, specify the fully qualified name of the object or a fully qualified wildcarded specification for multiple objects. For an Oracle multitenant container database or a SQL/MX database, the source object name must include the name of the container or catalog unless
SOURCECATALOG
is used.For the target object, specify only the
owner.object
components of the name, regardless of the database. Replicat can only connect to one Oracle container or SQL/MX catalog. Use a separate Replicat process for each container or catalog to which you want to load data.See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.
The
DEF
option specifies a definitions template. For more information about data definitions, see Section 13, 'Associating Replicated Data with Metadata'. See Reference for Oracle GoldenGate for Windows and UNIX for more information and options for theMAP
parameter.CATALOGEXCLUDE
SCHEMAEXCLUDE
MAPEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific source objects from a wildcard specification in the associated
MAP
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional Replicat parameters listed in the Reference for Oracle GoldenGate for Windows and UNIX.
Save and close the file.
View the Replicat parameter file to make certain that the
HANDLECOLLISIONS
parameter is listed. If not, add the parameter to the file.On the source system, start change extraction.
(Oracle, if replicating sequences) Issue the
DBLOGIN
command as the user who hasEXECUTE
privilege onupdate.Sequence
.(Oracle, if replicating sequences) Issue the following command to update each source sequence and generate redo. From the redo, Replicat performs initial synchronization of the sequences on the target. You can use an asterisk wildcard for any or all characters in the name of a sequence (but not the owner).
From the directory where Oracle GoldenGate is installed on the source system, start the initial-load Extract.
UNIX and Linux:
Windows:
Where:
initial-load_Extract
is the name of the initial-load Extract that you used when creating the parameter file, andpath
is the relative or fully qualified name of the Extract report file.Verify the progress and results of the initial extraction by viewing the Extract report file using the operating system's standard method for viewing files.
Wait until the initial extraction is finished.
On the target system, start the initial-load Replicat.
UNIX and Linux:
Windows:
Where:
initial-load_Replicat
is the name of the initial-load Replicat that you used when creating the parameter file, andpath
is the relative or fully qualified name of the Replicat report file.When the initial-load Replicat is finished running, verify the results by viewing the Replicat report file using the operating system's standard method for viewing files.
On the target system, start change replication.
On the target system, issue the following command to verify the status of change replication.
Continue to issue the
INFO REPLICAT
command until you have verified that Replicat posted all of the change data that was generated during the initial load. For example, if the initial-load Extract stopped at 12:05, make sure Replicat posted data up to that point.On the target system, issue the following command to turn off the
HANDLECOLLISIONS
parameter and disable the initial-load error handling.On the target system, edit the Replicat parameter file to remove the
HANDLECOLLISIONS
parameter. This preventsHANDLECOLLISIONS
from being enabled again the next time Replicat starts.Caution:
Do not use theVIEW PARAMS
orEDIT PARAMS
command to view or edit an existing parameter file that is in a character set other than that of the local operating system (such as one where theCHARSET
option was used to specify a different character set). View the parameter file from outside GGSCI if this is the case; otherwise, the contents may become corrupted.Save and close the parameter file.
From this point forward, Oracle GoldenGate continues to synchronize data changes.
16.5 Loading Data from File to Database Utility
To use a database bulk-load utility, you use an initial-load Extract to extract source records from the source tables and write them to an extract file in external ASCII format. The file can be read by Oracle's SQL*Loader, Microsoft's BCP, DTS, or SQL Server Integration Services (SSIS) utility, or IBM's Load Utility (LOADUTIL
).
Note:
The objects and data types being loaded in this method must be supported by Oracle GoldenGate for your database and also by the database utility that is being used. For items that are supported for your database, see the Oracle GoldenGate installation and configuration documentation for that database. For items that are supported by the database utility, see the database vendor's documentation.During the load, the change-synchronization groups extract and replicate incremental changes, which are then reconciled with the results of the load. As part of the load procedure, Oracle GoldenGate uses the initial-load Replicat to create run and control files required by the database utility.
Any data transformation must be performed by the initial-load Extract on the source system because the control files are generated dynamically and cannot be pre-configured with transformation rules.
Description of the illustration initsyncbulk.jpg
To load data from file to database utility
Make certain to satisfy 'Prerequisites for Initial Load'.
On the source and target systems, run GGSCI and start Manager.
On the source system, issue the following command to create an initial-load Extract parameter file.
Enter the parameters listed in Table 16-3 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Extract parameter file for loading data from file to database utility.
Table 16-3 Initial-load Extract Parameters
Parameter Description Designates Extract as an initial load process that extracts records directly from the source tables.
Specifies database connection information.
SOURCEDB
specifies the source datasource name (DSN). See Reference for Oracle GoldenGate for Windows and UNIX for more information.USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.-E
converts ASCII to EBCDIC.-d
defs_file
specifies the source definitions file.
Specifies the target system, the port where Manager is running, and optional encryption of data across TCP/IP.
The
PARAMS
clause is necessary when loading with IBM's Load Utility, because Oracle GoldenGate will need to refer to the source definitions file.Encrypts the data in the remote file. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.
BCP
is used for BCP, DTS, or SSIS.SQLLOADER
is used for Oracle SQL*Loader or IBM Load Utility.
Directs output to be formatted as ASCII text rather than the default canonical format. This parameter must be listed before
RMTFILE
. For information about limitations and options, see Reference for Oracle GoldenGate for Windows and UNIX.path
is the relative or fully qualified name of the fileMAXFILES
creates a series of files that are aged as needed. Use if the file could exceed the operating system's file size limitations.MEGABYTES
designates the size of each file, up to 2MB.
Specifies the extract file to which the load data will be written. Oracle GoldenGate creates this file during the load. Checkpoints are not maintained with
RMTFILE
.Specifies the fully qualified name of an object or a fully qualified wildcarded specification for multiple objects. If the database is an Oracle multitenant container database, the object name must include the name of the container unless
SOURCECATALOG
is used. See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.CATALOGEXCLUDE
SCHEMAEXCLUDE
TABLEEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific objects from a wildcard specification in the associated
TABLE
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional Extract parameters listed in Reference for Oracle GoldenGate for Windows and UNIX.
Save and close the parameter file.
On the target system, issue the following command to create an initial-load Replicat parameter file.
Enter the parameters listed in Table 16-4 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Replicat parameter file for loading data from file to database utility.
Table 16-4 Initial-load Replicat Parameters
Parameter Description Generates run and control files for the database utility. For instructions on using this parameter, see Reference for Oracle GoldenGate for Windows and UNIX.
Specifies database connection information.
TARGETDB
specifies the target datasource name (DSN) or Oracle container. See Reference for Oracle GoldenGate for Windows and UNIX for more information.USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.path
is the relative or fully qualified name of the fileUse
EXTTRAIL
only if you used theMAXFILES
option of theRMTFILE
parameter in the Extract parameter file.
Specifies the extract file specified with the Extract parameter
RMTFILE
.Use
SOURCEDEFS
if the source and target tables have different definitions. Specify the relative or fully qualified name of the source-definitions file generated by DEFGEN.Use
ASSUMETARGETDEFS
if the source and target tables have the same definitions.
Specifies how to interpret data definitions. For more information about data definitions files, see Chapter 13, 'Associating Replicated Data with Metadata.'
SOURCECATALOG
Specifies a default source Oracle container in a multitenant container database for subsequent
MAP
statements. Enables the use of two-part names (schema.object
) where three-part names otherwise would be required. You can use multiple instances of this parameter to specify different default containers for different sets ofMAP
parameters.Specifies a relationship between a source object or objects and a target object or objects.
MAP
specifies the source object, andTARGET
specifies the target object.For the source object, specify the fully qualified name of the object or a fully qualified wildcarded specification for multiple objects. For an Oracle multitenant container database, the source object name must include the name of the container unless
SOURCECATALOG
is used.For the target object, specify only the
owner.object
components of the name, regardless of the database. Replicat can only connect to one Oracle container. Use a separate Replicat process for each container to which you want to load data.See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.
The
DEF
option specifies a definitions template. For more information about data definitions, see Section 13, 'Associating Replicated Data with Metadata'.See Reference for Oracle GoldenGate for Windows and UNIX for more information and options for theMAP
parameter.CATALOGEXCLUDE
SCHEMAEXCLUDE
MAPEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific source objects from a wildcard specification in the associated
MAP
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional Replicat parameters listed in Reference for Oracle GoldenGate for Windows and UNIX.
Save and close the parameter file.
View the Replicat parameter file to make certain that the
HANDLECOLLISIONS
parameter is listed. If not, add the parameter to the file.On the source system, start change extraction.
(Oracle, if replicating sequences) Issue the
DBLOGIN
command as the user who hasEXECUTE
privilege onupdate.Sequence
.(Oracle, if replicating sequences) Issue the following command to update each source sequence and generate redo. From the redo, Replicat performs initial synchronization of the sequences on the target. You can use an asterisk wildcard for any or all characters in the name of a sequence (but not the owner).
From the directory where Oracle GoldenGate is installed on the source system, start the initial-load Extract.
UNIX and Linux:
Windows:
Where:
initial-load_Extract
is the name of the initial-load Extract that you used when creating the parameter file, andpath
is the relative or fully qualified name of the Extract report file.Verify the progress and results of the initial extraction by viewing the Extract report file using the operating system's standard method for viewing files.
Wait until the initial extraction is finished.
On the target system, start the initial-load Replicat.
UNIX and Linux:
Windows:
Where:
initial-load_Replicat
is the name of the initial-load Replicat that you used when creating the parameter file, andpath
is the relative or fully qualified name of the Replicat report file.When the initial-load Replicat is finished running, verify the results by viewing the Replicat report file using the operating system's standard method for viewing files.
Using the ASCII-formatted extract files and the run and control files created by the initial-load Replicat, load the data with the database utility.
Wait until the load into the target tables is complete.
On the target system, start change replication.
On the target system, issue the following command to verify the status of change replication.
Continue to issue the
INFO REPLICAT
command until you have verified that Replicat posted all of the change data that was generated during the initial load. For example, if the initial-load Extract stopped at 12:05, make sure Replicat posted data up to that point.On the target system, issue the following command to turn off the
HANDLECOLLISIONS
parameter and disable the initial-load error handling.On the target system, edit the Replicat parameter file to remove the
HANDLECOLLISIONS
parameter. This preventsHANDLECOLLISIONS
from being enabled again the next time Replicat starts.Caution:
Do not use theVIEW PARAMS
orEDIT PARAMS
command to view or edit an existing parameter file that is in a character set other than that of the local operating system (such as one where theCHARSET
option was used to specify a different character set). View the parameter file from outside GGSCI if this is the case; otherwise, the contents may become corrupted.Save and close the parameter file.
From this point forward, Oracle GoldenGate continues to synchronize data changes.
16.6 Loading Data with an Oracle GoldenGate Direct Load
To use an Oracle GoldenGate direct load, you run an Oracle GoldenGate initial-load Extract to extract the source records and send them directly to an initial-load Replicat task. A task is started dynamically by the Manager process and does not require the use of a Collector process or file. The initial-load Replicat task delivers the load in large blocks to the target database. Transformation and mapping can be done by Extract, Replicat, or both. During the load, the change-synchronization groups extract and replicate incremental changes, which are then reconciled with the results of the load.
Description of the illustration initsync_dirload.jpg
To control which port is used by Replicat, and to speed up the search and bind process, use the DYNAMICPORTLIST
parameter in the Manager parameter file. Manager passes the list of port numbers that are specified with this parameter to the Replicat task process. Replicat first searches for a port from this list, and only if no ports are available from the list does Replicat begin scanning in ascending order from the default Manager port number until it finds an available port.
This method supports standard character, numeric, and datetime data types, as well as CLOB
, NCLOB
, BLOB
, LONG
, XML
, and user-defined datatypes (UDT
) embedded with the following attributes: CHAR
, NCHAR
, VARCHAR
, NVARCHAR
, RAW
, NUMBER
, DATE
, FLOAT
, TIMESTAMP
, CLOB
, BLOB
, XML
, and UDT
. Character sets are converted between source and target where applicable.
This method supports Oracle internal tables, but does not convert between the source and target character sets during the load.
To load data with an Oracle GoldenGate direct load
Make certain to satisfy 'Prerequisites for Initial Load'.
On the source and target systems, run GGSCI and start Manager.
Note:
In a Windows cluster, start the Manager resource from the Cluster Administrator.On the source, issue the following command to create the initial-load Extract.
Where:
initial-load_Extract
is the name of the initial-load Extract, up to eight characters.SOURCEISTABLE
designates Extract as an initial-load process that reads complete records directly from the source tables. Do not use any of the otherADD EXTRACT
service options or datasource arguments.
On the source system, issue the following command to create an initial-load Extract parameter file.
Enter the parameters listed in Table 16-5 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Extract parameter file for an Oracle GoldenGate direct load.
Table 16-5 Initial-load Extract Parameters for Oracle GoldenGate Direct Load
Parameter Description Specifies the initial-load Extract.
Specifies database connection information.
SOURCEDB
specifies the source datasource name (DSN) or SQL/MX catalog. See Reference for Oracle GoldenGate for Windows and UNIX for more information.USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.Specifies the target system, the port where Manager is running, and optional encryption of data across TCP/IP.
initial-load_Replicat
is the name of the initial-load Replicat group
Directs Manager on the target system to dynamically start the initial-load Replicat as a one-time task.
Specifies the fully qualified name of an object or a fully qualified wildcarded specification for multiple objects. If the database is an Oracle multitenant database or a SQL/MX database, the object name must include the name of the container or catalog unless
SOURCECATALOG
is used. See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.CATALOGEXCLUDE
SCHEMAEXCLUDE
TABLEEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific objects from a wildcard specification in the associated
TABLE
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional Extract parameters listed in Reference for Oracle GoldenGate for Windows and UNIX.
Save and close the file.
On the target system, issue the following command to create the initial-load Replicat task.
Where:
initial-load_Replicat
is the name of the initial-load Replicat task.SPECIALRUN
identifies the initial-load Replicat as a one-time run, not a continuous process.
On the target system, issue the following command to create an initial-load Replicat parameter file.
Enter the parameters listed in Table 16-6 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Replicat parameter file for an Oracle GoldenGate direct load.
Table 16-6 Initial-load Replicat parameters for Oracle GoldenGate Direct Load
Parameter Description Specifies the initial-load Replicat task to be started by Manager. Use the name that you specified when you created the initial-load Replicat.
Specifies database connection information.
TARGETDB
specifies the target datasource name (DSN), Oracle container, or SQL/MX catalog. See Reference for Oracle GoldenGate for Windows and UNIX for more information.USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.Use
SOURCEDEFS
if the source and target tables have different definitions. Specify the source-definitions file generated by DEFGEN.Use
ASSUMETARGETDEFS
if the source and target tables have the same definitions.
Specifies how to interpret data definitions. For more information about data definitions files, see Chapter 13, 'Associating Replicated Data with Metadata.'
SOURCECATALOG
Specifies a default source Oracle container or a source SQL/MX catalog for subsequent
MAP
statements. Enables the use of two-part names (schema.object
) where three-part names otherwise would be required for those databases. You can use multiple instances of this parameter to specify different default containers or catalogs for different sets ofMAP
parameters.Specifies a relationship between a source object or objects and a target object or objects.
MAP
specifies the source object, andTARGET
specifies the target object.For the source object, specify the fully qualified name of the object or a fully qualified wildcarded specification for multiple objects. For an Oracle multitenant container database or a SQL/MX database, the source object name must include the name of the container or catalog unless
SOURCECATALOG
is used.For the target object, specify only the
owner.object
components of the name, regardless of the database. Replicat can only connect to one Oracle container or SQL/MX catalog. Use a separate Replicat process for each container or catalog to which you want to load data.See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.
The
DEF
option specifies a definitions template. For more information about data definitions, see Section 13, 'Associating Replicated Data with Metadata'.See Reference for Oracle GoldenGate for Windows and UNIX for more information and options for theMAP
parameter.CATALOGEXCLUDE
SCHEMAEXCLUDE
MAPEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific source objects from a wildcard specification in the associated
MAP
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional Replicat parameters listed in Reference for Oracle GoldenGate for Windows and UNIX.
Save and close the parameter file.
On the source system, start change extraction.
View the Replicat parameter file to make certain that the
HANDLECOLLISIONS
parameter is listed. If not, add the parameter to the file.(Oracle, if replicating sequences) Issue the
DBLOGIN
command as the user who hasEXECUTE
privilege onupdate.Sequence
.(Oracle, if replicating sequences) Issue the following command to update each source sequence and generate redo. From the redo, Replicat performs initial synchronization of the sequences on the target. You can use an asterisk wildcard for any or all characters in the name of a sequence (but not the owner).
On the source system, start the initial-load Extract.
Note:
Do not start the initial-load Replicat. The Manager process starts it automatically and terminates it when the load is finished.On the target system, issue the following command to find out if the load is finished. Wait until the load is finished before going to the next step.
On the target system, start change replication.
On the target system, issue the following command to verify the status of change replication.
Continue to issue the
INFO REPLICAT
command until you have verified that Replicat posted all of the change data that was generated during the initial load. For example, if the initial-load Extract stopped at 12:05, make sure Replicat posted data up to that point.On the target system, issue the following command to turn off the
HANDLECOLLISIONS
parameter and disable the initial-load error handling.On the target system, edit the Replicat parameter file to remove the
HANDLECOLLISIONS
parameter. This preventsHANDLECOLLISIONS
from being enabled again the next time Replicat starts.Caution:
Do not use theVIEW PARAMS
orEDIT PARAMS
command to view or edit an existing parameter file that is in a character set other than that of the local operating system (such as one where theCHARSET
option was used to specify a different character set). View the parameter file from outside GGSCI if this is the case; otherwise, the contents may become corrupted.Save and close the parameter file. From this point forward, Oracle GoldenGate continues to synchronize data changes.
16.7 Loading Data with a Direct Bulk Load to SQL*Loader
To use Oracle's SQL*Loader utility to establish the target data, you run an Oracle GoldenGate initial-load Extract to extract the source records and send them directly to an initial-load Replicat task. A task is a process that is started dynamically by the Manager process and does not require the use of a Collector process or file. The initial-load Replicat task interfaces with the API of SQL*Loader to load data as a direct-path bulk load. Data mapping and transformation can be done by either the initial-load Extract or initial-load Replicat, or both. During the load, the change-synchronization groups extract and replicate incremental changes, which are then reconciled with the results of the load.
To control which port is used by Replicat, and to speed up the search and bind process, use the DYNAMICPORTLIST
parameter in the Manager parameter file. Manager passes the list of port numbers that are specified with this parameter to the Replicat task process. Replicat first searches for a port from this list, and only if no ports are available from the list does Replicat begin scanning in ascending order from the default Manager port number until it finds an available port.
This method supports standard character, numeric, and datetime data types, as well as CLOB
, NCLOB
, BLOB
, LONG
, XML
, and user-defined datatypes (UDT
) embedded with the following attributes: CHAR
, NCHAR
, VARCHAR
, NVARCHAR
, RAW
, NUMBER
, DATE
, FLOAT
, TIMESTAMP
, CLOB
, BLOB
, XML
, and UDT
. VARRAYS
are not supported. Character sets are converted between source and target where applicable.
This method supports Oracle internal tables, but does not convert between the source and target character sets during the load.
Description of the illustration initsync_dirbulk.jpg
To load data with a direct bulk load to SQL*Loader
Make certain that you have addressed the requirements in 'Prerequisites for Initial Load'.
Grant
LOCK ANY TABLE
to the Replicat database user on the target Oracle database.On the source and target systems, run GGSCI and start Manager.
On the source system, issue the following command to create the initial-load Extract.
Where:
initial-load_Extract
is the name of the initial-load Extract, up to eight characters.SOURCEISTABLE
designates Extract as an initial-load process that reads complete records directly from the source tables. Do not use any of the otherADD EXTRACT
service options or datasource arguments.
On the source system, issue the following command to create an initial-load Extract parameter file.
Enter the parameters listed in Table 16-7 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Extract parameter file for a direct bulk load to SQL*Loader.
Table 16-7 Initial-load Extract Parameters for a Direct Bulk Load to SQL*Loader
Parameter Description Specifies the initial-load Extract.
Specifies database connection information.
USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.Specifies the target system, the port where Manager is running, and optional encryption of data across TCP/IP.
initial-load_Replicat
is the name of the initial-load Replicat group.
Directs Manager on the target system to dynamically start the initial-load Replicat as a one-time task.
Specifies the fully qualified name of an object or a fully qualified wildcarded specification for multiple objects. If the database is an Oracle multitenant container database, the object name must include the name of the container unless
SOURCECATALOG
is used. See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.CATALOGEXCLUDE
SCHEMAEXCLUDE
TABLEEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific objects from a wildcard specification in the associated
TABLE
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional parameters.
Save and close the file.
On the target system, issue the following command to create the initial-load Replicat.
Where:
initial-load_Replicat
is the name of the initial-load Replicat task.SPECIALRUN
identifies the initial-load Replicat as a one-time task, not a continuous process.
On the target system, issue the following command to create an initial-load Replicat parameter file.
Enter the parameters listed in Table 16-8 in the order shown, starting a new line for each parameter statement. The following is a sample initial-load Replicat parameter file for a direct load to SQL*Loader.
Table 16-8 Initial-load Replicat Parameters for Direct Load to SQL*Loader
Parameter Description Specifies the initial-load Replicat task to be started by Manager. Use the name that you specified when you created the initial-load Replicat.
Specifies database connection information.
TARGETDB
specifies the target Oracle container. See Reference for Oracle GoldenGate for Windows and UNIX for more information.USERID
andUSERIDALIAS
specify database credentials if required. See Chapter 11, 'Configuring Oracle GoldenGate Security' for more information about security options.Directs Replicat to interface directly with the Oracle SQL*Loader interface. See Reference for Oracle GoldenGate for Windows and UNIX for more information.
Use
SOURCEDEFS
if the source and target tables have different definitions. Specify the source-definitions file generated by DEFGEN. New punjabi duet song mp3 download.Use
ASSUMETARGETDEFS
if the source and target tables have the same definitions.
Specifies how to interpret data definitions. For more information about data definitions files, see Chapter 13, 'Associating Replicated Data with Metadata.'
SOURCECATALOG
Specifies a default source Oracle container for subsequent
MAP
statements. Enables the use of two-part names (schema.object
) where three-part names otherwise would be required. You can use multiple instances of this parameter to specify different default containers for different sets ofMAP
parameters.Specifies a relationship between a source object or objects and a target object or objects.
MAP
specifies the source object, andTARGET
specifies the target object.For the source object, specify the fully qualified name of the object or a fully qualified wildcarded specification for multiple objects. For an Oracle multitenant container database, the source object name must include the name of the container unless
SOURCECATALOG
is used.For the target object, specify only the
owner.object
components of the name, regardless of the database. Replicat can only connect to one Oracle container. Use a separate Replicat process for each container to which you want to load data.See Section 4.5, 'Specifying Object Names in Oracle GoldenGate Input' for guidelines for specifying object names in parameter files.
The
DEF
option specifies a definitions template. For more information about data definitions, see Section 13, 'Associating Replicated Data with Metadata'. See Reference for Oracle GoldenGate for Windows and UNIX for more information and options for theMAP
parameter.CATALOGEXCLUDE
SCHEMAEXCLUDE
MAPEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Parameters that can be used in conjunction with one another to exclude specific source objects from a wildcard specification in the associated
MAP
statement. See Reference for Oracle GoldenGate for Windows and UNIX for details.Enter any appropriate optional Replicat parameters listed in Reference for Oracle GoldenGate for Windows and UNIX.
Save and close the parameter file.
On the source system, start change extraction.
View the Replicat parameter file to make certain that the
HANDLECOLLISIONS
parameter is listed. If not, add the parameter to the file.(Oracle, if replicating sequences) Issue the
DBLOGIN
command as the user who hasEXECUTE
privilege onupdate.Sequence
.(Oracle, if replicating sequences) Issue the following command to update each source sequence and generate redo. From the redo, Replicat performs initial synchronization of the sequences on the target. You can use an asterisk wildcard for any or all characters in the name of a sequence (but not the owner).
On the source system, start the initial-load Extract.
Caution:
Do not start the initial-load Replicat. The Manager process starts it automatically and terminates it when the load is finished.On the target system, issue the following command to determine when the load is finished. Wait until the load is finished before proceeding to the next step.
On the target system, start change replication.
On the target system, issue the following command to verify the status of change replication.
Continue to issue the
INFO REPLICAT
command until you have verified that Replicat posted all of the change data that was generated during the initial load. For example, if the initial-load Extract stopped at 12:05, make sure Replicat posted data up to that point.On the target system, issue the following command to turn off the
HANDLECOLLISIONS
parameter and disable the initial-load error handling.On the target system, edit the Replicat parameter file to remove the
HANDLECOLLISIONS
parameter. This preventsHANDLECOLLISIONS
from being enabled again the next time Replicat starts.Caution:
Do not use theVIEW PARAMS
orEDIT PARAMS
command to view or edit an existing parameter file that is in a character set other than that of the local operating system (such as one where theCHARSET
option was used to specify a different character set). View the parameter file from outside GGSCI if this is the case; otherwise, the contents may become corrupted.Save and close the parameter file.
From this point forward, Oracle GoldenGate continues to synchronize data changes.
16.8 Loading Data with Teradata Load Utilities
The preferred methods for synchronizing two Teradata databases is to use any of the Teradata data load utilities. The recommended utility is MultiLoad.
This procedure requires Extract and Replicat change-synchronization groups to be available and properly configured for Teradata replication. For more information, see Chapter 14, 'Configuring Online Change Synchronization.'
If you are using multiple Extract and Replicat groups, perform each step for all of them as appropriate.
To load data with a Teradata load utility
Create the scripts that are required by the utility.
Start the primary Extract group(s).
Start the data pump(s), if used.
Open the Replicat parameter file(s) for editing.
Caution:
Do not use theVIEW PARAMS
orEDIT PARAMS
command to view or edit an existing parameter file that is in a character set other than that of the local operating system (such as one where theCHARSET
option was used to specify a different character set). View the parameter file from outside GGSCI if this is the case; otherwise, the contents may become corrupted.Add the following parameters to the Replicat parameter file(s):
END RUNTIME
directs Replicat to terminate normally when it reads an Oracle GoldenGate trail record that has a timestamp that is the same as, or after, the time that Replicat was started.HANDLECOLLISIONS
directs Replicat to overwrite duplicate records and ignore missing ones, as a means of resolving errors that occur from collisions between transactional changes and the results of the copy.
Save and close the Replicat parameter file(s).
Start the load utility.
When the load completes on the target, start the Replicat process(es).
When each Replicat process stops, remove the
HANDLECOLLISIONS
andEND RUNTIME
parameters from the parameter file.Restart the Replicat process(es). The two databases are now synchronized, and Oracle GoldenGate will keep them current through replication.
This is asking a lot but could anyone help check this short load order (of my entire 160ish pugins). This is the bottom half and the only parts that give me errors in xEdit. These mods obviously conflict and theres no going around that but I just want to make SURE they are in the most beneficial order.
Also any insight into the errors of xedit, mainly the navmesh ones and if they would cause crashes or just 'simply' result in empty voids in my game where something was replaced.
THANKS
This is my current load order for the mods that matter:
AKSKyrimUnderground
Alternate Start
RAO - Alternate Start Patch
Immersive Citizens - AI Overhaul
Open Cities
JK's Whiterun-Open cities SSE
Immersive Citizens - OCS patch (open cities)
SoS ImmersiveCitizens Patch
RLO - IC Patch
Relationship Dialogue Overhaul
RDO - AFT
RDO - USSEP Patch
RealisticWaterTwo
RW2 + Verdant patch
Palaces Castles Enhanced
PCE - USSEP
PCE - Live Another Life
PCE - Immersive Citizens
PCE- Open Cities
PCE - RDO
PCE - RLO
SSEMerged
Bashed Patch
BELOW ARE THE XEDIT ERRORS (Probably can ignore the abandonedprisonCELL in SKYRIMUNDERGROUND as I'm almost certain that's the same cell as Alternate Start)
[00:03] Checking for Errors in [0A] AKSkyrimUnderground.esp
[00:05] [REFR:0A5314B9] (places FarmBTrapdoor02 'Trapdoor' [DOOR:000E9E08] in GRUP Cell Persistent Children of DarkwaterCrossingGoldenrockMine 'Goldenrock Mine' [CELL:00013A8A])
[00:05] REFR XNDP - Navigation Door Link Teleport Marker Triangle -> <Warning: Navmesh triangle not found in '[NAVM:00061DCB] (in GRUP Cell Temporary Children of DarkwaterCrossingGoldenrockMine 'Goldenrock Mine' [CELL:00013A8A])'>
[00:07] [REFR:0A35C0B3] (places ImpDoorExtHoleDoorLoad01 'Wooden Door' [DOOR:0004E2C2] in GRUP Cell Persistent Children of AbandonedPrison01 'Abandoned Prison' [CELL:00021594])
[00:07] REFR XNDP - Navigation Door Link Teleport Marker Triangle -> <Warning: Navmesh triangle not found in '[NAVM:0005819D] (in GRUP Cell Temporary Children of AbandonedPrison01 'Abandoned Prison' [CELL:00021594])'>
[00:08] Checking for Errors in [0B] Alternate Start - Live Another Life.esp
[00:09] Checking for Errors in [0C] RAO - Alternate Start Patch.esp
[00:09] Checking for Errors in [0D] Immersive Citizens - AI Overhaul.esp
[00:16] NPCOWhiterunLuciaSitNearBanneredMare19x2 [PACK:0DFCC0AE]
[00:16] PACK Package Data Data Input Values Value PTDA - Target Target Data Target -> Target is not persistent
[00:17] Checking for Errors in [0E] Open Cities Skyrim.esp
[00:17] [REFR:00077323] (places SHouseDoor01 'Door' [DOOR:000368CE] in GRUP Cell Persistent Children of SolitudeCastleDourTower 'Castle Dour, Tower' [CELL:00077289])
Alternate Start Requiem Load Order
[00:17] REFR XNDP - Navigation Door Link Teleport Marker Triangle -> <Warning: Navmesh triangle not found in '[NAVM:000772CD] (in GRUP Cell Temporary Children of SolitudeCastleDourTower 'Castle Dour, Tower' [CELL:00077289])'>
[00:19] Checking for Errors in [0F] JK's Whiterun-Open cities SSE.esp
[00:19] Checking for Errors in [10] Immersive Citizens - OCS patch.esp
[00:19] Checking for Errors in [11] SoS_ImmersiveCitizens_Patch.esp
[00:19] Checking for Errors in [12] RLO - IC Patch.esp
[00:19] Checking for Errors in [13] Relationship Dialogue Overhaul.esp
[00:23] Checking for Errors in [14] RDO - AFT v1.66 Patch.esp
[00:24] Checking for Errors in [15] RDO - USSEP Patch.esp
[00:24] Checking for Errors in [16] RealisticWaterTwo.esp
[00:24] Checking for Errors in [17] RealisticWaterTwo+-+Verdant+Patch.esp
[00:24] Checking for Errors in [18] RealisticWaterTwo - Watercolor.esp
[00:24] Checking for Errors in [19] Palaces Castles Enhanced.esp
[00:24] _CuNPCGuardWeak 'Solitude Guard' [NPC_:191EECA5]
[00:24] NPC_ Tint Layers Layer TINI - Tint Index -> <Tint layer index 24 not found in NordRace 'Nord' [RACE:00013746]>
[00:24] NPC_ Tint Layers Layer TINI - Tint Index -> <Tint layer index 31 not found in NordRace 'Nord' [RACE:00013746]>
[00:27] [REFR:0007C3DF] (places RiftenDoorLoad01 'Door' [DOOR:0004D6ED] in GRUP Cell Persistent Children of [CELL:00016BE0] (in RiftenWorld 'Riften' [WRLD:00016BB4]) at 42,-25)
[00:27] REFR XNDP - Navigation Door Link Teleport Marker Triangle -> <Warning: Navmesh triangle not found in '[NAVM:000F7E94] (in GRUP Cell Temporary Children of RiftenCitySouth [CELL:00042245] (in RiftenWorld 'Riften' [WRLD:00016BB4] at 42,-25))'>
[00:28] Checking for Errors in [1A] PCE - USSEP Patch.esp
Alternate Start Dawnguard
[00:28] Checking for Errors in [1B] PCE - Live Another Life Patch.esp[00:28] Checking for Errors in [1C] PCE - Immersive Citizens AI Overhaul Patch.esp
Alternate Start Load Order Tool
[00:28] Checking for Errors in [1D] PCE - Open Cities Patch.esp[00:28] Checking for Errors in [1E] PCE - RDO Patch.esp
[00:28] Checking for Errors in [1F] PCE - RLO Patch.esp
[00:28] All Done!
Edited by imaqtpiesub, 11 October 2017 - 04:18 AM.