- Posted by Gavin Soorma
- On October 20, 2017
- 1 Comments
- 12.3, chunk, goldengate 12c release 3, new feature, parallel replicat, parallelism
One of the new features introduced in GoldenGate 12c Release 3 (18.104.22.168) is the Parallel Replicat feature.
So now in addition to the Classic Replicat, Co-ordinated Replicat and Integrated Replicat options, we also have another replicat option available as well.
The Parallel Replicat on the surface appears to be very similar to the Integrated Replicat in the sense that we can control the number of applier processes manually and also the apply process is auto-tuned as well where additional applier processes are added on the fly based on the workload being performed by the replicat process. This is managed by the Parallel Replicat parameters APPLY_PARALLELISM, MIN_APPLY_PARALLELISM and MAX_APPLY_PARALLELISM.
In addition, similar to the EAGER_SIZE which was used in the Integrated Replicat to help define what a ‘large’ transaction was, now in the Parallel Replicat we have something quite similar called CHUNK_SIZE.
We also have a parameter called SPLIT_TRANS_RECS which we can use to break a large transaction into logically smaller pieces which can then be applied in parallel. Dependencies are managed and maintained as well.
But what is different from the Integrated Replicat is that there is no requirement to set the STREAMS_POOL_SIZE and no Log Miner Server related processing happening inside the database.
Let us look at an example of using the Parallel Replicat feature.
The example assumes the following:
- Oracle database software is 12c Release 2 and the source and target databases have been configured appropriately for Oracle GoldenGate replication
- Oracle GoldenGate 12c Release 3 Micro Services software has been installed
- A deployment called test_ogg_123 has been created via Oracle GoldenGate 12.3 Service Manager
- Credential Store has been configured
- TRANDATA has been configured at the schema level
- Checkpoint Table has been created
- SOURCE and TARGET schemas have been created
- MYSALES table has been created in both schemas (script below)
SQL> create table mysales (id number, flag number , product varchar2(20), channel_id number, cust_id number , amount_sold number, order_date date, ship_date date) ;
We will see how to use the web interfaces as well as the command line Admin Client to configure Parallel Replicat.
Launch Service Manager
Create a Classic Extract
Add the MYSALES table to the extract parameter file
Create the distribution path – very similar to creating the Extract Pump process in the Classic Architecture.
Create the Parallel Replicat. In this case we are creating a non-integrated Parallel Replicat.
Add the MYSALES table to the replicat parameter file
We next use the Admin Client to add some other parameters to the replicat parameter file.
We are changing the value of the parameter MAP_PARALLELISM from the default value of 2 to 4 – this controls the number of mapper processes which will scan or process the trail file.
The default value for APPLY_PARALLELISM is 4 which controls the number of apply processes.
The parameter SPLIT_TRANS_RECS will break up the transaction into units of 10000 rows each and these will be applied in parallel.
[oracle@rac03 bin]$ ./adminclient Oracle GoldenGate Administration Client for Oracle Version 22.214.171.124.0 OGGCORE_126.96.36.199.0_PLATFORMS_170721.0154 Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved. Linux, x64, 64bit (optimized) on Jul 21 2017 07:16:02 Operating system character set identified as UTF-8. OGG (not connected) 1> connect http://rac03.localdomain:9001 deployment test_ogg_123 as oggadmin password oracle OGG (http://192.168.56.102:9001 test_ogg_123) 29> info all Program Status Group Lag at Chkpt Time Since Chkpt ADMINSRVR RUNNING DISTSRVR RUNNING PMSRVR RUNNING RECVSRVR RUNNING EXTRACT RUNNING EXT1 00:00:00 00:00:09 REPLICAT RUNNING REP1 00:00:00 00:00:10 OGG (http://192.168.56.102:9001 test_ogg_123) 30> edit params rep1 replicat rep1 useridalias oggadmin domain OracleGoldenGate MAP_PARALLELISM 4 SPLIT_TRANS_RECS 10000 MAP source.mysales, TARGET target.mysales;
Check that the extract and replicat process are both up and running.
On the source database, issue the INSERT statement which will populate the MYSALES table with 200,000 rows and commit the transaction.
SQL> insert into mysales select rownum, rownum + 1, 'Samsung Galaxy S7', mod(rownum,5), mod(rownum,1000) , 5000, trunc(sysdate - 10000 + mod(rownum,10000)), trunc(sysdate - 9999 + mod(rownum,10000)) from dual connect by level<=2e5 ; 200000 rows created. SQL> commit;
View the statistics of the extract process – similar to STATS ext1 LATEST command. Do the same for the Parallel Replicat.
Note the position in the trail files the extract and distribution server process pump1 are writing to.
We can see that the parallel replicat process is processing trail file rt000000000 and the location of the trail file now in the Micro Service architecture environment is under the specific deployment name top level folder in the /var/lib/data sub-directory.
OGG (http://192.168.56.102:9001 test_ogg_123) 40> info rep1 No EXTRACT groups found, but some coordinated threads may have been excluded REPLICAT REP1 Last Started 2017-10-19 16:55 Status RUNNING Parallel Checkpoint Lag 00:00:00 (updated 00:00:02 ago) Process ID 10459 Log Read Checkpoint File /u01/app/oracle/test_ogg_123/var/lib/data/rt000000000 2017-10-19 21:06:15.715609 RBA 42758112
Connect to the Performance Metrics Server home page from Service Manager home page and we can see the individual performance related metrics for REP1 parallel replicat process as well as the 4 mapper processes REP1M0* and 4 applier processes REP1A0*.