Softpedia >Mac >Business >Tungsten Replicator >  Changelog

Tungsten Replicator Changelog

What's new in Tungsten Replicator 2.2.0

Jan 6, 2014
  • Tungsten Replicator 2.2.0 is a bugfix and feature release that contains a number of key improvements to the installation and management of the replicator:
  • tpm is now the default installation and deployment tool; use of tungsten-installer, configure, configure-service, and update are deprecated.
  • tpm incorporates support for both INI file and staging directory deployments. See Section 4.4.4, “tpm INI File Configuration”.
  • Deployments are possible using standard Linux RPM and PKG deployments. See Section 2.2.2, “Using the RPM and DEB package files”.
  • tpm has been improved to handle heterogenous deployments more easily.
  • New command-line tools have been added to make recovery easier during a failure. See Section 4.7, “The tungsten_provision_slave Script”, Section 4.8, “The tungsten_read_master_events Script”, Section 4.9, “The tungsten_set_position Script”.
  • Improvements to the core replicator, including identification and recovery from failure.
  • New multi_trepctl tool for monitoring multiple hosts/services.
  • Behavior Changes:
  • The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:
  • The thl info command has been updated so that the output also displays the lowest and highest THL file, sizes and dates.
  • Issues: 471
  • For more information, see Section 4.2.4, “thl info Command”.
  • The following commands to trepctl have been deprecated and will be removed in a future release:
  • trepctl start has been replaced with trepctl load
  • trepctl stop has been replaced with trepctl unload
  • trepctl shutdown has been deprecated; use Section 2.20, “Starting and Stopping Tungsten Replicator” to stop the replicator.
  • Issues: 672
  • For more information, see Section 4.5.3.8, “trepctl load Command”, Section 4.5.3.21, “trepctl unload Command”, Section 2.20, “Starting and Stopping Tungsten Replicator”.
  • The tpm command has been updated to be the default method for installing deployments using the cookbook. To use the old tungsten-installer command, set the USE_OLD_INSTALLER environment variable.
  • Improvements, new features and functionality:
  • Installation and Deployment:
  • For heterogenous deployments, three new options have been added to tpm:
  • --enable-heterogenous-master
  • This options applies a range of settings, including --mysql-use-bytes-for=string=false, --java-file-encoding=UTF8, --mysql-enable-enumtostring=true, and --mysql-enable-settostring=true. This option also enables the colnames and pkey filters.
  • --enable-heterogenous-slave
  • This option disables parallel replication for hosts that do not support it, and sets the --java-file-encoding=UTF8 option.
  • --enable-heterogenous-service
  • Enables the --enable-heterogenous-master and --enable-heterogenous-slave for masters and slaves respectively.
  • Issues: 692
  • For more information, see Section 2.9.2, “Installing MongoDB Replication”, Section 2.11.2, “Installing Vertica Replication”.
  • Command-line Tools:
  • A new command-line tool, tungsten_set_position, has been created. This enables the position of either a master or slave to be set with respect to reading local or remote events. This provides easier control over during the recovery of a slave or master in the event of a failure.
  • Issues: 684
  • For more information, see Section 4.9, “The tungsten_set_position Script”, Section 3.2, “Managing Transaction Failures”.
  • A new command-line tool, tungsten_provision_slave, has been created. This allows for an automated backup of an existing host and restore of that data to a new host. The script can be used to provision new slaves based on existing slave configurations, or to recover a slave that has failed.
  • Issues: 689
  • For more information, see Section 4.7, “The tungsten_provision_slave Script”, Section 3.2, “Managing Transaction Failures”.
  • A new command-line tool, tungsten_read_master_events, has been created. This enables events from the MySQL binary log to be viewed based on the THL event ID.
  • Issues: 694
  • For more information, see Section 4.8, “The tungsten_read_master_events Script”, Section 3.2, “Managing Transaction Failures”.
  • The trepctl properties command has been updated to support a -values option that outputs only the values for filtered properties.
  • Issues: 719
  • For more information, see Section 4.5.3.12, “trepctl properties Command”.
  • The multi_trepctl command has been added. The tool enables status and other output from multiple hosts and/or services, providing a simpler way of monitoring a typical Tungsten Replicator installation.
  • Issues: 756
  • For more information, see Section 4.6, “The multi_trepctl Command”.
  • Oracle Replication:
  • The ddlscan tool and the ddl-mysql-oracle.vm template have been modified to support custom included templates on a per table basis.
  • The tool has also been updated to support additional paths for searching for velocity templates using the -path option.
  • Issues: 723
  • Core Replicator:
  • The block commit process has been updated to support different configurations. Two new parameters have been added, which affect the block commit size, and enable transactions to be committed to a slave in blocks either based on the number of events, or the time interval since the last commit occurred.
  • --repl-svc-applier-block-commit-size sets the number of events that will trigger a block commit. The default is 10.
  • --repl-svc-applier-block-commit-interval sets the time interval between block commits. The default is 0 (disabled).
  • Issues: 677, 699
  • For more information, see Section 7.1, “Block Commit”.
  • Filters:
  • The dropcolumn JavaScript filter has been added. The filter enables individual columns to be removed from the THL so that personal identification information (PII) can be removed on a slave.
  • Issues: 716
  • For more information, see Section 6.4.2.6, “dropcolumn.js Filter”.
  • Bug Fixes:
  • Installation and Deployment:
  • When performing a Vertica deployment, tpm would fail to create the correct configuration parameters. In addition, error messages and warnings would be generated that did not apply to Vertica installations. tpm has been updated to simplify the Vertica installation process.
  • Issues: 688, 781
  • For more information, see Section 2.11.2, “Installing Vertica Replication”.
  • tpm would allow parallel replication to be configured in heterogenous environments where parallel replication was not supported. During deployment, tpm now reports an error if parallel configuration parameters are supplied for datasource types other than MySQL or Oracle.
  • Issues: 733
  • When configuring a single host to support a parallel, multi-channel deployment, tpm would report that this operation was not supported. tpm has now been updated to support single host parallel apply configurations.
  • Issues: 737
  • Configuring an installation with a preferred path for MySQL deployments using the --preferred-path option would not set the PATH variable correctly, this would lead to the tools from an incorrect directory being used when performing backup or restore operations. tpm has been updated to correctly set the environment during execution.
  • Issues: 752
  • Command-line Tools:
  • When using the -sql option to the thl, additional metadata and options would be displayed. The tool has now been updated to only output the corresponding SQL.
  • Issues: 264
  • DATETIME values could be displayed incorrectly in the THL when using the thl tool to show log contents.
  • Issues: 676
  • An incorrect RMI port could be used within a deployment if a non-standard RMI port was specified during installation, affecting the operation of trepctl. The precedence for selecting the RMI port to use has been updated to use the -port, the system property, and then service properties for the selected service and/or trepctl executable.
  • Issues: 695
  • Backup and Restore:
  • During installation, tpm would fail to check the version for Percona XtraBackup when working with built-in InnoDB support in MySQL. The check has now been updated and validation will fail if XtraBackup 2.1 or later is used with a MySQL 5.1 and builtin InnoDB support.
  • Issues: 671
  • When using xtrabackup during a restore operation, the restore would fail. The problem was due to a difference in the interface for XtraBackup 2.1.6.
  • Issues: 778
  • Oracle Replication:
  • When performing an Oracle deployment, tpm would apply incorrect parameters and filters and check MySQL specific environment information. The following changes have been made:
  • The colnames filter is no longer added to Oracle master (extractor) deployments.
  • Incorrect schema value would be defined for the replicator schema.
  • The check for mysqldump is still performed on an Oracle master host; use --preferred-path to set a valid location, or disable the MySQLDumpCheck validation check.
  • Issues: 685
  • Core Replicator:
  • DECIMAL values could be extracted from the MySQL binary log incorrectly when using statement based logging.
  • Issues: 650
  • A null pointer exception could be raised by the master, which would lead to the slave failing to connect to the master correctly. The slave will now retry the connection.
  • Issues: 698
  • A slave replicator could fail when synchronizing the THL if the master goes offline. This was due to network interrupts during a failure not being recognised properly.
  • Issues: 714
  • In certain circumstances, a replicator could apply transactions that had been generated by itself. This could happen during a failover, leading to events written to the THL, but without the trep_commit_seqno table having been updated. To fix this problem, consistency checks on the THL contents are now performed during startup. In addition, all replicators now write their currently assigned role to a file within the configuration directory of the running replication service, called static-servicename.role.
  • When the replicator goes online, a static-servicename.role file is examined. If the current role identified in that file was a master, and the current role of the replicator is a slave, then the THL consistency checks are enabled. These check the following situations:
  • If the trep_commit_seqno is out of sync with the contents of the THL provided that the last THL record exists and matches the source-id of the transaction.
  • If the current log position is different to the THL position, and assuming that THL position exists, then an error will be raised and the replicator will go offline. This behavior can be overrriden by using the trepctl online -force command.
  • Once the checks have been completed, the new role for the replicator is updated in the static-servicename.role file.
  • Important:
  • The static-servicename.role file must be deleted, or the THL files must be deleted, when restoring a backup. This is to ensure that the correct current log position is identified.
  • Issues: 735
  • An UnsupportedEncodingException error could occur when extracting statement based replication events if the MySQL character set did not match a valid Java character set used by the replicator.
  • Issues: 743
  • When using Row-based replication, replicating into a table on the slave that did not exist, a Null-Pointer Exception would be raised. The replicator now correctly raises an SQL error indicating that the table does not exist.
  • Issues: 747
  • During a master failure under load, the number of transactions making it to the slave before the master replicator fails.
  • Issues: 753
  • Upgrading a replicator and changing the hostname could cause the replicator to skip events in the THL. This was due to the way in which the source-id of events in the slave replicator checks the information compared to the remote THL read from the master. This particularly affect standalone replicators. The fix adds a new property, replicator.repositionOnSourceIdChange. This is a boolean value, and specifies whether the replicator should try to reposition to the correct location in the THL when the source ID has been modified.
  • Issues: 754
  • Running trepctl reset on a service deployed in an multimaster (all master) configuration would not correctly remove the schema from the database.
  • Issues: 758
  • Replication of temporary tables with the same name, but within different sessions would cause a conflict in the slave.
  • Issues: 772
  • Filters:
  • The PrimaryKeyFilter would not renew connections to the master to determine the primary key information. When replication had been running for a long time, the active connection would be dropped, but never renewed. The filter has been updated to re-connect on failure.
  • Issues: 670
  • For more information, see Section 6.3.16, “PrimaryKeyFilter”.

New in Tungsten Replicator 2.1.2 (Oct 8, 2013)

  • Behavior Changes:
  • There has been a significant change in the THL format used for storing timestamp information, as a result of changes to the processing required for supporting MySQL 5.6.When upgrading to the latest version, slaves must be upgraded before the master to ensure that they are able to cope with the timestamp storage format change before the information is written to the THL on the master.Once slaves have been updated, the masters can be updated.
  • The internal CRC check used by MySQL 5.6 was incompatible with the binary log extractor. Issues: 461
  • When operating using ROW-based replication, DATETIME columns would show time differences during a daylight savings time (DST) change.In addition, timestamps may have replicated with different values if the master and slave were configured with different timezones.
  • Timezone configurations on all hosts within a cluster must be set to the same timezone to prevent values drifting between hosts.
  • Individual changes to the timezone configuration of the system or MySQL may introduce differences.
  • Values inserted during daylight savings time changes should be correct during a time change; ensure your timezone is configured properly and your servers are synchronized.In addition to the configuration requirements, the format of the THL files has been changed. When upgrading:
  • Slaves should be upgraded first, this will ensure that the slaves can read the new THL format
  • Once the slaves have been upgraded, upgrade the master. Once upgraded and reconfigured, the information output by thl may indicate a different date/time value than either extracted from the master, or that will be applied to the slave, due to the way the information is stored within the THL. Time differences may show a multiple-hour difference compared to the applied value. Issues: 542
  • Support for MySQL 5.6 row-based replication has been added to Tungsten Replicator. Replication is now supported for the following MySQL releases:
  • MySQL 5.0, 5.1, 5.5, 5.6
  • Percona 5.5
  • MariaDB 5.5_Issues: 558
  • Improvements, new features and functionality:
  • Installation and Deployment:
  • The replicator has been updated to support secure communication between replicators and between administrative tools (trepctl) and the replicator processes. Security operates at two separate levels within the configuration:
  • SSL support for the THL transfer between replicator instances. This feature enables full SSL certificate encrypted transmission of the THL data.
  • SSL and authentication support for administration. Administration, both local and remote, through trepctl and other tools can be encrypted and secured through an authorized user for the Java RMI channel.Security can be enabled for either or both components, and existing installations can be updated to use the secure installation.Secure installation is only supported when using tpm to perform installations and updates. Issues: 508, 638, 656, 664, 665, 666, 667, 668
  • To provide better interaction with complex environments, including those created by the Cookbook system, tpm supports the setting of additional PATH locations to be searched before the standard path. The --preferred-path option specifies one or more directories to be prepended to the PATH by Tungsten Replicator, including backup/restore tools, Ruby, Java and other utilities. Issues: 582; Tags: tpm:preferred-path
  • The tpm command has been updated to support the option --timezone, which sets the corresponding JVM timezones. Issues: 596
  • Use of tungsten-installer for performing installations will be deprecated in a future release. The tpm command will be used instead. tpm has been updated to support reading, and updating from an existing configuration, migrating a service installed using tungsten-installer to use tpm. Issues: 641, 669
  • The tpm tool has been updated to make the installation of different complex topologies easier. The improvements involve the following major changes:
  • The --topology option now supports explicit settings for all-masters (multi-master topology), fan-in, and star.
  • A list of services can be supplied where multiple services are created, for example during multi-master, fan-in, or star configurations. The list of services to be created is specified by the --master-services option, accepting a comma-separated list of service names. For example, when creating a multi-master configuration with four hosts, --master-services=alpha,beta,gamma,delta will create four services for the corresponding list of masters, i.e. --masters=host1,host2,host3,host4 would create a service on host1 called alpha, on host2 called beta and so on.
  • The specification of hosts has been simplified:
  • The list of master hosts (comma-separated) can be specified using the --masters or --master option.
  • The list of slave hosts (comma-separated) can be specified using the --slaves option.
  • The list of members (i.e. masters and slaves) can continue to be specified using the --members option.tpm will calculate the appropriate list of slaves, masters, and services from this information and create the corresponding configurations during deployment. Issues: 623
  • Command-line Tools:
  • The trepctl has been improved to support the output of a list of connected slaves (clients) to a running master service. Issues: 635
  • Cookbook Utility:
  • Cookbook scripts have been updated to use the tpm command for installation of a cluster, in place of tungsten-installer. Issues: 620
  • The cookbook toolset has been expanded to include explicit tools that integrate more effectively with MySQL Sandbox. Specifically:
  • deploy_to_sandboxes has been renamed to deploy_sandboxes.
  • A remove_sandboxes command has been added to remove existing sandboxes. A corresponding tool, restore_user_values saves back the contents of the USER_VALUES.sh and USER_VALUES.local.sh scripts to versions that do not use the sandbox infrastructure.
  • The db_use tool has been created and provides a direct route into a MySQL command-line tool for a specific host. Use the -h command-line option to specify the host. For example: shell> db_use -h host1
  • When uninstalling an installation using the cookbook through clear_cluster, the process would run sequentially. The process has now been updated to run concurrently. Issues: 653
  • The cookbook has been updated to check and warn if an installation is started and USE_TPM has not been enabled. A warning is generated in the event of executing the installer without the USE_TPM variable being enabled and set to 1. The warning and delay can be disabled by setting INSTALLATION_DELAY=0 before executing the cookbook installation scripts. Issues: 675
  • Backup and Restore:
  • The backup scripts that work in combination with trepctl have been documented. The documentation includes details on how the backup process works, and how custom backup scripts can be created, including sample scripts. Issues: 590
  • During a restore operation, a replicator is not automatically placed into ONLINE mode, instead the replicator is placed in OFFLINE mode. Issues: 609
  • Oracle Replication:
  • The CDCMetadataFilter and ddlscan has been updated to support placing the CDC columns at the start of a row, instead of at the end of the row. Issues: 628
  • Bug Fixes:
  • Installation and Deployment:
  • tungsten-installer would fail ungracefully when required applications were missing. Issues: 280
  • When running the configure-service tool from an incorrect location could result in the error message Unable to run configure-service because this directory has not been setup . Advisory text has now been added to the error message. Issues: 288
  • Tungsten Replicator has been updated to compile correctly against Java 1.7. Issues: 619
  • Installation of a fan-in cluster using tungsten-installer would fail when the fan-in slave is not last host in the list of hosts. Issues: 652
  • Command-line Tools:
  • When using the thl command with the -headers option, the entire THL content, including the full event and transaction data would be deserialized, which was inefficient. Issues: 645
  • The inline help within trepctl contained some duplication and inconsistencies. Issues: 646
  • Cookbook Utility:
  • The test_cluster script within the cookbook did not check for the online state within deployed services. Issues: 626
  • Cookbook deployments used relayed binary logs in place of directly reading binary logs. Issues: 627
  • The cookbook tool would fail to delete and clear an existing installation if LOCAL_USER_VALUES had been used during installation. Issues: 637
  • When configuring a new installation, the IP detection procedure would fail when a host has multiple IP addresses configured. Issues: 654
  • Backup and Restore:
  • When starting a backup through trepctl when the backup directory was empty (for example, after a move), the process would fail instead of recreating the required directory structure and contents. Issues: 552
  • When performing a backup on the master and restoring this backup to a slave, the backup contents would be inconsistent with the replication state, causing intermittent replication errors. This was due to the asynchronous nature of the backup process. The backup process has been updated to correctly clear the status information during restore. Issues: 556
  • Due to changes in the operating method, Tungsten Replicator was not compatible with Percona Xtrabackup 2.1. Issues: 629
  • Oracle Replication:
  • Running the setupCDC.sh command without specifying a correct service name would cause ongoing failures in the replication setup. The tool has been updated to report an error if the service name has not been set. Issues: 622
  • Core Replicator:
  • Due to a minor change in the binlog format used by MySQL, session variables were not replicated correctly for ROW-based events. Issues: 624
  • In some situations, parallel replication threads responsible for reading THL fail silently, resulting in badly lagging channels. The issue was due to a fault in the way failing channels and pipelines were identified and reported. Issues: 636
  • When replicating ROW-based events, null values for keys could incorrectly be handled. Issues: 659
  • When using ROW-based replication, DATETIME values of 0 (zero) would cause a ClassCastException when applied to a dataserver. Issues: 679
  • When using ROW-based replication, TIME values with a microsecond component would be incorrectly replicated, removing the microsecond component. Issues: 681
  • Filters:
  • The RenameFilter file has been documented. Issues: 612
  • Unclassified:
  • Due to a previous change to allow selection of SQL_MODE variable settings to be ignored when processing events. The supported setting of enabling ALLOW_INVALID_DATES would be identified incorrectly as INVALID_DATES. Issues: 642

New in Tungsten Replicator 2.1.1 (Aug 22, 2013)

  • Behavior Changes:
  • The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:
  • There has been a significant change in the THL format used for storing timestamp information, as a result of changes to the processing required for supporting MySQL 5.6.When upgrading to the latest version, slaves must be upgraded before the master to ensure that they are able to cope with the timestamp storage format change before the information is written to the THL on the master.Once slaves have been updated, the masters can be updated. Issues:
  • The internal CRC check used by MySQL 5.6 was incompatible with the binary log extractor. Issues: 461
  • When operating using ROW-based replication, DATETIME columns would show time differences during a daylight savings time (DST) change.In addition, timestamps may have replicated with different values if the master and slave were configured with different timezones:
  • To address these issue the configuration of Tungsten Replicator installations must be updated
  • The issues occurs because a valid date/time replicated to the slave creates a an invalid time due to daylight savings time when applied to the MySQL database. In the process of trying to fix the incorrect date/time, the time is updated before it is applied into the database. This is due to inconsistent timezone configurations in MySQL and the system timezone used by the Java VM used by Tungsten Replicator.The inconsistent date/time information has been fixed, but requires configuration changes to ensure that information is not replicated incorrectly:
  • The MySQL and Java JVM timezone configurations must be the same. Ideally, the system timezone should also be the same. Differences in the timezone configuration of Java and MySQL will cause differences in the stored DATETIME values on the slave.To set the timezone for MySQL: mysql> SET GLOBAL time_zone = timezone;
To set the timezone for your operating system, on Ubuntu/Debian, update the value in the file /etc/timzezone. On CentOS/RedHat, create a symbolic link from the right file within /usr/share/zoneinfo to /etc/localtime.To set the timezone for the JVM for Tungsten Replicator, use the --timezone configuration option to tpm to update your configuration.
  • Timezone configurations on all hosts within a cluster must be set to the same timezone to prevent values drifting between hosts.
  • Individual changes to the timezone configuration of the system or MySQL may introduce differences.
  • Values inserted during daylight savings time changes should be correct during a time change; ensure your timezone is configured properly and your servers are synchronized.In addition to the configuration requirements, the format of the THL files has been changed. When upgrading:
  • Slaves should be upgraded first, this will ensure that the slaves can read the new THL format
  • Once the slaves have been upgraded, upgrade the master..Once upgraded and reconfigured, the information output by thl may indicate a different date/time value than either extracted from the master, or that will be applied to the slave, due to the way the information is stored within the THL. Time differences may show a multiple-hour difference compared to the applied value. Issues: 542 See also: 596.
  • Support for MySQL 5.6 row-based replication has been added to Tungsten Replicator. Replication is now supported for the following MySQL releases:
  • MySQL 5.0, 5.1, 5.5, 5.6
  • Percona 5.5
  • MariaDB 5.5
  • Issues: 558
  • Improvements, new features and functionality:
  • Installation and Deployment:
  • The replicator has been updated to support secure communication between replicators and between administrative tools (trepctl) and the replicator processes. Security operates at two separate levels within the configuration:SSL support for the THL transfer between replicator instances. This feature enables full SSL certificate encrypted transmission of the THL data; SSL and authentication support for administration. Administration, both local and remote, through trepctl and other tools can be encrypted and secured through an authorized user for the Java RMI channel.Security can be enabled for either or both components, and existing installations can be updated to use the secure installation.Secure installation is only supported when using tpm to perform installations and updates. Issues: 508, 638, 656, 664, 665, 666, 667, 668. See SSL Secured Replication and Administration for more information.
  • To provide better interaction with complex environments, including those created by the Cookbook system, tpm supports the setting of additional PATH locations to be searched before the standard path. The --preferred-path option specifies one or more directories to be prepended to the PATH by Tungsten Replicator, including backup/restore tools, Ruby, Java and other utilities. Issues: 582; Tags: tpm:preferred-path
  • The tpm command has been updated to support the option --timezone, which sets the corresponding JVM timezones. Issues: 596
  • The tpm tool has been updated to make the installation of different complex topologies easier. The improvements involve the following major changes:
  • The --topology option now supports explicit settings for all-masters (multi-master topology), fan-in, and star.
  • A list of services can be supplied where multiple services are created, for example during multi-master, fan-in, or star configurations. The list of services to be created is specified by the --master-services option, accepting a comma-separated list of service names. For example, when creating a multi-master configuration with four hosts, --master-services=alpha,beta,gamma,delta will create four services for the corresponding list of masters, i.e. --masters=host1,host2,host3,host4 would create a service on host1 called alpha, on host2 called beta and so on.
  • The specification of hosts has been simplified: The list of master hosts (comma-separated) can be specified using the --masters or --master option; Thee list of slave hosts (comma-separated) can be specified using the --slaves option. The list of members (i.e. masters and slaves) can continue to be specified using the --members option.tpm will calculate the appropriate list of slaves, masters, and services from this information and create the corresponding configurations during deployment. Issues: 623
  • Use of tungsten-installer for performing installations will be deprecated in a future release. The tpm command will be used instead. tpm has been updated to support reading, and updating from an existing configuration, migrating a service installed using tungsten-installer to use tpm . Issues: 641, 669.
  • Command-line Tools:
  • The trepctl has been improved to support the output of a list of connected slaves (clients) to a running master service. Issues: 635.
  • Cookbook Utility:
  • Cookbook scripts have been updated to use the tpm command for installation of a cluster, in place of tungsten-installer. Issues: 620
  • The cookbook toolset has been expanded to include explicit tools that integrate more effectively with MySQL Sandbox. Specifically:
  • deploy_to_sandboxes has been renamed to deploy_sandboxes.
  • A remove_sandboxes command has been added to remove existing sandboxes. A corresponding tool, restore_user_values saves back the contents of the USER_VALUES.sh and USER_VALUES.local.sh scripts to versions that do not use the sandbox infrastructure.
  • The db_use tool has been created and provides a direct route into a MySQL command-line tool for a specific host. Use the -h command-line option to specify the host. For example: shell> db_use -h host1
  • Issues: 640
  • When uninstalling an installation using the cookbook through clear_cluster, the process would run sequentially. The process has now been updated to run concurrently. Issues: 653
  • The cookbook has been updated to check and warn if an installation is started and USE_TPM has not been enabled. A warning is generated in the event of executing the installer without the USE_TPM variable being enabled and set to 1. The warning and delay can be disabled by setting INSTALLATION_DELAY=0 before executing the cookbook installation scripts. Issues: 675
  • Backup and Restore:
  • The backup scripts that work in combination with trepctl have been documented. The documentation includes details on how the backup process works, and how custom backup scripts can be created, including sample scripts. Issues: 590.
  • During a restore operation, a replicator is not automatically placed into ONLINE mode, instead the replicator is placed in OFFLINE mode. Issues: 609.
  • Oracle Replication:
  • The CDCMetadataFilter and ddlscan has been updated to support placing the CDC columns at the start of a row, instead of at the end of the row. Issues: 628.
  • Bug Fixes:
  • Installation and Deployment:
  • tungsten-installer would fail ungracefully when required applications were missing. Issues: 280
  • When running the configure-service tool from an incorrect location could result in the error message Unable to run configure-service because this directory has not been setup . Advisory text has now been added to the error message. Issues: 288
  • Tungsten Replicator has been updated to compile correctly against Java 1.7. Issues: 619
  • Installation of a fan-in cluster using tungsten-installer would fail when the fan-in slave is not last host in the list of hosts. Issues: 652
  • Command-lines Tools:
  • When using the thl command with the -headers option, the entire THL content, including the full event and transaction data would be deserialized, which was inefficient. Issues: 645
  • The inline help within trepctl contained some duplication and inconsistencies. Issues: 646
  • Cookbook Utility:
  • The test_cluster script within the cookbook did not check for the online state within deployed services. Issues: 626
  • Cookbook deployments used relayed binary logs in place of directly reading binary logs. Issues: 627
  • The cookbook tool would fail to delete and clear an existing installation if LOCAL_USER_VALUES had been used during installation. Issues: 637
  • When configuring a new installation, the IP detection procedure would fail when a host has multiple IP addresses configured. Issues: 654
  • Backup and Restore:
  • When starting a backup through trepctl when the backup directory was empty (for example, after a move), the process would fail instead of recreating the required directory structure and contents. Issues: 552
  • When performing a backup on the master and restoring this backup to a slave, the backup contents would be inconsistent with the replication state, causing intermittent replication errors. This was due to the asynchronous nature of the backup process. The backup process has been updated to correctly clear the status information during restore. Issues: 556
  • Due to changes in the operating method, Tungsten Replicator was not compatible with Percona Xtrabackup 2.1. Issues: 629
  • Oracle Replication:
  • Running the setupCDC.sh command without specifying a correct service name would cause ongoing failures in the replication setup. The tool has been updated to report an error if the service name has not been set. Issues: 622
  • Core Replicator:
  • Due to a minor change in the binlog format used by MySQL, session variables were not replicated correctly for ROW-based events. Issues: 624
  • In some situations, parallel replication threads responsible for reading THL fail silently, resulting in badly lagging channels. The issue was due to a fault in the way failing channels and pipelines were identified and reported. Issues: 636
  • When replicating ROW-based events, null values for keys could incorrectly be handled. Issues: 659
  • Filters:
  • The RenameFilter file has been documented. Issues: 612.
  • Unclassified:
  • Due to a previous change to allow selection of SQL_MODE variable settings to be ignored when processing events. The supported setting of enabling ALLOW_INVALID_DATES would be identified incorrectly as INVALID_DATES. Issues: 642

New in Tungsten Replicator 2.1.0 (Jun 17, 2013)

  • Behavior Changes:
  • Tungsten uses comments appended to statements to mark the replication service name. This allows Tungsten replicators to recognize the origin of statements and prevent statement loops. This feature is only required in multi-master replication topologies where statements are replicated into another master again then extracted from the log. It is not required in simple master/slave topologies or those master/master topologies in which updates are not logged into the database log. However, in star topologies the comment information is required to prevent duplication or re-application of statements. It is also required in master/master topologies where updates are logged when applied to another master.Comments are now disabled by default. Cookbook templates for star topologies have been updated to enable this feature by default. This option should be enabled when creating a star topology, or when advised to do so. However, care should be taken to ensure that character set definitions for their table data, definitions and environment matches to prevent issues with the addition of comments to existing statements. Issue 547
  • In previous releases, a restore operation using trepctl would automatically set the replicator ONLINE once the restore operation had been completed. In Tungsten Replicator 2.1.0, the replicator will remain in the offline state after a restore operation. Issue 450
  • The following improvements, new features, and functionality, are available in this release:
  • Command-line Tools and Scripts:
  • Added a thl mode that outputs headers on a single line and supports JSON based output. The new command-line option is -headers, which outputs only the header and metadata information (without the SQL or row data) on a single line. The additional -json alters the output formatting to use the JSON format, with a single record for each event sequence fragment. Issue 576
  • The trepctl command has been updated to support the -json command-line option for status and service output to make it easier to parse the output of information from the tool. Issue 499
  • The tpm command can be used instead of tungsten-installer for installing and configuring services.Cookbook recipes still use the tungsten-replicator command, and tpm is not yet certified for general use. Issue 501
  • Cookbook Utility:
  • A script has been added to the Cookbook system that automatically starts a load on the replication service using Bristlecone. The new script is cookbook/load_data.sh in the Tungsten Replicator directory. Issue 483
  • Deploy MySQL sandboxes in several hosts for Tungsten installations. Issue 485
  • Shortcuts to the Tungsten Replicator tools have been added to the Cookbook directory, allowing for easier execution of selected tools and operations, such as the replicator and configuration viewing and editing. Issue 525
  • The Cookbook cleanup script now provides a number of environment variables that can be used to configure and select which items should be cleaned during the scripts execution. Issue 533
  • The Tungsten Replicator Cookbook recipes have been improved so that they use the current known topology of the cluster and the names have been simplified to be easier to use and identify. Issue 541
  • Added a dry-run option to cookbook installation scripts to enable the operations that will be performed to be shown. Issue 527
  • Core Replicator:
  • The parallel apply functionality has been improved to support sharding the information by sequence number within the THL. This can be used to improve initial loads of THL data to a new hosts, particularly in heterogeneous deployments. Issue 478
  • Very rarely, the CRC for a THL event can becomed corrupted, causing loading and parsing of the THL and replication to fail. Parsing of the THL would fail, making it impossible to identify the location of the failure and the event causing the problem. Within the thl command, the -no-checksum option now enables you to view the THL ignoring checksum errors if they exist.Additionally, the trepctl command has been updated to support the -no-checksum option. This switches off both CRC checking when reading the file, and generation of the CRC when replication data is written to the THL. In the event of a CRC failure, the THL should be examined and the CRC checksum switched off if the event is safe to be processed. Once processed, replication should be stopped and restarted without the -no-checksum option to enable checksum on THL events.
  • Replicators can now be configured as archive hosts. These download the THL but do not apply the THL to the database. Archive hosts can be used to act as a record of the THL which can help during recovery. Issue 549
  • The xtrabackup command used within the replicator restore procedure can now restore files directly to the MySQL data directory. Issues 568, 487
  • A general purposes Ruby library has been provided that supports basic API operations into the core Tungsten Replicator. General Tungsten scripting Ruby library. More information can currently be located within the cluster-home/lib/ruby/tungsten/README file within the distribution. Issue 569
  • Filters:
  • A new filter has been added, RenameFilter, which allows for easy renaming of schemas, tables and columns for ROW-based replication. Issue 464
  • Oracle Replication:
  • Support has been added to allow master extraction of the Oracle data for use in replication. This enables heterogeneous replication both to and from Oracle databases. Issue 551
  • The ddlscan tool includes a number of new features to make operations easier when scanning the Oracle database. These options include the ability to import additional templates, user-defined template options, interface for determining reserved words and integrated use of RenameFilter CSV file. Issue 462
  • Some sample scripts for easing the provisioning of MySQL databases to Oracle. Issue 585
  • Other Issues:
  • The ddlscan command templates can be used to generate schema and staging table DDL for Vertica. Issue 467
  • Backup storage agent moves files instead of copying them. This reduces the space requirement during backup. Issue 262
  • During installation and configuration, the THL port numbers for all services are compared to ensure that there are no duplicate port specifications on each host. A warning is generated during configuration. This affects tpm only. Issue 337
  • Concurrent garbage collection support has been added to the running replicator service. This can be configured by using the --java-enable-concurrent-gc=true option to the tungsten-installer or configure commands. Issue 412
  • The port number in a THL host configuration (--master-thl-host) specification can be specified on the command line using the host:port format. Issue 472
  • The replicator service can now be started without automatically going into the online state. To start, or restart, the replicator but start in the offline mode, use the -offline option: shell> replicator start -offline
  • Issue 553:
  • A --service-name argument has been added to tpm to specify the service name to override the default service name. Issue 563
  • A dump command has been added as an alias for the reverse command to the tpm command. Issue 564
  • Bug Fixes:
  • The following fixes and updates are included in this release:
  • Cookbook Utility:
  • When installing a cluster using cookbook, the cookbook tools could not be used from the installed directory, only from the staging directory. directoryIssue 532
  • The cookbook test script would not detect missing services, reporting a success even if no services were identified. Issue 540
  • The cookbook/load_data script would send data to the master as configured in the initial installation, instead of the current master within the replication service. Issue 544
  • Backup and restore operations would fail when using xtrabackup within the cookbook. The cookbook installer scripts now accept the option --datasource-boot-script to point to the required boot script for restarting the database server. Issue 578
  • The cookbook scripts did not work correctly with MySQL 5.6 due to the change in command-line password acceptance. Issue 581
  • The cookbook recipes for multi-master installation had hard coded values for the BINLOG_DIRECTORY. Issue 480
  • Filters:
  • The Eventmetadata filter would assign an empty string shard ID instead of #UNKNOWN, which would cause parallel apply failures. Issue 477
  • The ReplDBMSFilteredEvent filter would force commits on batch processing which would severely slow batch loading of data. Issue 592
  • Installation and Configuration:
  • Previously, a replicator configuration installed with the start-and-report option would return a 0 exit code, even if the replicator fails to start properly. The tungsten-installer now returns a non-zero result if the replicator fails to start after installation. Issue 277
  • The installer will now report an error if InnoDB has not been enabled on a MySQL server, is not the default storage engine, and if the required tables for Tungsten were not created as InnoDB tables. Issue 279
  • The installer has been updated to warn about unsafe values for the MySQL innodb_flush_log_at_trx_commit configuration value. Issue 482
  • The MySQL-to-MongoDB replication configuration has been fixed. A fault in the definition of the required filters and parallel apply functionality caused replication to MongoDB to fail on startup. Issue 572
  • Oracle Replication:
  • The setupCDC.sh script could not update an existing configuration when there were schema changes. The command has been updated to update the information during changes. Issue 522
  • The updateCDC.sh script would generate an nondescript error if the CDC tables did not exist. Issue 560
  • Other Fixes:
  • Tungsten replicator did not work correctly with Vertica 6 because the wrong commands and configuration settings were being used. Issue 463
  • Slave replicator would stall intermittently when parallel apply was enabled. Issue 466
  • On systems with slow filesystems, the replicator would generate error messages about needing to read more bytes than available from the filesystem. The identification and message have been changed. Issue 470
  • The replicator would fail to reduce the number of tasks properly when parallel apply is running and applier has been running for a long time. Issue 476
  • Replicator becomes unresponsive after OutOfMemoryError but does not indicate the error in the log. Issue 484
  • Taking a slave replicator offline immediately after process restart would result in the slave channel positions resetting to an earlier sequence no. Issue 493
  • Slave applier can fail to log error when DBMS fails due to exception in cleanupIssue 537
  • A master failure would cause partial commits on the slave configured with single channel parallel apply. Issue 546
  • The replicator would fail to go online if last transaction was at the end of a THL log file. Issue 570
  • The replicator does not correctly pick up the preferred master role when replicating from a Tungsten cluster after a switch. Issue 605
  • Multi table delete would not be detected correctly by the replicator and would lead to data inconsistencies. Issue 399
  • Documentation has been added about the EPOCH number stored in the THL. See EPOCH DescriptionIssue 444
  • The Java Service Wrapper has been modified to make it easier to define additional parameters. Issue 447
  • On certain operation system configurations some installation commands could fail, such as scp. The installation scripts have been updated to handle differences in operating system versions and support. Issue 455
  • The replicator command does not return the correct result code when running the command as a different user. Issue 469
  • The replicator installation would break when working in --direct mode. Issue 486
  • Using the deploy_to_sandboxes command within the cookbook, the data directory would not be updated in the USER_VALUES option. Issue 488
  • Using row-based replication, updates of binary data did not behave correctly for short values. Issue 489
  • The internal JMX management port would use a random port instead of a fixed port, making it difficult to configure firewall values accordingly. This has been updated to use the configured management (RMI) port (default 10000) + 1. Issue 490
  • The replicator would generate invalid SQL on a DROP TEMPORARY TABLE statement if the Preventative filter is configured to operate twice on the replication stream. Issue 491
  • The replicator startup script would return 0 even when the replicator process is not running. Issue 531
  • The CDC generating filter has been updated to support a different starting point for the CDC sequence number (replicator.filter.customcdc.sequenceBeginning property). The CDC has also been updated to support using a single schema for the CDC information using the replicator.filter.customcdc.toSingleSchema property. Issue 534
  • The cookbook installer for fan-in topology did not install slave services correctly. Issue 538
  • The utilities.sh script gets duplicate slaves when installing a multi-master topology. Issue 539
  • The exception message generated when a statement fails to be applied would get truncated to 1000 characters making it difficult to identify the statement contents. Issue 543
  • The load data script concurrent_evaluator.pl would stop only one instance instead of all of them during executing. Issue 545
  • Replication between MySQL and MongoDB could fail with a Null Pointer Exception. Issue 548
  • MySQL TEXT tables would not be replicated to MongoDB. Issue 567
  • Backup using trepctl could fail. Issue 577
  • Removing a service using the Oracle extractor would fail. Issue 586
  • When replicating a statement that included the UNHEX function within MySQL, an extra space would be added to the data. Issue 601
  • The Slave replicator would time out very slowly on connection to master when network interface is down, which could cause problems during auto failover. Issue 603

New in Tungsten Replicator 2.0.7 (Feb 23, 2013)

  • Issue 258 - Statement transactions lack SQL parsing metadata for filtering after they are logged
  • Issue 270 - Problems running installer on Solaris
  • Issue 304 - Replicator hangs on connection to dead network interface
  • Issue 323 - UPDATEs and DELETEs on a slave which update zero rows should fail
  • Issue 370 - Add support for CVS batch loading using external utilities like Calpont cpimport
  • Issue 371 - Parallel replication may lock up intermittently when Amazon instances are too small
  • Issue 374 - Implement purge operation
  • Issue 382 - Replication commits partial transaction following application error or OutOfMemory condition
  • Issue 386 - Add support for fast batch loading using direct inserts and minimal staging table for deletes
  • Issue 396 - Add option to make replicators read preferentially from a slave rather than a master
  • Issue 397 - Replicator has problems finding filtered events in log, leading to slave restart failures
  • Issue 398 - options for mysqldump backup are non-transactional safe
  • Issue 401 - Channel assigments for round-robin selection are not cleared when replicator goes offline cleanly
  • Issue 405 - Add Tungsten Replicator Cookbook to the standard tarball
  • Issue 408 - Incorrect configuration of THL parameter causes potentially large memory usage
  • Issue 411 - tools/configure fails if the $CONTINUENT_ROOT/tungsten directory already exists and is empty
  • Issue 414 - Parallel apply may stop replicating databases if channel number is adjusted after unclean restart
  • Issue 416 - Slave replicator stalls when parallel replication is enabled with on-disk queues
  • Issue 417 - Table rename filter with regex support
  • Issue 421 - Add filter to convert MySQL zero dates to null value
  • Issue 422 - Omit installation of parallel queues when 'none' is the replication type
  • Issue 424 - Filter to generate change rows for incoming transactions
  • Issue 425 - Enable Tungsten replication into Amazon RDS slave
  • Issue 426 - Add relative latency to enable users to tell when replicator is stalled or slave may be out of date
  • Issue 430 - Position reported as -1
  • Issue 431 - Implement ./ddlscan utility for flexible schema transformations
  • Issue 433 - NULL values ini extract_timestamp or update_timestamp cause NPE
  • Issue 434 - The xtrabackup-incremental script fails if $CONTINUENT_ROOT/backups/xtrabackup doesn't exist
  • Issue 440 - tungsten does not read variables with underscores
  • Issue 442 - BidiRemoteSlaveFilter fails with NullPointerException on filtered events
  • Issue 443 - shardfilter does not fail replication on wrong shard
  • Issue 445 - Tungsten installation fails with MySQL 5.6
  • Issue 446 - Tungsten installer MySQL 5.6 checksum awareness
  • Issue 448 - The xtrabackup methods ignore innodb_data_home_dir and innodb_log_group_home_dir
  • Issue 449 - SetToStringFilter to convert SET to a VARCHAR list
  • Issue 456 - Reduce the output of tools/configure and tools/configure-service
  • Issue 457 - Consistency checking fails on keyword-named columns

New in Tungsten Replicator 2.0.6 (Nov 6, 2012)

  • Issue 8 - Race condition when a DDL statement is immediately followed by a DML statement on the same object Durability
  • Issue 39 - Auto-commit DDL statements break block commit Durability
  • Issue 51 - PostgreSQL Slony Log Extractor: POC
  • Issue 90 - waitForAppliedSequenceNumber can return before the sequence number event was actually committed
  • Issue 129 - Certify MySQL->PostgreSQL support
  • Issue 168 - MySQL extractor is slow due to large number of calls to slow disk metadata commands
  • Issue 172 - './trepctl online -from-event' is dangerous
  • Issue 181 - Parallel replication does not use all channels
  • Issue 183 - Clean up code, unit tests, and build scripts to speed up builds and eliminate warnings and/or hung builds
  • Issue 238 - Implement an applier for Vertica data warehouse
  • Issue 256 - Master replicator unable to connect to MariaDB 5.2.9 after installation
  • Issue 291 - DROP TABLE loops in multi-master environment on MYSQL 5.5
  • Issue 299 - Installer support for AWS EC2 Linux
  • Issue 305 - Add installation support for logging slave updates
  • Issue 308 - Tungsten generates warning and skips prefetch on rows that contain MySQL "zero" dates of the form '0000-00-00 00:00:00'
  • Issue 317 - Filters should not drop parts of fragmented events, and even less the last fragment
  • Issue 328 - When reconnecting, slaves do not close sockets for connection to master if the master goes away/offline, leading to file descriptors leak.
  • Issue 330 - configure_deployment_handler.rb uses -C option for rsync, .so files fail to be copied during installation
  • Issue 333 - Tungsten does not correctly compute restart point after restore of hot backup on slave with parallel apply enabled
  • Issue 334 - Unable to restore the same backup to a slave and then a slave of a slave
  • Issue 336 - EnumToString filter URL is not properly configured with MySQL schema name
  • Issue 339 - In row replication, NULL blobs are badly handled and can lead to 'java.sql.SQLException: You need to set exactly X parameters on the prepared statement' when applied on slave
  • Issue 343 - Remove FSM (Finite State Machine) library source code and use JAR instead
  • Issue 346 - Add support for round-robin THL connection using a list of THL URLs instead of a single THL
  • Issue 355 - Oracle UPDATEs that change no columns lead to OptimizeUpdatesFilter removing too much
  • Issue 359 - Clean-up : remove classes in Commons project that are not needed by Tungsten Replicator
  • Issue 360 - Pipeline locks up when attempting to serialize #UNKOWN shard using disk parallel queue
  • Issue 362 - Installer support for round-robin THL connection list
  • Issue 364 - Option for applier to use column names from the underlying THL event
  • Issue 368 - tungsten-installer ability to install master extracting from a remote DBMS
  • Issue 372 - Add Oracle event ID type
  • Issue 373 - trepctl online -from-event does not work from slave
  • Issue 376 - OpenReplicatorManager doesn't have correct Method Descriptions
  • Issue 377 - Tungsten replicator generates a large number of build warnings
  • Issue 378 - Parallel replication locks up when waiting for #UNKOWN transaction to commit
  • Issue 380 - JMX configuration file misses the database port in its "resourceJdbcUrl"
  • Issue 381 - Timestamp value of 0 fails to replicate on MySQL when using row replication.
  • Issue 383 - Add support for an incremental xtrabackup method
  • Issue 384 - Tungsten generates invalid SQL when updating DROP TABLE statements to prevent replication loops
  • Issue 385 - Protobuf fails to deserialize events that are too large (more than 64MB)
  • Issue 387 - Replication has excessive latency in recent versions
  • Issue 389 - Replicator default heap size is too low
  • Issue 390 - replicator does not commit large RBR transactions Usability
  • Issue 392 - Replicator hangs on large transactions with multiple fragments
  • Issue 393 - installer accepts invalid option --log-slave instead of --log-slave-updates Issue 394 - Rename Tungsten commons library and Eclipse projects

New in Tungsten Replicator 2.0.5 (Apr 19, 2012)

  • Fixed:
  • Issue 17 - ./trepctl stop doesn't stop the Replicator process
  • Issue 67 - tungsten-installer does not detect already installed instances
  • Issue 73 - trepctl status does not show the host name and RMI port
  • Issue 74 - Null point exception when thl is used on an empty file after installing in --direct mode
  • Issue 88 - trepctl services does not display the latest sequence number with parallel replication
  • Issue 147 - Enable THL to regenerate itself after an error
  • Issue 148 - Enable backwards propagation of purge points
  • Issue 149 - Enable round-robin assignment of shards to channels.
  • Issue 152 - Support direct extract of MySQL binlog across network
  • Issue 153 - Improve usability of message logs.
  • Issue 174 - ./trepctl online -seqno X leads to deferred offline request
  • Issue 186 - Installation fails on master only.
  • Issue 189 - date 0000:00:00 does not replicate in ROW binlog format correctly.
  • Issue 200 - Tungsten Replicator removes log files before their contents are processed
  • Issue 201 - Purging all records from log using 'thl purge' hangs replication service
  • Issue 210 - tungsten installer does not recognize "~" as $HOME
  • Issue 217 - Regression: Installation fails with error concerning backup methods
  • Issue 218 - Extractor error during flush JMX call causes replicator to hang
  • Issue 220 - tungsten-installer --property can't insert an empty property
  • Issue 221 - thl list assumes 0 as lower seqno in every file
  • Issue 226 - Add 'version' to trepctl help
  • Issue 229 - Implement installation support for MySQL to MongoDB replication
  • Issue 231 - Installation does not support log fsync settings that are fully crash-safe
  • Issue 233 - Offline operation is very slow on slaves running parallel apply
  • Issue 235 - MySQL event extraction fails intermittently when using relay log downloading
  • Issue 239 - Improve DatabaseTransformFilter in order to handle more than one regular expression
  • Issue 240 - Installer should check replicator JMX ports are not boundIssue
  • 41 - Develop generic infrastructure for testing replication within and across DBMS types
  • Issue 245 - TableMapLogEvent are badly extracted for table with large number of columns
  • Issue 248 - OptimizeUpdatesFilter can fail with NPEIssue 249 - Docs: add which to prerequisites
  • Issue 251 - Slave restart may fail if the last applied transaction was from a filtered event
  • Issue 252 - CatalogManager does not correctly set the session log level
  • Issue 254 - ReplicateFilter gets NullPointerException if a filter string is omitted
  • Issue 255 - Add a new applier that can be used to prefetch data into database cache
  • Issue 257 - Replicator may fail to process binlog files that are more than 2 GB large
  • Issue 259 - Direct pipeline start-up after clearing logs fails
  • Issue 260 - Building with OpenJDK 7 Fails in DiagnosticWizard.java sun.management.ManagementFactory is not public…)
  • Issue 265 - Requirements are not complete (rsync missing)
  • Issue 267 - MySQL extraction fails on binlog file open when using using relay log downloading
  • Issue 269 - THLStorageCheck breaks multiple master installation
  • Issue 272 - Infinite loop in options parsing in configure-service
  • Issue 290 - Replicator fails to delete temp files on slaves when processing LOAD DATA LOCAL NFILE
  • Issue 295 - Using Load Data Infile into temporary table breaks replication
  • Issue 297 - Enable slave replicators to restart from a specific event ID on the master

New in Tungsten Replicator 2.0.4 (Feb 29, 2012)

  • Installation procedures have changed substantially. The old tungsten.cfg is not compatible with the new tungsten.cfg file format, which is JSON-based. The new installation uses ./tools/tungsten-installer, which is also a validation tool.
  • Installing a direct slave (useful for parallel replication) is simpler and straightforward. Taking over from MySQL native replication and handing back over to the old slave is also easy.
  • The trep_commit_seqno table includes additional columns. The table can be upgraded by running ALTER TABLE `trep_commit_seqno` ADD COLUMN `shard_id` VARCHAR(128) NULL DEFAULT NULL; ALTER TABLE `trep_commit_seqno` ADD COLUMN `extract_timestamp` TIMESTAMP NULL DEFAULT NULL;
  • There is a new trep_shard table that can be created by running CREATE TABLE `trep_shard` ( `name` varchar(128) NOT NULL DEFAULT '', `master` varchar(128) DEFAULT NULL, `critical` tinyint(4) DEFAULT NULL, PRIMARY KEY (`name`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8
  • The replicator logs are now integrated by a user.log, a more concise and user friendly log.
  • The installation tools can handle all values in the properties file. In addition to being a single-command installation, the procedure can also customize the replicator properties from the start.

New in Tungsten Replicator 2.0.2 (May 26, 2011)

  • Issue 1 - Clean up javadoc build warnings
  • Issue 2 - Configure script does not set up disk log
  • Issue 3 - Integrate Hudson build number to release tarball
  • Issue 5 - Replicator does not correctly assign #UNKNOWN shard when updates cross schema boundaries within a single transaction
  • Issue 6 - When using disk logs, thl list can show a NullPointerException error
  • Issue 11 - Add a timestamp column into trep_commit_seqno table in order to know when this table was updated
  • Issue 15 - Some records get lost in multiple master replication
  • Issue 19 - Thl unable to read disk log
  • Issue 20 - Need installation support for direct pipeline where master is on a remote host