Personal tools

Replication:Replication Monitor 1.0

From Adapt

Jump to: navigation, search

Overview

The SRB Replica monitor is a simple webapp that will watch registered directories and ensure that copies exist at designated mirrors. The monitor stores enough information to know if files have been removed from the master site and when the last time a file was seen. In addition any action that the webapp takes on files is logged. The monitor does NOT do any type of integrity checking, this is the responsibility of additional components.

This project has been obsoleted by version 2.0 located here


Files


Quick Setup

1. Requirements
2. Create database
Create a new database called 'srbmon', grant permissions and setup table structure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2 to server version: 4.1.20

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create database srbmon;
Query OK, 1 row affected (0.01 sec)

mysql> grant all on srbmon.* to monitor@localhost identified by 'YOUR_PASSWORD';
Query OK, 0 rows affected (0.00 sec)

mysql> use srbmon;
Database changed
mysql> source WHERE_YOU_DOWNLOADED_SCHEMA.sql;
ERROR 1146 (42S02): Table 'srbmon.ACTIVITY_LOG_ENTRY' doesn't exist
ERROR 1146 (42S02): Table 'srbmon.ACTIVITY_LOG_ENTRY' doesn't exist
...
...
Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> exit
Bye

Change WHERE_YOU_DOWNLOADED_SCHEMA.sql to correspond to the location you downloaded the schema listed below. Set YOUR_PASSWORD to a password to use for this database.

3. Install webapp
Install tomcat on your server and verify it's working. Shutdown the server and place a copy of the srb-monitor.war file in your TOMCAT/webapps directory. Place a copy of the configuration file srb-monitor.xml in your TOMCAT/conf/Catalina/localhost directory. Open up the configuration file and edit the following settings.
...
    <Parameter name="hibernate.connection.username" value="monitor" />
    <Parameter name="hibernate.connection.password" value="YOUR_PASSWORD" />
    <Parameter name="hibernate.connection.url" value="jdbc:mysql://localhost/srbmon" />
...
...
    <Realm className="org.apache.catalina.realm.JDBCRealm"
        driverName="org.gjt.mm.mysql.Driver"
        connectionURL="jdbc:mysql://localhost/srbmon" 
            connectionName="monitor" connectionPassword="YOUR_PASSWORD"
        userTable="USERS" userNameCol="username" userCredCol="password"
        userRoleTable="USER_ROLES" roleNameCol="rolename"
    />
...
  • connectionPassword and hibernate.connection.password should be set to the password you used above
  • if your database is running on a different server, or you used a username other than monitor when creating your database, update the above sections accordingly

If you haven't already done so, you will need to place a copy of the mysql connector in your tomcat TOMCAT/common/lib directory so the monitor can use the database for authentication.

You can now restart tomcat and browse to http://localhost:8080/srb-monitor where localhost is the name of the machine you installed the monitor on.

4. Register your first collection
When the monitor starts up, you will see a status screen with no collections.
  • Click on SRB Configuration. You will be prompted for a username and password, enter 'admin' for both. Fill in the six blanks with an account that has read access to your collection. Click Add when finished.
  • Click on Collection Configuration. Enter a name of your collection, and choose the path in the SRB where you collection is located. When finished click Add.
  • Click on Add Replica on the left. Select a resource that will store your replica, and then select the destination path for your collection. Click Add when finished.
5. Start the sync
Click on the Status tab, click your collection, then click Synchronize Mirrors at the bottom. After a few seconds, you should see files in your collection listed, and the counts for replicated items start to increase.


Usage

There are five main sections to the replication monitor.

  • Status - view current status and browse collections
  • Collections - register new collections and replica's for those collections
  • SRB Configuration - Register accounts that will be used to access collections.
  • Event Log - detailed event list of everything the replica monitor has done.
  • Accounts - Control access to the replica monitor.

Status

The status screen shows an overview of all monitored collections. Clicking on a collection will bring up an expanded view of the collection.

Mon1-status-web.png
  • Collection Name: descriptive name of the collection, preceded with a red icon showing no syncing is in progress or a green icon showing a sync is actively running.
  • Site Status - State of the master site as of the last sync.
  • Total Files - Count of monitored files, this will not be accurate until after the first complete sync has finished.
Mon1-status-idle-sync-web.png
Mon1-status-active-sync-web.png
  • Stop / Start sync - Start or start the syncing of master site to replica sites.
  • Collection Configuration - Go to the collection configuration screen for this collection.
  • Browse - browse and display the current status of any file in the collection

SRB Setup

The replication monitor allows for different collections to use different SRB accounts to replicate data.

Mon1-srb-add-account-web.png

Collection Configuration

There are two parts to configuring a collection for monitoring in the replica monitor. First, you need to configure a master collection that will be used as the source of data. Second you need to configure a set of replica sites that are used to receive copies of data from the master site. There are a few simple requirements for the replica collections. They include

  • replica sites and master site must be federated.
  • Account listed in the master site is the one used for replication to all child sites. It must exist on all replica sites

A common practice that works for replicating is to create a second account rooted in the replica zone, give this account read access to the master collection, use this account for the master and replica account. This provides a degree of safety in that this account cannot change the original copy.

Mon1-collection-modify-master-web.png
  • Name - descriptive name for the collection
  • SRB Account - account that will be used to replicate this collection. Account must have read access to master collection
  • path - SRB directory containing the master collection.
Mon1-collection-modify-replica-web.png
  • SRB Account - account to use for browsing and selecting a resource only (not used during replication)
  • resource - destination resource
  • path - destination path.

Event Logs

The event logs track every action that occurs while syncing a collection. As the logs can grow quite large, there are a number of filtering options available. The beginning and end of every sync is recorded as well as any new files or new replicas that were created and any errors that may have occurred. For a new file, you will generally see 3 entries per file, an entry for a new master file discovered, and two entries for the beginning and ending of the replica creation.

Mon1-event-web.png
  • Errors - show only errors that occurred during transfer. This is any error from master site to replica
  • Missing Files - Logged when a master file has gone missing.
  • New Master Items - Logged when a new master file has been found
  • New Replica - show when a new replica was created.
  • Sync Start/Stop - show when syncs started and stopped, good for showing over time involved in syncing a collection
  • Collections - limit the displayed errors to a given collection.


  • Show per page: display 20, 50, or 100 log items per page
  • |< >| - start at beginning or end of logs
  • << >> - show previous or next page

Account Management

  • accounts-new-web-small.png:
Mon1-accounts-new-web.png
  • accounts-roles-web-small.png:
Mon1-accounts-roles-web.png

System Configuration

System configuration is done in the TOMCAT/conf/Catalina/localhost/srb-monitor.xml file for the webapp. This included database connection properties and some properties controlling logging and how replica sites are checked.

The settings that most will want to change are:

    <!-- Time after which a replica check is needed, in hours -->
    <Parameter name="edu.umiacs.checkTime" value="168" />
    <!-- Max number of running replica's, if set to 0 or blank, then no limit -->
    <Parameter name="edu.umiacs.maxSyncThreads" value="5" />
    <!-- Max number of attempts to copy a file if we get an error -->
    <Parameter name="edu.umiacs.maxRetry" value="5" />
    <!-- Should we schedule sync's or require them to be run manually -->
    <Parameter name="edu.umiacs.autoSync" value="false" />


Implementation

The Monitor consists of three parts, first a database backend that tracks all monitored files, configuration, and logging information. Second a sync thread that runs across each collection, looking for new items and logging any changes. Third an interface for configuring collections and viewing the results of a synchronization.

Replication Workflow

Logging

TODO

  • what happens when replicas are removed when a thread is running.
  • Checkpointing during sync of collections in case of restart or crash. This should be done per directory.
    • low priority, startup cost even in large (2m collections) is only a few hours.
  • Allow collections and replica's to have excluded directories
  • Highlight currently selected item in status window
  • sync partial directories (dearp)
  • remove individual directories from monitoring. (dearp)
  • Create report showing which items are not currently active. (dearp)
  • Fix ordering of items on status screen. (dearp)
  • Link errors to directory or file (dearp)
    • logging handles this, just need to finish gui. masterfile and replicafile in gui finished, replica site and master site needs done still.
    • Filter logs based on collection
  • login for admin functions (web.xml filter...), and accounts creation page (dearp)
  • Add stop replication button
  • when determining what replica to run, choose the one w/ the greatest time since last checked.
  • Fix bug where files don't get marked active if they are replicated
  • Check hourly for stuff that has passed it's check time, while respecting max run
  • get total file count prior to starting replica
  • Make interface IE proof
  • Move hibernate config into context.xml