Advanced Config Guide: Pagely Clone Tool

This article is part of our Advanced Configuration Series. We'll walk you through the steps of creating a cloning profile and running the tool from CLI or a web interface. We assume you are comfortable with using SSH command line and have a familiarity with basic PHP syntax to set up the cloning profiles. If you're ever unsure or need assistance, our Support Team is just a few clicks away! 

Note: You will also need to contact support for the initial configuration of this tool. Additionally, a test run from the command line will need to take place to ensure a new clone profile works as intended. This is a new feature and we'll be making incremental improvements to it. We'll do our best to keep this guide current as that happens.

Glossary

1. About the Pagely Cloning Tool
   1. Visual overview
   2. Video topics and Timestamps
2. Basic stuff
3. Configuring a cloning profile
4. Configuration Options
   1. Path or File Options
   2. Single Site Options
   3. Multisite Options
   4. Notification Options
   5. WP CLI Command Options
5. Using the clone tool
   1. From the one-click-cloning site
   2. From CLI over SSH

About the Pagely Cloning Tool

Pagely's Cloning Tool makes Pagely Sync more repeatable and more flexible. In addition to the capabilities already present in sync, you are able to define an entire workflow and then call it with a more simple SSH command or initiate the sync from a web interface.

This system is extremely flexible and even makes it easier to empower the less technical members of your team to perform complex operations on their own. Even if everyone on your team is able to use the standard pagely sync commands, being able to wrap all of the extra steps into a single command is a huge time-saver!

This article covers all you will need to understand the cloning profile, but for a visual representation you can take a look at the video below:

Video Topics and Timestamps

(Please note that we have changed the "example" directory to ".example", which makes it a hidden directory in linux)

• Creating a Copy of the Cloning Profile Example
• Required Predesigned Parameters
• Additional Predesigned Parameters
• Custom Parameters (CLI Commands)
• Ignoring Paths, Files, and Tables
• Parameters for WordPress Multisite
• Example Cloning Tool Parameters
• Cloning Tool Web Interface Showcase

The Basics

Before you can configure or use a clone:

1. Contact our Support Team to have the feature enabled.
2. Have your SSH Access configured and ready to go.
3. Make sure that your key has been added to your SSH agent.
• Mac/Linux
  1. To check for existing keys within your SSH agent:

     ssh-add -L

  2. If your key isn't present, you can add it with these two commands:

     eval "$(ssh-agent -s)"

     ssh-add -K ~/.ssh/id_rsa

• Windows/PuTTY
  1. Open PuTTY.
  2. Go to your connection settings and navigate to Connection > SSH > Auth.
  3. Check to ensure that your key is present. If not, you'll need to add it.
4. Make sure you are connecting to your VPS with SSH Agent Forwarding enabled:
   

   ssh -A client_EXAMPLEUSER@example.com

Return to Glossary

Configuring a Cloning Profile

Before you can run the actual tool you have to create a configuration file. This specifies the source and destination app IDs as well as any other parameters, like wp search-replace or various wp options that should be set after the sync is done.

Note: whole DB Syncing should generally only target non-production environments. The reason for this is a database sync is dropping each table as it imports the new data, which can cause random issues with the production site while the operation is in progress. If it becomes necessary then it is possible to target production for a database sync, but generally it's safer to target specific content to sync (such as posts) instead using plugins or custom scripts.

You will create a folder and config file for each cloning scenario (prod to stage, stage to prod would be two separate workflows to set up)

There is an example located at /data/s####/scripts/.example/conf.php. Make a copy of this by running the following:

cp -a /data/s####/scripts/.example /data/s####/scripts/mysite_prod-to-stage

Then edit /data/s#####/scripts/mysite_prod-to-stage/conf.php to specify the parameters.

We recommend creating your first config file with a couple of testing sites to be sure you have the right flow, then repeat for your actual use case.

Return to Glossary

Configuration Options

The configuration options only require the following parameters to be updated in order for it to work...

• source - the source applications absolute file path
• dest - the destination the source application will be clone to. The absolute file path should be provided here as well
  
  Note: If you type ll ~/sites you'll see the absolute file path of all your applications
• home and siteurl - destination applications home and siteurl.
• pagely_cdn_enabled - whether you want the cloned application to utilize PressCDN

Beyond this, there are many permutations of options available with this tool. It's designed to let you automate the steps you would usually be taking manually when copying a site. We've designed it in a way to facilitate updating options, copying files into place, running WP CLI commands, multisite blog name mappings, and a whole lot more. That degree of flexibility does require a little know-how about what has to happen for your site as well and our team will be happy to assist you in getting the right things dialed in. Just ask!

By default, these additional options are turned off as seen in the image below. A description of each option is below the screenshot as well...

You can sync uploads between environments, but they won't necessarily show up in the Media Library unless you sync the database as well. Instead of syncing uploads, we also have the ability to configure rules to serve the requested image from production if they don't exist under staging/dev paths. This method also saves a free disk space on your VPS.

Path or File Options

• copy - copy files that might be unique to an application or not specific to any WordPress application.
  Note: if the wp-config.php file is included in this, it MUST be placed in the /data/s#####/scripts/name-of-clone-profile. It must reside in the same directory as the conf.php you are editing.
• ignore-paths - ignores the specified path during the cloning process

Single Site Options

• only-recent - Will only sync the last 100 posts from wp_posts (and children) and matching rows to post_meta
• reset-target-db - Resets the target database, ensuring the source database has a clean slate. Useful when the source has a different table prefix, or the target has extra tables
• scheme - changes the protocol scheme from http to https
• skip-db - ignores the database and only clones files
• skip-file - ignores files and only clones the database
  Note: skip-db and skip-file could both be used. A use case for this is if you just want to run specific cli commands, option updates, file copies, etc.
• skip-plugins - will still sync plugin files, but will ignore direct queries that sometimes cause wp cli or pagely sync errors and/or conflicts. This is the same as the wp cli flag --skip-plugins
• skip-themes - will still sync theme files, but will ignore direct queries that sometimes cause wp cli or pagely sync errors and/or conflicts. This is the same as the wp cli flag --skip-themes
• ignore-tables - will ignore the specified tables.

Multisite Options

This is one of the biggest advantages to the Pagely Clone Tool. Pagely sync is not equipped to handle some of the granular settings that are necessary for multisite out of the box. These features handle the manual steps Pagely sync required with multisite applications

• multisite - enables the cloning tool for multisite sync
• blogmap - will search for the first argument, and replace it with the second argument
• limit-to-multisite - will limit the sync to only the specified subsites

Notification Options

Notifications can be sent to email addresses or Slack webhooks when a sync process starts, succeeds, or fails.

Notifications are configured by passing an array of notifications to the notification option. Here's an example:

'notifications' => [
  [
    'slack_webhook' => 'https://hooks.slack.com/workflows/...',
    'emails' => [
      'user@example.com',
    ],
    'notify_on' => ["start,fail"],
  ],
  [
    'slack_webhook' => 'https://hooks.slack.com/workflows/...',
    'notification_templates' => [
      "fail_slack_pretty" => 'Your clone $name has failed! Check out: $clonetool_url',
    ],
    'notify_on' => ["fail"],
  ],
]

Let's go over the different parameters:

• slack_webhook - Accepts a Slack webhook workflow URL.
• emails - An array of email addresses that the notification will be sent to.
• notify_on - A comma-separated list of when the notification should be triggered. Possible options are start, success, and fail.
• notification_templates - Optionally, you can define a template for the notification. Templates are named by combining the status followed by the template type.
  
  The available notification templates are the following:
  • start_email_subject
  • start_email_body
  • success_email_subject
  • success_email_body
  • fail_email_subject
  • fail_email_body
  • start_slack_pretty
  • success_slack_pretty
  • fail_slack_pretty

WP CLI Command Options

You must connect to the server with SSH Agent Forwarding enabled in order for the clone tool to work. This can be done by running:

ssh -A client_EXAMPLE@EXAMPLE.vps.pagelyhosting.com

Before the the predesigned custom options you'll see the following line...

This section will go uncalled, unless the array cli_cmds is provided options. You'll find the array towards the beginning of the file...

A basic use case for this example is if you want to clear object cache, we can append wp cache flush as a command...

Return to Glossary

Using the Clone Tool

This tool is pretty flexible. While it does involve a little bit of initial configuration, once you have it dialed in it's super easy to use and repeatable.

From the Web with 1-Click Cloning

Our Support Team will provide you with the URL to use in order to access the web interface. Once you're in, you'll see all of your workflows that you defined in the configuration section. Clicking clone for the corresponding workflow will get the process started. You can navigate away from the page after you start the process, it won't be interrupted if you do.

From CLI over SSH

You must connect to the server with SSH Agent Forwarding enabled in order for the clone tool to work. This can be done by running:

ssh -A client_EXAMPLE@EXAMPLE.vps.pagelyhosting.com

Once you have your configs set up the way you like, you can issue commands like:

• clone mysite_prod-to-stage

• clone mysite_stage-to-dev

• clone mysite_qa-to-stage

The names and the order of flows are completely arbitrary, you're welcome to go with what works best for your approach.

Return to Glossary