Import & Migration

WooCommerce bulk import and export with WP All Import, WP-CLI and EDD migration

Nightly supplier syncs of a 250,000-SKU catalogue, an Easy Digital Downloads to WooCommerce migration that kept every active licence, and weekly feeds of 40,000 membership records. Each import is safe to re-run and can resume after a failure.

Client
Distributor, plugin shop and membership site (three clients)
My role
WordPress and WooCommerce developer
  • WooCommerce
  • WP All Import
  • WP-CLI
  • Easy Digital Downloads
  • EDD Software Licensing
  • WooCommerce Subscriptions
  • PHP
  • Server cron
SKUs imported
250,000

SKUs imported

full import time
6 hrs → 90 min

full import time

nightly delta sync
≈ 12 min

nightly delta sync

duplicate products in ≈ 180 nightly runs
0

duplicate products in ≈ 180 nightly runs

active licence keys migrated
≈ 4,200

active licence keys migrated

The challenge

Three different clients had the same problem: their data lived outside WordPress, and the import had to be safe to run twice, able to resume after a failure, and schedulable. A distributor had a catalogue of about 250,000 SKUs (variations, stock, price and tier pricing) that had to sync every night from a supplier ERP.

A plugin shop on Easy Digital Downloads needed its licensing to survive a move to WooCommerce without breaking a single active licence. A membership client received a weekly CSV feed of about 40,000 contact and entitlement records.

What I built

  1. 01

    WP All Import at catalogue scale

    Variation mapping combined by SKU, custom PHP functions for price tiers and supplier attributes, image sideloading, and a nightly scheduled import with an "only changed records" filter so 250,000 rows are not touched every night.

  2. 02

    Custom WP-CLI importer for the ERP feed

    Batched at 500 rows, resumable through a checkpoint option, idempotent through a unique key, with --dry-run and a per-batch log. Used where the client wanted no plugin dependency and full control over mapping.

  3. 03

    EDD Software Licensing

    Licence activation and deactivation hooks, updater integration through the EDD SL API for auto-updates, licence-to-order linkage, and a custom renewal reminder flow.

  4. 04

    EDD to WooCommerce migration

    Products became simple or variable products, legacy EDD payments became WooCommerce orders, and active licence keys kept their original expiry and activation counts. Customers were deduplicated and mapped to WC_Customer. EDD Recurring subscriptions were recreated as WooCommerce Subscriptions where the gateway supported it. It ran as a repeatable WP-CLI command with a --rollback that restored the pre-migration snapshot.

  5. 05

    Real server cron for scheduled imports

    WP-Cron is disabled and a server cron hits wp-cron.php, so nightly imports start on time on low-traffic sites.

  6. 06

    Exports for accounting and suppliers

    Scheduled CSV and XML exports for accounting and for the supplier's reconciliation, plus a "download my data" export wired to wp_privacy_personal_data_exporters.

Gotchas and how I fixed them

  • Re-imports create duplicates without a unique key

    WP All Import creates duplicates on re-import unless you set a unique key and use "Update existing" correctly. SKU is the right key for products. For custom post types, store a stable external ID as post meta and never use the title.

  • Variation mapping is its own tab

    Mapping parent attributes on the main tab without configuring the Variations tab produces a product with attributes but no variations.

  • Scheduled imports need a real cron

    At this volume DISABLE_WP_CRON plus a server cron hitting wp-cron.php is mandatory. WP-Cron's pseudo-cron on a low-traffic site will not run on time.

  • Import memory limits

    pmxi imports can pass memory_limit. Set it per import in the plugin's advanced options and run imports from the CLI where possible.

  • EDD 3.0 changed the data model

    Payments moved from wp_posts to the edd_orders and edd_order_items custom tables. A migration script written against EDD 2.x postmeta silently returns empty results on EDD 3.

  • Licence keys are not portable by default

    EDD SL stores activation counts and site URLs. Copying only the key means customers see "licence limit reached" on their live sites. Migrate the activation records as well as the key row.

  • Snapshot, dry run, rollback

    Never run a migration without a snapshot, a dry run and a rollback. That is the whole job.

Code from the build

Idempotent WP All Import: no duplicates, no wasted workphp
// Skip rows whose unique key already exists and hasn't changed.
add_filter( 'wp_all_import_is_post_to_create', function ( $continue, $import_id, $data ) {
    $sku = $data['sku'] ?? '';
    if ( ! $sku ) {
        return false; // never import a row with no key
    }
    $existing = wc_get_product_id_by_sku( $sku );
    return $existing ? false : $continue; // let the Update pass handle it
}, 10, 3 );

// After save: keep the external ID for future delta syncs.
add_action( 'pmxi_saved_post', function ( $post_id, $xml_node, $is_update ) {
    if ( ! empty( $xml_node['external_id'] ) ) {
        update_post_meta( $post_id, '_external_id', sanitize_text_field( $xml_node['external_id'] ) );
    }
}, 10, 3 );

Rows with a missing key are rejected and existing SKUs go through the update pass, so re-running an import cannot create duplicates.

Resumable batched WP-CLI importerphp
class My_Import_Command {
    public function __invoke( $args, $assoc_args ) {
        $batch   = (int) ( $assoc_args['batch'] ?? 500 );
        $dry_run = isset( $assoc_args['dry-run'] );
        $offset  = (int) get_option( 'my_import_offset', 0 );

        do {
            $rows = $this->fetch_rows( $offset, $batch );
            foreach ( $rows as $row ) {
                if ( $dry_run ) {
                    WP_CLI::log( "DRY: {$row['sku']}" );
                    continue;
                }
                try {
                    $this->upsert( $row );
                } catch ( Throwable $e ) {
                    WP_CLI::warning( "Row {$row['sku']}: " . $e->getMessage() );
                }
            }
            $offset += $batch;
            update_option( 'my_import_offset', $offset, false ); // checkpoint
            WP_CLI::log( "Imported up to offset {$offset}" );
        } while ( count( $rows ) === $batch );

        delete_option( 'my_import_offset' );
        WP_CLI::success( "Done at offset {$offset}" );
    }
}
WP_CLI::add_command( 'myplugin import', 'My_Import_Command' );

The checkpoint option makes a 250k-row import restartable after a timeout instead of a 6-hour restart.

Preserve EDD licence activations during migrationphp
// Copy the key AND its activations — otherwise live customers hit "limit reached".
$license = edd_software_licensing()->licenses_db->get_license( $old_license_id );

$new_key = $license->license_key; // keep the key stable for customers
update_post_meta( $new_order_id, '_edd_sl_license_key', $new_key );
update_post_meta( $new_order_id, '_edd_sl_expiration', $license->expiration );
update_post_meta( $new_order_id, '_edd_sl_activation_limit', $license->activation_limit );

$activations = edd_software_licensing()->get_license_activations( $old_license_id );
foreach ( $activations as $activation ) {
    // Re-register each site URL against the new licence record.
    my_migrate_activation( $new_key, $activation->site_name, $activation->site_url );
}

Most successful-looking migrations break here and the problem only shows up a week later in support.

Results

  • 250,000 SKUs imported; the first full import dropped from about 6 hours to about 90 minutes after batching, turning off per-row logging and removing a per-row remote image check.
  • Nightly delta sync takes about 12 minutes.
  • Zero duplicate products across about 180 nightly runs, using a unique key and the wp_all_import_is_post_to_create guard.
  • EDD to WooCommerce migration moved about 12,000 customers, about 8,500 orders and payments, and about 4,200 active licence keys, with no licence-support tickets in the first 30 days.
  • Import failures show up as a per-batch report instead of a white screen; time to diagnose a bad row fell from about 45 minutes to about 5.
  • The supplier reconciliation export runs in about 40 seconds, replacing a script that took about 9 minutes and was prone to timeouts.

Questions people ask

How do I import 250,000 products into WooCommerce without timeouts?

Batch the work, switch off per-row logging and drop per-row remote image checks. On one distributor catalogue of about 250,000 SKUs this cut the first full import from about 6 hours to about 90 minutes. After that, a nightly import that only touches changed records takes about 12 minutes.

How do I stop WP All Import from creating duplicate products?

Set a unique key and configure "Update existing" correctly. Use SKU for products and a stable external ID stored as post meta for custom post types, never the title. With that and a wp_all_import_is_post_to_create guard there were zero duplicates in about 180 nightly runs.

Can I migrate Easy Digital Downloads to WooCommerce without breaking licence keys?

Yes, if you migrate the activation records and not just the key. In one migration about 12,000 customers, about 8,500 orders and about 4,200 active licence keys moved over with original expiry and activation counts. There were no licence-support tickets in the first 30 days.

Why does my scheduled WooCommerce import not run on time?

WP-Cron only fires when someone visits the site, so low-traffic sites run late. Set DISABLE_WP_CRON and have a real server cron call wp-cron.php. At this import volume that is required.

WP All Import or a custom WP-CLI importer: which should I use?

WP All Import works well for most feeds. I write a custom WP-CLI importer when the client wants no plugin dependency and full control over mapping. It runs in batches of 500, resumes from a checkpoint, uses a unique key and supports --dry-run.