WooCommerce bulk import and export with WP All Import, WP-CLI and EDD migration
Nightly supplier syncs of a 250,000-SKU catalogue, an Easy Digital Downloads to WooCommerce migration that kept every active licence, and weekly feeds of 40,000 membership records. Each import is safe to re-run and can resume after a failure.
- Client
- Distributor, plugin shop and membership site (three clients)
- My role
- WordPress and WooCommerce developer
- WooCommerce
- WP All Import
- WP-CLI
- Easy Digital Downloads
- EDD Software Licensing
- WooCommerce Subscriptions
- PHP
- Server cron
- SKUs imported
- 250,000
- full import time
- 6 hrs → 90 min
- nightly delta sync
- ≈ 12 min
- duplicate products in ≈ 180 nightly runs
- 0
- active licence keys migrated
- ≈ 4,200
SKUs imported
full import time
nightly delta sync
duplicate products in ≈ 180 nightly runs
active licence keys migrated
The challenge
Three different clients had the same problem: their data lived outside WordPress, and the import had to be safe to run twice, able to resume after a failure, and schedulable. A distributor had a catalogue of about 250,000 SKUs (variations, stock, price and tier pricing) that had to sync every night from a supplier ERP.
A plugin shop on Easy Digital Downloads needed its licensing to survive a move to WooCommerce without breaking a single active licence. A membership client received a weekly CSV feed of about 40,000 contact and entitlement records.
What I built
- 01
WP All Import at catalogue scale
Variation mapping combined by SKU, custom PHP functions for price tiers and supplier attributes, image sideloading, and a nightly scheduled import with an "only changed records" filter so 250,000 rows are not touched every night.
- 02
Custom WP-CLI importer for the ERP feed
Batched at 500 rows, resumable through a checkpoint option, idempotent through a unique key, with --dry-run and a per-batch log. Used where the client wanted no plugin dependency and full control over mapping.
- 03
EDD Software Licensing
Licence activation and deactivation hooks, updater integration through the EDD SL API for auto-updates, licence-to-order linkage, and a custom renewal reminder flow.
- 04
EDD to WooCommerce migration
Products became simple or variable products, legacy EDD payments became WooCommerce orders, and active licence keys kept their original expiry and activation counts. Customers were deduplicated and mapped to WC_Customer. EDD Recurring subscriptions were recreated as WooCommerce Subscriptions where the gateway supported it. It ran as a repeatable WP-CLI command with a --rollback that restored the pre-migration snapshot.
- 05
Real server cron for scheduled imports
WP-Cron is disabled and a server cron hits wp-cron.php, so nightly imports start on time on low-traffic sites.
- 06
Exports for accounting and suppliers
Scheduled CSV and XML exports for accounting and for the supplier's reconciliation, plus a "download my data" export wired to wp_privacy_personal_data_exporters.
Gotchas and how I fixed them
Re-imports create duplicates without a unique key
WP All Import creates duplicates on re-import unless you set a unique key and use "Update existing" correctly. SKU is the right key for products. For custom post types, store a stable external ID as post meta and never use the title.
Variation mapping is its own tab
Mapping parent attributes on the main tab without configuring the Variations tab produces a product with attributes but no variations.
Scheduled imports need a real cron
At this volume DISABLE_WP_CRON plus a server cron hitting wp-cron.php is mandatory. WP-Cron's pseudo-cron on a low-traffic site will not run on time.
Import memory limits
pmxi imports can pass memory_limit. Set it per import in the plugin's advanced options and run imports from the CLI where possible.
EDD 3.0 changed the data model
Payments moved from wp_posts to the edd_orders and edd_order_items custom tables. A migration script written against EDD 2.x postmeta silently returns empty results on EDD 3.
Licence keys are not portable by default
EDD SL stores activation counts and site URLs. Copying only the key means customers see "licence limit reached" on their live sites. Migrate the activation records as well as the key row.
Snapshot, dry run, rollback
Never run a migration without a snapshot, a dry run and a rollback. That is the whole job.
Code from the build
// Skip rows whose unique key already exists and hasn't changed.
add_filter( 'wp_all_import_is_post_to_create', function ( $continue, $import_id, $data ) {
$sku = $data['sku'] ?? '';
if ( ! $sku ) {
return false; // never import a row with no key
}
$existing = wc_get_product_id_by_sku( $sku );
return $existing ? false : $continue; // let the Update pass handle it
}, 10, 3 );
// After save: keep the external ID for future delta syncs.
add_action( 'pmxi_saved_post', function ( $post_id, $xml_node, $is_update ) {
if ( ! empty( $xml_node['external_id'] ) ) {
update_post_meta( $post_id, '_external_id', sanitize_text_field( $xml_node['external_id'] ) );
}
}, 10, 3 );Rows with a missing key are rejected and existing SKUs go through the update pass, so re-running an import cannot create duplicates.
class My_Import_Command {
public function __invoke( $args, $assoc_args ) {
$batch = (int) ( $assoc_args['batch'] ?? 500 );
$dry_run = isset( $assoc_args['dry-run'] );
$offset = (int) get_option( 'my_import_offset', 0 );
do {
$rows = $this->fetch_rows( $offset, $batch );
foreach ( $rows as $row ) {
if ( $dry_run ) {
WP_CLI::log( "DRY: {$row['sku']}" );
continue;
}
try {
$this->upsert( $row );
} catch ( Throwable $e ) {
WP_CLI::warning( "Row {$row['sku']}: " . $e->getMessage() );
}
}
$offset += $batch;
update_option( 'my_import_offset', $offset, false ); // checkpoint
WP_CLI::log( "Imported up to offset {$offset}" );
} while ( count( $rows ) === $batch );
delete_option( 'my_import_offset' );
WP_CLI::success( "Done at offset {$offset}" );
}
}
WP_CLI::add_command( 'myplugin import', 'My_Import_Command' );The checkpoint option makes a 250k-row import restartable after a timeout instead of a 6-hour restart.
// Copy the key AND its activations — otherwise live customers hit "limit reached".
$license = edd_software_licensing()->licenses_db->get_license( $old_license_id );
$new_key = $license->license_key; // keep the key stable for customers
update_post_meta( $new_order_id, '_edd_sl_license_key', $new_key );
update_post_meta( $new_order_id, '_edd_sl_expiration', $license->expiration );
update_post_meta( $new_order_id, '_edd_sl_activation_limit', $license->activation_limit );
$activations = edd_software_licensing()->get_license_activations( $old_license_id );
foreach ( $activations as $activation ) {
// Re-register each site URL against the new licence record.
my_migrate_activation( $new_key, $activation->site_name, $activation->site_url );
}Most successful-looking migrations break here and the problem only shows up a week later in support.
Results
- 250,000 SKUs imported; the first full import dropped from about 6 hours to about 90 minutes after batching, turning off per-row logging and removing a per-row remote image check.
- Nightly delta sync takes about 12 minutes.
- Zero duplicate products across about 180 nightly runs, using a unique key and the wp_all_import_is_post_to_create guard.
- EDD to WooCommerce migration moved about 12,000 customers, about 8,500 orders and payments, and about 4,200 active licence keys, with no licence-support tickets in the first 30 days.
- Import failures show up as a per-batch report instead of a white screen; time to diagnose a bad row fell from about 45 minutes to about 5.
- The supplier reconciliation export runs in about 40 seconds, replacing a script that took about 9 minutes and was prone to timeouts.
Questions people ask
How do I import 250,000 products into WooCommerce without timeouts?
Batch the work, switch off per-row logging and drop per-row remote image checks. On one distributor catalogue of about 250,000 SKUs this cut the first full import from about 6 hours to about 90 minutes. After that, a nightly import that only touches changed records takes about 12 minutes.
How do I stop WP All Import from creating duplicate products?
Set a unique key and configure "Update existing" correctly. Use SKU for products and a stable external ID stored as post meta for custom post types, never the title. With that and a wp_all_import_is_post_to_create guard there were zero duplicates in about 180 nightly runs.
Can I migrate Easy Digital Downloads to WooCommerce without breaking licence keys?
Yes, if you migrate the activation records and not just the key. In one migration about 12,000 customers, about 8,500 orders and about 4,200 active licence keys moved over with original expiry and activation counts. There were no licence-support tickets in the first 30 days.
Why does my scheduled WooCommerce import not run on time?
WP-Cron only fires when someone visits the site, so low-traffic sites run late. Set DISABLE_WP_CRON and have a real server cron call wp-cron.php. At this import volume that is required.
WP All Import or a custom WP-CLI importer: which should I use?
WP All Import works well for most feeds. I write a custom WP-CLI importer when the client wants no plugin dependency and full control over mapping. It runs in batches of 500, resumes from a checkpoint, uses a unique key and supports --dry-run.