Merge Date-Range-Limited Shopify Exports Into One Master File
High-volume Shopify stores must pull data in monthly chunks due to the Admin UI's ~10,000 row export limit. But an order placed at 11:58 PM on March 31st that gets fulfilled after midnight will appear in both the March and April exports with different Fulfillment Status values ('unfulfilled' vs 'fulfilled'). Simply concatenating these files creates duplicate orders that inflate revenue. This workflow merges dozens of CSVs sequentially, identifies overlaps via the Name column, and keeps only the most recent version based on the 'Updated at' timestamp. It also flags schema drift if Shopify silently added or renamed fields between your January and June exports.
Why This Matters
A DTC supplement brand merged 14 months of orders in Excel via copy-paste, unknowingly creating 1,847 duplicate rows from orders updated across period boundaries. Reported revenue was inflated by $287,000—which flowed into their investor deck and triggered brutal questions during Series A due diligence. Manually resolving duplicates across 280,000 rows in Excel is a nightmare, and Google Sheets rejects imports exceeding 10 million cells. Let DuckDB-Wasm handle the merge and deduplication instantly.
Why Excel & Python Fail Here
The core problem is temporal overlap: Shopify's date-range filter uses 'created_at', but orders updated after export (e.g., fulfillment status changes) carry an 'updated_at' timestamp in the NEXT period. Both files contain the same order with stale vs fresh data. Excel has no native 'keep latest by timestamp' deduplication—VLOOKUP across 280K rows freezes. Python's drop_duplicates(keep='last') only works if files are loaded in chronological order, which users frequently get wrong.
| Criterion | Excel / Sheets | Python / Cloud | DataPrep (This Page) |
|---|---|---|---|
| Max Rows | ~150K (crashes) | RAM-limited | 1M+ via Wasm |
| Setup Time | Manual formulas | 30–60 min env setup | 0 seconds |
| Data Privacy | Local but fragile | Uploads to cloud | 100% in-browser |
| Cost | License fees | EC2 / SaaS $49+/mo | Free forever |
Step-by-Step Solution
We will use the CSV Merger to execute this entirely in your browser.
- 1
Upload all monthly export files in any order
Drag and drop all CSVs. The tool reads each file's headers and detects the schema version. Files do NOT need to be in chronological order—the tool uses 'Updated at' timestamps to determine recency, not file upload sequence.
- 2
Review overlap detection report
The tool identifies orders appearing in multiple files and displays a summary: total duplicates found, which files they span, and the timestamp delta. Orders with conflicting Fulfillment Status are highlighted for your review.
- 3
Export deduplicated master with schema-drift warnings
Download the clean master CSV. If any column was added or renamed between exports (e.g., 'Billing Country' → 'Billing Country Code'), a warning banner lists the discrepancies so you can update downstream automations.
Privacy Guarantee: Zero Bytes Uploaded
All processing happens in your browser via WebAssembly. Your data never touches a server. Try it: disconnect your Wi-Fi right now—the tool will still work.
Common Errors & Fixes
| Error / Symptom | Root Cause | Fix |
|---|---|---|
Revenue inflated by 5–15% after merge | Duplicate orders from period-boundary overlap counted twice in SUM() | Deduplicate by Order Name, keeping the row with the latest 'Updated at' value |
Column mismatch error on later files | Shopify silently added columns between exports (e.g., new 'Duty' field in 2025-Q2) | The tool auto-aligns schemas and fills missing columns with NULL—no manual header editing needed |
Frequently Asked Questions
Do I need to upload files in chronological order?
What happens if Shopify added a new column in June that doesn't exist in January?
Ready to clean your data?
100% local processing · Zero uploads · Blazing fast
Trusted by 2,400+ data teams · 18M+ rows processed monthly