Join Massive CSV Files Locally Without Silent Data Loss
Merging two CSV files with millions of rows — like joining a transactions.csv against a customers.csv on customer_id — seems straightforward until you hit Excel's row limit or VLOOKUP's performance cliff. Excel becomes unresponsive around 500K rows, eventually showing 'Process not responding' before crashing entirely. Python pandas handles the scale but introduces a subtler failure: automatic schema inference. If one file has customer_id values like '00123', '00456' (parsed as strings preserving leading zeros) and the other has 123, 456 (parsed as integers), a LEFT JOIN produces Unexpected NULL for every row where the types don't match. You lose data silently with no error message. This tool lets you write explicit SQL JOINs with CAST operations to enforce type consistency, running entirely in-browser via DuckDB-Wasm.
Why this matters?
A fintech analyst joined a 3.2 million row payment transactions CSV with a 890K row merchant reference file. The transactions file stored merchant_id as zero-padded strings ('000458921'), while the merchant file used bare integers (458921). Pandas inferred these as object and int64 respectively, and the merge silently dropped 412,000 rows where the string comparison failed — no exception, no warning, just a result set that was 12% smaller than expected. The analyst only discovered the discrepancy when the CFO noticed total transaction volume was $18M lower than the payment processor's statement. Excel VLOOKUP would have taken hours and likely crashed; the SQL approach with explicit CAST(merchant_id AS VARCHAR) completed the join in 6 seconds with full data integrity.
The 3-Step Solution
Follow this streamlined workflow to transform your raw data export into a clean, analysis-ready dataset. Each step leverages our browser-based tools to ensure your sensitive data never leaves your device.
By following these three steps, you eliminate manual data wrangling, reduce human error, and maintain full GDPR compliance throughout the process.
Ready to clean your data?
100% local processing. Zero uploads. Blazing fast.
Recommended Tools
Related Workflows
How to Query Nested JSON API Responses with SQL Directly in the Browser
Modern APIs return deeply nested JSON that's painful to parse in spreadsheets. This developer-focused guide demonstrates converting API response dumps into queryable tables and running SQL with JOINs, aggregations, and window functions — all without spinning up a local database.
Reconcile Facebook Ads Spend with Shopify Revenue
Attributing Facebook Ads spend to actual Shopify revenue is a nightmare because Shopify exports Name (e.g., #1042) and Order ID, but completely strips UTM parameters or Campaign IDs from the standard order CSV. Meanwhile, your Facebook Ads export groups spend by Campaign ID and Ad Set Name. You cannot directly VLOOKUP these two datasets. This workflow walks you through extracting UTM tags from Shopify's Note or Tags fields, parsing them with regex, and executing a memory-safe local VLOOKUP to bridge ad spend and realized revenue without touching a cloud server.
Isolate True Net Processing Fees from Stripe Balance Transactions
Stripe's Payouts export only shows the lump-sum deposit hitting your bank account — it tells you nothing about the per-transaction fee composition. To calculate your real effective processing rate, you need the Balance_Transactions.csv file, but this export dumps charges, refunds, adjustments, dispute reversals, and Stripe billing fees into a single flat table with no hierarchy. The Fee column contains negative values for processing fees, but refund-related fee reversals and dispute charges are interspersed without clear grouping, making a simple SUM() wildly inaccurate. This workflow filters the Type column to isolate charge, payment, refund, and dispute rows separately, recalculates the net fee per original transaction by grouping on Source ID, and produces a clean summary showing your true blended rate.
Reconcile Stripe Disputes and Refunds Against Original Charges
Annual Stripe exports for mid-volume merchants routinely exceed 500,000 rows, mixing successful charges, partial refunds, full refunds, dispute creations, dispute wins, and dispute losses in one monolithic CSV. The Type column uses cryptic values like dispute, dispute_reversal, and refund without linking back to the original charge amount in the same row. You must trace each dispute or refund back to its Source ID to find the original charge row and calculate net revenue impact. This workflow filters the export to only dispute and refund event types, joins them back to original charge rows using Source ID as the foreign key, and computes the net realized revenue per order after all adjustments.