Extracting data from extremely large csv files

by abigfatcat   Last Updated June 13, 2019 03:06 AM

I have a 40gb csv file with over 60 million rows for data analysis. Each row has a unique identifier (some numbers). For example, the first row's unique identifier will repeat approximately 150,000 rows later.

I would like to have a method to run through the entire file, and extract rows with the same identifier and write them into new csv files. Is there a good, automated way to do that? Please note that the file is very large and excel has problems opening it.

Tags : csv excel


Related Questions


Updated December 20, 2017 20:06 PM

Updated August 06, 2015 15:02 PM

Updated May 01, 2018 23:06 PM

Updated August 28, 2018 16:06 PM