I have un-managed data 5 text and 4 csv file, each file length is 1gb+ and some has 10gb or 20gb data. i want to combine all the data with matching row and column and store in single file like sql/nosql and then some data sorting or suggest any easy way to do so(that i can access easy big data in few minutes or do search operations).
12 freelancer chào giá trung bình₹8672 cho công việc này
I will use BigQuery and compile your documents all in your data lake. You can use SQL to query and make joins and aggregations as you like and train machine learning models using sql with your data.