ruby - Import CSV in batches of lines in Rails? -
I am using a fast CSV V to import an uploaded file into the model, and this small file Is working very well, though when I try to import a large dataset (21,000 lines), it takes ages and I get the browser timeout on the live server.
This is my current working code:
logon = 0 presence. FCSV.new transaction (file: header = & gt; true). Do do Row | Line [1] = date. Published (line [1], '% m /% d /% Y') record = @ event. attendees.new (: union_id = & gt; line [0],: dob = & gt; ; Line [1] ,: gender => line [2]) If recorded, logcode + = 1 end and end
I would love to use the background process , But the user is required
Therefore, I was thinking that I should use and read only a small number of rows, set a counter, then update that Considering the progression of C type, then run the method again using the previous counter as the starting point.
I can not seem to see how to read fast CSVs to read only a certain number of rows and also set offset for the starting point.
Does anyone know how to do this? Or is there any better way to make it better?
Have you tried to use bulk import? When you insert 1000 lines for DB, you will get impressive performance improvements. Travel them for more details.
Comments
Post a Comment