Parse Large XML and CSV files

Submitted by Fatihpk on Thu, 2016-05-12 22:11 in Magic Parser

Hi there, we are using your tool and we are parsing xml + csv files from many websites.

We have more than 500.000 products in our system and when parsing from different websites we are adding the products at run-time to the MySQL database.

But on a strange way some of the account who are using CSV files with a next 'csv|59|1|34' Format-String we always get a Internal Server Error.

We try to get the XML and CSV remote but also after downloading to our own server.

What is your advice for these kind or solutions?

Can i email you a file what have trouble with?

Hi, If you are able to

Submitted by support on Fri, 2016-05-13 07:49

Hi,

If you are able to selectively include an alternative version of MagicParser.php for this format, and you don't require multi-line quoted text support it might be worth giving the Faster CSV version a go otherwise sure, email me either a link to download or attached (ideally zipped if large) and I'll check it out for you...

Cheers,
David
--
MagicParser.com

Hi David, Thank you for the

Submitted by Fatihpk on Fri, 2016-05-13 17:40

Hi David,

Thank you for the fast support ;)

Do you have a good example how to use the MagicParserFASTCSV.php

It would be great. I also email you just seconds a go the URL location file.

This file is always downloaded to our server before parsing.

Hi, Thanks for the links -

Submitted by support on Sat, 2016-05-14 07:41

Hi,

Thanks for the links - I'll check those out as soon as I am back online properly but just quickly regarding MagicParserFASTCSV.php, usage is identical, and it still includes full XML support, the only difference being that CSV parsing uses PHP's fgetcsv() function which does not support multi-line records, however as a trade-off it is considerably faster than parsing by character...