You are here:  » larges files parsing


larges files parsing

Submitted by ludo42 on Sat, 2007-05-05 10:37 in

Hello,

I make a french price comparator (i'm french).
I would parse larges files (500mo).
The files are in csv, xml or txt.
I have 3 questions :
Magic Parser can parse them in spite of their size?
Magic parser read the files lines by lines or he charges the file in entirety to parse him ?
How long is necessary for him for parse a 500mo file ?

Thanks
ludo.

Submitted by support on Sun, 2007-05-06 09:01

Hello ludo,

Magic Parser has no internal limitation upon the size limit of files that you can parse, but you may find that there are limitations set by your hosting company on how long your server is allowed to run a script for. Some stop in 30 seconds.

You mentioned you want to make a price comparison site, have seen our other product - "Price Tapestry" which is a basic framework for a price comparison site which uses Magic Parser (which is included)....

http://www.pricetapestry.com/

Also, in the support forum for Price Tapestry is information about using very large files and the possible problems (timeout etc.)...

http://www.pricetapestry.com/node/582

Hope this helps!
Cheers,
David.