You are here:  » Processing speed


Processing speed

Submitted by johnrand on Fri, 2010-01-15 16:16 in

Just now, I am processing feeds at about 9800 records per hour. I am using feed formats. Is this what I should expect?

Submitted by support on Fri, 2010-01-15 16:24

Hello John,

Raw parsing shouldn't take that long, but ultimately it depends on the size of each record; and the intensity of the processing that is performed by your application. For example, when processing affiliate product feeds (which is a common application of Magic Parser) 10's of thousands of products can normally be inserted into a database in just a few minutes.

One test you could do to identify how long the parse is taking on its own is simply to return; from your myRecordHandler function immediately and not do anything; then the time the script takes (assuming it is not doing anything else) will be the raw parsing time...

Hope this helps,
Regards,
David.

Submitted by johnrand on Fri, 2010-01-15 20:32

Thanks David,

I checked our code, and found the problem (checking array elements with !empty instead of issset).

It is now VERY fast!

Thanks for your help.