Looks like magicparser uses all global variables (not object oriented) and probably doesn't initialize or free memory if it parses several files or strings in succession.
Is there a way out of this.
I am looping through a lot of csv files and parsing them. parsing is successful, but after about 20 files or about 40,000 records, magic parser throws the below error.
Err:
Allowed memory size of 75497472 bytes exhausted (tried to allocate 11576 bytes) in /home/user/lib/MagicParser.php on line 2
It doesnt happen if files are all small and only a less number of files are processed in succession.
I guess it has to do with initialization of magic parser variables everytime parsing is started. since they are global variables.
I found quite a number of global variables.
global $MagicParser_xml_name;
global $MagicParser_xml_path;
global $MagicParser_xml_depth;
global $MagicParser_xml_length;
global $MagicParser_xml_current_record;
global $MagicParser_xml_current_key;
global $MagicParser_xml_record_path;
global $MagicParser_xml_record_path_len;
global $MagicParser_xml_done;
but donno which ones to initialize in the loop.
Any help is appreciated.
Thanks,
Harish
Hello Harish,
The script does initialise all variables on each call to MagicParser_parse(). Where this error is likely to occur is if parsing a very large file with an incorrect format string; or the file is corrupt and record or string delimiters are broken, causing Magic Parser to effectively attempt to read the entire file into a string variable.
Does this occur with a particular large file, which would indicate that this might be the problem?
Cheers,
David.