My host has disbled url_fopen as my web sites have been receiving a large number of hacking bots.
Now my xml feeds parsed with Magic Parser do not work.
Is there a way around this problem?
Hi David,
Apparently, my host tells me that I should be able to use the curl option but I am having trouble following the example thread. Do you have a simple curl example where I can insert the xml url that I am using to test if curl is working ok on my site.
MagicParser_parse($url,"SortandCountItems","xml|PRODUCTS/PRODUCT/");
Hi,
I've never used CURL myself, but the following should work assuming that the code in the other thread is correct - I should have mentioned however that the first post is slightly wrong (the reason for the thread) - sorry about that. Try something like this:
<?php
function SortandCountItems($record)
{
}
$url = "YOUR_URL_HERE";
$ch = curl_init($url);
curl_setopt( $ch, CURLOPT_HEADER, 0 );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 0);
$xml = curl_exec ( $ch );
curl_close ( $ch );
MagicParser_parse("string://".$xml,"SortandCountItems","xml|PRODUCTS/PRODUCT/");
?>
Note in particular how this code uses the string:// handler that is built into Magic Parser, because with CURL you have loaded the remote XML document directly into a string rather than to a file on disk...
Hope this helps!
Cheers,
David.
Hi Dave,
Just tried the following and got a blank page. What am I doing wrong?
<?php
require $_SERVER['DOCUMENT_ROOT'].'/magic-parser/MagicParser.php';
function SortandCountItems($item)
{
echo "<p>
".$item["DESCRIPTION"]."
</p>";
}
$url = "MY FEED URL IS HERE";
$ch = curl_init($url);
curl_setopt( $ch, CURLOPT_HEADER, 0 );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 0);
$xml = curl_exec ( $ch );
curl_close ( $ch );
MagicParser_parse("string://".$xml,"SortandCountItems","xml|PRODUCTS/PRODUCT/");
?>
Hi,
Firsly you need to confirm that CURL is working properly and had downloaded your XML as expected. The easiest way to do this is to print it out using htmlentities() so that you can study it. Try this:
<?php
require $_SERVER['DOCUMENT_ROOT'].'/magic-parser/MagicParser.php';
function SortandCountItems($item)
{
echo "<p>
".$item["DESCRIPTION"]."
</p>";
}
$url = "MY FEED URL IS HERE";
$ch = curl_init($url);
curl_setopt( $ch, CURLOPT_HEADER, 0 );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 0);
$xml = curl_exec ( $ch );
curl_close ( $ch );
// MagicParser_parse("string://".$xml,"SortandCountItems","xml|PRODUCTS/PRODUCT/");
print htmlentities($xml);
?>
If that does't display your XML, it should at least give some indication of what has been returned by CURL and indeed whether it is working at all on your server...
Cheers,
David.
That returns an xml string to the browser.
That's good - but not sure why it's not working as it did before then. Do there seem to be any other characters than might have corrupted the XML - and also double check that it is the XML that you are expecting and that the format string looks correct (xml|PRODUCTS/PRODUCT/) for the document that it as returned.
If in doubt, one thing you could do is copy and paste the XML into the demo form on this website...
Cheers,
David.
I put the browser string into your demo and it returned the expected xml records.
Ah - my apologies;
I've just realised that you've been a Magic Parser user for over a year now - the string:// handler was only introduced quite recently! That's why it's not working!
If you download the script again it should then work fine.
Drop me an email if you've lost your registration code and i'll resend it for you.
Cheers,
David.
David,
Problem solved.
When I downloaded and updated Magic Parser, my scripts using curl now work ok.
How would I know in future if updates are made to Magic Parser?
Hi,
Glad to hear you're working. If there are any future updates to Magic Parser I will publish a changelog on the download page - but there isn't anything planned at this time, the script has been stable for several months now.
Cheers,
David.
Hi,
There are only 2 viable alternatives to using fopen() with a URL, however it is likely that you host will have them blocked also. The first is to use the CURL library. Another user has used this successfully as they also have fopen() for URLs blocked. You can see the sample code in the following thread:
http://www.magicparser.com/node/394
The second alternative is to exec() to a command line retrieval program such as wget, although again it is very unlikely that your server gives permissions to PHP to use this approach. What you need to do is create a subdirectory that has WRITE permissions for PHP, and then shell out to wget to download the URL into a temporary file (created in the subdirectory), and then delete the file afterwards. Here's an example of the BBC News headlines script but using wget to obtain the XML rather than using fopen():
<?phpheader("Content-Type: text/html;charset=utf-8");
require("MagicParser.php");
function myRecordHandler($item)
{
print "<h2><a href='".$item["LINK"]."'>".$item["TITLE"]."</a></h2>";
print "<p>".$item["DESCRIPTION"]."</p>";
}
print "<h1>BBC News Headlines</h1>";
$url = "http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml";
$tmp = "tmp/rss.xml";
$cmd = "wget -O ".$tmp." ".$url;
exec($cmd);
MagicParser_parse($tmp,"myRecordHandler","xml|RSS/CHANNEL/ITEM/");
unlink($tmp);
?>
The above script uses a directory called tmp/ which is expected to be in the same directory as your script, so if you want to test this, make sure that you create this directory first and make it "world writable" - you should be able to set the permissions through your FTP program...
Hope this helps,
Cheers,
David.