Hi,
i try to parse yahoo news feeds with magicparser.
First, i took a look on the forum and used this source code :
<?php
header("Content-Type: text/plain");
require("MagicParser.php");
function myRecordHandler($record)
{
// This is where you write your code to process each record, such as loading a database
// You can display the record contents using PHP's internal print_r() function:
print_r($record);
// The following code will print out each field in your sample data:
print $record["ITEM"];
print $record["TITLE"];
print $record["LINK"];
print $record["GUID"];
print $record["GUID-ISPERMALINK"];
print $record["PUBDATE"];
print $record["DESCRIPTION"];
}
$dir = "files/";
$tmp = $dir.uniqid();
$url = "http://news.search.yahoo.com/news/rss?p=labor+union&ei=UTF-8";
$cmd = "wget --user-agent='My Test Script' -O ".$tmp." \"".$url."\"";
print "[".$cmd."]";
system($cmd);
MagicParser_parse($tmp,"myRecordHandler","xml|RSS/CHANNEL/ITEM/");
?>This one is ok....
Now... i'm french.... and i would like to parse the french feed...
i changed the url in the source to have :
<?php
header("Content-Type: text/plain");
require("MagicParser.php");
function myRecordHandler($record)
{
// This is where you write your code to process each record, such as loading a database
// You can display the record contents using PHP's internal print_r() function:
print_r($record);
// The following code will print out each field in your sample data:
print $record["ITEM"];
print $record["TITLE"];
print $record["LINK"];
print $record["GUID"];
print $record["GUID-ISPERMALINK"];
print $record["PUBDATE"];
print $record["DESCRIPTION"];
}
$dir = "files/";
$tmp = $dir.uniqid();
$url = "http://fr.news.search.yahoo.com/rss?ei=UTF-8&p=chien&fr=news_sb_hd&source=yahoo";
$cmd = "wget --user-agent='My Test Script' -O ".$tmp." \"".$url."\"";
print "[".$cmd."]";
system($cmd);
MagicParser_parse($tmp,"myRecordHandler","xml|RSS/CHANNEL/ITEM/");
?>But it doesn't work....
what's the problem ?
Regards,
Gate
Thanks for your answer.
I tried but it didn't work...
I've just tested this way :
<?php
header("Content-Type: text/html;charset=utf-8");
require("MagicParser.php");
function myRecordHandler($record)
{
// This is where you write your code to process each record, such as loading a database
// You can display the record contents using PHP's internal print_r() function:
print_r($record);
// The following code will print out each field in your sample data:
print $record["ITEM"];
print $record["TITLE"];
print $record["LINK"];
print $record["GUID"];
print $record["GUID-ISPERMALINK"];
print $record["PUBDATE"];
print $record["DESCRIPTION"];
}
$homepage = file_get_contents('http://fr.news.search.yahoo.com/rss?ei=utf-8&p=chien&fr=news_sb_hd&source=yahoo');
$filename = 'test.txt';
// Assurons nous que le fichier est accessible en écriture
if (is_writable($filename)) {
// Dans notre exemple, nous ouvrons le fichier $filename en mode d'ajout
// Le pointeur de fichier est placé à la fin du fichier
// c'est là que $somecontent sera placé
if (!$handle = fopen($filename, 'a')) {
echo "Impossible d'ouvrir le fichier ($filename)";
exit;
}
// Ecrivons quelque chose dans notre fichier.
if (fwrite($handle, $homepage) === FALSE) {
echo "Impossible d'écrire dans le fichier ";
exit;
}
echo "L'écriture dans le fichier ($filename) a réussi";
fclose($handle);
} else {
echo "Le fichier n'est pas accessible en écriture.";
}
MagicParser_parse($filename,"myRecordHandler","xml|RSS/CHANNEL/ITEM/");
?>The output doesn't work but i took a look to the file text.txt
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss/">
<channel>
<title><![CDATA[Yahoo! News Search Results for chien]]></title>
<link></link>
<description><![CDATA[Yahoo! News Search Results for chien]]></description>
<language>fr</language>
<copyright>Yahoo! France</copyright>
<ttl>5</ttl>
<image>
<title><![CDATA[Yahoo! News Search Results for chien]]></title>
<width>144</width>
<height>18</height>
<link></link>
<url>http://l.yimg.com/a/i/us/nws/th/rss_news_logo.gif</url>
</image>
<incremental xmlns="http://purl.org/syndication/history/1.0">false</incremental>
<item>
<title><![CDATA[Un chien tire une balle dans les fesses de son propriétaire]]></title>
<link><![CDATA[http://fr.news.yahoo.com/55/20100713/tod-un-chien-tire-une-balle-dans-les-fes-17baed7.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/55/20100713/tod-un-chien-tire-une-balle-dans-les-fes-17baed7.html]]></guid>
<pubDate>Tue, 13 Jul 2010 05:47:00 -0700</pubDate>
<description><![CDATA[Nouvelle-Zélande - Un homme a été amené à l'hôpital de Whangarei afin de se faire soigner pour une balle reçue dans la fesse gauche. Son chien qui voyageait en sa compagnie a sauté sur son fusil qui chargé s'est déclenché.]]></description>
</item>
<item>
<title><![CDATA[Un chien blesse un homme avec un fusil... Quatre mois de colocation avec des serpents venimeux]]></title>
<link><![CDATA[http://fr.news.yahoo.com/82/20100713/tod-un-chien-blesse-un-homme-avec-un-fus-caa4346.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/82/20100713/tod-un-chien-blesse-un-homme-avec-un-fus-caa4346.html]]></guid>
<pubDate>Wed, 14 Jul 2010 06:24:00 -0700</pubDate>
<description><![CDATA[REVUE DE WEB - Retrouvez toutes les infos insolites de l'été...]]></description>
</item>
<item>
<title><![CDATA[Connaissez-vous le "beauty Dog"?]]></title>
<link><![CDATA[http://fr.news.yahoo.com/78/20100714/tfr-connaissez-vous-le-beauty-dog-9e39cb8.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/78/20100714/tfr-connaissez-vous-le-beauty-dog-9e39cb8.html]]></guid>
<pubDate>Wed, 14 Jul 2010 00:24:00 -0700</pubDate>
<description><![CDATA[Il a 3 ans et demi le pelage noir luisant et s'est rendu dimanche 11 juillet au salon international du chien à Villepinte. Témoignage d'un chien de concours.]]></description>
</item>
<item>
<title><![CDATA[Kellan Lutz : son chien le protège de ses fans]]></title>
<link><![CDATA[http://fr.news.yahoo.com/74/20100713/ten-kellan-lutz-son-chien-le-protge-de-s-3362cef.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/74/20100713/ten-kellan-lutz-son-chien-le-protge-de-s-3362cef.html]]></guid>
<pubDate>Tue, 13 Jul 2010 01:25:00 -0700</pubDate>
<description><![CDATA[Le ravissant acteur de "Twilight" également égérie de la marque Calvin Klein pour les sous-vêtements vient de révéler dans une interview que l'attention constante...]]></description>
</item>
<item>
<title><![CDATA[Human Rights Watch presse le président philippin]]></title>
<link><![CDATA[http://cf.news.yahoo.com/s/capress/100713/monde/philippines_droits_humains_1]]></link>
<guid isPermaLink="false"><![CDATA[http://cf.news.yahoo.com/s/capress/100713/monde/philippines_droits_humains_1]]></guid>
<pubDate>Mon, 12 Jul 2010 21:01:00 -0700</pubDate>
<description><![CDATA[MANILLE, Philippines - Un chien de garde international des droits humains a pressé mardi le président des Philippines Benigno Aquino III de respecter sa promesse de campagne de mettre fin aux présumés assassinats commandités par l'État.]]></description>
</item>
<item>
<title><![CDATA[Le prince Harry : Il vous présente sa nouvelle moitié !]]></title>
<link><![CDATA[http://fr.news.yahoo.com/51/20100714/ten-le-prince-harry-il-vous-prsente-sa-n-0111c6b.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/51/20100714/ten-le-prince-harry-il-vous-prsente-sa-n-0111c6b.html]]></guid>
<pubDate>Wed, 14 Jul 2010 03:58:00 -0700</pubDate>
<description><![CDATA[Le prince Harry est un jeune homme terriblement attachant. Ce jeune royal a un côté rebelle que tout le monde adore ! Il est humaniste souriant amoureux d'une très jolie fille - bien qu'il y ait]]></description>
</item>
<item>
<title><![CDATA[La saga 'Toy Story' racontée par un de ses créateurs]]></title>
<link><![CDATA[http://fr.news.yahoo.com/64/20100714/tcu-la-saga-toy-story-raconte-par-un-de-968b4de.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/64/20100714/tcu-la-saga-toy-story-raconte-par-un-de-968b4de.html]]></guid>
<pubDate>Wed, 14 Jul 2010 03:36:00 -0700</pubDate>
<description><![CDATA[A l'occasion de la sortie de 'Toy Story 3' en salles le 14 juillet Lee Unkrich son réalisateur qui a aussi travaillé sur les deux précédents films de la trilogie des studios Pixar revient images à l'appui sur une aventure de quinze ans.]]></description>
</item>
<item>
<title><![CDATA[Les femmes du nord de la France auraient une plus forte poitrine]]></title>
<link><![CDATA[http://fr.news.yahoo.com/82/20100713/tod-les-femmes-du-nord-de-la-france-aura-caa4346.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/82/20100713/tod-les-femmes-du-nord-de-la-france-aura-caa4346.html]]></guid>
<pubDate>Tue, 13 Jul 2010 06:24:00 -0700</pubDate>
<description><![CDATA[INSOLITE - Une étude a comparé les achats de sous-vêtements par Internet des Français...]]></description>
</item>
<item>
<title><![CDATA[Secret Story 4 : robin s'est grillé !]]></title>
<link><![CDATA[http://fr.news.yahoo.com/67/20100713/ten-secret-story-4-robin-s-est-grill-3f728ab.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/67/20100713/ten-secret-story-4-robin-s-est-grill-3f728ab.html]]></guid>
<pubDate>Tue, 13 Jul 2010 11:09:00 -0700</pubDate>
<description><![CDATA[Plus besoin de feinter le faux couple : Robin a révélé son secret à Anthony ! Du coup les deux faux-amoureux Robin et...]]></description>
</item>
<item>
<title><![CDATA[Emmanuelle Seigner : "C'est la fin d'un cauchemar"]]></title>
<link><![CDATA[http://fr.news.yahoo.com/74/20100713/ten-emmanuelle-seigner-c-est-la-fin-d-un-3362cef.html]]></link>
<guid isPermaLink="false"><![CDATA[http://fr.news.yahoo.com/74/20100713/ten-emmanuelle-seigner-c-est-la-fin-d-un-3362cef.html]]></guid>
<pubDate>Tue, 13 Jul 2010 02:40:00 -0700</pubDate>
<description><![CDATA[Hier la justice Suisse a refusé l'extradition du cinéaste Franco-Polonais vers les Etats-Unis lui rendant ainsi par la même occasion sa liberté. Si...]]></description>
</item>
</channel></rss><!-- fe2.news.search.ac4.yahoo.com uncompressed Wed Jul 14 06:51:24 PDT 2010 -->I putted it on this page :
http://www.magicparser.com/demo
> with the option 1, it doesn't work (by uploading the file)
> with the option 3, just cut and paste the content, it works....
i dont understand...
Gate
Hello Gate,
Opening the file with mode "a" will append to the file, which will result in a broken XML document after the first time! It should really use "w" - change this line:
if (!$handle = fopen($filename, 'a')) {...to:
if (!$handle = fopen($filename, 'w')) {...and fingers crossed that should fix it...
All the best,
David.
I see... but when i'm testing i was always flushing all the content of the txt file.
Why after writing the txt file magicparser doesn't parse the file ? (and why it works when you give the complete rss code on the demo page ?)
Fred
Hi Fred,
Do you actually need the intermediate file - could you try it like this:
<?php
header("Content-Type: text/html;charset=utf-8");
require("MagicParser.php");
function myRecordHandler($record)
{
// This is where you write your code to process each record, such as loading a database
// You can display the record contents using PHP's internal print_r() function:
print_r($record);
// The following code will print out each field in your sample data:
print $record["ITEM"];
print $record["TITLE"];
print $record["LINK"];
print $record["GUID"];
print $record["GUID-ISPERMALINK"];
print $record["PUBDATE"];
print $record["DESCRIPTION"];
}
$url = "http://fr.news.search.yahoo.com/rss?ei=utf-8&p=chien&fr=news_sb_hd&source=yahoo";
MagicParser_parse($url,"myRecordHandler","xml|RSS/CHANNEL/ITEM/");
?>
Cheers,
David.
David,
i created the intermediate file to see if the rss was ok because when you try with your script (it was the first i used) you have a blank page. That's why i searched on the forum and find a similar bug with the uk yahoo feed... In this case the user created the intermediate file but it doesn't work for the french version anymore :s...
Fred :)
Hello Fred,
It looks like there are utf8 encoding errors in the feed. I can make it work using utf8_encode() which cleans the data to valid utf-8. Please try this:
<?php
header("Content-Type: text/html;charset=utf-8");
require("MagicParser.php");
function myRecordHandler($record)
{
// This is where you write your code to process each record, such as loading a database
// You can display the record contents using PHP's internal print_r() function:
print_r($record);
// The following code will print out each field in your sample data:
print $record["ITEM"];
print $record["TITLE"];
print $record["LINK"];
print $record["GUID"];
print $record["GUID-ISPERMALINK"];
print $record["PUBDATE"];
print $record["DESCRIPTION"];
}
$xml = file_get_contents("http://fr.news.search.yahoo.com/rss?ei=utf-8&p=chien&fr=news_sb_hd&source=yahoo");
$xml = utf8_encode($xml);
MagicParser_parse("string://".$xml,"myRecordHandler","xml|RSS/CHANNEL/ITEM/");
print MagicParser_getErrorMessage();
?>
Hope this helps!
Cheers,
David.
Oh David ! It works !!! I will test it more later, time to BBQ now ;)
Thanskssssssssssss
Fred
Hello Gate,
Could you try it with the following code, which also sets the correct character set (UTF-8):
<?phpheader("Content-Type: text/html;charset=utf-8");
require("MagicParser.php");
function myRecordHandler($record)
{
// This is where you write your code to process each record, such as loading a database
// You can display the record contents using PHP's internal print_r() function:
print_r($record);
// The following code will print out each field in your sample data:
print $record["ITEM"];
print $record["TITLE"];
print $record["LINK"];
print $record["GUID"];
print $record["GUID-ISPERMALINK"];
print $record["PUBDATE"];
print $record["DESCRIPTION"];
}
$url = "http://fr.news.search.yahoo.com/rss?ei=UTF-8&p=chien&fr=news_sb_hd&source=yahoo";
MagicParser_parse($url,"myRecordHandler","xml|RSS/CHANNEL/ITEM/");
?>
Hope this helps!
Cheers,
David.