What I want to do is put all my feeds on one server. Then parse the feeds on multiple accounts. I have feed X and want to recreate the feed on 2 other accounts but the feeds will only add certain records from feed X. I can pull in the data I want into a mysql database but I don't want the entire feed recreated just certain categories and then I want to make those files appear in a directory. Like /feeds/newfeed.csv. I can do the part where I import only certain products but not sure how to make the new feeds. I will use price tapestry to process those feeds.
Your right one :) Except the virtual hosts will be on another server.
My goal is that some feeds are huge, say 14mb. I only want one section for a niche site. I thought maybe using Magicparser to fetch the feed and then parse it into the smaller one and have it placed in the feeds directory. Then automate the imports into price tapestry once a week.
Hi Don,
It's reasonably straight forward to write a splitting script based. Can you give an example of the sort of rules you would like to split based on; in other words; what logic do you want to use (within myRecordHandler) to determine whether the record should go into feed A or feed B?
Cheers,
David.
I really do not need to split the feed. Say I have a feed with 10,000 items. I only want "My widgets". So I would extract "My widgets" to create this new csv file. For another site I might want "Your widgets" and create a "your widgets" only feed for.
I thought there was a thread on creating an RSS feed from a datafeed, but I can't find it. I thought it woud be similar to doing that.
Hi Don,
Yes - it would be similar to creating an RSS feed. Here's a script which should give you something to work from. This will parse the feed in $input and create a new feed $output, which will contain records from the input feed based on the rules that you can see inside myRecordHandler.
$output must of course be writable by PHP. The script uses PHP's FOREACH operator to create a new row in the output CSV file by looping through each field of the current record (assuming that the rules are met that determine whether this record belongs in the new feed). This script won't generate a header row, and I've used the PIPE character to separate the fields in the new feed which is nicely autodetected by Magic Parser / Price Tapestry.
<?php
$input = "feed.csv";
$output = "newfeed.csv";
function myRecordHandler($record)
{
global $fp;
// inspect $record to see if we want it in the new feed
if ($record["FIELD2"] == "My Widgets")
{
// create an array $fields of just the field values
$fields = array();
foreach($record as $k => $v)
{
$fields[] = $v;
}
// construct the line by glueing $fields with the pipe character
$line = implode("|",$fields);
// write $line to the output file terminated in new-line
fwrite($fp,$line."\n");
}
}
// open the output file for writing
$fp = fopen($output,"w");
// parse the feed
MagicParser_parse($input,"myRecordHandler");
// close the output file
fclose($fp);
?>
Note that the test assumes that the source feed does not have a header row. If it does, you might need to perform the test something like:
if ($record["Merchant"] == "My Widgets")Hope this helps!
Cheers,
David.
Thanks David, it looks like it worked. Imported 11 records into a new feed in a test. I really appreciate you taking the time to write that script.
Hi David / all
I'm a php newbie :(
I wasn't sure whether to repost in this thread or not but bascially I am trying to use the code you posted here with an affiliate windows feed and I don't want to filter the records at this conversion stage but rather once they are loaded into my database. So I have taken off the if statement below (and the closing tag!) and left eveything else as is but I think the script is running for longer than it should - could you tell me how I should be looping through if I remove this if part as I think simply cutting this off is incorrect?
if ($record["FIELD2"] == "My Widgets")
{Cheers
Andy
Hi Andy,
That should be all you need to do. If the script is running forever there may be some other problem. If you could post or email me your script i'll take a look for you...
Cheers,
David.
Hi Don,
I'm afraid I don't completely follow what you want to do here. Is the general idea that:
a) Download Feed X from affiliate network
b) Parse Feed X and recreate 2 separate feeds A & B into 2 different virtual hosts on the same server
c) Feeds A & B will then be imported into separate instances of Price Tapestry.
If that's the case; is it possible to use Price Tapestry's filtering system to exclude the records that you don't want in each case? This would save having to write any custom code to split the feeds prior to loading into Price Tapestry.
Cheers,
David.