How can I count the number of items in an xml file?

Hiya,

Because the parsing is a serial process (one record after another); there is unfortunately no way of getting a record count "up front" without actually parsing the entire file.

If you are working with small feeds then it may not be too much of an overhead; but with large files it may take several seconds just to get the record count, and you would have to do this on every page.

The basic code to count records would be as follows:

My recommendation would be to design your pagniation without a requirement for knowing the total number of records (for example, displaying a "Next" link only rather than links to each page) as this will have far less of a performance overhead.

Hope this helps!
David.

Hi David,

Thank you for the prompt help. Using the information that you have provided, I have managed to count all of the items and paginate the results.

I do still have one problem in that the pagination is displaying 10 items on the first page, and then 11 items on all other pages. How can I correct the number of items displayed?

Here is the script that I currently have - you may notice that I have changed some of the descriptions of the variables.

Hi there,

Yes sorry about that - I've spotted the bug in my example code. The following line:

if ($counter < (($page-1)*$items_per_page)) return false;

should be:

if ($counter <= (($page-1)*$items_per_page)) return false;

I would also recommend that you provide your format string for the first call to MagicParser_parse() when you're counting the records, as this will save the autodetection from being carried out, in other worde just change your code where you call the parse function to this:

Cheers!
David.

Great David, That's just what I was looking for.

Here is my corrected code for those who wish to use it.

Note:
The code displays one large table with the table rows being repeated inside the table. You will need to change the xml elements xml|PRODUCTS/PRODUCT/ to whatever your xml elements are and you will need to insert your own table row data to suit the elements that you wish to display. The table is designed to be xhtml compatible, so you will need to alter some of the xhtml tags if it is to be html compatible. Where I have used css styles such as class="content-text", you will need to create your own styles for the display presentation.

David,

Just another thought. Is it possible to display the xml items in a set order such as lowest price first? If so, what would I need to do to the code to achieve this?

Hi David,

Great the pages display the correct number of items. However, I just noticed that the paging script that we have arrived at is actually displaying the last item from the previous page as the first item on a new page. Any ideas on what needs to be altered?

Also, is there any way to order the display of results by the price?

Hi,

Would you be able to double check that your paging code is the same as in the following test script. I couldn't spot an error so I just tried it, and it seems to be working correctly. Click here to view the live output from this script:

Sorry that I missed your earlier message about sorting by price. For the same reasons that the only way to count the number of records is to parse the entire file, in this scenario, your only option is to read the entire XML document into an array rather than displaying each item, and then sort the array by price, and then display items out of the array as per the current page.

However, as it happens you are reading the entire file anyway in order to get the record count; so that actually means you can achieve this without having to parse the file twice as you are doing at the moment.

Here's an example that will display the BBC News headlines sorted by headline. The code first parses the entire file into an array using the headline as the array key. Next, it uses ksort() to sort the array by key, and finally loops through each item in the array to generate the output. Click here to view the live output from this script:

Hope this helps!
Cheers,
David.

Hi David,

I managed to get the data to sort by price. However, when I have tried to page the results, the sorting by price has become unsorted. Can you please point me in the right direction as to where I am going wrong.

Here is what I have so far:-

Hi David,

I thought that I would clarify exactly what I am trying to do as it seems that I keep posting here.
I can paginate my xml feed, but the items are all mixed up as I wish to sort them by Price.
I can sort the feed items to display by Price, but the page displays 100 items.
I cannot get the two to work together to do what I want.

I have an xml feed with 100 items.
1. I wish to sort the items by Price.
2. I wish to display only 10 items per page.
3. I wish to display pagination as below.
<<PREVIOUS 1|2|3|4|5|6|7|8|9|10 NEXT>>

It is probably very simple to combine the two, but I just cannot get my head around it.

Hi,

I've tried to fixup your last modified version to apply the paging to the sorted set. Please note that I haven't been able to test this so it might not be quite there, but should give you a basis to work from.

What you need to do is combine the counter and the sorter into one parse; as all records have to be read in order to do the sort - there is no getting away from that fact because of the serial nature of XML.

Hope this helps...
Cheers,
David.

Hi Dave,

Thank you for the reply.
The script is better, but is still not working quite right.

The items are now being displayed sorted by price and can be paged, but approx 57 products are being displayed per page instead of 10 and each page seems to start with the following 10th product.

Instead of ten pages with ten products, I now have five pages showing 57 products and page 6 shows 7 products.

Any ideas where the problem lies?

Ah yes, the way I implemented this for you I simlated the record handler function mechanism, but the loop that called the function to build the display items array was not breaking out when the builder function returns true...

Instead of this:

...try...

That should do the trick.

Hi Dave,

Thank you again for the quick reply.
The modification has now paged the products at 10 per page. However, the pagination script is displaying the correct amount of 10 pages, but there are only 6 pages when the next page links are clicked. In order for you to help me achieve the display of 10 results per page over 10 pages, here is the feed that I am trying to parse.

Hi Dave,

Here is the latest script that I have.
I have commented the various parts to make it more reader friendly. I look forward to your help on resolving this problem. I would guess that most users of MagicParser will be trying to display xml/rss feeds using MagicParser and I believe that once sorted out, this will be a very useful script to provide as an example of a sorted and paged xml feed.

After further investigation, for some reason, items that have the exact same price are not being displayed. For example, there are 3 items with a price of 115.00 and only one item is being displayed in the paginated results.

Hi David,

Any chance that you could please have a look at this?

I have done a little more investigation.
The current script does not currently display all of the items and if any of the prices are the same, it only displays one of them. To do a simple test, I changed the price for the tdproductid which should all be unique. Upon testing this, all of the items were returned in the paging and all 10 pages had 10 items displayed.

How can I get the pages to display all of the items by price, even if they are duplicated prices. Can I display the items using the tdproductid and the price together?

Hiya,

The reason there is only one item with the same price is because during the count and sort process; the $sortedItems array is constructed using the price as the key, so there will only ever be one item with any given price.

To resolve this, independent arrays need to be constructed for every price, and then the price arrays merged before the script continues. To start with, the count and sort function needs to be as follows (PHP tags used for clarity - these are actually parts of the larger script above):

All that's been added there is the [] after $sortedItems[$item["PRICE"]]. What this does is create an array of every $item with the same price rather than just storing 1 item per price.

Then, more code is needed to sort the array. Instead of just:

Change this to:

If you follow through the code above you'll see how it works; i) sort the $sortedItems array as before; ii) loop through each item, which itself contains an array of items at the same price, iii) add each item to the $newSortedItems array, and finally iv) copy $newSortedItems back to $sortedItems for use in th rest of the code.

Hope this helps,
David.