The default Aggregator Drupal module does not work very well. There are several problems with the Drupal Core module, one of which we have not fixed in our version (i.e. the flatness of the item table.)
There is a list of the known issues and our comments and whether we fixed the problem:
| Problem | Solution in m2osw's version of Aggregator |
|---|---|
| Missing XML marker | The <?xml ... ?> marker is missing from some RSS feeds, add it as required |
| Spurious data | Some RSS feeds add spurious data before or after the <?xml ... ?> or the closing tag </rss> (the only valid thing after the closing root tag are comments.) |
| XML Stylesheet | The XML parser used does not like XML stylesheets; we remove them before parsing. |
| XML entities | RSS feeds include all sorts of characters that are not valid UTF-8 characters. We attempt to fix all those that we can fix. Most often we find the Windows 1252 apostrophe and different types of quotes. |
| Lone Ampersand | It is frequent to find a lone ampersand, instead of &. We fix those so the output becomes a valid &. |
| Date timezone | If we find "UT" instead of "UTC" in a date, change it. This happens and would completely fail parsing of the date. |
| Empty XML files | Some RSS feeds return an empty file instead of nothing when there was no additions since last check. |
| Large number of Blocks | In Core, each RSS feed added to the Aggregator adds a corresponding block. With this version, you must turn on a flag or the block is not made available, making the list of blocks a lot shorter if you have a really large number of RSS feeds. |
| Open in the same window | By default, the Core Drupal aggregator creates links that open all the destinations in the same window. Our version opens the links in a new window, meaning that your website stays open when your users visit that destination link. |
| Lack of logs | We enhanced the log capability by adding some watchdog() calls and also adding parameters so you know the name of the RSS feed that generated problems. |
| Limit the GUID | The database limits the item GUID to 255 characters: VARCHAR(255). We clamp that parameter to make sure we don't get an error (MySQL auto-clamps.) Although this could cause problems with duplicates, it doesn't seem to ever happen. |
| Limit author and title fields | Like the GUID, the author and title fields are limited to 255 characters. These two fields are clamped when saved in the database. |
| Deleting old items | Somehow, the Core Aggregator module doesn't properly get rid of older items. We palliate to that problem by adding a loop over all the existing items. This can be slow the first time if you already have many items in your database. Afterward, it should not be that bad since it only deletes the few old items. |
| Hidden feed identifier | With Core, the feed identifier can only be known by editing the feed. It is at times practical to know the identifier (i.e. to use it in insert_view) without having to click on edit or at least look at the edit link in your status bar. We added an ID column. |
| Default update time | The default update time is set to 1h which seems extremely frequent to us. Most RSS feeds do not get updated more than once a day (when not once a week!) So by default we set the update time to 1 day. |
| Never delete items | Support the concept of never deleting anything from your site. Although this is not recommended (it grows real fast to a size that makes your site slowdown...) it works. Choose the option "Never" in the Delete every feature. |
We also added support for transforming RSS items into nodes. There are many features in that regard. This is done at the time RSS items are retrieved, and for sites that already had many items, it can be applied to existing items via CRON.
Finally, we included an attempt in using Taxonomies instead of the usual Categories (only in the nodes created by the module.) At this time, the taxonomy items must be name the same as the aggregator categories.
You can download our version by clicking on the tarball below.
IMPORTANT NOTES
If you find any problem, don't hesitate to post a comment below, email us, or register and post a ticket. Thank you.
You can also have a look at Drupal issue #350667.
| Attachment | Size |
|---|---|
| aggregator-6.20-m2osw-1.1.tar_.gz | 23.19 KB |
| aggregator-6.20-m2osw-1.0.tar_.gz | 23.12 KB |