Manage Learn to apply best practices and optimize your operations.

Syndicated content: RSS aggregators and more

Ed Tittel discusses the client side of RSS feeds and how one goes about grabbing or searching syndicated Web content, and then trying to make sense of its contents.

Building and syndicating Web feeds using either version of RSS (versions 1.0 and 2.0 were discussed in an earlier tip) or Atom (the next topic in this ongoing series on Web syndication topics) is all well and good. But on the client side, users need some way to decide what feeds to grab and read. They often benefit from additional tools to search what's out there (often collectively called "blog space" or the "blogosphere") to find content items of potential interest.

In this tip, therefore, we'll dig into the client side of the equation and talk about how one goes about grabbing or searching syndicated Web content, and then trying to make sense of its contents.

The key to the syndication usually comes from one or more aggregators. An aggregator is client software that uses a Web feed to "tune into" specific types of syndicated Web content that may be considered analogous to a broadcast channel for information—namely, a specific, identifiable source for certain kinds of information that centers around some particular topic or collection of topics.

Syndicated content might include Web logs (or blogs), podcasts, virtual logs (vlogs) or even mainstream mass media Web sites (like those from major new outlets like CNN, broadcast companies like ABC, NBC, PBS or the BBC, and so forth). A search aggregator, on the other hand, plays the same role in searching syndicated content that a typical search engine like Google or Yahoo plays in searching more typical types of Web content.

Once a user subscribes to some kind of syndicated content feed, the aggregator checks the source for new content at regular, user-controlled intervals to retrieve any new content or updates that might have appeared since the last check. This means that aggregators use a "pull" model, wherein the subscriber initiates the check and any subsequent downloads that might occur, as opposed to a "push" model, where the source initiates downloads when and as new content appears.

Aggregator features can make themselves available in a variety of ways. Portal sites like those for the major search engines (Google, Yahoo, MSN, and so forth) will often offer feed subscription or regular aggregator capabilities, as well as search aggregator functions. Many modern Web browsers, including Firefox, Mozilla, Opera and even the latest Internet Explorer 7 beta include built-in RSS aggregation. Jane Kim, program manager for RSS in IE calls it "initial support for discovery and reading RSS feeds," but the software itself calls it "Web feeds".

Some e-mail programs, such as Mozilla Thunderbird also include aggregator capabilities and plug-ins like NewsGator make similar or equivalent functionality available for Microsoft Outlook. Apple's iTunes acts as a podcast integrator. Even devices that include certain mobile phones and Tivo video recorders as well as the Linux-based MythTV environment also incorporate XML aggregators or plug-ins that deliver this functionality.

Among many other sources, the Web site offers a comprehensive listing of leading RSS aggregators and related tools and plug-ins. Check this out for pointers to some very interesting tools and technologies to help you and your users tune into this powerful and interesting source of "never-ending XML content."

About the author

Ed Tittel is a full-time writer and trainer whose interests include XML and development topics, along with IT Certification and information security topics. Among his many XML projects are XML For Dummies, 4th edition, (Wylie, 2005) and the Shaum's Easy Outline of XML (McGraw-Hill, 2004). E-mail Ed at with comments, questions or suggested topics or tools for review.

Dig Deeper on Topics Archive

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.