Chapter 6. Putting It All Together

6.1.1. Data Model for RSS Classic

Here are the key parts of the RSS 0.91 DTD; it also incorporates the HTML 4.0 ISO Latin/1 character entities, which aren't shown here, and various other integrity rules that aren't expressed by this DTD:

<!ELEMENT rss (channel)>
<!ATTLIST rss
          version     CDATA #REQUIRED> <!-- must be "0.91"> -->

<!ELEMENT channel (title | description | link | language | item+
		| rating? | image? | textinput? | copyright?
		| pubDate? | lastBuildDate? | docs? | managingEditor?
		| webMaster? | skipHours? | skipDays?)*>

<!ELEMENT image (title | url | link | width? | height? | description?)*>

<!ELEMENT item (title | link | description)*>

<!ELEMENT textinput (title | description | name | link)*>

<!ELEMENT title (#PCDATA)>
<!ELEMENT description (#PCDATA)>
<!ELEMENT link (#PCDATA)>
<!ELEMENT url (#PCDATA)>
<!ELEMENT name (#PCDATA)>
<!ELEMENT rating (#PCDATA)>
<!ELEMENT language (#PCDATA)>
<!ELEMENT width (#PCDATA)>
<!ELEMENT height (#PCDATA)>
<!ELEMENT copyright (#PCDATA)>
<!ELEMENT pubDate (#PCDATA)>
<!ELEMENT lastBuildDate (#PCDATA)>
<!ELEMENT docs (#PCDATA)>
<!ELEMENT managingEditor (#PCDATA)>
<!ELEMENT webMaster (#PCDATA)>
<!ELEMENT hour (#PCDATA)>
<!ELEMENT day (#PCDATA)>
<!ELEMENT skipHours (hour+)>
<!ELEMENT skipDays (day+)>

In short, the DTD includes a wrapper that gives the version, one channel with some descriptive data, and a bunch of items. RSS 0.92 changes it slightly. Data length limits (which a DTD can't describe) are removed, and a bit more. If you're working with RSS, you should know that most RSS feeds incorporate at least a few of those 0.92 extensions and have your code handle the issues. And if you're generating an RSS feed for your web site, you'll want to know that many aggregators present the image as the channel's icon, along with the newest items and the text input box, to provide quick access to your site.

When you work with XML-based systems and SAX, one of the first things you'll want to do is decide on the data structures you'll use. Sometimes you'll have a pre-existing data structure that must be matched; in cases like this RSS code, you have the luxury of a blank slate to write on. I'm a big believer in designing appropriate data structures, rather than expecting some development tool to come up with a good answer; as a rule, a good "manual" design beats code generator output in any maintainable system. In the case of RSS Classic, simple structures like those shown in Example 6-1 can do the job:

Note that these classes didn't include any methods; methods can be added later, as application code determines what's really necessary. There are a variety of features that would be good to constrain this way, which you'll see if you look at the RSS specifications. Even pure "value objects" benefit from such internal consistency checks. For example, you may prefer to use beans-style accessor functions, but they would only complicate this example. (So would the class and field documentation, which has been deleted for simplicity.)

There's one type of code that is certainly needed but was intentionally put into different classes: marshaling data to RSS and unmarshaling it from RSS. Such choices are design policies; while it's good to keep marshaling code in one place, that place doesn't need to be the data structure class itself. It's good to separate marshaling code and data structure code because it's easier to support several different kinds of input and output syntax. Examples include different versions of RSS, as well as transfers to and from databases with JDBC. To display RSS in a web browser, different versions of HTML may be appropriate. Sometimes, embedding a stylesheet processing instruction into the XML text may be the way to go. Separate marshaling code needs attention when data structures change, but good software maintenance procedures will ensure that's never a problem.

6.1.2. Consuming and Producing RSS Parsing Events

Earlier chapters have touched on ways to marshal and unmarshal data with SAX. This section shows these techniques more completely, for a real-world application data model.

Example 6-2 shows what SAX-based unmarshaling code can look like, without the parser hookup. In this case it's set up to be the endpoint on a pipeline. This just turns infoset "atoms" into RSS "molecules" and stops. Note that it isn't particularly thorough in how it handles all the various types of illegal, or just unexpected, RSS that's found on the Web, although it handles many RSS Classic sites perfectly well. For example, the controls to skip fetches on particular days (perhaps weekends) or hours (nonbusiness hours) aren't usually supported, so they're just ignored here. With a more complex DTD, unmarshaling might not be able to rely on such a simple element stacking scheme; you might need to stack the objects you're unmarshaling and use a more complex notion of context to determine the appropriate actions to take.

If you think in terms of higher-level parsing events, rather than in terms of data structures, you might want to define an application-level event handler interface and package your code as an XMLFilterImpl, as shown in Example 6-3. This is the "atoms into molecules" pattern for handlers, as sketched in Chapter 3, "Producing SAX2 Events". In the case of RSS, both item and channel might reasonably be expected to be "molecules" that get reported individually as application-level events. If you report finer grained structures (like item) it might be it easier to assemble higher-level data structures, but we won't show that here.

A filter written in that particular way can be used almost interchangeably with the handler-only class shown earlier in Example 6-2. In fact it's just a bit more flexible than that, though it may not be a good pipeline-style component. That's because it doesn't pass the low-level events through consistently; the ContentHandler methods this implements don't pass their events through to the superclass, but all the other methods do. That's easily fixed, but it's likely that you'd either want all the XML atoms to be visible (extending the XML Infoset with RSS-specific data abstractions) or none of them (and use an RSS-only infoset).

Example 6-4 shows what the core marshaling code can look like, without the hookup to an XMLWriter or the XMLWriter setup. For simplicity, this example takes a few shortcuts: it doesn't marshal the channel's icon description or most of the other optional fields. But notice that it does take care to write out the DTD and provide some whitespace to indent the text. (It uses only newlines for end-of-line; output code is responsible for mapping those to CRLF or CR when needed.) Also, notice that it just generates SAX2 events; this data could be fed to an XMLWriter, or to the RssConsumer class, or to any other SAX-processing component.

Since this code implements the RssHandler interface shown earlier, an instance of this class could be assigned as the RSS handler for the XMLFilter shown here. That could be useful if you wanted to round-trip RSS data. Round-tripping data can be a good way to test marshaling and unmarshaling code. You can create collections of input documents, and automatically unmarshal or remarshal their data. If you compare inputs and outputs, you can ensure that you haven't discarded any important information or added inappropriate text.

6.1.3. Building Applications with RSS

One of the most fundamental things you can do in an RSS application is act as a client: fetch a site's summary data and present it in some useful format. Often, your personal view of a web site is decorated with pages or sidebars that summarize the latest news as provided by other sites; they fetch RSS data, cache it, and reformat it as HTML or XHTML so your web browser shows it. That is, the web server acts as a client to RSS feeds and generates individualized pages that you can read on and click on the latest headlines.

Example 6-5 is a simple client that dumps its output as text. It's simple to write a servlet or JSP that does this for a set of RSS feeds, formatting them as nice XHTML sidebar tables so that a site's pages will be more useful.[26]

[26]If you do this in a server, you should handle one very important task that's not shown here: cache the RSS data! Do not make servers fetch the summary before each page view. That makes for a very slow user experience and can overload remote RSS feeds.

There are two basic techniques to use to create such a cache. One is to put a caching proxy between your server and all the RSS feeds. The other is to write a page cache module, preferably one that uses HTTP "conditional GET" (the If-Modified-Since HTTP header field) to avoid excess cache updates. You can save RssChannel data or store channel information in a local database, as variants of the page cache technique.

One extremely important point shown here is this code uses a resolver to force the use of a local copy of the RSS DTD. Servers should always use local copies of DTDs. Some RSS applications got a rude reminder of that fact in April 2001, when Netscape accidentally removed the DTD when it reorganized its web site. Suddenly, those badly written applications stopped working on many RSS feeds! Of course, those that were properly set up with local copies of that DTD had no problems at all.

Besides servlets that present RSS data in HTML form to a web site's clients, another kind of servlet is important in the world of RSS applications: servlets that deliver a site's own RSS feed as XML. Servers often arrange that the current channel data is always ready to serve at a moment's notice. You've probably worked with sites that give you HTML forms to publish either short articles (web log entries or discussion follow-ups) or long ones (perhaps XML DocBook source that's then formatted). When such forms post data through a servlet, it's easy to ensure the servlet updates the site's RSS channel data when it updates other site data for those articles.

While the mechanics of such a servlet would be specific to the procedures used at a given web site, almost any site could use code like that in Example 6-6 to actually deliver the RSS feed. Notice the XML text is delivered with an encoding that any XML parser is guaranteed to handle, using CRLF-style line ends (part of the MIME standard for text/* content types), and this sets the Last-Modified HTTP timestamp so it supports HTTP caches based on either "conditional GET" or on explicit timestamp checks with the HEAD request.

As RSS 1.0 starts to become more widely supported and more RSS/RDF modules are defined, more clever RSS-based services will become available. For example, RSS aggregator services may begin to be able to dynamically create new channels with information filtered from many other channels. That is, you could be able to define a channel that the aggregator will fill with new articles on a particular topic, listed in any of several hundred RSS feeds. Today, you'd have to scan each feed yourself to do that. Such smarter services would also have better reasons to cache information. Today, such a service would have a hard time knowing which articles to remember while you were away on vacation, since there would be far too many articles to remember them all.