RSS for News Gathering

• 8 minute read

What is RSS

RSS is a way to get updates from a website, without visiting the site, on a feed reader of your choice. In technical terms, it’s an open standard format for a site to provide a stream/feed of content. There are tons of feed readers, but I use Feedbin.

A feed can be pretty much anything: a blog, a magazine, a newspaper, a twitter account, twitter search results, a subreddit, a reddit user, a newsletter, emails to you that match a filter, a podcast, a youtube channel, a google alert, an instagram account, any url - and of course much more.

Here’s why that’s cool:

  • Instead of you going to the content, the content comes to you. You can bring all your feeds together under one roof. No more “making the rounds” to a handful of sites. This is a radical inversion that puts you in control of the content.
  • Your information is organized by topic, not platform. Instead of checking reddit and then checking twitter, you’re checking your folder of art feeds or your folder of science feeds, which include content across subreddits, twitter accounts, blogs, etc.
  • No ads, no algorithm, no recommendations. With RSS, you get just the content. You get your youtube subscriptions without the recommendations, the trends, and the autoplay. You get your tweets without the algorithm (content comes chronologically), without the ads, and without the endless feed to scroll.

How to switch to RSS

It’s easy. Get the feed reader of your choice (I like Feedbin), and load in some of the content you like.

Most readers let you organize your feeds in folders. I particularly like my folder setup:

  • Aggregators - These are sites/blogs/etc that aggregate links (eg. Hacker News, Marginal Revolution, weekly twitter roundups, etc). I like to keep these separate since the content tends to be all over the place.
  • Twitter - All the twitter accounts that I particularly like.
  • News - I like to keep time-sensitive, raw reporting separate (eg. newspapers, press releases, TechMeme, fundraising roundups, etc). I have a few domain-specific news folders (eg. “world-news”, “venture-news”, etc.).
  • Analysis - This is for anything that’s not one of the above - mostly blogs. Like my news folders, I have them organized by domain.
  • I also have folders for a few narrow interests I have that aren’t big enough to have both news and analysis (eg. a deep tech field, a niche intellectual field).

It’s essential to establish a habit of checking your RSS reader at least daily. This is easy. Load in your favorite twitter accounts, youtube channels, and subreddits - then totally block twitter, youtube, and reddit. You will soon be firmly in the habit of checking your RSS reader.

Getting content into your reader

It sometimes takes a bit of up-front work to get an RSS with the content you want, because RSS support is spotty. But the up-front work is what separates passive content consumers from people who use the internet like a tool. Tools requires care.

  • RSS icon. Many sites that support RSS will have an RSS icon somewhere on the site (eg. the footer, the ‘news’ page). Click on that to be taken to the URL that you need to put into your feed reader.
  • Google around for it. Some sites support RSS but don’t have the RSS icon on their webpage, so you have to google around to find how its RSS support works. Examples:
  • The Wall Street Journal, The New York Times, and SEC all have dedicated pages with directories of their RSS feeds. Major news publishers post many dozens of articles daily, so subscribing to their main feeds will immediately overwhelm you. (For both NYT and WSJ, I subscribe to their “print edition” newsletter, which is just a list of headlines in the day’s paper, which I route to RSS. I find that I strongly prefer this once-a-day delivery.)
  • Reddit - Add .rss to the end of a subreddit, topic, or user. (Example #1, #2, #3).
  • Pinterest - Add /feed.rss to a user’s url or a board url. (Example #1)
  • Wikipedia - Wikipedia has great RSS support. You can get a feed of all new articles, all new changes, contributions from a user, changes to an article (notice that minor changes are hidden), changes to a file (eg. the [Ukraine war situation map]( 1,503 words

https://commons.wikimedia.org/w/index.php?title=File:2022_Russian_invasion_of_Ukraine.svg&action=history&feed=rss)), and much more.

  • YouTube - You can get a channel’s videos by RSS if you know their channel id. The channel id is often in the url on their channel page, but more “official” channels often don’t have the channel id visible in the url. This tool will give you the channel id.
  • Github - You can get a project’s releases, tags, and commits via RSS.
  • Scraping services. If a website doesn’t offer RSS support, you could scrape the webpage and serve the content in RSS format at a url that your RSS feed reader could access. There are a few quality, reliable services I’ve used (RSS.app and fetchrss.com) that offer this in a no-code interface.
  • Email. You can route emails to Feedbin by sending them to a unique email address (found here under “Newsletter Address”). If you get an email that belongs in Feedbin, create a filter that forwards emails from that sender to the Feedbin email address and auto-archives the copy in gmail. You can also filter emails to a specific address (eg. [email]+rss@gmail.com), so when you sign up for content you can route it to RSS by providing that email. In my opinion, separating content and correspondence is very important. Previously I would leave newsletters unread in my inbox because I intended to read them in full later, but this cluttered my inbox. Since I’ve made my RSS reader my home for all content, staying on top of email has been much easier.
  • Monitoring webpages. If the site doesn’t even have a newsletter (or the signal:noise in the newsletter is unacceptable), the last resort can be to monitor a webpage for visual or text content changes using visualping.io or changetower.com. Those services send updates to email, which you can route to your RSS reader. (I use this to monitor changes to the FDIC list of bank failures).

The true power of RSS

So now you have all your favorite content, and only your favorite content, in one place, organized chronologically in folders. Is it worth adopting a new tool just for that? Maybe. But I haven’t even told you about the true power of RSS yet.

The real power of RSS is that it’s a foundation that you can build on to easily improve your media diet. For example, there are probably tons of sites that you’d like to check periodically if only you could get into the habit. With RSS, you can effortlessly start checking every update from sites that you’d never otherwise check.

RSS helps me keep up with infrequently-updated blogs, some academic publications, and some aggregators that I wasn’t previously checking. But there are tons of other, less obvious use cases.

  • Organization press releases. Lots of organizations put out interesting/useful information, but we only hear about it if it’s picked up by a news organization. With RSS, it’s feasible to be directly monitoring a large number of organizations’ press releases. (Now the “news” page on random dry institutional websites becomes an exciting window into the organization; if I’m interested in an organization, I’ll often add their press releases to my RSS for a few months, just to get a feel for the pulse of the place). Here are some examples:
  • Search results. You can get updates to search results via RSS.
    • eBay - eBay offers RSS support for turning search results into an RSS.
    • Google - You can get new items in Google search results via Google Alerts emails.
    • YouTube - Using a scraping service, you can get YouTube search results via RSS.
    • Twitter - Using a scraping service, you can get Twitter search results via RSS.