RSS to JSON: convert any feed, then get only the new items

Four formats, one idea

Find a site's feed URL

Most sites declare their feed in the page head. This prints the declaration:

curl -s https://blog.cloudflare.com/ | grep -o '<link[^>]*rel="alternate"[^>]*>'

<link rel="alternate" type="application/rss+xml" title="Cloudflare Blog" href="/rss/">

If nothing is declared, try the platform patterns:

Convert a feed to JSON with Python

The widely used feedparser library reads RSS and Atom (including broken XML) and gives every format the same field names:

import json, sys
import feedparser

feed = feedparser.parse(sys.argv[1])
items = [
    {
        "title": e.get("title"),
        "url": e.get("link"),
        "publishedAt": e.get("published") or e.get("updated"),
        "id": e.get("id") or e.get("link"),
    }
    for e in feed.entries
]
print(json.dumps({"feed": feed.feed.get("title"), "items": items}, indent=2))

uv run --with feedparser python rss_to_json.py https://blog.cloudflare.com/rss/

Output on 2026-10-12, first item:

{
  "feed": "Cloudflare Blog",
  "items": [
    {
      "title": "Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash",
      "url": "https://blog.cloudflare.com/clef-faster-cheaper-multimodal/",
      "publishedAt": "Fri, 09 Oct 2026 18:27:34 GMT",
      "id": "01M4GV0DTKYJTCM5GPYBCME1T4"
    }, …

Note the date is still in RSS's RFC 822 style; use published_parsed to normalise it. To make this a monitor you also need: HTML stripped from summaries, JSON Feed support (feedparser does not read it), a store of item IDs already sent per feed, conditional requests with ETag and Last-Modified so unchanged feeds cost almost nothing, retries for 429 and 5xx, and a scheduler.

Read many feeds without code

1. Open RSS Feed Reader & Monitor on Apify.
2. Paste feed URLs or plain website URLs into Feed or website URLs, one per line (up to 5,000). For a website it reads the declared feed links, then RSS-looking links on the page, then common paths allowed by robots.txt.
3. Optionally add Keywords (whole words, launch* wildcards), Exclude keywords, Published after (24 hours, 7 days, 2026-10-01) or Max items per feed.
4. Click Start and export JSON, CSV or Excel, or use the Items and Podcasts and media views.

A row from a real run on 2026-10-04 with the website URL https://blog.cloudflare.com/ as input:

{
  "title": "Streamline: custom video pipelines with Cloudflare Stream and Workers",
  "url": "https://blog.cloudflare.com/streamline/",
  "publishedAt": "2026-10-02T16:13:45Z",
  "authors": ["Willi Geiger", "Taylor Smith"],
  "categories": ["Birthday Week", "Cloudflare Stream", "Containers", "Developer Platform", "Durable Objects", "Workers"],
  "summary": "Streamline demonstrates how to build long-running, continuous video processing pipelines by pairing Cloudflare Workers and Durable Objects with a containerized media engine.",
  "feedTitle": "Cloudflare Blog",
  "feedUrl": "https://blog.cloudflare.com/rss/",
  "feedType": "RSS 2.0",
  "inputUrl": "https://blog.cloudflare.com/",
  "discoveredVia": "link-alternate"
}

Podcast feeds fill the media fields: enclosureUrl, enclosureType (audio/mpeg), enclosureLength in bytes and durationSecs. A feed that cannot be read appears only in the SUMMARY record with a readable reason, for example DNS_ERROR: the domain name does not resolve.

Get only new items on a schedule

{
  "feeds": [
    "https://blog.cloudflare.com/",
    "https://github.com/apify/apify-sdk-python/releases.atom",
    "https://www.gov.uk/search/news-and-communications.atom",
    "https://feeds.npr.org/510289/podcast.xml"
  ],
  "onlyNew": true,
  "outputOnFirstRun": false,
  "keywords": [
    "security",
    "outage",
    "launch*"
  ],
  "stateStoreName": "my-feed-alerts"
}

• Only new items stores the IDs already seen per feed (items still in the feed plus 90 days of history, up to 5,000 per feed) in a named key-value store in your Apify account.
• Output current items on the first run off means the first run only remembers the backlog, so your first alert is not 500 old posts.
• Unchanged feeds are checked with a conditional request (ETag and Last-Modified) and return nothing.
• Give each independent monitor its own Memory name.

Worked example (2026-10-04). The default input (3 feeds) returned 40 items. A first monitoring run on 6 feeds returned 95 items; running it again straight away found all 6 feeds unchanged and returned 0 items.

Why not just "published in the last hour"? A date filter misses items that arrive late with an older date and repeats items when runs overlap. Comparing IDs returns every item exactly once, whatever the schedule.

Route the run's dataset to Slack, email, Google Sheets, Zapier, Make or a webhook with Apify's integrations.

What each row contains

Honest limits

• Only what the feed carries. Many feeds hold the newest 10 to 50 items and a summary. Older items and full article text are not fetched from the website.
• Blocked feeds are reported, not bypassed. It identifies itself as RSSFeedReaderMonitor/1.0, sends at most 2 parallel requests per site and uses no proxy. Feeds behind bot protection come back as http_error with the status code.
• Items without a date are dropped when Published after is set.
• Private and internal addresses are refused, and responses over 15 MB are skipped.
• Copyright stays with publishers. Linking back is usually fine; republishing full articles usually needs permission.

All input fields

Generated from the actor's live input schema (build 1.0.3).

Call it from code

Start a run and get the results in one HTTP request with the Apify API (your token is under Console, Settings, API & Integrations):

curl -X POST "https://api.apify.com/v2/acts/adamhaleworks~rss-feed-reader-monitor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"feeds":["https://github.blog/feed/"],"maxItemsPerFeed":10}'

The run's Output tab offers 2 ready-made views; every view exports to CSV, Excel, JSON or XML:

• Items: publishedAt, title, feedTitle, author, summary, categories, imageUrl, url
• Podcasts and media: publishedAt, title, feedTitle, durationSecs, enclosureType, enclosureLength, enclosureUrl, url

What it costs

With the planned prices, feeds that fail and items removed by filters or already seen would not be charged. The default 3-feed run would cost $0.0095, and 10 feeds checked every hour for a month with about 300 new items about $3.66.

FAQ