Feed Readers

There are 71 posts filed in Feed Readers (this is page 1 of 8).

Making your own media

A few days ago, Dave Winer talked about “making your own media”, meaning that individuals could create lists of sources and distribute them. Dave called for Democratic podcasts, but he has a podcast “river of news” site with multiple podcasts. I thought – why not create a river of news site for some other list of sources as an example?

I recently became aware of a blogging challenge called Blaugust (during the month of August, natch!). On their media page, they had a link to an OPML file of all the participants. I copied that file, added it to my River5 subscription list, and created a river of news site for that list of bloggers. I have instructions on how to do this for yourself – so get busy!

Updating my River5 install

I have had some problems recently in getting the web reader feature in my River5 RSS reader install to work. Since I recently revised my main subscription list to create a blogroll OPML file, I decided it would be good to perform a clean install and start fresh with a smaller set of feed lists. I am recording the following steps to document my work.

  • Completed backup of current river5 folder from server
  • Saved my latest feed list as readinglist.txt
  • Found post about River5 callbacks, could not find my callback script
  • Looked at artcasting site, saw how I used a modified form of riverbrowser.js to display the pictures
  • Deleted river5 install on server
  • Uploaded clean River5 install from Github to river5 folder on server
  • Updated package.json to recent changes (allowing latest Node.js version)
  • Updated config.json to current content (removing initial tab setup from Github, setting max number of items in a river, setting podast download flag to FALSE
  • Updated lists folder
  • Added exampleRiver4Callback.js from C:\NodeApps\Rivers to callbacks\addToRiver folder on server
  • Did npm install, no issues
  • Did forever stop river5.js (had to, since server would not allow me to run new app)
  • Did node river5.js, watched startup in Putty, saw web view of lists
  • Did forever run river5.js
  • Checked my reading list app, saw new river file, all is well!

Using ChatGPT to convert a list of feed URLs to OPML

After seeing the interest in blogrolls by Dave Winer, I decided to spend some time updating my RSS subscription list and learning about using ChatGPT as a programmers’ assistant at the same time. I use the River5 feed reader, along with a single page web app to display my feeds. River5 supports having a feed list as a text file with one feed URL per line. I wanted to see how well ChatGPT could create an OPML subscription list from this text file.

Here was my first prompt:

Create a Javascript program to run under Node.js. The program should take as input a text file that contains one URL on each line. The program should output an OPML file with the URLs added to the outline such that the file can be uploaded to a feed reader that uses OPML as an input source for subscriptions. In addition, create a package.json file for this program.

This created an app which ran without an error, and created a basic OPML file. Each element had the type attribute set to “rss”. However, the text and xmlUrl attributes were both set to the feed URL. I decided to experiment to see how much further ChatGPT could go.

Here was my second prompt:

Create a Javascript program to run under Node.js. The program should take as input a text file that contains one URL on each line. The program should output an OPML file with the URLs added to the outline such that the file can be uploaded to a feed reader that uses OPML as an input source for subscriptions. The program should also fetch a copy of each URL, and if a feed is available, get the value of the <title> element within the <channel> element and set the text attribute of each entry in the OPML file to the <title> element for that feed/URL. If the feed is not available, set the text attribute of each entry in the OPML file to the URL value. In addition, create a package.json file for this program.

I was more specific here, asking that each feed URL be read to try to get the title of the feed and set the text attribute to the feed title. This version had a small programming error, but I easily fixed it, and the script was able to run without a problem. The output OPML file was able to fill in a lot of the titles, but some feeds still had problems (in particular, Blogspot blogs and other blogs using Atom instead of RSS). Also, this version created a separate “_attributes” element within the outline element, which did not have the same look as an example OPML file I reviewed.

Based on my review, I created a third prompt:

Create a Javascript program to run under Node.js. The program should take as input a text file that contains one URL on each line. The program should output an OPML file with the URLs added to the outline such that the file can be uploaded to a feed reader that uses OPML as an input source for subscriptions. The program should also fetch a copy of each URL, and if a RSS feed is available, get the value of the <title> element within the <channel> element and set the text attribute of the entry in the OPML file to the <title> element for that feed/URL. If an Atom feed is available, get the value of the <title> element and set the text attribute for that entry in the OPML file to the <title> element for that feed/URL. If the feed is not available, set the text attribute of the entry in the OPML file to the URL value. Each <outline> element should contain the following attributes: type = rss, htmlUrl = URL of the feed, xmlUrl = URL of the feed, text = value of the <title> element within the <channel> element if the feed is in RSS format, or the value of the <title> element if the feed is in Atom format. In addition, create a package.json file for this program.

This version went back to creating elements within the OPML file as single XML elements with all the attributes contained within the element. I tried some further refinements of this prompt, but had some issues with the generated code, so I then began manually iterating on the code generated by the third prompt. I had a WordPress feed which was redirecting to google.com, so I added some error checking if the data read from the feed was undefined. I also did some experimenting with the Blogspot feeds to find the correct HTML element to get the title of the feed and the site URL. Finally, I did an overall cleanup of feed URLs in the input file during my script troubleshooting.

Eventually, I was able to get all the data for the Blogspot feeds read correctly. The remaining Atom feeds were all generated from custom code, so there was too much variation to be able to easily get the title and the site URL from those feeds. I then hand-edited the output OPML file to add the remaining data I wanted.

Lessons learned:

  • It was easy to experiment with different prompts
  • Adding more detailed would refine the code generated by ChatGPT
  • For my app, there was a point of diminishing returns (got harder to specify the desired behavior – but could be due to lack of experience)

I have uploaded the app to this Github repo – give it a try!

Not just a blogroll – it’s a feedroll

In mid-March 2024, Dave Winer rolled out a new feature on Scripting News – a blogroll with dynamic content. I think it is interesting, but it is more than a “traditional” blogroll, in that it is not just a list of sites, but is presenting content from those sites, and ordering them in terms of the sites most recently updated at the top of the list. Blogrolls of the past were a static list of sites. In his podcast announcing the rollout, Dave Winer comments that future development directions will be up to users (about 8 minutes into the podcast), as it should be.

Until recently, though, it was unclear how other users would be able to access this feature. However, some one at Automattic developed a WordPress plugin to display the Dave Winer blogroll on a WordPress site. Yesterday, the first new user (Doc Searls) announced that he had the blogroll plugin working on his site.

I think it is great when people are adding features to their sites. I also think it is great that Dave Winer is working to migrate his ideas to other platforms like WordPress, which have a large base of users. The base technology of this feature is based on the FeedLand service, so there is still a linkage between Dave Winer’s tool and deployment of this feature – something for users to keep in mind.

My last observation on this is perhaps a subtle one, but still significant. Following comments from Dave Winer over the years about RSS, and that new format creators should use a different name than RSS, I think there should be a different name for this feature, as it is a significant enhancement from traditional blogrolls. When I looked at Doc Searls’ implementation, I saw that the title of his was “Feedroll”. I think this is a great name for this new feature – let’s use it!

What is “interop” for social media apps?

I have been busy lately, but something Dave Winer wrote recently about interop bothers me, and now I have a little time to discuss it. Dave said this on December 13th:

Andrew Hickey is one of my favorite bloggers, and it’s somewhat weird because..

  • He blogs on Bluesky but it works because..
  • John Spurlock added RSS 2.0 feeds for Bluesky, built on their API, I am able to subscribe Hickey’s observations in FeedLand, because..
  • Of course FeedLand understands RSS.

And it all happened without getting complicated. No federation needed. Just plain old RSS.

from https://feedland.blog/2023/12/13/the-power-of-open-formats/

So – let’s break this down. Someone (in this case, Andrew Hickey) is using the social media app/service, Bluesky. Another person (John Spurlock) has created a web service that uses the Bluesky API to allow a user of the web service firesky.tv to create on demand an RSS feed of posts for a Bluesky user. This RSS feed can then be read by any feed reader supporting RSS. In the post, Dave Winer links to the feed for Andrew Hickey’s account as presented by his feed management system/application, FeedLand. The display of the items in the feed are attractive and easy to read. However, since the feed is public, anyone could create a similar display (like this one). To me, this demonstrates the interoperability of RSS – anyone can take an RSS feed, consume it however they wish, and present the feed however they wish (either privately (like someone using Feedly or Inoreader) or publically (like the FeedLand representation or my representation)). I agree that no “federation” was needed, but that service from John Spurlock was needed, otherwise there would have been no RSS feed for FeedLand or my app to read.

Now, this example does not, in my opinion, demostrate any “interop” with the Bluesky application or service. John Spurlock’s application is using data from Bluesky using its API, but the interaction is one way – Bluesky to firesky.tv. That, in and of itself, is fine. There are many applications in the world that provide an API to allow other applications to get data from the application (and presumably to send data to the application). There is a list of such applications for Bluesky (https://atproto.com/community/projects), so no one is stopping innovation in this area. My impression is that Bluesky is not charging for use of its API (unlike X/Twitter), so this also should foster innovation in applications using data from Bluesky and client development for Bluesky.

Now, returning to the Dave Winer post linked at the top of this article, he is putting out the call to add feed support to social media systems (i.e., outbound feeds, publishing the posts that users make using Bluesky as feeds, separate from what someone would see using a Bluesky client, such that someone could keep up with posts on Bluesky without using the native app or any other app besides a feed reader):

I am lobbying everyone I know to add great feed support to social media systems, so we can get out of the mode of dominant platforms before Threads becomes the dominant platform.

from https://feedland.blog/2023/12/13/the-power-of-open-formats/

In pushing for this, if all social media services produced feeds, then this could be used to support moving content between services, and for other uses. Mastodon natively supports outbound RSS feeds, but Bluesky does not (John Spurlock’s Firesky.tv service appears to be the main way to read RSS feeds of Bluesky posts).

So, what about peering? Currently, there is no peering of data between Bluesky and Mastodon or any other social media service. It is, of course, possible to send content from Bluesky to Mastodon or the other way around, or any other service, as long is there is a protocol or API that supports that sharing of content. Again, those protocols do exist (AT Protocol, Activity Pub, and Mastodon API), so content could be moved around (if someone wanted to invest in doing that work).

So, what about federation? This is where different servers and/or services could communicate with each other. The Mastodon documentation has a summary:

“Unlike a traditional website, Mastodon websites can interoperate, letting their users communicate with each other; just like you can send an email from your Gmail account to someone from Outlook, Fastmail, Protonmail, or any other email provider, as long as you know their email address, you can mention or message anyone on any website using their address.”

from https://docs.joinmastodon.org/#federation

This is part of the nirvana promised by the concept of federation – you can post anywhere, mention a user, and have that user be notified. The mechanics of that, however, can be tricky, and depends on multiple servers/apps using the same protocol.

Even though there seems to be a lot of heat/activity in trying to make federation work, what if that was not the key feature that users want? Dave Winer comments here:

“Imho, what’s valued is the ability to publish something quickly, and without much fuss, and follow others, again easily without having to have a deep understanding of how these things are architected.”

from http://scripting.com/2023/12/13.html#a135218

In my opinion, this is the use case that could benefit from universal RSS support across social media applications. If that was in place, the ability to flow content via RSS between social media apps would be straightforward, and perhaps not require any effort on the part of the social media apps to federate. The world will have to wait and see….

Open letter to Dave Winer’s call to develop feed-based social media apps

On December 3, Dave Winer published a “call to develop” for a feed-based social media app (he mentioned Mastodon in his post, but could be any social media app out there in my opinion). Sounds great to me! I have a feed-based app using rssCloud for notification called MyStatusTool (link to Github repo and my install). It is implemented using Node.js, Express, and Embedded Javascript for templating. It inspired Colin Walker to develop a PHP implementation of the tool (Github repo, his install). We were able to use our tools to interop with each other (have a real-time conversation), and to follow other feeds using rssCloud for notification. I have tested my implementation with the rssCloud user feeds from FeedLand and WordPress.com in addition to Colin Walker’s MyStatusTool implementation, and verified the real-time performance (new posts showed up almost immediately).

Now, is MyStatusTool a fully-developed thing? Nope! It’s a bootstrap. It does use a number of Node packages developed by Dave Winer for feed generation and feed reading, and uses Andrew Shell’s rssCloud server for notification, so I think it falls into the “working together” category that Dave Winer wrote about in 2015: “When you have a choice, instead of re-inventing someone else’s work, use it.”

I am ready to work for interop with whatever Dave develops. I don’t have any illusion that MyStatusTool is the epiphany of feed-based social media tools, but I developed it to see if one could be developed, and it spawned another implementation, so I think that is pretty cool. Let’s get busy and see what happens!