rssCloud, WordPress, FeedLand, and Dave Winer

This is an attempt to capture some work/events I have been involved with over the last few weeks regarding the rssCloud protocol. I hope this will provide some insights into software development and technical communication, and will be of use to others using the rssCloud protocol. I hope it will also demonstrate that it can be difficult to identify/isolate problems between multiple apps/web services/web sites, and that all involved in the discussion should adopt a mindset of “follow the data” and be open to input from others.

rssCloud is a notification protocol that uses a “cloud” element in a RSS feed. This element contains information on a cloud server that is supporting this feed. A feed reader app can read the RSS feed and register with the server listed in the cloud element to be notified when the feed updates. The producer of the RSS feed can then “ping” the server to let it know that updates are available, and the server can inform apps that registered that feed that updates are available. The cloud element was included in version 2.0 of the RSS specification.

The protocol has been supported in the blogging tools that Dave Winer developed since 2001 (Radio Userland (feed reader/blogging) and Manila (blogging)). Since that time, several other tools added support for adding the cloud element to RSS feeds (Django, Perl XML-RSS, and syndic-rss2). In 2009, Matt Mullenweg announced that WordPress.com blogs would support rssCloud, and that a plugin would be available for WordPress.org blogs. A unique feature of the WordPress implementation is that the WordPress blog itself is the cloud server. A result of this was that a user did not have to find a rssCloud server for their feed to reference – their own site could handle feed registrations and notifications to feed reader apps. This created a supply of literally millions of weblogs that could support quick notifications to their readers if their reading app supported the use of rssCloud. However, for the most part, only feed readers and apps developed by Dave Winer supported the protocol (River2 (OPML Editor app), River4 and River5 (Node.js apps), Radio3 (Node.js) and now FeedLand (Node.js)).

Some users noted issues with rssCloud support back in 2009 as WordPress.com rolled out its rssCloud support. One of the posts from this period highlighted the problems with registering an application with the rssCloud server associated with the WordPress.com blog. I began looking at rssCloud in 2015 and 2016 with limited success, and tried to develop a separate plugin, but had issues. I have been aware for several years that there were problems with River5 working with WordPress.com blogs and WordPress.org blogs in terms of rssCloud support, but I did not have the Node.js knowledge to adequately figure out the issue.

After the rollout of FeedLand (Dave Winer’s latest feed application), a user, Colin Walker, started asking questions about rssCloud so that he could support it in his custom blog/reader application, and Dave Winer said he wanted people to ask questions about rssCloud, so I thought this was the right time to bring up the WordPress.com problem and document it to the best of my ability. I also tried to not say that it was a FeedLand problem, but to me the end result was that FeedLand was not doing what WordPress.com wanted to support rssCloud. Since Joseph Scott indicated that the WordPress.com implementation would not be changing, it was FeedLand that had to change (and other rssCloud apps that might want to say that they work with WordPress.com blogs). I didn’t get much, if any, credit for bringing this up, but I did identify the problem and tried to provide the level of info that Dave Winer seems to want/need before looking into a problem. During the investigation, Andrew Shell determined that the WordPress cloud server would only respond to requests from ports 80, 443, and 8080. This fact alone would be a valid explanation of why River5 (which ran on a default port of 1337) would not be able to register with a WordPress site or get notifications. However, this was not documented in any of the WordPress posts listed above regarding rollout of RSS Cloud support.

Dave Winer then published a reference app for testing rssCloud implementations (https://github.com/scripting/reallysimple/tree/main/demos/clouddemo). I tried to comment on the reallySimple repo, but I saw this:

Screen capture from Github repo issue

…so I posted my report on my own blog.

I checked, and saw that Github does have moderation features which allow repo owners to block people. I assume this is what has happened. I am also going to assume that this is from my comment on the earlier Github thread where I called out Dave’s negative request for help with rssCloud issues. Being his repo, he can do what he wants. However, to me that does not give the impression that Dave is open to feedback, even though he asked for feedback at the top of the issue (I guess it was just my feedback he did not want). I later checked and saw that I saw the same response on other repos within github.com/scripting, so I assume I am blocked from all of those repos.

This is not the first time I have identified problems in how Dave Winer approaches comments about him or his products. I know he has “guidelines” to handle feedback, and that is his right. At this time in my life (I turned 60 earlier this year, I know Dave is 67-68), I tend more to “call it like I see it” in as factual a manner as I can. If that is what Dave Winer thinks of as “trolls, spammers, and abusers“, I can’t do anything about that. However, I don’t think I fit into any of those categories.

With the help of Andrew Shell, Dave Winer was able to update FeedLand so that it supports display of WordPress.com and WordPress.org blog posts in near-real-time. I am seeing some RSS feed caching issues with WordPress, which I am continuing to document on a Github gist thread. Joseph Scott of Automattic has been very helpful with ideas. I was able to demonstrate that the new version of the rssCloud plugin (0.5.0) supported registration and notification with FeedLand, but subsequent tests with test scripts are showing the RSS feed caching problem I saw last week.

I have a rssCloud project in mind that I would like to develop, but I want all WordPress users to be able to take advantage of rssCloud. Hopefully, this will become a reality soon.

Test results for rssCloud app from Dave Winer

I couldn’t leave a comment on the reallySimple repo for clouddemo.js (it appears to be locked), so I am writing a blog post. I was able to run the app on port 443 and successfully register a WordPress.com site and a WordPress.org site using version 0.5.0 of the WordPress RSS Cloud plugin. When I ran my separate test script on the WordPress.org site, it worked, but I figured out that the WordPress.org site is caching the RSS feed, which explains why feed updates were not seen by FeedLand.

The code for clouddemo.js looks good, I am pleased that it could run on port 443, since I already had an app running on port 80 on my server and did not want to change it.

I was able to confirm this morning that rssCloud support in FeedLand worked for my WordPress.com test site – hurray! Many thanks to Andrew Shell and Dave Winer for working things out, along with support from Joseph Scott at Automattic. Joseph Scott released a new version of the RSS Cloud plugin today for WordPress.org sites, I did an initial test, but FeedLand did not get a rssCloud update from my test site. I got some debug ideas from Joseph Scott, working on those now.

Lazyweb request – does anyone know of other feed readers besides FeedLand and River5 that support the rssCloud protocol? I am trying to collect data regarding the WordPress rssCloud implementation. Thanks.

I have been collaborating with Andrew Shell on the WordPress.com rssCloud issue. Andrew added a note to my support forum post that WordPress.com and the RSS Cloud plugin only accept aggregators running on port 80, 443, and 8080. I have confirmed that the RSS Cloud plugin will accept a registration from an app running on ports 443 and 8080 and provide the correct rssCloud response when the source feed updates. Andrew will be contacting Joseph Scott (plugin developer) at Automattic with this information.

RSS Cloud support in WordPress.com not working

UPDATE: Here is the link to the support forum post I made at WordPress.com

This text is reposted from my test WordPress.com site:

I am doing some testing of the RSS Cloud protocol (http://home.rsscloud.co/) in WordPress.com (https://wordpress.com/support/rsscloud/, https://wordpress.com/blog/2009/09/07/rss-in-the-clouds/) . I think there is a problem in the implementation, here is my bug report.

What I did:

1. Created a test WordPress.com site ( https://rsscloud4.wordpress.com), checked the feed (https://rsscloud4.wordpress.com/feed/), made sure that the rssCloud element was present.

2. Subscribed to the site in FeedLand (http://feedland.org/) (best RSS reader for RSS Cloud support)

3. Reviewed Joseph Scott original blog post on rssCloud support in WP and using WP plugin https://blog.josephscott.org/2009/09/07/rsscloud-for-wordpress/

4. Modified my rssCloud test script (https://gist.github.com/andysylvester/d41a6ee6b3a7d0860c039ccbe3c6147f) for Node.js RSS Cloud server (http://rpc.rsscloud.io:5337/) to access the RSS Cloud setup for my site (https://rsscloud4.wordpress.com)

a. var urlRssCloudServer = “https://rsscloud4.wordpress.com/?rsscloud=notify“;

b. var urlHackerNews = “https://rsscloud4.wordpress.com/feed/

5. Modified function pleaseNotify to display the text of the response from the rssCloud server (my WordPress.com site).

What I saw:

1. Ran the test script (node testscript.js), got the following response:

root@AndyDO-03:~/rssCloudSimple# node rssCloudTestOriginal05.js

testRssCloud v0.40 running on port 2222.

response: == <?xml version=’1.0′?>

<notifyResult success=’false’ msg=’Error testing notification URL : A valid URL was not provided.’ />

.

pleaseNotify: success == undefined, msg == “undefined”

2. Made a post (https://rsscloud4.wordpress.com/2022/12/04/test-post-007/) at 2:18 pm, no response to my test script.

3. Checked in FeedLand for new item, did not see it until 2:28pm (10 minutes later). In testing of other tools creating RSS feeds with rssCloud support, the item would appear in FeedLand within a few seconds. I have noticed that feed readers seem to take up to 30 minutes to detect that there has been a change in a RSS feed from WordPress.com or WordPress.org sites.

What I expected:

1. My script would get a notification that the registration was successful.

2. My script would get a notification from the WordPress.com site when a new post was made.

3. A new post from my WordPress.com site would appear within a few seconds in FeedLand, since I had seen this behavior for other tools creating RSS feeds with rssCloud support

I did some searching on rssCloud problems with WordPress.com/WordPress.org, and found this post from 2009 (http://www.xn--8ws00zhy3a.com/blog/2009/10/rss-cloud-fail) which described issues with notification URLs:

“Still, we were getting somewhere. There were only two remaining unexplained failures. These were WordPress feeds that were somehow failing to connect to my notification URL. Eventually, after digging through various releases of the WordPress source code, I discovered that these servers were actually running a buggy, older version of the WordPress plugin. Regardless of what the client specified, this version of the plugin would always attempt connections on port 80. “

My app is running on port 2222, the notification URL that the WordPress.com site should be trying to contact is http://fedwiki.andysylvester.com:2222/feedupdated. Could someone shed some light on this? Thanks.

Lazyweb request – I am trying to make sure WordPress is not caching RSS feeds, I tried these things but it did not work. WordPress devs, please get in touch if you have any ideas! I am trying to test a RSS Cloud plugin I developed.

Demo of rssCloud protocol and reallySimple NPM module 

In this era of moving away from Twitter, I have been reviewing the rssCloud protocol and think about its potential. I had not done anything about it, but saw this Twitter thread and really liked this comment by Preslav Rachev regarding building on RSS: “And best of all, everyone is free to build their own apps and tooling on top of it without restrictions, or stepping onto each other’s toes.”. I then decided to start playing around….

I read through the walkthrough document on the rssCloud site and decided that the simplest test would be to create some RSS feeds that have rssCloud elements, register them with a rssCloud server, then get the server to contact a server when the feeds updated, then display some information from the feeds. I started with this script from Dave Winer, updated it to provide web output, then ran the script twice to register two different feeds. I then made a video of running the script, posting to feeds using Drummer and FeedLand, then displaying the title or description of the most recent item in the feed, along with the feed name. The video is shown below. Source code and more instructions are available on Github. Let me know what you think!

rssCloud testing – Day 3

Decided to start to try to send a HTTP POST to rsscloud.andysylvester.com using the Curl application. I was able to run Curl from the Node.js command prompt window.

After lots of experiments, I came up with the following test command based on the cloud element content of the RSSCloud test site feed:

curl –data “port=80” –data “path=/?rsscloud=notify” –data “protocol=’http-post'” –data “url1=http://rsscloud.andysylvester.com/feed/” http://rsscloud.andysylvester.com/?rsscloud=notify > test.txt

The resulting text file content (test.txt) was as follows:

<?xml version=’1.0′?>
<notifyResult success=’false’ msg=’Only http-post notifications are supported at this time.’ />

I had tried using “https” in the fields in the test command, but got the same response text as before. I think my next step is to look at the RSSCloud WordPress plugin.

Resources:

Everything Curl book: https://ec.haxx.se/

Curl http scripting: https://curl.se/docs/httpscripting.html

Curl command line command for HTTP POST: https://superuser.com/questions/149329/what-is-the-curl-command-line-syntax-to-do-a-post-request

Blog post on release of RSSCloud WordPress plugin (has some example PHP code): https://blog.josephscott.org/2009/09/07/rsscloud-for-wordpress/

Andrew Shell RSSCloud server docs: http://rpc.rsscloud.io:5337/docs