Feeds

There are 115 posts filed in Feeds (this is page 10 of 12).

The way ahead for RSS and podcasting

Dave Winer has released some new tools/apps using RSS this week, and referenced some recent posts on podcasting concerning innovation in that space. Both of these deserve some review and context.

The RSS tools Dave initially demonstrated showed mailbox-reader styles of displaying the content of a RSS feed. Next, Dave talked about two-way RSS as a method to connect outliners with other publishing systems. I voiced an opinion that I did not understand why RSS needed to be involved. With the Twitter + Markdown + RSS unveiling, my view of “the way ahead” is becoming a little clearer. A tool is used to create tweets which contain Markdown markup. Those tweets are sent to Twitter. At the same time, a RSS feed of the tweets is created containing Markdown markup. Finally, a feed reader app is consuming that RSS feed and rendering the Markdown markup in the presentation of the RSS feed. Now, where is the value added by this workflow/system? Twitter is not making use of the enhanced RSS feed, but another app is using those features. And – those features are enabled by the use of a namespace (the source namespace), a feature in RSS that has been there for a long, long time. This is a demonstration that new apps can be built around RSS and using RSS. Now, what else could be done with this RSS feed? Well, at some point, who needs Twitter? Someone could create an app or service that uses the RSS feed as the content container and notifies users when the feed has been updated. Historically, feed reader apps do this when you start the app (highlight new entries), or rivers of news display the new items at the top of the page. But it could be done in a much quicker manner, perhaps using some other protocol like Micropub/Microsub or XMPP. Dave Winer has written about this in 2016 (The Internet’s Twitter), 2011 (fractional horsepower Twitter feed), 2009 (Fractional Horsepower Twitters), again in 2009 (A Billion Twitters). I also wrote a series of posts in 2016 where I also put forward that RSS could be the basis of an open messaging system.

Now, I will turn to podcasting. In the past week, Michael Mignano wrote about the lack of innovation in podcasting. Dave Winer commented that this is not the case. Alberto Betella then had an excellent reply to Mignano, pointing to the work Adam Curry has done on his Podcasting 2.0 initiative, supporting micropayments and other cool features. Again, the vehicle for innovation was using the RSS namespace feature. This is how the rest of the world can overcome the juggernauts of Spotify and others (which I have written about here, here, and here).

So – nothing keeping people from being innovative here – let’s get to work then!

Playing with includes in opmlPackage

This afternoon, I spent some time experimenting with opmlPackage from Dave Winer, focusing on the new feature to include other OPML files within an OPML file. The includes feature is only part of the Node version of the package (other features can be used in a browser). I changed the OPML file that had example includes to use three reading list files that I had in OPML format. I then changed the demo script to add logic to write out the new outline as a file (you can see it in this Gist) and ran it locally on my laptop. Finally, I created another copy of the OPML client to display the new outline (you can see it here). Looks like the includes feature is working fine!

Further explorations of the TweetsToRss tool

I am using the TweetsToRss tool created by Dave Winer to get a set of tweets for a Twitter user and convert it to a RSS feed (wrote a previous blog post comparing this tool and Granary). When I started using the tool, I saw that it only listed tweets created by the user (no replies). I decided to look at how to add replies as part of the RSS feed.

Twitter has an API reference page for data contained in a user timeline. I decided to review TweetsToRss to see if some of this data was present. In tweetstorss.js, there is an array called “params”, which sets screen_name to the variable username, and trim_user to “false”. I was thinking that I would have to add some additional parameters to this array based on the API reference page. However, upon further review of the source code, there was some logic to not add replies (lines 437-441):

[cc lang=javascript]
if (flSkipReplies) {
if (thisTweet.in_reply_to_status_id != null) { //it’s a reply
flInclude = false;
}
}
[/cc]

The value of flSkipReplies is set to true in line 36. I changed this line to set flSkipReplies to false, and was able to see replies in the RSS feed – yay!

One last thing – I tested having flSkipReplies set to both true and false, and saw that the version of the RSS feed with replies had 20 items, where the version of the RSS feed with no replies had only 10 items. May still be a thing or two to play around with here….

Creating an aggregator for Portland protest news

Recently, I tried out two tools for creating RSS feeds from Twitter account timelines. My next step was to put this into use for tracking news on a topic. Since I live in the Portland, Oregon area, I thought it would be good to be able to collect Portland protest news from Portland news media, reporters, protest groups, and city/county government into a single easy-to-access source.

My first step was to collect RSS feeds or Twitter handles from news sources. Sadly, the main newspaper in Portland (The Oregonian) does not seem to offer RSS feeds, but I did find some for several other newspapers. What I did find, though, is just about everyone news org is using Twitter to broadcast links to stories. I decided to make TweetsToRss my tool of choice for turning those Twitter timelines into RSS feeds.

Once I had a set of feeds, I made a copy of the single page app for my normal RSS feed reading, and made a few changes to the template. Since I had quite a few feeds, I decided to group them into four categories:

  • News Orgs – Newspapers, TV news
  • Reporters – Reporters from news orgs as well as freelancers and other people covering the protests
  • Groups – Protest groups
  • Government – Portland city government (mayor, city council, police bureau, police union) and Multnomah County government (county sheriff dept)

I then modified my template to add tabs for each of these categories. You can see the result here. I am interested in feedback on the design and in suggestions for additions to the feed list. If you have feedback, send it to andy at andysylvester dot com. Thanks!

 

 

Huginn is a system for building agents that perform automated tasks for you online. They can read the web, watch for events, and take actions on your behalf. Huginn’s Agents create and consume events, propagating them along a directed graph. Think of it as a hackable version of IFTTT or Zapier on your own server. You always know who has your data. You do.

See also this post on Medium

How to create a RSS feed from a Twitter user timeline

Introduction

I have used services before like TwitRSS.me to generate RSS feeds from Twitter user timelines. However, I recently noticed that the one feed I was using from this service did not seem to be updating. I took a look at the home page, and it appears that the service might not be working anymore due to changes at Twitter.com. Based on that, I decided I needed to look for another tool, and found two that appear to work well.

Granary

The first is Granary.io, which calls itself “the social web translator”. This site runs an app written in Python, and creates multiple feed types for Twitter, Facebook, Instagram, and several other social media sites/services. The home page features a set of buttons for a user to choose which service to work with. I clicked on the Twitter button, which then started an authentication with my Twitter account (I was logged in, so I did not have to take any action), then was replaced by a set of text boxed/drop-down menus:

I then changed the first drop-down menu from “@all” to “@self”, then changed the format dropdown from “as2” to “rss”. I pressed the “Go” button, which then generated a list of my last ten posts in RSS format. The app generated a link which I am adding to my feed reader. Next, I changed the “username” text box to another Twitter user handle “davewiner” (did not include the “@” symbol). Clicking on the “Go” button again gave a list of the last 10 tweets from davewiner. I did notice that the list was not in reverse-chronological order.

TweetsToRss

The second tool I reviewed was tweetsToRss by Dave Winer. This is a Node.js app which uses the node-twitter-api NPM package to get data from a Twitter user’s timeline. In order to use the app, the user needs to get an approved Twitter developer account to get a set of keys for this app. Fortunately, I had a set of keys from a previous Twitter app that I was able to reuse for this app.

I had Git, Node.js, and NPM already installed on a server, so I did a git command to download the app to my server:

git clone https://github.com/scripting/tweetsToRss

The app was copied into a folder (tweetsToRss). I then changed directories to that folder, then made some edits to the main app file tweetstorss.js:

  • Added the four keys as specified on the app README.md page (decided to edit the parameters to specify the keys directly rather than set them up as environment variables.
  • Set pathRssFile to “”
  • Set twitterScreenName to “davewiner”

Next, I did a NPM command to install the required Node.js modules:

npm install

Finally, I went ahead and ran the app with the command:

node tweetstorss.js

The app started and appeared to run, but did not create a RSS file. After some experimenting, I figured out that pathRssFile needed to be set to undefined:

var pathRssFile = undefined;

With that last change, the app ran and created a RSS file with the 10 most recent tweets from davewiner (Dave Winer, creator of the app). The last change I made was to add a JSON file with a list of several Twitter usernames to create RSS files for each one (the README file links to an example that I modified). My final step with the app was to run it with the forever command:

forever start tweettorss.js

The app was now running continuously and calling the Twitter API once a minute to check the Twitter user timelines and update the RSS files. My last step was to reuse a script to FTP the feeds to a web server to be able to access them from my feed reader.

Viewing the feeds

To read the RSS files, I set up two single page apps using River5 and the RiverBrowser toolkit:

Conclusions

Pros:

  • The Granary version shows more of the text from the tweets
  • Able to use Granary API without having a server
  • The TweetsToRss app better shows which of the Twitter user timelines are being shown
  • The TweetsToRss app runs on my server, so I am not dependent on another service to get the Twitter data (like I was for TwitRSS.me)

Cons:

  • TweetsToRss needs more setup that Granary

Both tools provide the desired result – the ability to follow Twitter users using a feed reader instead of a Twitter app or Twitter.com. Enjoy!

References