A live example: bwkfanboy.herokuapp.com

bwkfanboy is a converter from a raw HTML to an Atom feed. You can use it to watch sites that do not provide its own feed.

The converter is not a magick tool: you’ll need to write a plugin (in Ruby) for an each site you want to watch. bwkfanboy gives guidelines and a general assistance.

Version 2.0.0 was rewritten form scratch. Plugins from previous versions will not work. See github.com/gromnitsky/bwkfanboy_old repo for old versions.

NAME                 VER DESCRIPTION

bwk                    2 Brian Kernighan's articles from Daily Princetonian
freebsd-ports-update   3 News from FreeBSD ports
inc                    1 Articles (per-user) from inc.com
econlib                1 Latest articles from econlib.org

Install bundler:

# gem install bundler

Install all dependencies. You have 2 choices:

1. Usual via

   # bundle install
   

2. Or only for this program

   % bundle install --path vendor/bundle

   You’ll have to prefix other commands with ‘bundle exec’ though.

Run application:

% rackup -E production

Point a browser to 127.0.0.1:9292

Plugins can be in system

`gem env gemdir`/gems/bwkfanboy-x.y.z/plugins

or user’s home

~/.bwkfanboy/plugins

directory.

The program consists of 3 parts:

1. bwkfanboy script that takes 1 parameter: the name of a file in plugin directories (without the .rb suffix). So, for example to get an atom feed from dailyprincetonian.com, you type:

   % bwkfanboy bwk

   and it will load /usr/local/lib/ruby/gems/1.9/gems/bwkfanboy-2.0.0/plugins/bwk.rb file on my FreeBSD machine, fetch and parse the html from dailyprincetonian.com and generate the required feed, dumping it to stdout.

   The script is just a convenient wrapper for 2 separate utils below. (Although it doesn’t executes them, but uses their internal api.)

2. bwkfanboy_parse

   Takes 1 parameter: a plugin name.

   This util reads stdin, expecting it to be a html, parses it and dumps the result to stdout as MessagePack object in UTF-8.

3. bwkfanboy_generate

   Reads stdin expecting it to be a proper MessagePack object.

   The result will be an Atom feed dumped to stdout in UTF-8.

So, without the wrapper all this together looks like:

% curl http://example.org | bwkfanboy_parse myplugin | bwkfanboy_generate

bwkfanboy includes a small sinatra app. See the installation phase.