So, I ended-up writing my own tool to do this. But, first, some back-story…

Existing Tools

In doing some searches for variations around ”svn repo filter”, I found a lot of people pointing to the ”svndumpfilter” utility as the tool-of-choice. Sadly, it doesn’t seem to be quite “smart” enough to do what I’d like it to do. It seems to be aimed at (namely) filtering an svnadmin dump stream, taking only certain paths in the SVN “filesystem”. That works fine if you’re trying to take an isolated folder/project from the repository, but if that folder/project has ever been merged into from any paths outside of the target filter path, then things fall apart. The “trouble” are the copy-from’s…

For example, say that you have a repo with a typical trunk/tags/branches setup. Say you create a new topic branch (e.g. svn copy /trunk /branches/my-fix && svn co /branches/my-fix) and happily work on your sandboxed branch. Say that you decide to rename some of the pre-existing files/folders, so you run the appropriate svn move commands and happily commit those changes to your branch. Once everything is working happily, you go to merge these changes into /trunk and that all works great. After the commmit, if you run a svn log -v -l1 /trunk to look at the details of the most recent commit to trunk, you’ll see something like this:

A /trunk/Project/RenamedFolder (from /branches/my-fix/Project/RenamedFolder@12345)

…which only describes the (top-level) folder rename, not any add/modifications/renames/etc that might have happened inside that folder on the branch. At the full repo-level, SVN can get away with just doing a ”svn copy” from the branch to trunk, so that when you do an ”svn log” on the new “RenamedFolder” path it will walk from trunk back into that originating topic branch and follow the rename (copy or move) that happened within.

The svnadmin dump will show the same “copy-from” action as svn log -v did, since the dump stream will need to be able to recreate that same ancestry. When svndumpfilter sees this, it throws up its hands and returns an error because it doesn’t know how to recreate the logical history that happened between when that branch originally forked from trunk versus the final state of that branch. That is, assuming the copy-from path even has any ancestry back to trunk…

If You Want Something Done Right…

In my searches, I had found a few folks that had hacked together there own solutions. None of those quite fit my problem at hand, so naturally I decided to start working on my own solution.

The closest fit I found was a project named ”svn2svn” hosted on Google Code. It had copied parts of the ”hgsvn” project (sychronizing between Mercurial and Subversion repositories) and slapped them together in a way that worked for the original author’s needs. It used ”svn log” to walk the entire history of a given path in a source repository and then manually replayed those changes to a working-copy of some path in a target repository. Revision by revision, it would replay the delta, recreating the history of just the source path in the target repo. This was oh-so-close to what I wanted, except that it also didn’t correctly handle the copy-from case. And the repository I really wanted to replay was littered with copy-from’s, since I’m trying to replay just the history from /trunk and we create topic-branches for everything.

Enter svn2svn

So, I started with the svn2svn project, got familiar with the code, and started hacking to extend the code to solve my problem at hand.

The net-result is my own (nearly completely rewritten) take on the problem, a project which I’m also calling svn2svn.

I’ve made several enhancements to the original script:

• Full ancestry (copy-from) support. This was the tricky part. It took several different iterations/rewrites to get something which worked for all the different edge-cases. The general idea here is that we can use svn log to walk backwards through the ancestry on a copy-from case: if we can trace back our ancestry to same source path we’re replaying, then we can do an svn copy from that original parent and then do svn export to update the contents of all the files to match the final copy-from verison. There’s also some extra recursion that needs to happen here, to handle cases where child files got renamed inside of a parent-renamed folder. There are other edge-cases like files getting replaced inside a parent-renamed folder.

• Use revprops for source-tracking and resume support. Subversion has revision properties, key+value pairs that are associated with a particular revision/commit. I’m setting some svn2svn:* rev-props to track the source URL, source repository UUID (i.e. in case source URL now points at a physically different repo), and source revision #. This is all needed for proper resume support, since for the ancestry support we have to maintain a mapping-table of source_rev -> target_rev, so that when we find a copy-from from some revision # in the source repo, we can map this to the equivalent revision # in the target repo so we can do an equivalent svn copy command.

• Better verbosity output, and optional debug output. As I was playing with the rewrite, I quickly found I needed better debug output, to see which shell commands were being run and ot display just general debug/status messages as we do all this new complicate ancestry-walking logic. Bonus: the debugging messages have colored output, using ANSI escape codes which all self-respecting terminal emulators should respect.

• All commits (including initial import) go through the same central code-path, which means we could run a (client-side) pre-commit script to scrub the contents of the target working-copy before each commit. This is where the power of doing the manual replay of changes really starts to shine. We have full control over the pending changes, which means that if your original trunk history had some files which you didn’t want to transition into the new replayed repo then you could easily svn rm those files from the working-copy before the commit happens. That could be as simple as excluding certain fixed paths, but it could be a lot more flexible like searching the entire working-copy tree and removing any files which match a certain file-name. Heck, you could even modify the working-copy file-contents at this point…or add brand-new files if you want. We’re replaying the SVN history here will full control of the target content, so this opens a lot of interesting options.

Check-out the svn2svn project page for more details.

Also, I have the project mirrored on Github so please feel free to fork the project, send me issues/enhancements, or send me pull-requests for any tweaks you’ve made.

I learned a lot of neat stuff along the way, including some config options which I never knew were there and some tricks which really optimized my command-line shell experience.

Using Git

I found it to be a very natural fit to use Git to track the edits to my dot-files. It makes it dead-easy to see what changes I’ve made since the last commit and easy to commit those changes and push them to a central Git repository.

Git also makes it dead-easy to “bootstrap” my dot-files environment into a brand-new home directory:

1
2
3
4
5
6
7

# Clone the repo into ~/dotfiles/
git clone git://github.com/tonyduckles/dotfiles.git dotfiles
# Move all the contents (including .git, but excluding the specal "." and ".." directories)
# into ~, overwriting any pre-existing files
mv dotfiles/.??* .
# Clean-up the now-orphaned ~/dotfiles/ directory
rm -rf dotfiles

~/bin/

Ryan has a ~/bin/ directory in his dot-files repo with all kinds of nifty little utilities.

There’s a bunch of great Git-related utilities in there, e.g.:

• git-up - Like git-pull but show a short and sexy log of changes immediately after merging (git-up) or rebasing (git-reup).
• git-incoming/git-outgoing - Shows difference between the current branch vs. the upstream branch.

And there’s also some great general-purpose utilities in there, e.g.:

• ack - A tool like grep, designed for programmers with large trees of heterogeneous source code. Very handy for recursive file-searching.

~/.gitconfig

Ryan has lots of nifty tricks in his .gitconfig file, tying in short-cuts for calling the helper utilities in ~/bin/. Also, you can setup Git to use ANSI colors for various sub-commands (diff, status), which is very handy.

~/.inputrc

There’s a lot of neat options in the .inputrc file, for binding key-sequences to various command-line options. For example:

1
2

C-n: history-search-forward
C-p: history-search-backward

…are the two single-most time-saving key-bindings that I stumbled upon. This lets you type a partial command and then use Ctrl-n and Ctrl-p to completion-match the rest of the line based on your command history. Awesome!

I find this incredibly useful to completion-match hostnames for ssh commands or other longer commands which I often type. Completion-matching for-the-win!

Vim: .vimrc and .vim/

Before setting out on this project, I had used Vi some but hadn’t really explored (or understood) the full-power that was Vim. Digging into the .vimrc file, reading-up on all the various config options, and digging into all the ~/.vim/ plugins which Ryan had was an eye-opening experience. I came to realize just how much you could extend and customize Vim to meet your needs. Simple things like getting syntax-highlighting and color-schemes set by default made for an oh-so-much more pleasant Vim experience.

This all led me to learning a lot more about Vim and becoming much more proficent using Vim. It’s now my text-based editor of choice. Once you get to know the key-bindings and the various commands, the command-chaining that you can do in Vim is incredibly powerful.

Sharing is Fun!

If you’re at all interesting in learning more about customizing your Unix shell environment, looking at other people’s dot-files is a great learning experience. And publishing your dot-files on Github is a great way to share your shell-environment with the world so that others can learn and explore.

Octopress is an obsessively designed framework for Jekyll blogging. It’s easy to configure and easy to deploy. Sweet huh?

Sweet indeed! The more I read about it, the more intrigued I became. After a few hours spent exploring the code, importing the posts/pages from my Wordpress install into Octopress, and tweaking the default theme to my taste, I’ve taken the jump and moved my site over to using the Octopress platform.

Octopress is a simple blogging platform aimed at hackers. It’s built around the Jekyll engine, which is the blog-aware static site generator that powers Github Pages. The “database” for the site is simply a collection of flat text files using Markdown (or Textile) mark-up. You can setup either “posts” or “pages”, all as you’d expect. When you’re ready to deploy, you use Rakefile automation to “generate” the static site files.

It also has a great default theme (IMHO) – very clean and simple – which was actually my original attraction. It was only after reading more about the project that I realized it had huge appeal to my geek-side too.

Why Octopress?

It was Matt Gemmell’s ”Blogging With Octopress” post which resonated the most with me:

WordPress is excellent, but it’s over-featured for what I need, and its PHP/MySQL guts are opaque. I don’t really like the idea of all my writing being inside a big database either; it’s a single point of failure, and that makes me uneasy.

For what I do with my website, Wordpress was overkill. It ended-up being Yet-Another-Thing-I-Needed-To-Maintain-Security-Patches-On™.

I was immediately attracted to the simplicity which Octopress provides: editing pages and posts as plain-text text-files, all static files so no security-patches or vulnerabilities to worry about, and easily portable and backup-able. It combines a lot of things which I have grown to love: I can keep all my content in a Git repository so that I can publish from multiple locations (if need be), I can work on the site entirely in text (vim + screen), and it’s dead easy to deploy and backup.

Migrating from Wordpress

It really didn’t take that long to migrate everything over to Octopress.

It took a few hours to get a cleaned-up import of my old Wordpress content. Based on the recommendation of others, I used exitwp to create Jekyll-style posts based on my Wordpress content. This went fine for the most part, but there were some parts in the Wordpress export file that I needed to fiddle to keep the exitwp script from crashing. The end result was that I had a directory full of simple *.markdown files which represented all the content from my Wordpress site. I spent a few hours going through those files and cleaning up the Markdown syntax until it matched what I wanted. Some of my original Wordpress posts were a mixture of Wordpress mark-up and raw HTML, and that threw the exitwp parser for a loop in some places. It did a great job overall though.

From there, I spent some time looking through the guts of the Octopress source code getting acquainted with things and seeing how I might be able to fiddle with some of its inner fiddly-bits. It’s written with native support for customization, trying to keep a clear separation between the core “code” of the site (which can be overwritten during a future upgrade) versus any “customization” files which the user might make changes too. It’s a fantastic paradigm and one that you don’t often see too much of in projects.

Last but not least, I spent some time tweaking the theme to my tasting (…a bit of pepper here, a bit of salt there…). As I mentioned above, it natively carves out files which are earmarked for user-customizations, where you can tweak the CSS (SASS), colors, page/sidebar dimensions, etc. It’s all just extremely well-thought-out and a joy to use and extend.

Ending Thoughts

This has been a fun pet project for me. It’s the first time I’ve used a Ruby environment. It’s also the first time I’ve played with SASS (*.scss), and boy is it going to be hard to go back to writing plain-old CSS. SASS’s color functions are ridiculously slick: being able to do things like desaturate(lighten($nav-bg, 8), 15) in a *.scss file is awesome and makes tweaking a site’s color-scheme oh-so-much easier.

I’m a bit sad to have lost the lifestream (wp-lifestream) functionality from my old Wordpress site. But then again I’m already doing full exports of most of my lifestream’d websites (Google Reader, Delicious), so it might not be too hard to generate static lifestream pages using a cronjob or something. That’s a project for another day…

Useful Links

• Blogging With Octopress - A fantastic and well-thought-out introduction and overview of Octopress.
• Switching to Octopress - A great end-to-end summary of the Wordpress-to-Octopress transition process.
• Octopress Hidden Features - Pointing out some of subtle features of Octopress.
• exitwp - Tool for converting Wordpress export XML to the Jekyll blog engine.
• vim-octopress - Vim syntax highlighting mode for Octopress-flavored *.markdown files.

In reading through different online forums, the topic of ZFS-based systems kept coming-up again and again. I had heard the term “ZFS” thrown around before but I had never really spent the time to read-up on it. It’s just another filesystem, right? How fancy can it be?

Well, ZFS is just damn cool. ZFS is lot more than ”just another filesystem“…

• It’s a filesystem, logical volume manager, and software RAID subsystem all wrapped-up into one end-to-end system.
• It does end-to-end checksums, making sure the raw data read off the physical disk matches the same data that was previously written to disk.
• It is a transactional file system, which guarantees a consistent on-disk data-state even with catastrophic failures like a power loss.
• You can do data scrubbing online, as opposed to the normal Linux solution of ext2/ext3/ext4 where you can only “fsck” a filesystem offline.
• It uses a copy-on-write strategy which makes doing filesystem snapshot’ing a breeze. For example, you could setup a cronjob to take nightly snapshots of your filesystem to facilitate incremental backups, i.e. allowing you to go back in time and recover old-states of files.
• It has RAID-Z support, which is similar to RAID-5/RAID-6 except without the potentially-fatal ”RAID-5 write hole” problem, in part thanks to the copy-on-write strategy.
• You can replicate ZFS filesystems (zfs send and zfs receive), snapshots and all, making it dead-easy to backup ZFS filesystems to remote machines.

Very, very cool stuff! The more I read about it, the more I wanted it for my next-generation file-server solution. In particular, I really love the idea of end-to-end checksums. I want to make darn sure that the OS can realize when the disks are serving up garbage-data or if somehow bits have mysteriously been corrupted on-disk (as unlikely as that might be).

In reading about all these various historical problems that ZFS sets out to address, I’m really surprised that more OSes haven’t tried to adopt superior filesystem technology. The classic problem for so many Windows-users seems to be that their installs get “corrupt”, likely usually due to folks forcibly powering-off their computers before the hard-drives have a chance to flush their write-buffer to disk. And then folks get to cope with BSOD’s or broken Windows installs. Sad times for everyone. Filesystems seem like such a foundational part of the OS that you’d expect companies would spend the time to make them as robust as possible. I read some rumors that Apple was planning to incorporate ZFS into OSX a few years back, but apparently that fell apart for some reason. Sad…

Kudos to the Sun folks for designing such a fantastic filesystem and then open-sourcing it!

I run an Ubuntu box at home and it was easy to install the dovecot-imapd package to get an IMAP server installed. Since my box is behind my router/firewall, I’m wasn’t that concerned with tweaking Dovecot’s default configuration, but I’m sure you could fiddle with the config to ensure that Dovecot only binds to 127.0.0.1.

From there, it’s just a matter of using imapsync, just like I ended-up using previously to initially transfer all my e-mail to my Google Apps account.

Here’s the script:

1
2
3
4
5
6
7
8
9
10
11

 # Sync e-mail from "username@yourdomain.com" to localhost, using IMAP
 imapsync --host1 imap.gmail.com --user1 username@yourdomain.com
          --passfile1 /path/to/gmail_mirror.passfile1 \
          --host2 localhost --user2 username
          --passfile2 /path/to/gmail_mirror.passfile2 \
          --ssl1 \
          --useheader 'Message-Id' --skipsize --allowsizemismatch \
          --syncinternaldates --noauthmd5 -nofoldersizes\
          --split1 100 --split2 100 \
          --regextrans2 's/\[Gmail\]/username\@somedomain/' \
          --include "All Mail|Sent Mail" --delete2 --expunge2

The --regextrans2 option rewrites IMAP folder-names on-the-fly, so that my local IMAP folder structure can be different than the structure on Gmail’s server. For example, the top Gmail IMAP folder is [Gmail] which wasn’t all that useful for me, so instead I rewrote that top-level folder to be username@somedomain so that the local folder name (e.g. in ~/mail/) would match the source e-mail address.

You can also use the --include option to decide which IMAP folders to copy. I opted to just copy “All Mail” and “Sent Mail”, which gives me a copy of all my mail but doesn’t preserve any information about the labels I might have had assigned to those messages in Gmail.

The initial copy will definitely take a few hours (or more), depending on how much e-mail you have in your Gmail account. But this works great for me and stores the mail in “mbox” format locally so I can even access the mail locally via mutt/alpine/etc.

http://gemal.dk/blog/2008/04/08/completed_the_gmail_migration/

I adapted that script for my own needs, and I was able to successfully copy all the mail from my regular Gmail account to my new Google Apps account.

Here is the final script I ended-up with:

1
2
3
4
5
6
7
8
9
10
11
12

 #!/bin/sh
 # Copy all e-mail from source@gmail.com to target@yourdomain.com (Google Apps)
 imapsync \
   --host1 imap.gmail.com --port1 993 --user1 source@gmail.com \
   --passfile1 /path/to/gmail_mirror.passfile1 --ssl1 \
   --host2 imap.gmail.com --port2 993 --user2 target@yourdomain.com \
   --passfile2 /path/to/gmail_mirror.passfile2 --ssl2 \
   --useheader 'Message-Id' --skipsize \
   --syncinternaldates --noauthmd5 -nofoldersizes \
   --split1 100 --split2 100 \
   --maxage 50 \
   --regexmess 's/Delivered-To: source\@gmail.com/Delivered-To: target\@yourdomain.com/g'

The --syncinternaldates, --useheader 'Message-Id', and --skipsize options are all recommended by the imapsync FAQ (search for “Gmail”):

http://www.linux-france.org/prj/imapsync/FAQ

I opted to use the --passfile1/passfile2 options rather than passing in a plain-text password in via a command-line param for two reasons: first because anyone with access to your system can use ps to view active processes and hence would see your password plain-as-day; second because it just better abstracts the script-logic from the password-text, and we can control the file-permissions of those password files.

All your Gmail labels should sync-over automagically. Since this is going Google-to-Google, because Google presents labels as separate IMAP folders in their Gmail IMAP implementation, the process above should sync and preserve all your Gmail labels for free.

It’s been a two years since I’ve used this script, but I seem to remember it working pretty painlessly. I wanted to share it here for anyone else who might be looking to do this same thing.

(via kotaku.com)

(via autocompleteme.com)

Google auto-complete hilarity.

(via appolicious.com)

After hearing some rave-reviews from various places over the past few days, I picked-up GeoDefense for the iPhone this evening. Wow, it’s a blast! Nice balanced game-play, good amount of challenge, great graphics. Well worth the $2.

I find that I don’t always have the inspiration or motivation to write blog posts that often, but I do have a Web 2.0 foot-print that other people might want to watch, whether it be sharing interesting news tidbits on Google Reader to bookmarking useful webpages on Del.icio.us, to what songs I love on Last.fm.

Hence, the birth of this lifestream: all my various online activities aggregated and presenting in a unified “lifestream”. This gives others an easy way to check-out what I’m up to. If you have any comments or feedback, drop me an e-mail!

More and more lately, I’ve been trying to find web apps that can replace the desktop applications I use on a daily basis. It’s just so incredibly convenient to have the portability to be able to do everything you need from inside a web-browser, and more importantly to have the persistent-state data stored on the server-side rather than on each individual machine I use throughout the day. I’m already a huge fan of webapps like del.icio.us (for tracking all my bookmarks) and Google Reader (for all my RSS aggregation and reading needs), and finding a web-based password manager would be ideal. It really becomes a chore remembering what username or e-mail address I registered with at various websites: Amazon, eBay, Newegg, Google, Yahoo, credit cards, travel sites, online bill pay, banks, etc. (Oy!) It’s just a lot of identities to juggle. I’ve thought about using KeePass or something, but that’s a desktop app and I’d have to keep my database in-sync between work and home.

What really interested me about Clipperz is their claim at being a “zero- knowledge” web-application: they only store encrypted data on their server- side, and use Javascript to decrypt the data client-side based on the username and password which you supply. Your password is never sent to the server at all; it’s merely used to decrypt the data locally. That’s a pretty slick idea, IMO. But, I’m also a little nervous about trusting some third-party website will all my personal sensitive online account information. So, I’m tempted to sit down and write my own webapp to recreate that wheel, except that it’s all code I can vouch for and know that it’s not secretly sending my data off to some third-party site. You can already find open-source Javascript-based implementations of the various crypto algorithms that I’d need to use. And it seems like a simple AJAX-based webapp to save/load data from the server.

Also really intriguing is Clipperz’ one-click-sign-in feature. They have a Javascript bookmark which pulls HTML FORM information off a given login page and then allows you to link the various FORM INPUT fields with the appropriate data-fields on the “card” for that website (e.g. username, password). The one- click sign-in then just has to do a HTTP POST to the desired web-address with the correct FORM data to act just like the real login page. It’s simple enough, but that seems like the real time-saver here. Not only could you have a webapp which stores all your personal data, but it also provides a quick launchpad to login any site you need. That just takes it a step further and makes the webapp almost a blackbox: you don’t really need to know what your username and password is anymore because the webapp database knows it and can feed it to the target website’s login form for you.

This just seems like a really awesome idea, and I’m really tempted to just use Clipperz natively so that I don’t have to re-invent the wheel, but I’m still just nervous about using someone else’s website as a database to store all my personal sensitive data. It’s basically just a trust issue, and I don’t think I have any good reason to trust that their Javascript code will never do anything malicious. I’d much rather control the data myself on my private webserver. If only Clipperz was a SourceForge project… ;)

I already use a quasi-RAID system: I have two identical hdd’s in my file- server box and I rsync the “master” drive to the “slave” drive every so often. Not only does this provide me some amount of rollback-ness (i.e. because I only rsync so often), but the decision to not RAID-mirror the drives was intentional: if the filesystem or partition table on the RAID somehow became corrupted, all my data could be lost.

But…what if my house burns down? (*gasp* Oh noes!!) Yes, that would be sad indeed! So, rather than just providing a single layer of redundancy locally, if I really want to invest in the survival of my important data, I really need to spread that data around; I need to diversify.

I’ve looked around at various web-based back-up solutions like Amazon’s S3 service, but those don’t seem very optimal for me because of the amount of data I want to backup (~150GB) and because it seems like it would be a PITA to do a full restore over my home cable internet pipe. Not to mention the monthly fees, paying someone else to store my data safe and sound. But, dare I trust my important (and partially sensitive) data to a stranger?

Currently, I’m tempted by a seemingly simple solution: just get an external USB hdd, mirror my data once locally, and then throw it at a friend’s house and use rsync to keep it up-to-date. The main cost involved is the cost of the new hdd; there’s no monthly fee because I already need to pay for my internet- access. And I can even return the favor by hosting drives on my end too. And this could even be expanded to a multiple people, if you wanted to back-up your data in multiple off-sites. This seems almost too easy to me, but it seems perfectly effective. Anything, it’s making use of the hidden geek- factor: you’re a geek and you have geek friends, so why not make the most of it and use them for geeky endeavors like helping each other backup each other’s data? ;)

The main problem I have with this plan is that my data wouldn’t be encrypted at all. I’m not sure how paranoid I really need to be about my friends snooping around my data. Though, I think this is basically the general idea as: what would happen if someone stole your computer? So, that’s really more an argument that I should be locally encrypting my data so that even if prying eyes were to get at my local/master copy, they still wouldn’t be able to do much with it.

Has anyone else given thought to getting more serious about backing up their data?

http://kotaku.com/gaming/clips/minesweeper-the-movie-286639.php

Then I found out that someone else had already done the same thing and they had done it better. Enter del.icio.us. (Ooh, the pretty pretty tag-clouds! Yes, my tag-cloud is quite delicious indeed…) Tags are such a simple concept, but oh-so effective. It just makes so much sense to have a webapp where you save bookmarks and can assign arbitrary tags to each entry. del.icio.us then makes it very easy to navigate around your list of links by tags, even providing nifty “easy” URLs like http://del.icio.us/tduckles/madison, which shows me everything I’ve tagged with “madison”.

Even better, people out there have written even slicker tools to access and navigate around your del.icio.us bookmarks.

John Vey wrote this amazingly cool tool called del.icio.us direc.tor. In it’s native form, it’s meant to be a bookmarklet which you use at api.del.icio.us after you’ve done all your HTTP-Auth stuff, so that the JavaScript code doesn’t need to worry about auth when trying to access your bookmarks.

Not only is this a uber-slick UI for navigating around your tag-tree and for quickly finding a link based on (real-time!) searching, but the page itself also goes into the design and architecture of direc.tor. This was a really fascinating read for me, from a web-dev perspective. What really blew my mind about this is the whole concept of using AJAX methods to grab the del.icio.us XML DOM, and then use XSLT and XPath features which are built-in to most modern browsers to do all the searching in real-time on the client-side. So, really, the whole bottom pane of direc.tor is the output of an XSTL on the del.icio.us XML DOM of your bookmarks. I had never realized that you could do XSLT and XPath operations via Javascript before. This was a watershed moment for me, because a whole new perspective on web-development opened up for me: you could use XMLHttpRequest to grab an XML document, and then use Javascript to do an XSLT to transform that XML data into HTML or whatever presentation- layer you wanted. In the past, I had always only thought of AJAX calls returning some chunk of HTML which you could then insert somewhere into the document DOM.

Enter Google Reader. Being the show-off’s they are, Google has created a fantastic RSS reader web-app. I’ve ditched my home-based reBlog setup for using Google Reader for all my RSS feed reading needs. Here are the things which really won me over:

• Super-slick UI, with fantastic keyboard navigation support.
• Nifty AJAX tricks, like only loading enough items into the ListView to fit on screen initially and then fetching more as you scroll down. (Of course, they do this for performance reasons, but it’s still implemented very cleanly and just “feels” nice.)
• You can “star”/flag entries that are useful, and then go back and look at all the articles you’ve flagged. It remembers the timestamp of when you flagged the entry, which I find handy. (Though most useful links I consume in a del.icio.us-fashion these days…)
• You can easily sort either by Oldest or Newest, and it saves that preference on each “view” (i.e. viewing entries from just one feed vs all entries) you have.
• Has an “infinite” scrollback feature, to allow you to scroll backwards through all the posts which Google has ever fetched for that feed.
• It’s free. ;)

I would definitely recommend Google Reader to anyone that uses a fat-client RSS reader/aggregator, especially if you want something that does the aggregating 24-7 for you and is accessible from any computer.

http://www.collegehumor.com/tag:streetfighterthelateryears

The fan community for Firefly seems to be alive and strong, and it looks like a fan-compiled Firefly documentary is on the verge of being released. I’m still debating picking it up… I know I can get fanatical about some things, but I’m not sure if I’m too that point with Firefly yet. I certainly wish that the TV series hadn’t been cancel mid-1st- season.

http://www.donetheimpossible.com/

This weekend, I was suddenly struck with the idea that it would be cool to have an RSS reader with a web-interface, and the RSS fetcher could run as a crontab job. This has the benefits of being able to be accessed anywhere, and to fetch new posts automagically, without needing some desktop computer somewhere to handle that.

At first, I thought it might be a cool home project to write up. Using some AJAX techniques, I could come up with a pretty slick interface. But, surely someone else must have had this same idea too, so I did some searching for existing open-source projects like this. And, sure enough, enter reBlog. This article gives a good overview of the feature-set, and also has a Google Video giving a short workflow run-through.

It uses a lot of AJAX tricks, and it even has full keyboard navigation shortcuts. It uses “,” and “.” as the default row up/down, but I was able to fiddle with the source code to map “j” and “k” to these same functions, so I could use it vi-style. ;) So, it’s really easy to scroll through posts, mark them as read or flag them for follow-up. All the posts you flag (or “Publish” in reBlog terminology) go to a different bucket, which can be rendered to a RSS feed itself, I guess so you can point some other RSS reader to the list.

I’m just very impressed with the project. It’s really just what I was looking for. The 2.0 version is still in beta, and I’m hoping by the final version they offer more flexibility/customizing. So, if you’ve ever wanted a web-based RSS reading solution, check out reBlog.