I think the mod_rewrite problem was caused by there being no $ at the end of the match pattern. It should be:
RewriteRule ^(index|atom[0-9]*|rss|b2rss|b2rdf).(rss|rdf|rss2|xml)$ rss.php?file=$1&ext=$2
That should make it match end of line, so even first, rss won’t get matched, as the 2 in rss2 would prevent it.