Restoring Democracy to the U.S. Congress Act of 2004 - Amends the Rules of the House of Representatives to address: (1) the time limit on roll call votes; (2) actual voting required in conference committee meetings; (3) waiver of the germaneness requirement for conference reports; (4) removal of Members from committee meetings; (5) a limit on redistricting after an apportionment of Representatives; (6) availability of legislation on the Internet; (7) bribery on the House floor; (8) hacking into other Members' computer files; (9) capping committee mailing expenses; and (10) a requirement that at least one-third of committee budget be provided to the minority ranking member.