Home Mortgage Tax Simplification Act of 2004 - Amends the Internal Revenue Code to allow a tax deduction in the current taxable year of points paid for the refinancing of a principal residence to the extent that the resulting indebtedness does not exceed the sum of: (1) the amount of debt refinanced; plus (2) the lesser of $10,000 or the points paid.