Women's Sustainability Recovery Act of 2004 - Establishes FY 2004 funding priorities for women's business development centers by: (1) reserving a certain percentage of available funds for sustainability grants under the Small Business Act notwithstanding current limits on the use of unawarded funds; (2) setting a maximum funding amount for such grants; and (3) specifying a formula for continuation grants to centers that received FY 2003 funding.