At-Home Infant Care Act of 2004 - Amends the Child Care and Development Block Grant Act of 1990 to authorize appropriations for the Secretary of Health and Human Services to make grants to eligible States to provide financial assistance for up to 24 months (in the aggregate) to enable an eligible parent to become the caregiver for an eligible infant at home.