Specially Adapted Housing Grants Improvements Act of 2005 - Provides that if a veteran who is totally and permanently service-disabled (due to loss of, or loss of use of, one or more limbs or organs) resides, but does not intend to permanently reside, in a residence owned by a member of that veteran's family, the Secretary of Veterans Affairs may assist such veteran in acquiring such residence adaptations as necessary due to such disability. Provides assistance limits. Prohibits any veteran from receiving more than three assistance grants.