Passenger Security Act of 2004 - Amends Federal transportation law to repeal the security screening opt-out program (under which an airport operator may apply to the Under Secretary of Transportation for Security of the Transportation Security Administration (TSA) to have the screening of passengers and property at the airport be carried out by the screening personnel of a qualified private screening company).