Safe and Fair Enforcement and Recall for Meat, Poultry and Food Act of 2006 or SAFER Meat, Poultry and Food Act of 2006 - Amends the Federal Meat Inspection Act, the Poultry Products Inspection Act and the Federal Food, Drug, and Cosmetic Act to: (1) require a person, other than a household consumer, who has reason to believe meat, poultry, or food products to be adulterated or misbranded to so notify the Secretary of Agriculture or the Secretary of Health and Human Services, as appropriate; (2) provide the Secretary of Agriculture with authority for voluntary and mandatory nondistribution and recall, withdrawal of inspectors from violating facilities, consumer and health official notification, and imposition of civil penalties; and (3) provide the Secretary of Health and Human Services with authority for voluntary and mandatory nondistribution and recall, consumer and health official notification, and imposition of civil penalties.