Schwartz has been a fan of Thoroughbred racing since his childhood in New York. He purchased his first horse in 1978 through a friendship with Carl Rosen, owner of the champion filly Chris Evert. Schwartz is a successful owner/breeder and many of his top runners come from his 700-acre Stonewall Farm in Granite Springs, N.Y. He is the former CEO of Calvin Klein Inc. He turned a $10,000 investment in 1968 into a multibillion-dollar corporation with Calvin Klein, his childhood friend. Schwartz was as a member of the board of trustees of the New York Racing Association from 1994 to 2004 and served as chairman of the board of trustees and CEO of NYRA from 2000 to 2004. He is a member of The Jockey Club. In nearly 35 years as an owner, Schwartz has campaigned numerous stakes winners, including Degenerate Jon, Boom Towner, Only Queens, Perfect Poppy, Seeking the Sky, Fire King, Turnofthecentury, Dr. Root, Beru, Great Intentions, Fernando Po, and Acorn winner Three Ring.