That's a good point. It's not a tax in the traditional sense, but the money does go to the government and it does unfairly affect poor people. In my mind, there's not really a difference between that and a cigarette tax.
edit: When buying lotto tickets, you're giving money to the government. If the Fed prints more money, you're not actually giving any more money to the government.