It is happening at all JoAnn's stores, everywhere.
But, hey, you're talking about two years ago. Inflation happens. Prices go up, demand fluctuates, need changes. Less people sew now than they did even 20 years ago, but the stores selling fabric and making fabric still need to earn enough - and the cost of living has risen since then, too. Is it fun? Nope. But it's just a part of life. I'm sorry you won't be able to pay the new price, it's never enjoyable dropping a hobby that you can't afford any more. Perhaps you should start looking in thrift stores for sheets, skirts, shirts, etc. that you could rip apart for the fabric?
Edit: I've already said I'm sorry it's too much for you, but too much for you may be not even enough for them. If you increase your prices, yes, you may lose sales. But if prices go up, they go up. I understand you're upset (and you have every right to be), but saying "Well I sell the things I make!" isn't going to make prices go down. If prices go up for fabric you have to raise prices for the items you sell. Maybe you will lose sales, but you can't charge less than you spend. But you already know that.
I didn't know that, at thrift stores around here full length skirts are $2-4. Perhaps try different thrift stores if you have more in the area or look at online fabric stores (though as the other answer said prices have gone up everywhere, or will soon). Other than that, maybe you just need to be patient with rummage sales. Sure some people get rid of things that really should be trashed, but not everyone does. And often the good items are hidden under the trash items, you have to dig. Good luck.