Electric heat costs more, but gas heat only heats your whole house. Using electric heaters, you can choose to heat only the room you're in and let the rest of the house's temperature drift.
NOTE: whatever you do, don't fall for the idiotic trap of those expensive electric heaters. They don't do anything more effectively than any other electric heater. Electric heaters convert electricity to heat, and they all do it at the same rate: ONE WATT = 3.4 BTU. That's it. No heater is going to do better than that. Heat pumps can do better, but they're a different technology that moves heat, not converts electricity.
You can throw away over $100 for the Duraflame electric heater, or you can spend about $40 for an electric radiator which does the exact same job with the exact same conversion rate. Or you can look at thrift stores for used ones for even less - that's where I get mine.
At my house my wife uses a variation on the Japanese kotatsu, a table with a heater under it and a heavy quilted skirt all the way around it. In the Japanese style the table is very low and the quilt spreads a couple of feet around it; my wife's version is a conventional dining-height table with the skirt falling straight down (but with pleats so it can flare out to cover her lap when she's tucked in at the table). With an electric radiator set on LOW and the thermostat controlling how hot the radiator gets, it's extremely cozy under the table and it keeps her entire lower body warm, giving her a comfortable place to work on her art projects no matter how cool the house is (she doesn't notice until the house gets down to around 55 degrees or so).
When the kotatsu is on, you can't get the cat off the table for love or money. It's warm and comfortable and she's in the way of whatever my wife is doing, so it ticks all the boxes on a cat's to-do list.
Another thought: sweaters.
I'm not joking. We've allowed ourselves to be trained to believe that comfort is just a twist of the knob away and you don't have to do anything. But you could expend just a tiny bit more effort and pull on a sweater or stocking cap and be warm without turning any knobs at all. Thin, lightweight long underwear like hikers use makes a BIG difference too, keeping your lower extremities warm without adding a lot of bulk and making your pants feel tight.
Your first, best money is spent on insulation, specifically insulation that YOU wear and keep your heat inside you. Then add some to windows, then start thinking about moving a small electric heater around to keep just YOU warm and not trying to heat the whole house.
Good luck with it.