Rest assure that it's HIGHLY IMPROBABLE for a hacker to remotely turn on & access your computer freely as there's a bunch of conditions that would have to happen here.
1) You would have to enable your computer to "Wake on LAN" in Windows & BIOS / UEFI -- This is typically disabled by default & would require you to enable it in both spots. In order to modify the BIOS / UEFI settings, you have to physically disrupt your computer's boot settings (typically by holding down a specific key. While it can vary by motherboard maker, it's typically F2).
With this setting on, the majority of your computer will be off, but a small part will remained powered in order to check for a "magic packet" (which is the special piece of data sent to your system) that will tell your computer to turn on. This is similar to what your TV does with it's remote control.
On the practical side, enabling "Wake on LAN" can be helpful when you want to remotely access your computer without physically touching it. If you have a Steam Link connected to a TV & want to play a game through your computer, the Steam Link will send that magic packet to turn on your computer for you.
2) Your computer would need to be on a wired connection -- "Wake on LAN" only works with the ethernet port on your computer & NOT with WiFi connections as the WiFi card & the antenna array powers down when you turn it off.
3) You have port forwarding enabled on your router -- Any time you have an inbound connection request from an outside source, your router needs to know which computer to send that request to. While this is typically done when you're hosting a server (be it a private web, file or game server), this would need to be done for the "Wake on LAN" request as well.
4) You KNOW your LAN's external IP address (or have a DNS address for it) -- Whenever you're hosting a server, you need to know the IP address that your ISP gave you for your connection. This will be something like 192.168.1.1 (NOTE: If you're a total noob, this is NOT my IP address... this is just one of the "Internal" IP addresses that routers typically use), but it'll be something a bit more unique for you. Doing a google search of "What is my IP address?" will provide this.
If you're a bit more savy, you may choose to use a DNS to convert your IP address to something easier to remember like "MyServer.MyDomain.com" so you don't have to memorize or give out your actual IP address. If your ISP does dynamic IP addresses (where it changes every couple of days or so), utilizing a service like No-IP.com & their sync tool to keep things working.
5) You have a "Wide Area Network Wake-up on LAN" (WAN WoL) tool on your remote access device, like your smartphone or another computer -- This app actually sends the magic packet to wake up your computer.
6) You know your ethernet card's MAC address -- The MAC (Media Access Control) address is the ID number for your networking device (be it wired or wireless). This is needed to ensure that magic packet is going to the right device.
7) You have a remote access application / tool that loads up at boot -- This could be something like TeamViewer (for more general usage), Steam (geared more for gaming) or Remotr (geared for gaming) that has some type of login requirement to connect remotely. Nvidia even has a GameStream feature built into their graphics card to work with the Nvidia Shield (again, mostly for gaming), which has limited use.
8) You have granted permission for said remote access applications to communicate with remote & local systems with your firewall -- Most trusted security suites & firewall makers will typically have an automatic rule to allow the well-known apps (like those noted above) to connect to other computers so that you don't have to deal with it (unless you really want to). However, if the program is unknown or questionable, your firewall will typically block the connection while it's asking you for permission.
If the hacker has done everything on this list... then you would have a legitimate concern. HOWEVER, either you did this yourself for your own personal reasons OR somebody you know would have to physically access your computer as well as connect to your router to permit this.
Considering the hassle involved configuring the WAN WoL stuff, hackers will more likely go after systems that are already on & running as they're easier targets to hit which could allow them access into other items / systems as a result.
Therefore, if you're 100% paranoid about hackers, turning off your computer is most likely the safest protective measure you can do.