Judy, Judy, Judy!
You know the rules! The number of nights in each parents home is the ONLY thing that the IRS cares about. Legal custody is meaningless. It would be one thing if she was away at school but it's another thing entirely since she's living with her father. She's not "temporarily away" from anything. I certainly would not be putting MY signature on anything that didn't comply with a strict reading of Pub 17, Pub 501, and Pub 596.
Absent any 8332s:
Dad claims 17 year old who lived in his home for more than half of the year, files as HoH if he's considered single for HoH purposes and paid more than half of the cost, and claims the EIC if eligible. No CTC due to age.
Mom claims the younger sibling, files HoH, EIC, CTC, ACTC, yadda yadda.
If it's mom's year per agreement to claim the kids, dad give her a Form 8332 listing the 17 year old. She files HoH if eligible & claims both kids as dependents but only claims the EIC, etc. for the younger sibling. No CTC for the 17 year old due to her age. Dad claims the EIC and files as HoH based upon the older sibling living in his home. If his income is too high for EIC, too bad.
Of course there's the matter of mom claiming SNAP based in part on a family member who does not reside with her and the child attending school in a district where she does not live. You certainly have a duty to warn her of the fraud potential there. I'll let you make up your own mind on the moral duty to the state and other taxpayers on how far you go with that.