I found this description of someone who I think was looking for the same book:
Driving me crazy trying to find this book: it was a romance, Historical I think… . heroine is in Hero’s mansion/ home, he is believed to be deceased. But he is just injured, His portrait hangs in the bedroom and he comes to the room at night pretending to be a ghost and appears to walk out of the portrait. They talk each night, he is trying to drive her out of his home, but she has no choice but to stay as she has been sent there almost as a prisoner or exile.”
In addition, I think the “caregiver” eventually sends this woman to live/work in a brothel. I think it’s set in Korea?
Any help would be greatly appreciated. Thanks
​
​
by Repulsive-Mistake496