Help finding a book

I remember reading a book about a boy that was raised by a maid in secret. The boy lived with the maid in a house next to a field of flowers. The boy was not allowed outside but could only look at the flowers from the window. One day he went outside and saw a girl from the mansion. Later when the boy was discovered he was locked away in the mansion stables and treated like an animal. That is the best description I have unfortunately if anyone can help

by Coxymoron22