The turning point of the movie is when Sam takes Lucy to a Halloween party, and her friends make fun of Sam for being himself and call him retarded and this embarrasses Lucy. Lucy says that she is actually adopted which raises suspension with some of the parents. This calls into play a social worker who deems that Sam is incapable of taking care of her. And the state takes her away from Sam. When this occurs Sam takes action by hiring a big name attorney to help win the case. All the court battle goes through and Sam loses custody of Lucy to a great family who is planning on adopting her. Lucy runs away from her new family to see Sam, and eventually the family gives custody to Sam, and the woman form the family helps to raise Lucy with Sam.
If I were in the shoes of a judge or a social worker, I would not take Lucy away from Sam, but would do what happened at the end of the movie. Sam is going to need help raising Lucy into her teens. So I would make the situation where a social worker or someone such as that would help Sam with Lucy. He is capable of caring for her, but not capable of giving the enrichment she needs for a growing mind.