If you only watched Harry Potter movies, the answer to this question might be a bit tricky for you. No shame in it, it is obscured and not thoroughly explained in the movies, and by that, I mean not explained at all.
While there are strong indications that Voldemort hated Harry specifically and knew about him from the moment he was born, it is never answered how and why he was there, killing his mom and dad, Lilly, and James, and attempting to kill Harry, which ended up in a well-known fiasco.
Voldemort was there due to the prophecy regarding him and harry, the one he’s after in the fifth movie. And, that prophecy was in fact passed on to Voldemort by none other than our dear Severus Snape, or at least the first part of it.
The prophecy in question, the one which Harry and the others are after in Harry Potter: The Order of the Phoenix, was given to Dumbledore first by our resident prophet prof. Sybill Trelawney. She is masterfully portrayed in the movies by Emma Thompson, which completely disappears in the role.
She was meeting Dumbledore for an interview for a possible position at Hogwarts, just as Harry was born. So, this was in 1981 in the books. The two met in a pub in Hogsmeade.
“On a cold, wet night sixteen years ago, in a room above the bar, at the Hog’s Head Inn.” – Albus Dumbledore
This is how he begins his story, as he finally decides to tell Harry what he’s been hiding from him for so many years. Dumbledore was going there to interview Sybill Trelawney for the Divination post.
He confesses he was considering dropping Divination completely from the school program, but Sybill was a great great granddaughter of a famous, gifted seer, and he thought he should be polite and meet her still.
Dumbledore was disappointed with her and even told Sybill that he thinks she would not be suitable for the post. He then extracts his memory and puts it into the pensive. A figure of Professor Trelawney herself rises in smoke from the pensive and speaks the prophecy:
“The one with the power to vanquish the dark lord approaches.
Born to those who have thrice defied him.
Born as the seventh month dies.
And the dark lord will mark him as his equal.
But he will have power the dark lord knows not.
And, either must die from the hand of the other, for neither can live as the other survives!
The one with the power to vanquish the dark lord will be born as the seventh month dies”
After finishing the prophecy narrative, the image of prof. Trelawney disappears back into the pensive. Now, Dumbledore continues to explain that the prophecy could have been referring to two people. One is Harry and the other one is Neville Longbottom.
But the main problem he who must not be named, had was that he did not get the full version of the prophecy. As they were overheard, prof Dumbledore and Trelawney, the listener, were caught mid-way and thrown out from the Hog’s Head.
Who was listening in? Non-other than Severus Snape!
Severus was in the dark lord’s service at that moment, and he transmitted the first part of the prophecy to him. It was because of that that Voldemort decided to visit the home of Lily and James in Godric’s Hallow, so many years ago.
And, Severus, unknowingly, condemned the love of his life, to death. It was a mistake that he never forgave himself for and which defined his entire life from that point on and determined his death. But that story we’ll leave for another time.