Yes, believe it or not, the set does matter.
First, winning a chess game does not only depend on one's chess knowledge and skill, but also depend on his/her psychological strength. It's hard to measure or tell, but it's there. Like in any sports, competing in a familiar environment makes one feel more comfortable and confident. That's why there are home game and away game, to be fair. For chess, other than playing venue, the set is a critical part because you look at it during the entire game. USCF rule actually stipulates that black player has the right to pick a set.
Second, similar to the first point, touching the piece is also important, and sometimes it gives you inconscious confidence boost. In fact, GO (Weiqi) players love to touch their pieces all the time, even when they are not working on any critical position, same are poker players, maybe also the basketball players and other ball players.
Third, not all chess pieces are made same, just like printing fonts. Most chess sets sold in toy stores are not manufactured precisely according to any standard. Some pieces are difficult to distinguish because of their smaller sizes, such as Bishop vs Pawn, and Queen vs King. It creates unnecessary difficulty for studying and playing.
Fourth, a standard set will make your child feel more formal, and take his/her games more serious, then s/he can learn better. Chess is not a toy for little baby anymore. This is the first step s/he makes for maturity.
A funny tidbit, the formal World Champion Anand doesn't have any set at his home. When a reporter wanted to play a game against Anand, he was told to bring in a set. Anand is very good with computer since its early age, and he mostly use his computer and more importantly his brain. He can see and feel everything in his brain.