I'm guessing that when you say you'll enter France with a Long Stay French Visa, you mean you'll have a "visa de long séjour"?
All the details you should need are on the website of the l'Office Francais de l'Immigration et de l'Integration (the OFII), though they're only available in French. The page that should cover your situation is this one, and there's a very detailed Wikipedia.fr page on it too.
Under Schengen rules, a French resident (who isn't also an EU citizen) is allowed to visit other Schengen countries for up to 90 days at a time. The problem is that an un-validated Visa de Long Séjour doesn't seem to qualify you as a resident, that only happens when you go through the OFII to get it validated. Once validated, they'll issue you with a sticker (vignette) which goes in your passport by your visa. If you don't have the vignette, then if you leave France (including to visit another Schengen country) then technically you're not entitled to return until you get a Visa de Retour Consulaire
So, it looks like you're not technically supposed to leave France until such time as you've applied to the OFII for your vignette, you've filled out the appropriate forms and completed the appropriate linguistic tests / medical exams / interviews (as required), and it's been issued by the OFII. Could you get away with it? Maybe, especially if you don't fly, but it's pretty risky as if found you could potentially be deported home from the country you're in (assuming you didn't also get a short term visa for there), and then have to get a Visa de Retour before you can fly back to France...
One thing to be aware of is that the rules changed on the 1st of September 2009, but a lot of the resources you'll find on the internet about the "visa de long séjour" pre-date that, and talk about a residence permit (carte de séjour). You should check with the OFII website / your nearest French embassy to be sure that you're covered by the new rules where you don't need to go down that route (from your question it looks like you don't need it), and then make sure any websites you're reading about the process are new enough to cover you!