A friend of mine has some friends who've been holding a small music festival on their ranch somewhere in Oregon every June. Apparently they hire 3 or 4 local bands from the Portland area and then have some people who come and build a stage and do a whole sound and light set-up and everything. Everyone pitches in about $30 to cover the cost of the music, some drinks and a collective meal that is prepared on the Saturday. I think this year will be their fourth year. Last year my friend said about 200 people showed up, with some coming from as far as San Francisco, and that's just from word of mouth.
Obviously, it's still pretty grassroots right now but I think that's the kind of way to start something. It really just requires having the right mix of people at the beginning who can provide certain elements, like land, technical and musical hook-ups, etc. Another friend of mine hosts an electronic festival in the Oakanagan every summer (forget what it's called but a few thousand attend now), and he started it in essentially the same manner, just having his own farm and friends that put raves on in the city who could do the lights and sound.