The guild levels give guild-wide perks, like reduced hearthstone cooldowns and being able to summon your guild bank. Buyable rewards are items like mounts, tabards, heirlooms, and pets and have less to do with your level (I think) than your achievements.
For small or casual guilds, I'd try to hit at least guild level 22. As we know from the Goblins, time is money and that means Bountiful Bags is amazing. More herbs, enchanting mats, ore, and skins per gather means more crafting items and thus more gold from the AH.
Level 11 is when you get the final experience buff for leveling alts.
I expect that leveling your guild will require less effort than collecting rewards.
Guild rewards, like mounts and pets and tabards and such will be buyable with gold when the guild manages to acquire them. We do not, as far as I know, have any clue what all the rewards will be or what specific tasks are required to collect them.
When we do have a listing, be sure that I'll let you know.
Rewards, though, will likely be more difficult to collect than guild levels. Rewards are for cooperative effort (probably from guild achievements), and if your guild is small or casual, I expect rewards to be difficult.
As a mount and pet collector, though, I'll be keeping you apprised of how to get rewards for, say, a guild that has only 1-5 people in it.
All of this really makes me wish Horde had a race I'd feel comfortable changing Birdfall to. I joked back before race changes had even been hinted at about making Bird a Goblin, but I honestly don't think I could move my Alliance alts over until Horde has a bevy of races I'd really enjoy. If ever.
But the temptation is there. The thought of benefiting from my guild, being able to help on Birdy, being able to do stuff at all. . . . Tempting. Quite tempting.