El's post on the matter, here, inspired me to make a list of my own.  I totally agree with the "start small" mantra. You want your first season or two to be successful, and be inspired to keep going. Even now, I have to curb my impulses in spring so that I don't make myself crazy in late summer.  Keep it fun.   And remember that every garden had humble beginnings.

Our yard when we moved in:  A blank slate!  The only "useful" plant we had was that patch of rhubarb in the corner

First year's garden, I can hardly remember what was in there:  tomatoes, and lots of herbs I think.  Lots of weeds! We grew birdhouse gourds on the clothesline pole and they took over...

1. Go cheap or free as much as you can.  Do take advantage of your energy levels in spring, and use it to save some cash:  Look for municipal compost sites, or free manure on craigslist.  Leaves and grass clippings are easy to come by.  Don’t invest in expensive infrastructure at the start of a new garden:  you may be rethinking or expanding your layout, so committing to fixed fencing or permanent raised beds may not be the best bet for the first season or two.  Share seeds with family or friends.  Tools can be found at garage sales.  Start-up can cost a bit, so go small and improvise when you have to.  
2. Ideas and Advice are often free:  Be observant at the farmer’s market:  Talk to your vendors, or if you’re shy, take a careful look at what they are selling. They are growing things that do well in your particular climate.  If you see something you've never eaten--buy some and give it a try.  Also—having a bad year with a certain veggie?  Maybe it’s not just you.  Until you have earned a few seasons of experience, use someone else’s!  Same goes with neighbors and community gardeners—most people like to talk and share about their gardens. University extension offices have great information suited for your climate, check their websites for free publications.  I read a lot of bulletins meant for market gardeners.
3. Find your own way:  Related to the above, beware of gardening advice that says you HAVE to do things a specific way.  Every area and gardener is different, and it will take you a while to find the style that fits you best.  Like a healthy diet, there is rarely one size that fits all.  So watch and learn, and use the resources you find in your area.  I find this is like bread baking—there are lots of techniques and one will click for you. And be flexible yourself, be willing to try new things, my gardening style has changed SO much over  the last 10+ years.
4. Don’t torture your plants, especially when starting plants indoors.  Better to get started a little later and have small sets than have tall, leggy, root-bound plants that will take longer to recover after transplanting into the garden.  Smaller plants often do better and catch up faster than big ones.  And if you buy sets, smaller plants cost lest too!  Same thing applies here: take good care of them, don’t let them languish for weeks before planting. 
5. Harden off!  It’s a little early to talk about this, but I see a lot of new gardeners that plant out tomatoes in the middle of a 70-degree day, and they shrivel up or break in the first blast of wind.  Plants have to acclimate from their sheltered early lives:  get them used to changes in temperature and sunlight, and let them feel some gentle wind to make strong stems.  Here’s where a little torture is okay to get them used to the harsh world.  Plant on a cloudy day if you can, or after the heat of the day has mostly passed, to make their first day out less stressful.
6. Hold back some reserves:  We almost always get some nasty hailstorms in late spring.  By holding back some sets of plants, I have a backup in case of a catastrophic loss. Leave those extra plants in a sheltered spot, and in some shade to keep them from getting too big or pot bound. 
7. Do take some risks. It’s okay to plant one or two plants early, just to experiment, especially when you have some spares in reserve (see above).  Make an improvised cloche or cold frame, and see how things do. 
8. Finally, keep notes.  Watch what happens.  Take pictures.  Make a garden journal or get a planner/calendar (whatever works for you) and write down what you do and what you see.  Spring weather, consecutive plantings, draw MAPS of where you plant things.  Update on how things did.  In future years, you can look back and see what worked for you, and you'll have a record of the history of your very own micro-climate.  And you'll see how much you've grown as a gardener!

A few years in:  crummy fencing, and the start of raised beds.  I was told it looked like a graveyard!

Most recent addition, a couple of springs ago.  It looks better filled with veggies1