I have a 2 year old boy and a 4 year old girl for whom I am planning a combined birthday party. Their guests are the same so I thought I would just do the party together this year. Neither of them really watch a lot of TV but they do have some similar interests (not many).
If you have a Hobby Lobby close by, they often have their t-shirts on sale half off which would be 1.50 each. You could do a treasure hunt in the yard and let the children paint their own treasure chest and decorate them. They sell these small wooden boxes at Walmart for about 1.50 each. They are plain unstained wood and have a closing clasp on them. You can also hide treasures in a dough type ball made of coffee grounds and flour. I seen it yesterday on PBS, but they did not give exact measurements on the recipe.
How about a "color" theme. Do a RED party this year, blue next, etc. By the time you run out of easy to do colors they'll be old enough to want their own parties with their school friends.
I know that you said the children don't watch much TV, but you can still get a lot of ideas, printables, etc from www.dltk.com website, as well as others. When my twins were 4, I did a princess themed party where we made our own medieval style hats with flowing fabric (used solid color bday hats & threaded cheap sheer type material thru the end). I made a castle cake (which I saw in a magazine -- still have the picture/pattern for it). You could even include your son (king or prince) and his "knights" (male friends).
That's just a few ideas from a party I did...there are LOTS more on the web. Just decide on a theme -- princess, "ocean/beach fun", "farm fun", "cowboys", etc. and do a web search on "children's parties" + whatever theme you decide.
Good luck