to save a few pennies, only get a flip type or candy bar type phone.
i don't suggest a smart phone - they cost more and need a data plan
when you use your email - unless it is disposable too, you have just tagged the burner as belonging to you.
the flip phones are more rugged and don't have a large fragile screen.
battery life on a flip phone is longer.
make sure it is AWS capable - there are 3 main cellular bands in the US. (850MHz, PCS and AWS bands) best to be able to use all three. if you live outside the US, make sure it will work with 900, 1800 and the UMTS band. outside the US you can get cheap phones and SIM cards in vending machines in airports. In the US, many convenience stores sell them.
keep the battery out of the phone, unless it is in actual use. - like any other radio - set up a regular contact schedule to minimize the amount of time your phone is on. you can pick up your text messages and voice mail then. don't use the phone in a location easily traceable to you. (use a blootooth headset with the phone in a pocket - cuts down on the likelyhood a cc camera will tag you in a public area as the user.