I went through this whole train of thought when I tore mine down. I tossed around the idea of the cam... and if we pulled the engine or did it cab on I would have done it. It would require the removal of the trans, but it could of been done. The cam was $600, then I figured another $600-$800 for pushrods, valve springs and new rockers, or somewhere in that price range. I decided it was money I didn't feel like spending at the time, and I don't necessarily regret it. We ended up doing cab off anyway so the cam would have been more work.
I thought about just doing upgraded pushrods as well, but ultimately decided against it. After talking with a few guys who had been down that road, I realized there was no point in doing the upgraded pushrods. They're made to be the first part to fail for a reason. The thought behind the beefier pushrods was also that you're adding extra mass to the pushrods by going with a beefier design and actually making it easier to float the valves. How much easier? Not really sure, I never crunched any numbers on it, but the theory makes sense. More mass means the valve springs has more force it's working against. If you upgraded the valve springs it wouldn't be an issue, but then you have stiffer valve springs pushing harder on your cam. I know in the four wheeler world when we built those, it was a no-no to do the stiffer valve springs on a setup that wasn't turning any more RPM and didn't have a hardened camshaft because it just wore the cam out quicker. Again, probably not a big deal, but just something that wasn't necessary and seemed to have more cons than pros I guess, even if they were small cons.
My vote is either just use what you have now, or do it all at once, cam, pushrods, valve springs and run it.