When the heads were off did you by chance inspect the valve-guide clearances? Comparing an early year repair manual I have to my '08 manual there was a change resulting in a .0004" increase in the exhaust valve-guide clearance (went from .0006" min to .001" min). I don't have an 04 manual to see if it also lists the tighter clearance but no matter what manufacturing could have still missed their proper clearances on it.
If all else fails, since you comfirmed SVS
it might be worth going back in to re-inspect those clearances, and if needed hone or ream the guides to achieve .001" minimum exhaust clearance. If you measure yours at .0008" -.0009" you're probably going to be fine since the old minimum was .0006, unless you have something else going on like a slightly bent valve. Don't rule out intake clearances as an issue either. Check them all.
I certainly hope you find a different cause, like finding a timing issue or something else. There's nothing worse than having to go in and repeat work, other than not fixing the problem.
best of luck