I would secondary only if adding ingredients such as in your case, or when doing a complex dry-hop with multiple hop charges. Otherwise I'll crash the primary after completion of fermentation is confirmed by at least two consecutive days at the same SG, plus a couple more days to ensure the yeast has cleaned up after themselves. No need to add extra steps if not necessary - just increases the chances of infection, oxidation, etc.
I suppose you could crash it in the primary prior to racking onto the fruit in the secondary, but I'd think you'd want the beer to be at warmer than freezing temps to infuse the fruit characters - could be off on this one, never tried it.
Me? I crash a couple of days prior to packaging but I keg my brews. If you are bottling, you'd want the beer to be at fermentation temperatures or the yeast will not eat the priming sugar and carbonate your brew. Even if you DID crash it in the primary to get most everything to drop out, you'd still have plenty of yeast available for the bottle, but I'd let it come back up to fermentation temperatures prior to bottling to ensure the yeasties are wide awake for their next meal.
Crashing to clear your beer can be done, with time being the most effective factor. Where time is an issue, there is always gelatin. I just tried it for the first time on my last IPA and I gotta say; it really works! But then again; I keg so I'm not relying on residual yeast to carb my beer. Sitting in the bottle carbonating and conditioning for a week or three can do the same thing - you'd be surprised how clear your bottled beer can become by the time it's ready to enjoy.