Didn't pay enough attention to that last part where you said "When I transfer over to the secondary in a day or two..." I would not be so quick to transfer to secondary and here's why; after 4 days, even with a fairly active fermentation, the yeast are still doing their thing whether finishing up on the attenuation tasks or keeping house by cleaning up after themselves.
Provided you keep an eye on temperatures and have a good healthy pitch of yeast, typical fermentation activity for ales will peak in 2 or 3 days but that is no indication it's done doing what it's supposed to do.
Check the gravity with a hydrometer (or similar instrument) after about 4 days or when the bubbling slows significantly, then check it again the next day to see if you get the same number, and perhaps even again in another day to be sure fermentation is over. But wait; there's more...
Even after the bubbles have slowed and the gravity is stable, the yeast are still cleaning up for at least another day or two getting rid of the byproducts of fermentation such as diacetyl and acetaldehyde. Letting them do this will avoid off flavors in your finished beer, the resulting product will likely be much better.
