First on the suspect list is the famous ligustrum bush that bursts into bloom all over the city and countryside. Ligustrum has a very fragrant scent and I have been told (although I have yet to authenticate this fact) that it is a relative to the lilac plant which also evokes some of the same reactions from folks with an allergic factor at work.
The next culprit may well be the beautiful and sometimes undiscovered "sweet olive" plant. I am not sure what its real name is, but anyone who has lived in New Orleans knows exactly what I am talking about here. I once actually thought that a candle with the scent of sweet olive would be wonderful to remind me of my home town. I gave up on that and adopted the Magnolia instead and I am sure all my friends are thankful.
Rounding out the trio here is the Jasmine plant - I believe that this one may well be the variety that is called Confederate Jasmine, but don't hold me to it. I will have to go and research this for us. Right now though, I am using the names that were bandied about the neighborhoods where I grew up.
Is there any plant that you remember from your childhood that makes you remember things you had long forgotten? Another that I am waiting for is the ginger plant - butterfly ginger grew right under my window and I still look forward to its blooming today! Go take a stroll around your garden and see what memories you can conjure up!