My biggest trigger is fatty foods and olive oil... Being Italian, oil is in practically all the foods we eat. So I watch what I eat and how it's cooked and because of this I rarely go out for dinner. Just to avoid the pain and suffering that may occur after the meal. If I'm staying home say a Saturday night I'll indulge in the some of the foods I love but that's once in awhile. But I know I'll pay the price for it after. I stop drinking espresso and coffee, my alcohol consumption (I love wine) is next to zero. Like you, I also have an issue with milk products, being lactose intolerant.
Having a different diet when you were younger wouldn't have changed a thing. IBS can hit males at anytime usually between the ages of 19-50 years old. 20% of men suffer from some form of IBS. Being that the stomach is like a second brain, anything can affect the bowels. That's why doctors can't make heads or tails of the cause and no magic pill can fix it either if the unknown is causing the issue.
Are you taking any or tried taking a pre and/or probiotic? Fish Oil? Peppermint Oil? Ginger? Fennel?