Can we talk about what my old manager said after I quit?

I left my warehouse job in Baltimore last month and ran into my old manager at a gas station. He said, "I would have fought harder for you if I knew you were looking." That line has been stuck in my head for two weeks. I never told anyone I was unhappy because I thought it would get back to him and make things weird. I just quietly applied to places and took the first offer, which pays $2 less an hour but is closer to home. Now I keep wondering if I left money and a promotion on the table just because I was scared to have a hard talk. Has anyone here actually told their boss they were thinking about leaving, and did it work out or blow up in your face?