I have a heavy traffic carpet area that I just can't get all the way clean. I have a Bissel Steam Cleaner. I have even sprayed it with Oxy-Clean laundry pre-treatment spray before using the steamer, but it still looks dirty compared to the surrounding area.
We just bought another Hoover Steamvac, our old one broke after many many years of use. Our carpet had horrible high traffic areas. We took some of the industrial type carpet cleaner from Home Depot and put it in a garden sprayer, and mixed with water. We pre-treated carpet by spaying several times with the garden sprayer and let sit for about an hour. We put very very hot water into the tank and used the same cleaner in the machine. It turned out like new carpet! The product we used is called ProForce Carpet Extraction cleaner. Worked great!
Try SPOT SHOT carpet spray. It comes in an orange and blue aerosol can. It's made by WD-40 I think. It is amazing. We used it in a rental and didn't have to replace a carpet that I thought was ruined.
I have the same problem. I've found that the cleaner remaining in the carpet gives the dirt something to cling to, especially in the high traffic areas. Try using your machine with just water and 2 ounces of fabric softener. This will pull the residual soap out, and the fabric softener will kill the suds.
Try using shaving cream. Spray a bit, wait, then use a rag to scrub a little.
Use a plastic runner. It doesn't look classy, but once you get it clean, it will stay that way.