One way that you can smooth out your fabric when using fusible fleece is to first fuse a layer of Shape Flex to the fabric, then follow it with a layer of fusible fleece.
Pellon SF-101 Shape Flex Fusible Woven
I use this in 100% of my bags. I might use it in different ways, but bar none, it is the most important interfacing in my stash. I fuse Shape Flex to every single pocket that I make. I fuse it to facings in garments. I for sure fuse it to an area around a zipper to help reinforce that area.
What Sara has to add - Shape Flex will make a quilt-weight cotton have the feel, once fused, of a decor-weight or canvas-weight fabric. It will give the fabric a bit of sturdiness. As with the previous interfacings, the rough tacky side should go against the wrong side of your fabric before ironing. It is perfect as a stand-alone interfacing if you are making a small pouch or other smaller accessory. You can even combine it with other interfacings. Someone asked me the other day what is the most interfacing I have used on one piece of fabric, and the answer is 3. I like to experiment a lot, and there is no right or wrong answer. I might use a layer of Shape Flex, one of Thermolam, and another of 520.