Once you start seeing flowers, you should be about 4 weeks away from harvesting… or so I have read.
You can start getting the potatoes out of the ground as soon as they are the right size for you. If you planned from slips, this will be 3 or 4 months after planting. If you are like me, and started your sweet potatoes from seed, you may have a bit longer. As a general rule of thumb… you should let the leaves start turning yellow, but you can leave them in the ground up until the fall frost.