Preheat oven to 325 degrees. Mix together flour, baking powder, and nutmeg. In a saucepan, combine milk, lemon balm, and lemon thyme. Bring to a boil, then remove from heat. Let stand, covered, until cool.
In a separate bowl, beat butter until creamy. Add sugar gradually, beating constantly until light and fluffy. Add eggs, one at a time, beating well after each. Add flour mixture alternately with milk mixture, beginning and ending with the flour. Stir in lemon zest. Pour into a greased and floured 5x9-inch loaf pan.
Bake 50 minutes or until a toothpick inserted near the center comes out clean. Cool in the pan 10 minutes. If glaze is desired, pour half of the glaze over the top of the cake before removing to a wire rack to cool completely. Finish with the rest of the glaze.
Stir together sugar and lemon juice until mixture is smooth.