In most traffic signal optimization problems, pedestrian traffic at an intersection receives minor consideration compared to vehicular traffic, and usually in the form of simplistic and exogenous constraints (e.g., minimum green time). This could render the resulting signal timings sub-optimal especially in dense urban areas with significant pedestrian traffic, or when two-stage pedestrian crosswalks are present. This paper proposes a convex (quadratic) programming approach to optimize traffic signal timings for an isolated intersection with one- and two-stage crosswalks, assuming undersaturated vehicular traffic condition. Both vehicle and pedestrian traffic are integrated into a unified framework, where the total weighted delay of pedestrians and vehicles at different types of crosswalks (i.e. one- or two-stage) is adopted as the objective function, and temporal and spatial constraints (e.g. signal phasing plan and spatial capacity of the refuge island) are explicitly formulated. A case study demonstrates the impacts of incorporating pedestrian delay as well as geometric and spatial constraints (e.g., available space on the refuge island) in the signal optimization. A further analysis shows that a two-stage crosswalk may outperform a one-stage crosswalk in terms of both vehicle and pedestrian delays in some circumstances.