Connected and autonomous vehicles (CAVs) are predicted to alleviate traffic congestion, particularly at road intersections, which are the major bottleneck of the urban road network. This paper proposes a signal-vehicle coupled optimal control strategy for mixed traffic flows of CAVs and human-driven vehicles. The method follows a two-layer architecture, which formulates the signal-vehicle control tasks as two cascaded optimization problems by a notion of mixed platoons so that they can be efficiently solved by the central coordinator. In particular, the upper layer is designed to minimize the total waiting time of all vehicles in the intersection, while the lower layer is formulated to minimize the aggregated vehicle energy consumption by adequately exploiting the signal plan, number of crossing vehicles and target crossing speed obtained in the upper layer. Extensive simulation results are provided to examine the performance of the proposed signal-vehicle joint control framework and to reveal the impact of the introduction of the new algorithm at different CAV penetration rates, traffic demands and electric vehicle ratios. The comparisons with existing methods demonstrate the benefit of the proposed method in terms of fuel usage and traffic throughput.