Electricity prices on the European market have decreased significantly over the past few years, resulting in a deterioration of Swiss hydropower firms’ competitiveness and profitability. One option to improve the sector’s competitiveness is to increase cost efficiency. The goal of this study is to quantify the level of persistent and transient cost efficiency of individual firms by applying the generalized true random effects (GTRE) model introduced by Colombi et al. (Journal of Productivity Analysis 42(2): 123–136, 2014) and Filippini and Greene (Journal of Productivity Analysis 45(2): 187–196, 2016). Applying this newly developed GTRE model to a total cost function, the level of cost efficiency of 65 Swiss hydropower firms is analyzed for the period between 2000 and 2013. A true random effects specification is estimated as a benchmark for the transient level of cost efficiency. The results show the presence of both transient as well as persistent cost inefficiencies. The GTREM predicts the aggregate level of cost inefficiency to amount to 21.8% (8.0% transient, 13.8% persistent) on average between 2000 and 2013. These two components differ in interpretation and implication. From an individual firm’s perspective, the two types of cost inefficiencies might require a firm’s management to respond with different improvement strategies. The existing level of persistent inefficiency could prevent the hydropower firms from adjusting their production processes to new market environments. From a regulatory point of view, the results of this study could be used in the scope and determination of the amount of financial support given to struggling firms.