Cognitive dysfunction is a common comorbidity in people with epilepsy, but its causes remain unclear. It may be related to the etiology of the disorder, the consequences of seizures, or the effects of antiepileptic drug treatment. Genetics may also play a contributory role. We investigated the influence of variants in the genes encoding neuron-restrictive silencer factor (NRSF) and brain-derived neurotrophic factor (BDNF), proteins previously associated with cognition and epilepsy, on cognitive function in people with newly diagnosed epilepsy. A total of 82 patients who had previously undergone detailed neuropsychological assessment were genotyped for single nucleotide polymorphisms (SNPs) across the NRSF and BDNF genes. Putatively functional SNPs were included in a genetic association analysis with specific cognitive domains, including memory, psychomotor speed, and information processing. Cross-sectional and longitudinal designs were used to explore genetic influences on baseline cognition at diagnosis and change from baseline over the first year since diagnosis, respectively. We found a statistically significant association between genotypic variation and memory function at both baseline (NRSF: rs1105434, rs2227902 and BDNF: rs1491850, rs2030324, rs11030094) and in our longitudinal analysis (NRSF: rs2227902 and BDNF: rs12273363). Psychomotor speed was also associated with genotype (NRSF rs3796529) in the longitudinal assessment. In line with our previous work on general cognitive function in the healthy aging population, we observed an additive interaction between risk alleles for the NRSF rs2227902 (G) and BDNF rs6265 (A) polymorphisms which was again consistent with a significantly greater decline in delayed recall over the first year since diagnosis. These findings support a role for the NRSF–BDNF pathway in the modulation of cognitive function in patients with newly diagnosed epilepsy.