Short-scale local adaptation is a complex process involving selection, migration, and drift. The expected effects on the genome are well grounded in theory but examining these on an empirical level has proven difficult, as it requires information about local selection, demographic history, and recombination rate variation. Here, we use locally adapted and phenotypically differentiated Arabidopsis lyrata populations from two altitudinal gradients in Norway to test these expectations at the whole-genome level. Demography modeling indicates that populations within the gradients diverged <2 kya and that the sites are connected by gene flow. The gene flow estimates are, however, highly asymmetric with migration from high to low altitudes being several times more frequent than vice versa. To detect signatures of selection for local adaptation, we estimate patterns of lineage-specific differentiation among these populations. Theory predicts that gene flow leads to concentration of adaptive loci in areas of low recombination; a pattern we observe in both lowland-alpine comparisons. Although most selected loci display patterns of conditional neutrality, we found indications of genetic trade-offs, with one locus particularly showing high differentiation and signs of selection in both populations. Our results further suggest that resistance to solar radiation is an important adaptation to alpine environments, while vegetative growth and bacterial defense are indicated as selected traits in the lowland habitats. These results provide insights into genetic architectures and evolutionary processes driving local adaptation under gene flow. We also contribute to understanding of traits and biological processes underlying alpine adaptation in northern latitudes.