Histone methylation is believed to provide binding sites for specific reader proteins, which translate histone code into biological function. Here we show that a family of acidic domain-containing proteins including nucleophosmin (NPM1), pp32, SET/TAF1 beta, nucleolin (NCL) and upstream binding factor (UBF) are novel H3K4me2-binding proteins. These proteins exhibit a unique pattern of interaction with methylated H3K4, as their binding is stimulated by H3K4me2 and inhibited by H3K4me1 and H3K4me3. These proteins contain one or more acidic domains consisting mainly of aspartic and/or glutamic residues that are necessary for preferential binding of H3K4me2. Furthermore, we demonstrate that the acidic domain with sufficient length alone is capable of binding H3K4me2 in vitro and in vivo. NPM1, NCL and UBF require their acidic domains for association with and transcriptional activation of rDNA genes. Interestingly, by defining acidic domain as a sequence with at least 20 acidic residues in 50 continuous amino acids, we identified 655 acidic domain-containing protein coding genes in the human genome and Gene Ontology (GO) analysis showed that many of the acidic domain proteins have chromatin-related functions. Our data suggest that acidic domain is a novel histone binding motif that can differentially read the status of H3K4 methylation and is broadly present in chromatin-associated proteins.