Despite significant advances in surgery, chemotherapy, radiotherapy, endocrine therapy, and molecular-targeted therapy, breast cancer remains the leading cause of death from malignant tumors among women. Immunotherapy has recently become a critical component of breast cancer treatment with encouraging activity and mild safety profiles. CAR-T therapy using genetically modifying T cells with chimeric antigen receptors (CAR) is the most commonly used approach to generate tumor-specific T cells. It has shown good curative effect for a variety of malignant diseases, especially for hematological malignancies. In this review, we briefly introduce the history and the present state of CAR research. Then we discuss the barriers of solid tumors for CARs application and possible strategies to improve therapeutic response with a focus on breast cancer. At last, we outlook the future directions of CAR-T therapy including managing toxicities and developing universal CAR-T cells.