Wine label information was found to have a positive effect on consumer choice. However, it is not easy for consumers to read the information on wine labels since these texts are usually not print in English. Wine label information system with visual recognition function seems to be important as an auxiliary reading tool for customers. In this paper, we proposed an efficient wine image label recognition method using convolutional neural network(CNN) algorithms. Data augmentation techniques for machine learning with different image manipulation such as cropping, rotation and hue modification were applied to enlarge the dataset. Experimental results show a high accuracy on 130 different wine label images using the proposed CNN model with mixing different image manipulation.