In this article we apply information theory to the multivariate data analysis problem. Given a dependent variable Y and any independent variable X, we can associate with X a number that would indicate the importance of X with respect to Y. This procedure enables us to select variables based on their importance. Two sets of data (upper abdominal pain and character recognition) are analyzed by this approach. The results indicate that this approach is feasible.