Feature Engineering
I want to go through the Wikipedia series on Machine Learning and Data mining. Data mining is the process of extracting and discovering patterns in large data sets involving methods at the intersection of machine learning, statistics, and database systems.
References
Notes
Feature engineering is a preprocessing step in supervised machine learning and statistical modeling which transforms raw data into a more effective set of inputs. Each input comprises several attributes, known as features. By providing models with relevant information, feature engineering significantly enhances their predictive accuracy and decision-making capability.
Feature engineering is also used in:
- Physics in the construction of dimensionless numbers: c
- Reynolds number in fluid dynamics
- Nusselt number in heat transfer
- Archimedes number in sedimentation
One of the applications of feature engineering has been clustering of feature-objects in a dataset.
Feature engineering in machine learning and statistical modeling involves selecting, creating, transforming, and extracting data features. Key components include feature creating from existing data, transforming and imputing missing or invalid features, reducing data dimensionality through methods like PCA, and selecting the most relevant features for model training based on importance scores and correlation matrices.
Features vary in significance. Feature selection can reduce the number of features to prevent a model from becoming too specific to the training data set (overfitting). Feature explosion occurs when the number of identified features is too large for effective model estimation or optimization. Common causes include:
- Feature templates - implementing feature templates instead of coding new features
- Feature combinations - combinations that cannot be represented by a linear system
Automation of feature engineering is a research topic that dates back to the 1990s. Machine learning software that incorporates automated feature engineering has been commercially available since 2016.
Comments
You have to be logged in to add a comment
User Comments
There are currently no comments for this article.