Abstract
The trend of writing fake reviews has recently increased with the rapid growth of e-commerce websites. This research addresses the challenge of detecting fake reviews in Roman Urdu, a low-resource language, by introducing novel user behavioral features alongside traditional textual and lingual features. We propose a comprehensive methodology that combines machine learning and deep learning approaches to enhance fake review detection.
Our study evaluates multiple experimental configurations including class imbalance handling techniques such as LSTM-based text generation, random undersampling (RUS), and oversampling (ROS). Through empirical comparison of machine learning and deep learning algorithms, we demonstrate a 3% accuracy improvement over baseline studies using Gradient Boosting.
Key Contributions
🎯 Novel Feature Engineering
Identification of three types of discriminative features: Review Textual Features (RTF), Review Lingual Features (RLF), and Review Behavioral Features (UBF).
⚖️ Class Imbalance Handling
Evaluation of LSTM-based text generation, random undersampling (RUS), and oversampling (ROS) techniques for balanced classification.
🔬 Comprehensive Evaluation
Empirical comparison of machine learning and deep learning algorithms with systematic experimental configurations.
📈 Performance Enhancement
3% accuracy improvement over baseline studies using Gradient Boosting with novel behavioral features.
Methodology
.png)
Research Methodology: Comprehensive 6-step pipeline for detecting fake reviews in Roman Urdu
Pipeline Overview:
- Text Preprocessing: Remove special characters, whitespace, and single character words; Apply Roman Urdu stopword removal
- Feature Extraction: Extract Review Textual Features (RTF), Review Lingual Features (RLF), and User Behavioral Features (UBF)
- Data Split: 90% for training and validation, 10% for testing
- Resampling: Apply hybrid oversampling and undersampling techniques
- Classification: Use both Deep Learning and Machine Learning models including Gradient Boosting
- Model Evaluation: Assess performance using accuracy, precision, recall, F-score, and confusion matrix
Feature Types
📝 Textual Features (RTF)
- TF-IDF vectors
- N-gram features
- Text length and structure
🔤 Lingual Features (RLF)
- Language-specific patterns
- Roman Urdu linguistic characteristics
- Sentiment indicators
👤 Behavioral Features (UBF)
- User review patterns
- Review frequency analysis
- Rating behavior
- Temporal patterns
Experimental Settings
Multiple Configuration Evaluation:
- Setting I: Default train-test split with ML algorithms
- Setting II: Cross-validation evaluation
- Setting III: Feature selection techniques
- Setting IV: Class imbalance handling (ROS, RUS, SMOTE)
- Setting V: Feature selection + Undersampling (Chi-square, IG, RFE)
Key Results
Best Performance: Gradient Boosting Classifier achieved the highest accuracy with textual + lingual + behavioral features using random undersampling and feature selection.
🏆 Best Algorithm
Gradient Boosting Classifier
🎯 Feature Combination
Textual + Lingual + Behavioral
⚙️ Best Preprocessing
Random Undersampling + Feature Selection
📊 Accuracy Improvement
3% over baseline studies
Dataset distribution and statistics
Word cloud of cleaned review data
Experimental Results
Key Findings: User behavioral features play a vital role in detecting fake reviews. Text generation is ineffective for balancing textual data. Gradient Boosting outperformed other models with a 3% accuracy improvement over baseline studies.
| Experimental Setting | Technique | Best Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|---|---|---|---|---|---|
| Setting I | Default ML | Gradient Boosting | 0.92 | 0.91 | 0.93 | 0.92 |
| Setting III | Feature Selection | Random Forest | 0.89 | 0.88 | 0.90 | 0.89 |
| Setting IV | RUS | Gradient Boosting | 0.91 | 0.90 | 0.92 | 0.91 |
| Setting IV | SMOTE | Support Vector Machine | 0.88 | 0.87 | 0.89 | 0.88 |
| Setting V | RFE + RUS | Gradient Boosting | 0.90 | 0.89 | 0.91 | 0.90 |
Implementation & Usage
Open Source Implementation: All experimental code, datasets, and results are available for reproducibility and further research in Roman Urdu fake review detection.
Repository Contents:
-
Dataset: Processed Roman Urdu review data
with extracted features (
clean_train.csv,clean_test.csv) -
Notebooks: Feature extraction
(
Experiment_Feature_Extraction.ipynb) and baseline model implementations (Train_and_test_baseline.ipynb) -
Experiments: Complete experimental code for
ML and DL approaches in
ExperimentalCodes_updated/ -
Results: Detailed performance metrics and
visualizations in
Experimental results/
Quick Start:
# Clone the repository
git clone https://github.com/AbdulManaf12/Roman-Urdu-fake-review-detection.git
cd Roman-Urdu-fake-review-detection
# Install requirements
pip install pandas numpy matplotlib seaborn scikit-learn
pip install nltk imbalanced-learn tensorflow keras
# Run feature extraction
jupyter notebook Experiment_Feature_Extraction.ipynb
# Train baseline model
jupyter notebook Train_and_test_baseline.ipynb
Future Directions
Research Extensions:
- Cross-lingual Studies: Extend to other low-resource languages
- Real-time Detection: Develop online fake review detection systems
- Multi-modal Analysis: Incorporate images and ratings data
- Explainable AI: Develop interpretable models for fake review detection
- Advanced Deep Learning: Explore transformer-based models for Roman Urdu
Contact & Citation
Research Team
Nimra Mughal: nimra.mscsf19@iba-suk.edu.pk
Abdul Manaf: abdul.manaf@iba-suk.edu.pk
Ghulam Mujtaba: mujtaba@iba-suk.edu.pk
This research contributes to the growing field of fake review detection in low-resource languages and demonstrates the importance of user behavioral features in enhancing detection accuracy for Roman Urdu e-commerce reviews.