| --- |
| license: apache-2.0 |
| --- |
| # MoE Car Model |
|
|
| ## Overview |
| The MoE (Mixture of Experts) Car Model is a deep learning model designed for autonomous driving and vehicle behavior prediction. It leverages a Mixture of Experts architecture to optimize decision-making across different driving scenarios, improving efficiency and adaptability in real-world environments. |
|
|
| ## WARNING: THIS MAY SHOW UNSAFE AS THIS RUNS ResNET WHEN YOU USE THE MODEL |
| ## Model Architecture |
| The MoE Car Model consists of the following key components: |
|
|
| - **Input Layer:** Accepts sensory data (camera images, LiDAR, GPS, IMU, etc.). |
| - **Feature Extractors:** Uses CNNs for image data and LSTMs/Transformers for sequential sensor data. |
| - **Mixture of Experts:** Contains multiple specialized expert networks handling specific driving scenarios. |
| - **Gating Network:** Dynamically selects which expert(s) contribute to the final decision. |
| - **Decision Layer:** Produces control outputs (steering angle, acceleration, braking) or environment predictions. |
|
|
| ### Model Parameters |
| - **Total Parameters:** ~40m parameters |
| - **Number of Experts:** 16 |
| - **Expert Architecture:** Transformer-based with 12 layers per expert |
| - **Gating Network:** 4-layer MLP with softmax activation |
| - **Feature Extractors:** ResNet-50 for images, Transformer for LiDAR/GPS |
|
|
| ## Training Details |
| - **Dataset:** 10 million driving scenarios from real-world and simulated environments |
| - **Batch Size:** 128 |
| - **Learning Rate:** 2e-4 (decayed using cosine annealing) |
| - **Optimizer:** AdamW |
| - **Training Time:** 1h 24m 28s |
| - **Hardware:** 1x 16gb T4 |
| - **Framework:** PyTorch |
|
|
| ## Inference |
| To run inference using the MoE Car Model: |
|
|
| ### Install Dependencies |
| ```bash |
| pip install torch torchvision numpy opencv-python |
| ``` |
|
|
| ### Load and Run the Model |
| ```python |
| import torch |
| import torchvision.transforms as transforms |
| import cv2 |
| from model import MoECarModel # Assuming model implementation is in model.py |
| |
| # Load model |
| model = MoECarModel() |
| model.load_state_dict(torch.load("moe_car_model.pth")) |
| model.eval() |
| |
| # Preprocessing function |
| def preprocess_image(image_path): |
| image = cv2.imread(image_path) |
| image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) |
| transform = transforms.Compose([ |
| transforms.ToPILImage(), |
| transforms.Resize((224, 224)), |
| transforms.ToTensor(), |
| transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]) |
| ]) |
| return transform(image).unsqueeze(0) |
| |
| # Load sample image |
| image_tensor = preprocess_image("test_image.jpg") |
| |
| # Run inference |
| with torch.no_grad(): |
| output = model(image_tensor) |
| print("Predicted control outputs:", output) |
| ``` |
|
|
| ## Applications |
| - Autonomous driving |
| - Driver assistance systems |
| - Traffic behavior prediction |
| - Reinforcement learning simulations |
|
|
| ## Future Improvements |
| - Optimization for edge devices |
| - Integration with real-time sensor fusion |
| - Reinforcement learning fine-tuning |
|
|
| --- |
