Update README.md

README.md

@@ -295,11 +295,8 @@ print(f"Peak Memory Usage: {mem:.02f} GB")
 |                                  | Phi-4 mini-Ins | Phi-4-mini-instruct-FP8         | 
 | latency (batch_size=1)           | 1.61s          | 1.25s (1.29x speedup)           |
 | latency (batch_size=256)         | 5.16s          | 4.89s (1.05x speedup)           |
-| serving (num_prompts=1)          | 1.37 req/s     | 1.66 req/s (1.21x speedup)      |
-| serving (num_prompts=1000)       | 62.55 req/s    | 72.56 req/s (1.16x speedup)     |
 
 Note the result of latency (benchmark_latency) is in seconds, and serving (benchmark_serving) is in number of requests per second.
-Note the result is not using fbgemm kernels, (no `fbgemm-gpu-genai` installed), fbgemm kernels has less speedup when num_prompts is 1000 currently.
 
 <details>
 <summary> Reproduce Model Performance Results </summary>
@@ -334,41 +331,6 @@ python benchmarks/benchmark_latency.py --input-len 256 --output-len 256 --model
 VLLM_DISABLE_COMPILE_CACHE=1 python benchmarks/benchmark_latency.py --input-len 256 --output-len 256 --model pytorch/Phi-4-mini-instruct-FP8 --batch-size 1
 ```
 
-## benchmark_serving
-
-We benchmarked the throughput in a serving environment.
-
-Download sharegpt dataset:
-
-```Shell
-wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
-```
-
-Other datasets can be found in: https://github.com/vllm-project/vllm/tree/main/benchmarks
-
-Note: you can change the number of prompts to be benchmarked with `--num-prompts` argument for `benchmark_serving` script.
-### baseline
-Server:
-```Shell
-vllm serve microsoft/Phi-4-mini-instruct --tokenizer microsoft/Phi-4-mini-instruct -O3
-```
-
-Client:
-```Shell
-python benchmarks/benchmark_serving.py --backend vllm --dataset-name sharegpt --tokenizer microsoft/Phi-4-mini-instruct --dataset-path ./ShareGPT_V3_unfiltered_cleaned_split.json --model microsoft/Phi-4-mini-instruct --num-prompts 1
-```
-
-### FP8
-Server:
-```Shell
-VLLM_DISABLE_COMPILE_CACHE=1 vllm serve pytorch/Phi-4-mini-instruct-FP8 --tokenizer microsoft/Phi-4-mini-instruct -O3
-```
-
-Client:
-```Shell
-python benchmarks/benchmark_serving.py --backend vllm --dataset-name sharegpt --tokenizer microsoft/Phi-4-mini-instruct --dataset-path ./ShareGPT_V3_unfiltered_cleaned_split.json --model pytorch/Phi-4-mini-instruct-FP8 --num-prompts 1
-```
-
 </details>
 
 # Paper: TorchAO: PyTorch-Native Training-to-Serving Model Optimization