Kafka topic partitions to Spark streaming : <p>I have some use cases that I would like to be more clarified, about Kafka topic partitioning -> spark streaming resource utilization.</p>

<p>I use spark standalone mode, so only settings I have are "total number of executors" and "executor memory". As far as I know and according to documentation, way to introduce parallelism into Spark streaming is using partitioned Kafka topic -> RDD will have same number of partitions as kafka, when I use spark-kafka direct stream integration.</p>

<p>So if I have 1 partition in the topic, and 1 executor core, that core will sequentially read from Kafka.</p>

<p>What happens if I have:</p>

<ul>
<li><p>2 partitions in the topic and only 1 executor core? Will that core read first from one partition and then from the second one, so there will be no benefit in partitioning the topic?</p></li>
<li><p>2 partitions in the topic and 2 cores? Will then 1 executor core read from 1 partition, and second core from the second partition?</p></li>
<li><p>1 kafka partition and 2 executor cores?</p></li>
</ul>

<p>Thank you.</p>
