Improve PySpark DataFrame.show output to fit Jupyter notebook : <p>Using PySpark in a Jupyter notebook, the output of Spark's <code>DataFrame.show</code> is low-tech compared to how Pandas DataFrames are displayed. I thought "Well, it does the job", until I got this:</p>

<p><a href="https://i.stack.imgur.com/1k6OP.png" rel="noreferrer"><img src="https://i.stack.imgur.com/1k6OP.png" alt="enter image description here"></a></p>

<p>The output is not adjusted to the width of the notebook, so that the lines wrap in an ugly way. Is there a way to customize this? Even better, is there a way to get output Pandas-style (without converting to <code>pandas.DataFrame</code> obviously)?</p>
