Spark-Shell Startup Errors : <p>I am seeing errors when starting <code>spark-shell</code>, using <code>spark-1.6.0-bin-hadoop2.6</code>. This is new behavior that just arose.</p>

<p>The upshot of the failures displayed in the log messages below, is that sqlContext is not available (but sc is).</p>

<p>Is there some kind of Derby lock that could be released? <code>Another instance of Derby may have already booted the database /root/spark-1.6.0-bin-hadoop2.6/bin/metastore_db.</code></p>

<pre><code>&lt;console&gt;:16: error: not found: value sqlContext
         import sqlContext.implicits._
                ^
&lt;console&gt;:16: error: not found: value sqlContext
         import sqlContext.sql

16/05/25 11:00:00 ERROR Schema: Failed initialising database.
Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@c2191a8, see the next exception for details.
org.datanucleus.exceptions.NucleusDataStoreException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@c2191a8, see the next exception for details.


16/05/25 11:06:02 WARN Hive: Failed to access metastore. This class should not accessed in runtime.
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient


16/05/25 11:06:02 ERROR Schema: Failed initialising database.
Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@372e972d, see the next exception for details.
org.datanucleus.exceptions.NucleusDataStoreException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@372e972d, see the next exception for details.

Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@c2191a8, see the next exception for details.
        at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
        ... 134 more
Caused by: java.sql.SQLException: Another instance of Derby may have already booted the database /root/spark-1.6.0-bin-hadoop2.6/bin/metastore_db.
        at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
        ... 131 more
Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /root/spark-1.6.0-bin-hadoop2.6/bin/metastore_db.
</code></pre>
