#Install spark on windows with jupyter install
Perhaps you want to install with `sudo` or `-user`? In Linux if you get an error: Permission denied: '/usr/local/share/jupyter' !jupyter-kernelspec install c:\users\youruserid\appdata\local\continuum\anaconda3\lib\site-packages\sparkmagic\kernels\sparkkernel Next run the following commands in the Jupyter notebook: Then we will need to run the above command from a linux terminal using root login. Note: If you get an error in linux: PermissionError: Permission denied: ‘/opt/anaconda3/etc/jupyter/nbconfig/notebook.json’ !jupyter nbextension enable –py –sys-prefix widgetsnbextensionĮnabling notebook extension jupyter-js-widgets/extension… Next run the below command in the Jupiter Notebook: Summary: SparkMagic: Spark execution via LivyĬopy the example_config.json into ~/.sparkmagic/config.json (Usually in Windows the folder location will be C:\Users\youruserid\.sparkmagic\config.json ). Change the username, password and url of the Livy server.Ĭlick Windows->Start->Anaconda Navigator and Launch Jupiter Lab. Also if you see the error: distributed 1.21.8 requires msgpack, which is not installed. Then you may need to run the $ pip install sparkmagic after login as root in a linux terminal. Consider using the `–user` option or check the permissions. Note: If you get an error under linux that says:Ĭould not install packages due to an EnvironmentError: Permission denied: ‘/opt/anaconda3/lib/python3.6/site-packages/pbr’.
It gives a successful install message in the output that Sparkmagic-0.12.5 is installed along with few other packages. Run the following command on the notebook. On the Jupyterlab browser notebook and tried to load sparkmagic using:ĥ. After successful install of the package on Windows click on the Anaconda Navigator in the Windows Start. Download the Anaconda installer from which was Anaconda 5.0.1 For Windows Installer Python 3.6 version 64-bit.ģ.
We strongly recommend installing Python and Jupyter using the Anaconda Distribution, which includes Python, the Jupyter Notebook, and other commonly used packages for scientific computing and data science.Ģ.
#Install spark on windows with jupyter windows 10
We will first install Anaconda and Sparkmagic on Windows 10 to install Jupyter Notebook using Anaconda. Kerberos based Hadoop cluster needs different setup and these instructions wont work. Anaconda parcel installed using Cloudera Manager (see other blog on this website to install Anaconda parcel on CDH).Cloudera CDH 5.12.x running Livy and Spark (see other blog on this website to install Livy).
0 kommentar(er)
