Running PySpark & Jupyter With Docker

Calendar Jun 8, 2023
Setup
Spark, Docker, Jupyter

Thanks to the Jupyter community, it’s now much easier to run PySpark on Jupyter using Docker. There are two ways you can do this : 1. the “direct” way and 2. the customized way.

The “direct” way#

  • verify your local settings are aligned with the pre-requisites to run this container, grosso modo make sure docker is installed, of course !

    You have to have about 4 GB of free space