Skip to content

Fix CONDA_DIR and give priority to pyspark #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
keineahnung2345 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Fix CONDA_DIR and give priority to pyspark #3

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ RUN apt-get update && \
apt-get clean && \
rm -rf /var/tmp /tmp /var/lib/apt/lists/*

ENV CONDA_DIR="/conda"
ENV CONDA_DIR="/root/anaconda3"
ENV PATH="$CONDA_DIR/bin:$PATH"
RUN curl -sSL -o installer.sh https://repo.continuum.io/archive/Anaconda3-4.4.0-Linux-x86_64.sh && \
bash /installer.sh -b -f && \
rm /installer.sh

ENV PATH "$PATH:/root/anaconda3/bin"
ENV PATH "$CONDA_DIR/bin:$PATH"

# SPARK
ARG SPARK_ARCHIVE=https://d3kbcqa49mib13.cloudfront.net/spark-2.2.0-bin-hadoop2.7.tgz
Expand All @@ -41,6 +41,7 @@ ENV PATH $PATH:$SPARK_HOME/bin
# CUSTOM CONDA-ENV
RUN conda create -n pyspark python=3.5
RUN ["/bin/bash", "-c", "source activate pyspark; apt-get install libstdc++; conda install jupyter;conda install ipykernel; python -m ipykernel install --user --name pyspark --display-name pyspark; source deactivate"]
ENV PATH "$CONDA_DIR/envs/pyspark/bin:$PATH"

# JUPYTER CONFIG
COPY jupyter_notebook_config.py /root/.jupyter/
Expand Down