andrew-chan1181 - Tumblr blog

[andrew-chan1181]

I Back

I don’t know there has anyone can know or read my tumblr blog, but I think here is my new thing to study

[andrew-chan1181]

安裝 jupyter (Python 執行程式＿利用瀏覽器執行)

1.環境建置

brew install python3

      安裝Python3

pip3 install jupyter

      用pip安裝jupyter套件

jupyter notebook

2.爬蟲用套件

pip3 install requests

pip3 install beautifulsoup4

       Python3用beautifulsoup4抓網頁

sudo pip3 install HTMLParser

      Python3用，抓網頁

 pip3 install urlopen

pip3 install parse

pip3 install urlparse

pip3 install urljoin

install urljoin

sudo pip3 install urlparse

sudo pip3 install urllib

pip3 install urllib3

3. 結巴套件

pip3 install jieba

      Python3用jieba

jupyter notebook

[andrew-chan1181]

OSX 單機版 Spark

Macbook pro 版本：MacBook Pro (13-inch, Mid 2012)

OSX  版本：10.12.3

安裝日期： 2017/02/21

1. 安裝 brew ,替代 apt-get（Mac 沒這個功能） :

 curl -LsSf http://github.com/mxcl/homebrew/tarball/master | sudo tar xvz -C/usr/local --strip 1 

2. 因為要安裝scala 以及其他的軟體，所以要轉移權限

sudo chown -R $(whoami) /usr/local

參考資料：https://segmentfault.com/q/1010000007575129

3. brew install scala

4.wget http://apache.mirrors.ionfish.org/spark/spark-2.1.0/spark-2.1.0-bin-hadoop2.7.tgz

5.tar -zxvf spark-2.1.0-bin-hadoop2.7.tgz

6.cd spark-2.1.0-bin-hadoop2.7

   cd bin/

   ./spark-shell