Python-powered Analysts @WesMackinn

  • Author for “Python for Data Analysis”
  • Python’s role in 2014 for analysis
  • BI
    • Query 
    • Report ordinary
    • Alert when trend change
  • ETL (Extract Transform Load)
    • moving data between storage format
    • Normalization
  • How did we get here?

    • Good develop tool —  iPython

    • Learning resource:

    • nbviewer.ipython.org

      • static html share space
    • The libraries packaging tech become more maturing.
  • Why did pandas succeed?

    • tasteful and consistent solutions

    • passionate user base

    • An API optimized for 

    • terminal-freindliness

    • composability 
  •  Tools before Data Analysis

    • Cython

    • NumPy

    • matplotlib

    • ipython

  • Python might not best language for Data Analysis, but everyone can using python to discussion.

  • Scikit-learn http://scikit-learn.org/stable/

    • Machine learning in python.
  • Pandas: The good and bad

    • Ecosystem compatibility

    • Design for broad appeal

    • The big data problem.

  • Big Data Trend

    • Python/R/Julia growing 

    • JVM-base big data ecosystem

    • Concurrent / multicore programming challenges

  • Staying competitive

    • “Enterprise money” is still stay in JVM-Ecpsystem
  • A bit about DataPad http://www.datapad.io/

    • Exploratory analytics and report-building environment

    • Powerful analysis tool

    • No coding

    • Collaboration and sharing tools for teams

  • Note - FAQ:

    • What DB is DataPad.io

    • DB is propitiatory 

d3 with Python

https://pyconapac2014.hackpad.com/Real-time-visualization-with-Python-and-d3.js-73-LGLyhjzr5qY

 


Python to LiveScript

A speech for how to jump LiveScript to Python

  • LiveScript

    • Concise Syntax

    • say bye to bad parts of javascript

    • many improvement for OOP

    • many features of FP

  • Language Change:

    • JS+Ruby = CoffeeScript

    • CoffeeScrip

  • Some note:

    • $ is “this”

    • Bond range for indent

    •  string until space  =>    abc = “abc”

    • comment

  • Check how to use module (or check API)

    • Python

    • dir(path)

    • LiveScript

    • Object.keys path
  • Object access

    • Python [0]

    • LS: .0 


LiveMedia and ffmpeg

https://pyconapac2014.hackpad.com/VapourSynth-comes-does-it-indicate-that-AviSynth-will-shutdown-29-Dr34xdWrkzz

 


Docker shipping Python project by Docker


Python and Science http://fperez.org/

  • The ideal and reality of science

    • Using code to validate 

    • More cited in CV

  • Open Source Software in this context

    • Open, collaborative by definition

    • Response

  • The second best program language in the world

    • Maybe it is not the best solution but everyone can use it to communicate.
  • SciPy, NumPy are tool for scientist 

  • Python for scientists

    • separation of concerns

    • huge collaborate

  • Software-carpentry.org http://software-carpentry.org/

  • Data Science in Python CS 109 in harvad 

  • iPython

    • Has graphic UI

    • kernel can parse any content (HTML/PDF)

    • Can join Julia code and Python code.

    • Deep language integration

    • iPython can integrate in http part as a listener to hook web site to run python code and return result instantly.

    • One line code to get integrate UI for python result.

    • Can establish blog system

  • iPython on OSS

    • Enthoughy

    • Wakari.io

    • MSFT VS2013 has iPython integrated

    • StarCluster

    • IBM Watson (machine learning)

    • Authorea

    • Plotly (iPython with d3)

    • NBDiff.org http://nbdiff.org/

    • Google Research and iPython experiment

    • Collaborator Notebook (like http://nbviewer.ipython.org/ collaborate version)

小海嚴選

  • 為何想做小海嚴選

    • 個人興趣

    • 工作需求(老婆是經紀人)

  • FB “Like” is hard to collect and summarized (only in activity log)

  • 資訊爆炸的年代 (策展時代)

    • 資訊爆炸有人要你過濾資訊
  • FQL

    • select object_id from like where user_id is “me”
  • How to development

    • Using iPad for prototyping 

    • Prompt

    • Django framework

    • 會員系統  pip install django-userena

    • API djangorestframework

    • django-oauth2-provider
  • 動態縮圖

    • thumbor + CDN
  • 免費營運

    • 0元營運~

    • Heroku dyno 1

  • API 應用

  • Hubot plugin https://hubot.github.com/

Python in VIM

  • Python interface to VIM

    • :help python

    • :python :py   REPL python

    • :pydo return str(sum(map(int, line.split())))

    • 累加每一行

    • line 是每一行的代表

    • :pyf :pyfile

    • run current python code with python.
  • vim module

    • :py import vim

    • import vim as module using for

    • vim windows

    • vim.tabpages

    • vim.current

    • vim.current

    • vim buffer

    • vim.command(str)

    • using vim to operate this file or other file

    • vim.eval

    • the same with python.eval

    • vim.vars

    • Windows Object
  • example - Conque

  • example - python-mode

    • It could run python directly (lead key+R) lead key is assign by self

    • file syntax check


OpenStack 簡介

  • What is OpenStack

    • OpenStack build your own cloud

    • Restful api

  • Trystack

    • http://trystack.org/

    • Need FB account

  • devStack

  • Component

    • keystone - auth and service catalog 

    • nova - VM

    • glance - image service

    • cinder - block storage service

    • swift - object storage service

    • neutron -  network

  • Project

    • trove - database as service

    • sahar - provision a Hadoop cluster on top of OpenStack

    • taskflow - look like sql transaction

    • oslo messaging -

  • Component communication

    • REST api

    • AMQP


Evan

Attitude is everything