Data Science Notes: Difference between revisions
Jump to navigation
Jump to search
Adamcolton (talk | contribs) |
Adamcolton (talk | contribs) |
||
| Line 31: | Line 31: | ||
=== The Course : 1 === | === The Course : 1 === | ||
http://static.usenix.org/event/osdi04/tech/full_papers/dean/dean.pdf | |||
Hadoop was an early MapReduce system that did away with old database standards, but in the 4 years after it's release, most SQL standards were brought back into the fold. The abstractions are important not just the tools. | |||
MapReduce contributed the following the SQL | |||
* Fault tolerance | |||
* schema-on-read | |||
* user defined functions that don't suck | |||
=== The Course : 2 === | === The Course : 2 === | ||
=== eScience === | === eScience === | ||
Revision as of 16:13, 1 May 2013
https://class.coursera.org/datasci-001/class/index
Introduction
Context
Bill Howe venn diagram. "Danger Zone" is hacking + expertise without statistics.
Mike Driscolls 3 DS skills
- Statistics
- Data Munging
- Visualization
Dimensions
DS is not (or not quite)
- Business Intelligence
- Statistics
- Data(base) Management
- Visualization
- Machine Learning
If you're a → You need to learn
- DBA → unstructured data
- statistician → data that exceeds memory
- software engineer → statistical modeling
- business analyst → algorithms and tradeoffs at scale
The Continuums
- Tools vs Abstractions
- Desktop vs cloud
- hackers vs analysts
- structures vs statistics
The Course : 1
http://static.usenix.org/event/osdi04/tech/full_papers/dean/dean.pdf
Hadoop was an early MapReduce system that did away with old database standards, but in the 4 years after it's release, most SQL standards were brought back into the fold. The abstractions are important not just the tools.
MapReduce contributed the following the SQL
- Fault tolerance
- schema-on-read
- user defined functions that don't suck