01_introduction.ipynb 2.93 KB
Newer Older
Eva Zangerle's avatar
Eva Zangerle committed
1
2
3
4
5
6
7
{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "500bd02c-eee8-45e6-a301-3482638767de",
   "metadata": {},
   "source": [
8
    "# Data Engineering and Analytics\n",
Eva Zangerle's avatar
Eva Zangerle committed
9
    "Master Software Engineering\n",
10
    "\n",
Eva Zangerle's avatar
Eva Zangerle committed
11
12
    "Eva Zangerle\n",
    "\n",
Eva Zangerle's avatar
Eva Zangerle committed
13
14
15
16
17
18
19
20
21
22
    "## Overview of Notebooks\n",
    "* Datasets: [3_datasets.ipynb](3_datasets.ipynb)\n",
    "* Data Preparation and Quality: [4_data_preparation_quality.ipynb](4_data_preparation_quality.ipynb)\n",
    "* Feature Engineering: [5_feature_engineering.ipynb](5_feature_engineering.ipynb)\n",
    "* Dataset Analyses: [6_dataset_analyses.ipynb](6_dataset_analyses.ipynb)\n",
    "* Hypotheses and Evaluation: [7_hypotheses_evaluation.ipynb](7_hypotheses_evaluation.ipynb)\n",
    "* Modeling and Prediction: [8_modeling_prediction.ipynb](8_modeling_prediction.ipynb)\n",
    "* Reproducible Research: [9_reproducible_research.ipynb](9_reproducible_research.ipynb)\n",
    "\n",
    "\n",
Eva Zangerle's avatar
Eva Zangerle committed
23
24
    "## General Notes\n",
    "* Code is partly taken from further sources, such as books.\n",
25
    "* Sources are annotated (and acknowledged!) as follows:\n",
Eva Zangerle's avatar
Eva Zangerle committed
26
    "    * (CleaningData): Cleaning Data for Effective Data Science: Doing the other 80% of the work with Python, R, and command-line tools; David Mertz; Packt Publishing, 2021; [Github repo](https://github.com/PacktPublishing/Cleaning-Data-for-Effective-Data-Science/)\n",
27
    "* Unless marked otherwise, code was written by Eva Zangerle.\n",
Eva Zangerle's avatar
Eva Zangerle committed
28
    "* I deliberately mix different Python packages (e.g., for visualization matplotlib, pandas and seaborn) to showcase their use.\n",
29
30
31
32
33
34
35
36
37
38
39
40
41
    "\n",
    "\n",
    "\n",
    "## Virtual environments\n",
    "\n",
    "![xkcd python environment](https://imgs.xkcd.com/comics/python_environment.png)\n",
    "\n",
    "[Comic taken from XKCD Comics https://xkcd.com/1987/ (CC-BY)]\n",
    "\n",
    "\n",
    "\n",
    "Good tutorial on pipenv and jupyter(-lab): https://towardsdatascience.com/virtual-environments-for-data-science-running-python-and-jupyter-with-pipenv-c6cb6c44a405#\n",
    "\n",
Eva Zangerle's avatar
Eva Zangerle committed
42
43
    "\n",
    "## Useful python stuff\n",
44
45
46
47
48
    "* Startup files: https://ipython.readthedocs.io/en/stable/interactive/tutorial.html#startup-files\n",
    "* tqdm progress bars (also for Jupyter): https://github.com/tqdm/tqdm\n",
    "\n",
    "## Further tools\n",
    "* jq command linen json processor: https://stedolan.github.io/jq/\n"
Eva Zangerle's avatar
Eva Zangerle committed
49
   ]
Eva Zangerle's avatar
Eva Zangerle committed
50
51
52
53
54
55
56
57
58
59
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6860d72-11e4-4574-b20b-7884a6653abc",
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
Eva Zangerle's avatar
Eva Zangerle committed
60
61
62
63
  }
 ],
 "metadata": {
  "kernelspec": {
64
   "display_name": "Python 3 (ipykernel)",
Eva Zangerle's avatar
Eva Zangerle committed
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}