Skip to content

GitLab

Commit 9af9c3f7 authored by Eva Zangerle's avatar Eva Zangerle
Browse files

added imputation examples

parent cc611254
Expand all Show whitespace changes
Inline Side-by-side
Showing with 1299 additions and 444 deletions
data/dermatology.data
View file @ 9af9c3f7
This diff is collapsed.
data/dermatology.names 0 → 100644
View file @ 9af9c3f7
1. Title: Dermatology Database
2. Source Information:
(a) Original owners:
-- 1. Nilsel Ilter, M.D., Ph.D.,
Gazi University,
School of Medicine
06510 Ankara, Turkey
Phone: +90 (312) 214 1080
-- 2. H. Altay Guvenir, PhD.,
Bilkent University,
Department of Computer Engineering and Information Science,
06533 Ankara, Turkey
Phone: +90 (312) 266 4133
Email: guvenir@cs.bilkent.edu.tr
(b) Donor: H. Altay Guvenir,
Bilkent University,
Department of Computer Engineering and Information Science,
06533 Ankara, Turkey
Phone: +90 (312) 266 4133
Email: guvenir@cs.bilkent.edu.tr
(c) Date: January, 1998
3. Past Usage:
1. G. Demiroz, H. A. Govenir, and N. Ilter,
"Learning Differential Diagnosis of Eryhemato-Squamous Diseases using
Voting Feature Intervals", Aritificial Intelligence in Medicine,
The aim is to determine the type of Eryhemato-Squamous Disease.
4. Relevant Information:
This database contains 34 attributes, 33 of which are linear
valued and one of them is nominal.
The differential diagnosis of erythemato-squamous diseases is a real
problem in dermatology. They all share the clinical features of
erythema and scaling, with very little differences. The diseases in
this group are psoriasis, seboreic dermatitis, lichen planus,
pityriasis rosea, cronic dermatitis, and pityriasis rubra pilaris.
Usually a biopsy is necessary for the diagnosis but unfortunately
these diseases share many histopathological features as
well. Another difficulty for the differential diagnosis is that a
disease may show the features of another disease at the beginning
stage and may have the characteristic features at the following stages.
Patients were first evaluated clinically with 12 features.
Afterwards, skin samples were taken for the evaluation of 22
histopathological features. The values of the histopathological features
are determined by an analysis of the samples under a microscope.
In the dataset constructed for this domain, the family history feature
has the value 1 if any of these diseases has been observed in the
family, and 0 otherwise. The age feature simply represents the age of
the patient. Every other feature (clinical and histopathological) was
given a degree in the range of 0 to 3. Here, 0 indicates that the
feature was not present, 3 indicates the largest amount possible,
and 1, 2 indicate the relative intermediate values.
The names and id numbers of the patients were recently
removed from the database.
5. Number of Instances: 366
6. Number of Attributes: 34
7. Attribute Information:
-- Complete attribute documentation:
Clinical Attributes: (take values 0, 1, 2, 3, unless otherwise indicated)
1: erythema
2: scaling
3: definite borders
4: itching
5: koebner phenomenon
6: polygonal papules
7: follicular papules
8: oral mucosal involvement
9: knee and elbow involvement
10: scalp involvement
11: family history, (0 or 1)
34: Age (linear)
Histopathological Attributes: (take values 0, 1, 2, 3)
12: melanin incontinence
13: eosinophils in the infiltrate
14: PNL infiltrate
15: fibrosis of the papillary dermis
16: exocytosis
17: acanthosis
18: hyperkeratosis
19: parakeratosis
20: clubbing of the rete ridges
21: elongation of the rete ridges
22: thinning of the suprapapillary epidermis
23: spongiform pustule
24: munro microabcess
25: focal hypergranulosis
26: disappearance of the granular layer
27: vacuolisation and damage of basal layer
28: spongiosis
29: saw-tooth appearance of retes
30: follicular horn plug
31: perifollicular parakeratosis
32: inflammatory monoluclear inflitrate
33: band-like infiltrate
8. Missing Attribute Values: 8 (in Age attribute). Distinguished with '?'.
9. Class Distribution:
Database: Dermatology
Class code: Class: Number of instances:
1 psoriasis 112
2 seboreic dermatitis 61
3 lichen planus 72
4 pityriasis rosea 49
5 cronic dermatitis 52
6 pityriasis rubra pilaris 20
data/digits.npy 0 → 100644
View file @ 9af9c3f7
File added
src/4_data_preparation_quality.ipynb
View file @ 9af9c3f7
......@@ -12,7 +12,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 2,
"id": "792af709-0621-4d64-8166-8c8cc28cc73c",
"metadata": {},
"outputs": [],
......@@ -39,7 +39,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 3,
"id": "82672d3c-d574-47f1-b2f1-9a42b414e278",
"metadata": {},
"outputs": [],
......@@ -684,7 +684,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 6,
"id": "7b89a886-33ab-480c-bdf6-d2ac032991b5",
"metadata": {},
"outputs": [],
......@@ -751,9 +751,17 @@
"derm['Age'] = derm.Age.astype(float)"
]
},
{
"cell_type": "markdown",
"id": "45f6a92b-4b9c-4a81-bcb3-1fbac980f682",
"metadata": {},
"source": [
"**Todo for all:** have a look at dermatology.names and dermatology.data for further information on the data, its provenance and structure."
]
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": 7,
"id": "0348f05a-931a-43b1-8361-eb95e6a0118d",
"metadata": {},
"outputs": [
......@@ -786,33 +794,33 @@
" <tbody>\n",
" <tr>\n",
" <th>erythema</th>\n",
" <td>\"0</td>\n",
" <td>}</td>\n",
" <td>object</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>scaling</th>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>float64</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>definite borders</th>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>float64</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>itching</th>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>float64</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>koebner phenomenon</th>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>float64</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
......@@ -822,21 +830,21 @@
" </tr>\n",
" <tr>\n",
" <th>perifollicular parakeratosis</th>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>float64</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>inflammatory monoluclear inflitrate</th>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>float64</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>band-like infiltrate</th>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>float64</td>\n",
" <td>0</td>\n",
" <td>3</td>\n",
" <td>int64</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Age</th>\n",
......@@ -846,8 +854,8 @@
" </tr>\n",
" <tr>\n",
" <th>TARGET</th>\n",
" <td>None</td>\n",
" <td>None</td>\n",
" <td>cronic dermatitis</td>\n",
" <td>seboreic dermatitis</td>\n",
" <td>object</td>\n",
" </tr>\n",
" </tbody>\n",
......@@ -856,23 +864,36 @@
"</div>"
],
"text/plain": [
" min max dtype\n",
"erythema \"0 } object\n",
"scaling 0.0 3.0 float64\n",
"definite borders 0.0 3.0 float64\n",
"itching 0.0 3.0 float64\n",
"koebner phenomenon 0.0 3.0 float64\n",
"... ... ... ...\n",
"perifollicular parakeratosis 0.0 3.0 float64\n",
"inflammatory monoluclear inflitrate 0.0 3.0 float64\n",
"band-like infiltrate 0.0 3.0 float64\n",
"Age 0.0 75.0 float64\n",
"TARGET None None object\n",
" min max \\\n",
"erythema 0 3 \n",
"scaling 0 3 \n",
"definite borders 0 3 \n",
"itching 0 3 \n",
"koebner phenomenon 0 3 \n",
"... ... ... \n",
"perifollicular parakeratosis 0 3 \n",
"inflammatory monoluclear inflitrate 0 3 \n",
"band-like infiltrate 0 3 \n",
"Age 0.0 75.0 \n",
"TARGET cronic dermatitis seboreic dermatitis \n",
"\n",
" dtype \n",
"erythema int64 \n",
"scaling int64 \n",
"definite borders int64 \n",
"itching int64 \n",
"koebner phenomenon int64 \n",
"... ... \n",
"perifollicular parakeratosis int64 \n",
"inflammatory monoluclear inflitrate int64 \n",
"band-like infiltrate int64 \n",
"Age float64 \n",
"TARGET object \n",
"\n",
"[35 rows x 3 columns]"
]
},
"execution_count": 22,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
......
src/5_feature_engineering.ipynb
View file @ 9af9c3f7
This source diff could not be displayed because it is too large. You can view the blob instead.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment