Newer
Older
========================================================================
DONE
- change all VS to differential.xml
- changed GRU1 to GRU4
- permos/geosat adaptations
- gsn2 experimental download
- image download works
- check column naming gnss raw/derived data products (was iso T/F switch)
- check float_format=5 usage gnss, is it necessary?
- check all files have same ISO8601 timestamps (gnss derived!!!)
- column names with no units do not carry []
- rename some dirs/files and subroutines
- added switch for generating yearly files
- default behavior generating only total files (raw/derived)
- check tool default behavior with gsn1
- data_availability plots not showing binary file data
NEEDS MORE TESTING
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
- processing time should be formated as ISO 8601
- download recursive for >500'000 items
Statistics
========================================================================
<!--
more mh_data/gnss_data_raw/*/*.*O |grep G|wc - l
ls mh_data/gnss_data_raw/*/*.*D.gz|wc
du -sm mh_data/*
more mh_data/gnss_derived_data_products/*daily.csv|wc -l
find -type f -name '*.JPG'|wc -l
more mh_data/timeseries_data_raw/*.csv | wc -l
ls mh_data/timeseries_data_raw/|wc -l
more mh_data/timeseries_derived_data_products/*.csv | wc -l
ls mh_data/timeseries_derived_data_products/ | wc -l
ls mh_data/timeseries_plots/ | wc -l
ls mh_data/timeseries_plots/channelwise/ | wc -l
-->
## Matterhorn Data
| Directory | Data Description | Format | Data Points | Files | Size |
| -------------------------------- | ------------------------------- | ---------- | -------------- | ------ | -------- |
| gnss_data_raw | GNSS raw observations | RINEX 2.11 | 74'543'752 | 17'045 | 43668 MB |
| gnss_derived_data_products | daily position data | csv | 16'712 | 6 | 1 MB |
| timelapse_images | time lapse images | jpg | 49'348 | 49'348 | 67008 MB |
| timeseries_data_raw | raw primary sensor data | csv | 152'449'042 | 44 | 3743 MB |
| timeseries_derived_data_products | data after cleaning/aggregation | csv | 4'553'745 | 44 | 112 MB |
| timeseries_plots | standard plots for all data | png | - | 270 | 91 MB |
| matterhorn_nodepositions.xlsx | general metadata file | xlsx | - | 1 | - |
| README.md | - | md | - | 1 | - |
| -------------------------------- | ------------------------------- | ---------- | -------------- | ------ | -------- |
| Total | - | - | 231'612'599 | 66'487 | 114.5 GB |